Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On 24/08/12 06:09, Arnold Krille wrote:
> Hi,
>
> On Friday 24 August 2012 01:56:37 Adam Goryachev wrote:
>> I am using all software from Debian Stable with all updates/security
>> updates installed:
>> drbd8-utils 2:8.3.7-2.1
> I made good experience with compiling latest 8.3 from git and creating debs.
> Less bugs, more performance on debian stable.
I have a strong aversion to compiling from source, not because I can't,
but because the distro packaged versions are easier to maintain in
relation to "automatic" updates/etc
Would using the testing version 8.3.13 be up to date enough to help ?
>> Linux san1 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64
>> GNU/Linux
>>
>> My config file for DRBD is:
>> resource storage2 {
>> protocol A;
>> device /dev/drbd2 minor 2;
>> disk /dev/md1;
>> meta-disk internal;
>> on san1 {
>> address 172.17.56.1:7802;
>> }
>> on san2 {
>> address 172.17.56.2:7802;
>> }
>>
>> net {
>> after-sb-0pri discard-younger-primary;
>> after-sb-1pri discard-secondary;
>> after-sb-2pri call-pri-lost-after-sb;
>> max-buffers 8000;
>> max-epoch-size 8000;
>> unplug-watermark 4096;
>> sndbuf-size 512k;
>> }
>> startup {
>> wfc-timeout 10;
>> degr-wfc-timeout 20;
>> }
>> syncer {
>> rate 100M;
>> }
>> }
>>
>> root at san1:/etc/drbd.d# cat /proc/drbd
>> version: 8.3.7 (api:88/proto:86-91)
>> srcversion: EE47D8BF18AC166BE219757
>>
>> 2: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r----
>> ns:30035906 nr:0 dw:29914667 dr:42547522 al:49271 bm:380 lo:0 pe:0
>> ua:0 ap:0 ep:1 wo:b oos:0
>>
>> root at san1:/etc/drbd.d# dd if=/dev/zero of=/dev/mapper/vg0-testdisk
>> oflag=direct bs=1M count=100
>> 100+0 records in
>> 100+0 records out
>> 104857600 bytes (105 MB) copied, 2.48851 s, 42.1 MB/s
> With reads of 4MB or 8MB blocks you might get better values.
root at san1:/etc/drbd.d# dd if=/dev/zero of=/dev/mapper/vg0-testdisk
oflag=direct bs=4M count=25
25+0 records in
25+0 records out
104857600 bytes (105 MB) copied, 1.58077 s, 66.3 MB/s
However, the main point is that there is a massive difference when the
secondary is disconnected:
root at san1:/etc/drbd.d# dd if=/dev/zero of=/dev/mapper/vg0-testdisk
oflag=direct bs=8M count=25
25+0 records in
25+0 records out
209715200 bytes (210 MB) copied, 0.888038 s, 236 MB/s
> But beware: continuous reading is a very bad indicator for performance unless
> you do video-playback only. Better test with a real filesystem and a real
> benchmark, I am using dbench with great success as it gives the typical usage
> patterns of business computers.
In this case, there are two main types of "real usage", one is lots of
random small read/write as you suggest, though performance for this is
acceptable anyway. The other is a business process which effectively is
reading large files, and writing large files (basically cp file1 file2
file3 destdir). It is this case that we are seeing the very slow
performance, and dd replicates this slow performance very closely.
>> However, if I stop DRBD on the secondary:
>> Then I get good performance:
>> Should I increase the connection between the servers for sync to 2 x 1G
>> which would exceed max write speed of the disks on san2?
> First: get a dual-link, that made my disk be the limiting factor.
> Second: Use external metadata at least on the hdds and put the meta-data on a
> different disk. That made my dual-gigabit-link be the limiting factor again...
I would have thought a single ethernet should support closer to 100M/s,
but I'll arrange an additional crossover ethernet and see if that
improves things further.
>> Is there some other config option to say "I don't mind if san2 is behind
>> san1, just cache the changes and make them when you can", similar to
>> running DRBD over a slow remote connection?
> This is afaik what drbd-proxy does, but I leave it to the sales-people to
> explain what it does.
I thought that should be more useful for a real remote backup... Also, I
thought protocol A would allow for better performance even if the second
server is slower. Hopefully the second ethernet will improve the situation.
Regards,
Adam
--
Adam Goryachev
Website Managers
www.websitemanagers.com.au