Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello everyone,
I have a problem that google seems to have failed me on. Basically,
sync'ing seems to be plenty fast (~50MB/s), but when I simply write
to the mirrored partition things move much slower, on the order of
1MB/s. I tried al-extents at 257 and 907, didn't seem to make a
difference. A write to a non-drbd partition is blazing fast (~285MB/
s). iperf also shows full speed on the link (~984 Mbits/sec, jumbo
frames).
I'm completely at a loss at this point. Any ideas or information
would be greatly appreciated.
Thanks,
David
########### Vitals ###########
OS: Redhat EL4 2.6.9-42.ELsmp
DRBD: drbd-0.7.21-1.c4, kernel-module-
drbd-2.6.9-42.ELsmp-0.7.21-1.c4 (CentOS4 RPMs)
Connection: Intel(R) PRO/1000 Network Connection, e1000 driver,
MTU=9000, cat6 crossover.
IO: 3ware 9550SX-4LP (2.26.04.010) raid-5, 4 Seagate SATA participating
Proc: Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz, MemTotal: 2073840 kB
(2GB), MemFree: 1456452 kB
df: /dev/drbd0 841G 306M 798G 1% /shared
drbd.conf (complete)
------------------------------------------------------------------------
-----------------------
resource r0 {
protocol C;
incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep
60 ; halt -f";
startup {
degr-wfc-timeout 120;
}
disk {
on-io-error pass_on;
}
net {
on-disconnect reconnect;
max-buffers 32768;
max-epoch-size 16384;
sndbuf-size 1M;
unplug-watermark 32768;
}
syncer {
# set this as high as possible for now...
rate 500M;
group 1;
al-extents 907;
}
on federal-a {
device /dev/drbd0;
disk /dev/sda9;
address 10.2.2.1:7788;
meta-disk internal;
}
on federal-b {
device /dev/drbd0;
disk /dev/sda9;
address 10.2.2.2:7788;
meta-disk internal;
}
}
------------------------------------------------------------------------
-----------------------
[root at federal-a ~]# /sbin/ifconfig/eth0
eth0 Link encap:Ethernet HWaddr 00:30:48:89:F9:C6
inet addr:10.2.2.1 Bcast:10.1.1.255 Mask:255.255.255.0
inet6 addr: fe80::230:48ff:fe89:f9c6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:5621864 errors:0 dropped:0 overruns:0 frame:0
TX packets:7380589 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:590376286 (563.0 MiB) TX bytes:340753247 (324.9
MiB)
Base address:0x5000 Memory:e0300000-e0320000
########### Tests ###########
These tests obviously weren't done at the same time. The mirror was
"Consistent" at the time of the dd.
[djonas at federal-a ~]$ cat /proc/drbd
version: 0.7.21 (api:79/proto:74)
SVN Revision: 2326 build by buildsvn at build-i386, 2006-10-07 05:12:17
0: cs:SyncSource st:Primary/Secondary ld:Consistent
ns:583570960 nr:4 dw:4 dr:583618840 al:0 bm:90251 lo:11903 pe:
814 ua:11970 ap:0
[=========>..........] sync'ed: 46.2% (304311/564910)M
finish: 1:46:25 speed: 48,748 (52,384) K/sec
[root at federal-a ~]# time dd if=/dev/sda of=/shared/100M.bin bs=4096
count=25600
25600+0 records in
25600+0 records out
real 1m35.336s
user 0m0.004s
sys 0m0.019s
[root at federal-a ~]# scp /shared/100M.bin djonas at 10.2.2.2:
djonas at 10.2.2.2's password:
100M.bin
1
00% 100MB 33.3MB/s 00:03