Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
That does seem a bit odd. I would try running bonnie++ on the drbd
device to see if the results are consistent with dd. Can you also
respond with output from federal-b's ifconfig and ethtool on both GigE
interfaces?
Brandon Poyner
Network Engineer III
CCAC - College Office
412-237-3086
> -----Original Message-----
> From: drbd-user-bounces at linbit.com
> [mailto:drbd-user-bounces at linbit.com] On Behalf Of David Jonas
> Sent: Friday, October 20, 2006 9:01 PM
> To: drbd-user at linbit.com
> Subject: [DRBD-user] Fast sync, slow regular write
>
> Hello everyone,
>
> I have a problem that google seems to have failed me on. Basically,
> sync'ing seems to be plenty fast (~50MB/s), but when I simply write
> to the mirrored partition things move much slower, on the order of
> 1MB/s. I tried al-extents at 257 and 907, didn't seem to make a
> difference. A write to a non-drbd partition is blazing fast (~285MB/
> s). iperf also shows full speed on the link (~984 Mbits/sec, jumbo
> frames).
>
> I'm completely at a loss at this point. Any ideas or information
> would be greatly appreciated.
>
> Thanks,
>
> David
>
>
> ########### Vitals ###########
>
> OS: Redhat EL4 2.6.9-42.ELsmp
>
> DRBD: drbd-0.7.21-1.c4, kernel-module-
> drbd-2.6.9-42.ELsmp-0.7.21-1.c4 (CentOS4 RPMs)
>
> Connection: Intel(R) PRO/1000 Network Connection, e1000 driver,
> MTU=9000, cat6 crossover.
>
> IO: 3ware 9550SX-4LP (2.26.04.010) raid-5, 4 Seagate SATA
> participating
>
> Proc: Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz, MemTotal: 2073840 kB
> (2GB), MemFree: 1456452 kB
>
> df: /dev/drbd0 841G 306M 798G 1% /shared
>
>
> drbd.conf (complete)
> --------------------------------------------------------------
> ----------
> -----------------------
>
> resource r0 {
>
> protocol C;
> incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep
> 60 ; halt -f";
>
> startup {
> degr-wfc-timeout 120;
> }
>
> disk {
> on-io-error pass_on;
> }
>
> net {
> on-disconnect reconnect;
> max-buffers 32768;
> max-epoch-size 16384;
> sndbuf-size 1M;
> unplug-watermark 32768;
> }
>
> syncer {
> # set this as high as possible for now...
> rate 500M;
> group 1;
> al-extents 907;
> }
>
> on federal-a {
> device /dev/drbd0;
> disk /dev/sda9;
> address 10.2.2.1:7788;
> meta-disk internal;
> }
>
> on federal-b {
> device /dev/drbd0;
> disk /dev/sda9;
> address 10.2.2.2:7788;
> meta-disk internal;
> }
> }
> --------------------------------------------------------------
> ----------
> -----------------------
>
> [root at federal-a ~]# /sbin/ifconfig/eth0
> eth0 Link encap:Ethernet HWaddr 00:30:48:89:F9:C6
> inet addr:10.2.2.1 Bcast:10.1.1.255 Mask:255.255.255.0
> inet6 addr: fe80::230:48ff:fe89:f9c6/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
> RX packets:5621864 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7380589 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:590376286 (563.0 MiB) TX bytes:340753247 (324.9
> MiB)
> Base address:0x5000 Memory:e0300000-e0320000
>
>
> ########### Tests ###########
>
> These tests obviously weren't done at the same time. The mirror was
> "Consistent" at the time of the dd.
>
> [djonas at federal-a ~]$ cat /proc/drbd
> version: 0.7.21 (api:79/proto:74)
> SVN Revision: 2326 build by buildsvn at build-i386, 2006-10-07 05:12:17
> 0: cs:SyncSource st:Primary/Secondary ld:Consistent
> ns:583570960 nr:4 dw:4 dr:583618840 al:0 bm:90251 lo:11903 pe:
> 814 ua:11970 ap:0
> [=========>..........] sync'ed: 46.2% (304311/564910)M
> finish: 1:46:25 speed: 48,748 (52,384) K/sec
>
>
> [root at federal-a ~]# time dd if=/dev/sda of=/shared/100M.bin bs=4096
> count=25600
> 25600+0 records in
> 25600+0 records out
>
> real 1m35.336s
> user 0m0.004s
> sys 0m0.019s
>
>
> [root at federal-a ~]# scp /shared/100M.bin djonas at 10.2.2.2:
> djonas at 10.2.2.2's password:
> 100M.bin
>
>
> 1
> 00% 100MB 33.3MB/s 00:03
>
>
>
>
>
>
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>