[DRBD-user] Fast sync, slow regular write

Poyner, Brandon bpoyner at ccac.edu
Mon Oct 23 15:02:41 CEST 2006


That does seem a bit odd.  I would try running bonnie++ on the drbd
device to see if the results are consistent with dd.  Can you also
respond with output from federal-b's ifconfig and ethtool on both GigE
interfaces?

Brandon Poyner
Network Engineer III
CCAC - College Office
412-237-3086
 

> -----Original Message-----
> From: drbd-user-bounces at linbit.com 
> [mailto:drbd-user-bounces at linbit.com] On Behalf Of David Jonas
> Sent: Friday, October 20, 2006 9:01 PM
> To: drbd-user at linbit.com
> Subject: [DRBD-user] Fast sync, slow regular write
> 
> Hello everyone,
> 
> I have a problem that google seems to have failed me on. Basically,  
> sync'ing seems to be plenty fast (~50MB/s), but when I simply write  
> to the mirrored partition things move much slower, on the order of  
> 1MB/s. I tried al-extents at 257 and 907, didn't seem to make a  
> difference. A write to a non-drbd partition is blazing fast (~285MB/ 
> s). iperf also shows full speed on the link (~984 Mbits/sec, jumbo  
> frames).
> 
> I'm completely at a loss at this point. Any ideas or information  
> would be greatly appreciated.
> 
> Thanks,
> 
> David
> 
> 
> ########### Vitals  ###########
> 
> OS: Redhat EL4 2.6.9-42.ELsmp
> 
> DRBD:  drbd-0.7.21-1.c4, kernel-module- 
> drbd-2.6.9-42.ELsmp-0.7.21-1.c4 (CentOS4 RPMs)
> 
> Connection: Intel(R) PRO/1000 Network Connection, e1000 driver,  
> MTU=9000, cat6 crossover.
> 
> IO: 3ware 9550SX-4LP (2.26.04.010) raid-5, 4 Seagate SATA 
> participating
> 
> Proc: Intel(R) Core(TM)2 CPU  6300 @ 1.86GHz, MemTotal:  2073840 kB  
> (2GB), MemFree: 1456452 kB
> 
> df: /dev/drbd0            841G  306M  798G   1% /shared
> 
> 
> drbd.conf (complete)
> --------------------------------------------------------------
> ---------- 
> -----------------------
> 
> resource r0 {
> 
>    protocol C;
>    incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep  
> 60 ; halt -f";
> 
>    startup {
> 	degr-wfc-timeout 120;
>    }
> 
>    disk {
> 	on-io-error   pass_on;
>    }
> 
>    net {
> 	on-disconnect reconnect;
> 	max-buffers 32768;
> 	max-epoch-size 16384;
> 	sndbuf-size 1M;
> 	unplug-watermark 32768;
>    }
> 
>    syncer {
> 	# set this as high as possible for now...
> 	rate 500M;
> 	group 1;
> 	al-extents 907;
>    }
> 
>    on federal-a {
> 	device    /dev/drbd0;
> 	disk      /dev/sda9;
> 	address   10.2.2.1:7788;
> 	meta-disk internal;
>    }
> 
>    on federal-b {
> 	device    /dev/drbd0;
> 	disk      /dev/sda9;
> 	address   10.2.2.2:7788;
> 	meta-disk internal;
>    }
> }
> --------------------------------------------------------------
> ---------- 
> -----------------------
> 
> [root at federal-a ~]# /sbin/ifconfig/eth0
> eth0      Link encap:Ethernet  HWaddr 00:30:48:89:F9:C6
>            inet addr:10.2.2.1  Bcast:10.1.1.255  Mask:255.255.255.0
>            inet6 addr: fe80::230:48ff:fe89:f9c6/64 Scope:Link
>            UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
>            RX packets:5621864 errors:0 dropped:0 overruns:0 frame:0
>            TX packets:7380589 errors:0 dropped:0 overruns:0 carrier:0
>            collisions:0 txqueuelen:1000
>            RX bytes:590376286 (563.0 MiB)  TX bytes:340753247 (324.9  
> MiB)
>            Base address:0x5000 Memory:e0300000-e0320000
> 
> 
> ########### Tests ###########
> 
> These tests obviously weren't done at the same time. The mirror was  
> "Consistent" at the time of the dd.
> 
> [djonas at federal-a ~]$ cat /proc/drbd
> version: 0.7.21 (api:79/proto:74)
> SVN Revision: 2326 build by buildsvn at build-i386, 2006-10-07 05:12:17
> 0: cs:SyncSource st:Primary/Secondary ld:Consistent
>      ns:583570960 nr:4 dw:4 dr:583618840 al:0 bm:90251 lo:11903 pe: 
> 814 ua:11970 ap:0
>          [=========>..........] sync'ed: 46.2% (304311/564910)M
>          finish: 1:46:25 speed: 48,748 (52,384) K/sec
> 
> 
> [root at federal-a ~]# time dd if=/dev/sda of=/shared/100M.bin bs=4096  
> count=25600
> 25600+0 records in
> 25600+0 records out
> 
> real    1m35.336s
> user    0m0.004s
> sys 0m0.019s
> 
> 
> [root at federal-a ~]# scp /shared/100M.bin djonas at 10.2.2.2:
> djonas at 10.2.2.2's password:
> 100M.bin                                                      
>            
>                                                               
>          1 
> 00%  100MB  33.3MB/s   00:03
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
> 



More information about the drbd-user mailing list