[DRBD-user] slow sync speed of 12TB over 10GE

Lars Täuber taeuber at bbaw.de
Fri Jan 9 16:16:19 CET 2009


Hi there!

I'm new to the topic but managed to get a mirrored setup working.
The only problem is the sync speed:

# cat /proc/drbd 
version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by root at monosan, 2009-01-09 15:15:29
 0: cs:SyncTarget ro:Secondary/Secondary ds:Inconsistent/UpToDate C r---
    ns:0 nr:1275248 dw:1267056 dr:0 al:0 bm:77 lo:2050 pe:9009 ua:2048 ap:0 ep:1 wo:b oos:11599037748
	[>....................] sync'ed:  0.1% (11327185/11328422)M
	finish: 36:24:02 speed: 88,440 (90,796) K/sec

Both two hosts are connected through 10GE.
The underlying device is a RAID6 of 15+1 SATA disks with netto transfer rates of roughly 325-350 MB/sec.

Here is my drbd.conf:
common {
    protocol		C;

    handlers {
	split-brain	"/usr/lib/drbd/notify-split-brain.sh root";
    }
    syncer {
	rate		1150M;
    }
}

resource RAID61 {
    disk		/dev/md4;
    device		/dev/drbd0;
    flexible-meta-disk		internal;

    on duosan {
	address		10.9.8.6:7788;
    }
    on monosan {
	address		10.9.8.7:7788;
    }
}

There are 4 AMD Opteron(tm) Processor 2216 cores and 4GB of RAM on each machine.

Mem:   4052996k total,   930200k used,  3122796k free,    57100k buffers
Swap:        0k total,        0k used,        0k free,   289016k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                               
11121 root      20   0     0    0    0 R   53  0.0   2:07.88 drbd0_receiver                                        
 5514 root      15  -5     0    0    0 S   50  0.0 104:10.59 md4_raid5                                             
11128 root      20   0     0    0    0 R    7  0.0   0:18.44 drbd0_asender                                         
11111 root      20   0     0    0    0 S    4  0.0   0:06.36 drbd0_worker                                          

What can I do to speed it up a bit? Are there some tuning hints anywhere on the net?

Thanks for the drbd.
And a happy new year to all of you.
Lars



More information about the drbd-user mailing list