[DRBD-user] slow sync speed of 12TB over 10GE
lars.ellenberg at linbit.com
Mon Jan 12 18:02:24 CET 2009
On Mon, Jan 12, 2009 at 04:44:05PM +0100, Lars Täuber wrote:
> # cat /proc/mdstat
> Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
> md5 : active raid6 ram3 ram2 ram1 ram0
> 1023872 blocks level 6, 64k chunk, algorithm 2 [4/4] [UUUU]
> but the speed is still frustrating:
> # cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by root at monosan, 2009-01-09 15:15:29
> 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r---
> ns:0 nr:781524 dw:781448 dr:0 al:0 bm:44 lo:20 pe:9847 ua:19 ap:0 ep:1 wo:b oos:242356
> [==============>.....] sync'ed: 76.4% (242356/1023804)K
> finish: 0:00:01 speed: 130,240 (130,240) K/sec
what write trouput do you get on top of that,
-"- on the other node?
what does dmesg say about
raid5: using function
raid6: using algorithm
if you do a raid0 on ramdisk instead?
if you only use ram disk, no md raid?
if you use dm-zero for the disk?
does your tcp do funny things, "optimizing" for throughput, thereby
hurting latency? have a look at packet timings with tcpdump,
look out for gaps in packet timing.
if you cannot get more than 130 MB/sec on top of a ram disk and 10GbE,
there is something fishy going on,
and I doubt that drbd can do much about it.
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
please don't Cc me, but send to list -- I'm subscribed
More information about the drbd-user