[DRBD-user] slow sync speed of 12TB over 10GE

Ítalo Rossi adm.italo at digi.com.br
Mon Jan 12 20:24:05 CET 2009

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Mon, Jan 12, 2009 at 04:44:05PM +0100, Lars Täuber wrote:
> # cat /proc/mdstat
> Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
> md5 : active raid6 ram3[3] ram2[2] ram1[1] ram0[0]
>      1023872 blocks level 6, 64k chunk, algorithm 2 [4/4] [UUUU]

> but the speed is still frustrating:
> # cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by  
> root at monosan, 2009-01-09 15:15:29
> 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r---
>    ns:0 nr:781524 dw:781448 dr:0 al:0 bm:44 lo:20 pe:9847 ua:19 ap:0  
> ep:1 wo:b oos:242356
>        [==============>.....] sync'ed: 76.4% (242356/1023804)K
>        finish: 0:00:01 speed: 130,240 (130,240) K/sec


what write trouput do you get on top of that,
  drbd disconnected?
    -"- on the other node?
  drbd connected?

what does dmesg say about
raid5: using function
raid6: using algorithm

  if you do a raid0 on ramdisk instead?
  if you only use ram disk, no md raid?
  if you use dm-zero for the disk?

does your tcp do funny things, "optimizing" for throughput, thereby
hurting latency?  have a look at packet timings with tcpdump,
look out for gaps in packet timing.

if you cannot get more than 130 MB/sec on top of a ram disk and 10GbE,
there is something fishy going on,
and I doubt that drbd can do much about it.

: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
please don't Cc me, but send to list   --   I'm subscribed
drbd-user mailing list
drbd-user at lists.linbit.com

More information about the drbd-user mailing list