[DRBD-user] slow sync speed of 12TB over 10GE

[Tomasz Chmielewski]

Lars Ellenberg wrote:

> if you cannot get more than 130 MB/sec on top of a ram disk and 10GbE,
> there is something fishy going on,
> and I doubt that drbd can do much about it.

I don't see anything fishy here.

I guess someone is confusing the _read_ speed of a device:

> # hdparm -t /dev/md4
> 
> /dev/md4:
>  Timing buffered disk reads:  1102 MB in  3.01 seconds = 366.57 MB/sec
>  Timing buffered disk reads:  1040 MB in  3.00 seconds = 346.63 MB/sec


With its _write_ speed:

>         [==============>.....] sync'ed: 76.4% (242356/1023804)K
>         finish: 0:00:01 speed: 130,240 (130,240) K/sec

How fast can you write on that RAID-6 device (with DRBD sync to the 
other node disabled)?

i.e., do:

# sync
# time dd if=/dev/zero of=/mnt/mountpint/bigfile bs=64k count=100000
# time sync

Also, looking at "iostat -d 1 -k" output in the other terminal while the 
writes are going would be interesting.


-- 
Tomasz Chmielewski
http://wpkg.org