[DRBD-user] Performance puzzle

Felix Frank ff at mpexnet.de
Wed Nov 7 09:45:10 CET 2012

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On 11/06/2012 05:18 PM, Tor wrote:
> the drbd-device is
> on a HW-raid5 with eight drives (total of 6.4 TiB)

I'm not sure this is a very sound idea, performance-wise. The RAID5 in
and of itself should be fine, but you should really consider external
metadata IMHO.

> It should be close to the sync speed that I have set to 110
> MB/Sec (my NIC is 1Gbit/s).

As an aside - you should limit the sync speed to 30M. You do *not* want
the background sync to saturate your link and slow down live replication.

> The results puzzles me somewhat.  There is a large drop in performance
> when the size exceeds 510 MB (I have ran tests up to 2GB).  The speed
> increases when the second node is not connected, but the relative
> performance drop becomes even larger.  On the "raw" device there is no
> such drop in performance.  So my conclusion is that this has something
> to do with DRBD that I can not understand...

Another remark: What you're benchmarking is replication speed, not sync
speed. Sync speed is what's applied when you reconnect your nodes and
one of them (typically the one secondary) has to catch up to the other.

Replication is always done as fast as possible. There is no upper bound
you can configure (that would be silly).

> Any ideas?

I think you're missing the metadata interactions. The 500MB barrier
seems suggestive to me, although I have no clear idea at all what the
details might be here.

I suggest you first try and create a test device with external metadata.
The quicker the metadata disk the better (yes, for testing you might
even go for ramdisk, but don't tell anyone I wrote that). See if this
changes things.


More information about the drbd-user mailing list