[DRBD-user] Performance puzzle

Tor gentoo.linux.dude at gmail.com
Wed Nov 7 16:07:15 CET 2012

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Many thanks!  I put the metadata on mirrored SSD-disks and it made a
dramatic change in performance - the drop in performace for write size
> 510MB disappeared completely (almost x5 raise in speed).  The peak
sync rate increased from ~25,000 K/s to ~119,488 K/s (yes, I
understand that you dont want that high sync rate on a working
cluster, but now I just want to get them in sync as fast as possible).

I also found this
when googleing for external metadata, so I suppose that connecting a
battery backup unit to the controller should result in a similar
performance boost - and I have just ordered two such devices to
upgrade my controllers and will have the answer in a couple of weeks.

Thanks again!


On 09:45 Wed 07 Nov     , Felix Frank wrote:
> Hi,
> On 11/06/2012 05:18 PM, Tor wrote:
> > the drbd-device is
> > on a HW-raid5 with eight drives (total of 6.4 TiB)
> I'm not sure this is a very sound idea, performance-wise. The RAID5 in
> and of itself should be fine, but you should really consider external
> metadata IMHO.
> > It should be close to the sync speed that I have set to 110
> > MB/Sec (my NIC is 1Gbit/s).
> As an aside - you should limit the sync speed to 30M. You do *not* want
> the background sync to saturate your link and slow down live replication.
> > The results puzzles me somewhat.  There is a large drop in performance
> > when the size exceeds 510 MB (I have ran tests up to 2GB).  The speed
> > increases when the second node is not connected, but the relative
> > performance drop becomes even larger.  On the "raw" device there is no
> > such drop in performance.  So my conclusion is that this has something
> > to do with DRBD that I can not understand...
> Another remark: What you're benchmarking is replication speed, not sync
> speed. Sync speed is what's applied when you reconnect your nodes and
> one of them (typically the one secondary) has to catch up to the other.
> Replication is always done as fast as possible. There is no upper bound
> you can configure (that would be silly).
> > Any ideas?
> I think you're missing the metadata interactions. The 500MB barrier
> seems suggestive to me, although I have no clear idea at all what the
> details might be here.
> I suggest you first try and create a test device with external metadata.
> The quicker the metadata disk the better (yes, for testing you might
> even go for ramdisk, but don't tell anyone I wrote that). See if this
> changes things.
> Cheers,
> Felix

More information about the drbd-user mailing list