[DRBD-user] external metadata on ssd vs bcache

Arnold Krille arnold at arnoldarts.de
Wed Feb 27 23:13:07 CET 2013

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Wed, 27 Feb 2013 18:32:07 +0100 Lionel Sausin <ls at numerigraphe.com>
wrote:
> I wouldn't expect anything like the gains of
> bcache/flashcache/enhencio. Normally internal metadata are just as
> fast, thanks to the write cache of your disks and RAID adapter. Those
> are much faster than SSDs and metadata are small enough.
> However you may benefit from external metadata when your those caches 
> are saturated by writes (high throughput for a long time).
> If you do have an SSD and expect big writes, give it a try and please 
> tell us if it really makes a difference.

My experience with an ssd for (external) meta-data says that imrovement
is quite a lot!
You won't get faster continous writes, that is still limited by the
hdd. But you get much faster random-writes and the reason is this:
 - With internal meta-data on hdd, each write (or until each barrier)
   is followed by a disk-seek to the end of the disk where the
   meta-data lives followed by a seek back to where you are writing.
   And then you mix random writes at random positions...
 - With external meta-data on another hdd, your data-disk doesn't have
   to seek to the end of the disk anymore, step one of improvement.
 - With external meta-data on ssd, you are only left with the seeks
   during your normal random writes.

With todays disks and normal usage (unless you are netflix or google),
the real speed-improvement your users see/feel is not faster throughput
but lower latency.

Of course, using internal meta-data with the whole partition on ssd
gives you the best performance, but not everyone can buy enough ssds to
create a mirrored 6TB array of ssd.

3x2TB hdd + 160G ssd (for meta-data and the fast-loving databases)
times two on the other hand is actually affordable...

As to the original authors question: There is a manpage about drbdmeta
which describes the options to dump and restore the meta-data of an
offline drbd. So the action will be:
 - stop the drbd-volume
 - dump the meta-data
 - change the config to point the meta-data to the new place
 - restore the meta-data
 - restart the drbd-volume
 - wait for sync (only incremental, not a full sync) and repeat with the
   other node

I did that with several volumes when our ssds arrived. Test the steps
with a scrap-drbd-volume before doing the procedure on production-data
to be sure.

Have fun,

Arnold
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130227/4048a90b/attachment.pgp>


More information about the drbd-user mailing list