[DRBD-user] Question about using DRBD to do snapshots on AWS EBS volumes

Giles Thomas giles at pythonanywhere.com
Fri Feb 6 20:45:49 CET 2015

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Hi there,

We have DRBD set up to create live backups of important data on a file 
server on Amazon EC2.  It's a simple setup, with a DRBD primary and a 
secondary, both with a bunch of storage made up of EBS volumes bound 
together via RAID, with LVM on top of that, and DRBD on top of the 
LVM.   Or, more succinctly, EBS -> RAID -> LVM -> DRBD.

This setup works well to help us handle the risks of a disk failure, and 
also has made it easy for us to move between AWS availability zones.  
But we'd like to use it to do proper snapshots for backups to protect 
from user errors (deleting an important file etc.)   We have a couple of 
questions about the best way to do this.

We've read this blog post 
<http://blogs.linbit.com/p/277/use-backup-with-drbd/> about doing 
backups of DRBD volumes using LVM, which says:

    Essentially one would disconnect the Secondary, [use LVM to]
    snapshot the backing device, mount the snapshot, perform the
    backups, umount the snapshot, reconnect the Secondary.

We've tried this (the mickey-mouse test script is below), and it appears 
to work fine.

But: as far as we understand, the reason LVM is useful in this process 
is that it allows you to mount and access the backing device, as this 
would normally be inaccessible even when the server is disconnected, 
because DRBD is keeping it locked.  For example, in a test environment, 
on the secondary, with /dev/important_volume/real_data as the disk to 
which the DRBD resource r0 is syncing:

    root at giles-devbackup1:~# drbdadm disconnect r0
    root at giles-devbackup1:~# mkdir /mnt/foo
    root at giles-devbackup1:~# mount /dev/important_volume/real_data /mnt/foo
    mount: /dev/mapper/important_volume-real_data already mounted or
    /mnt/foo busy

Are we right in thinking that the purpose of using LVM is purely to get 
a mountable device that you can run the backup from?  After all, if the 
server you're running on has been disconnected from the primary, the 
volume won't be being written to, so it presumably isn't to avoid 
"blurring" as the disk changes underneath your feet while you're backing up.

If we're right in that belief, we think we don't need to use LVM 
snapshots for our specific environment.  We're using Amazon EBS, and 
that has its own snapshot functionality.   It doesn't work very well if 
data changes under its feet, but if we use "drbdadm disconnect" before 
it starts and reconnect when it's done, perhaps it will work?  So we're 
wondering if instead we can use this, perhaps simpler, backup methodology:

  * drbdadm disconnect the resource on the secondary
  * Use the AWS API to do their kind of snapshot on the underlying
    disks.  Wait for these to complete.
  * drbdadm connect the resource secondary
  * Make sure you wait for the primary to have synced fully before doing
    this again!

Again, we've done this in a testing environment and it looked like it 
worked -- but we're not sure if it will work in production.

Is anyone out there doing backups on Amazon like that?  Any thoughts?

All the best,


PS here's our mickey-mouse script to do backups using LVM snapshot:

    echo Start off with a clean backup disk
    mkfs.xfs -f /dev/xvdf
    mount /dev/xvdf /mnt/backup

    echo Disconnect us - the secondary
    drbdadm disconnect r0

    echo Snapshot the backing device
    SNAPSHOT_ID=`date +%Y%m%dT%H%M`
    lvcreate -L100M --snapshot --name $SNAPSHOT_ID

    echo Mount the snapshot
    mount /dev/important_volume/$SNAPSHOT_ID /mnt/snapshot/

    echo Perform the backup-- primitive version
    cp -pr /mnt/snapshot/* /mnt/backup/

    echo Unmount the snapshot
    umount /mnt/snapshot

    echo Remove the snapshot
    lvremove -f /dev/important_volume/$SNAPSHOT_ID

    echo Reconnect us -- the secondary
    drbdadm connect r0

    echo Umount the backup disk
    umount /mnt/backup

Giles Thomas <giles at pythonanywhere.com>

PythonAnywhere: Develop and host Python from your browser

A product from PythonAnywhere LLP
17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number OC378414.
Registered address: 28 Ely Place, 3rd Floor, London EC1N 6TD, UK

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20150206/d0a525ae/attachment.htm>

More information about the drbd-user mailing list