Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi there,
We have DRBD set up to create live backups of important data on a file
server on Amazon EC2. It's a simple setup, with a DRBD primary and a
secondary, both with a bunch of storage made up of EBS volumes bound
together via RAID, with LVM on top of that, and DRBD on top of the
LVM. Or, more succinctly, EBS -> RAID -> LVM -> DRBD.
This setup works well to help us handle the risks of a disk failure, and
also has made it easy for us to move between AWS availability zones.
But we'd like to use it to do proper snapshots for backups to protect
from user errors (deleting an important file etc.) We have a couple of
questions about the best way to do this.
We've read this blog post
<http://blogs.linbit.com/p/277/use-backup-with-drbd/> about doing
backups of DRBD volumes using LVM, which says:
Essentially one would disconnect the Secondary, [use LVM to]
snapshot the backing device, mount the snapshot, perform the
backups, umount the snapshot, reconnect the Secondary.
We've tried this (the mickey-mouse test script is below), and it appears
to work fine.
But: as far as we understand, the reason LVM is useful in this process
is that it allows you to mount and access the backing device, as this
would normally be inaccessible even when the server is disconnected,
because DRBD is keeping it locked. For example, in a test environment,
on the secondary, with /dev/important_volume/real_data as the disk to
which the DRBD resource r0 is syncing:
root at giles-devbackup1:~# drbdadm disconnect r0
root at giles-devbackup1:~# mkdir /mnt/foo
root at giles-devbackup1:~# mount /dev/important_volume/real_data /mnt/foo
mount: /dev/mapper/important_volume-real_data already mounted or
/mnt/foo busy
Are we right in thinking that the purpose of using LVM is purely to get
a mountable device that you can run the backup from? After all, if the
server you're running on has been disconnected from the primary, the
volume won't be being written to, so it presumably isn't to avoid
"blurring" as the disk changes underneath your feet while you're backing up.
If we're right in that belief, we think we don't need to use LVM
snapshots for our specific environment. We're using Amazon EBS, and
that has its own snapshot functionality. It doesn't work very well if
data changes under its feet, but if we use "drbdadm disconnect" before
it starts and reconnect when it's done, perhaps it will work? So we're
wondering if instead we can use this, perhaps simpler, backup methodology:
* drbdadm disconnect the resource on the secondary
* Use the AWS API to do their kind of snapshot on the underlying
disks. Wait for these to complete.
* drbdadm connect the resource secondary
* Make sure you wait for the primary to have synced fully before doing
this again!
Again, we've done this in a testing environment and it looked like it
worked -- but we're not sure if it will work in production.
Is anyone out there doing backups on Amazon like that? Any thoughts?
All the best,
Giles
PS here's our mickey-mouse script to do backups using LVM snapshot:
echo Start off with a clean backup disk
mkfs.xfs -f /dev/xvdf
mount /dev/xvdf /mnt/backup
echo Disconnect us - the secondary
drbdadm disconnect r0
echo Snapshot the backing device
SNAPSHOT_ID=`date +%Y%m%dT%H%M`
lvcreate -L100M --snapshot --name $SNAPSHOT_ID
/dev/important_volume/real_data
echo Mount the snapshot
mount /dev/important_volume/$SNAPSHOT_ID /mnt/snapshot/
echo Perform the backup-- primitive version
cp -pr /mnt/snapshot/* /mnt/backup/
echo Unmount the snapshot
umount /mnt/snapshot
echo Remove the snapshot
lvremove -f /dev/important_volume/$SNAPSHOT_ID
echo Reconnect us -- the secondary
drbdadm connect r0
echo Umount the backup disk
umount /mnt/backup
--
Giles Thomas <giles at pythonanywhere.com>
PythonAnywhere: Develop and host Python from your browser
<https://www.pythonanywhere.com/>
A product from PythonAnywhere LLP
17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number OC378414.
Registered address: 28 Ely Place, 3rd Floor, London EC1N 6TD, UK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20150206/d0a525ae/attachment.htm>