[DRBD-user] Documentation download & another DRBD question
linux at alteeve.com
Fri Feb 17 18:07:38 CET 2012
On 02/17/2012 11:42 AM, Brian O Mahony wrote:
> (Apologies for top posting - working off a different laptop as I broke mine this morning, and outlook is a pain)
> Jake/Maurits/Digimer, Thanks for the replies.
>> What it sounds like you need is LVM's snapshotting capability. DRBD is designed to be a block-level replication tool, not a backup tool per-se.
> We need to back up individual folders, some of which are 80+ GB. I don't think a snapshot allows me to do this?
LVM snapshotting works like this;
You have an LVM LV, with some space left over on the VG sufficient to
hold deltas created for the duration of time it takes to copy the data off.
So when you want to make a backup, you create a snapshot LV and mount
that somewhere. At the moment of the mount, the snapshot is identical to
the source LV. As time passes though and data on the source partition
changes, the old data is stored on the smaller snapshot partition,
allowing you to still see the data on the snapshot mount as it was at
the time the snapshot was made, without making the source partition
Thus, you need to have a rough idea of how long it takes to copy the
data off. With this time in mind, you must then estimate how much data
will change in that period of time. Add some headroom and you have the
size of the snapshot partition you will need.
The result of all this is that, for the life of the snapshot partition,
you have an unchanging, point-in-time view of the source LV. Meanwhile,
the source partition is never offline, frozen or otherwise unavailable.
If you need to stop processes to get the data into a backup-able state,
then you can simply; Stop your service -> mount the snapshot -> restart
the service and you should have no more than a few seconds of downtime.
>> As a secondary aside; Why use cp instead of rsync?
> Data on the primary server is on a SAN, with about 8 million files, most tiny. The backup dumps this to fast local disks (there is other things on the SAN also, and all our VMs etc get backed up nightly, meaning sometimes disk IO access can be slowed), this is then compressed locally, before being booted off to local storage on the second server. The department in charge of the data, and the backup process are not that well versed in this, and when I suggested rsync and the delta capabilities, they assumed it is going to take longer to check, and never tested it fully. Preferably, I would like them to rsync to the second machine, and then do the tar/gz there. But with managers involved, ive been told to look for another solution, as they "don't have the time to spend days looking at something that is potentially slower"
Sounds like you've got too many cooks in the kitchen.
No matter what you explore, there is a chance that it won't work. It may
well be the case that rsync isn't viable, but I would suggest that it is
a much quicker course to investigate than DRBD. This sounds to me like
someone read something, got excited and is now hell-bent on a given
solution to the exclusion of other options.
Don't get me wrong, DRBD is awesome and ridiculously useful, but I don't
think it fits this use-case.
>> (inducing a split-brain condition, which is generally something you want to avoid
> The second machine does nothing except store a copy of this data (why they need 2 6core procs & 128GB memory baffles me tbh!). It actually stores two copies, one which is a "site sync" in the software, and the other being the tar/gz copy. It is source code, so they are quite finicky about the data.
> I will download the DRBD manual, and have a good read through. As I said im a novice with it. I also haven't used LVM snapshotting much, so will investigate that also.
> Once again, thanks for the replies.
Happy to help.
E-Mail: digimer at alteeve.com
Papers and Projects: https://alteeve.com
More information about the drbd-user