[DRBD-user] DRBD inside KVM virtual machine

Nick Morrison nick at nick.on.net
Sat Oct 22 04:24:27 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On 21 Oct 2011, at 07:22, Arnold Krille wrote:

>> I have two physical servers, reasonably specced.  I have a 250GB LVM volume
>> spare on each physical server (/dev/foo/storage).  I would like to build a
>> KVM/QEMU virtual machine on each physical server, connect /dev/foo/storage
>> to the virtual machines, and run DRBD inside the two VM guests.
> Why would you want to do that?

I have two servers; I would like to provide shared storage, and run virtual machines.  I don't want to care which server the VM runs on.

I would prefer to use primary/secondary+nfs rather than dual-primary+gfs2/ocfs2, because of the hassle of split-brain.  I mount nfs-floating-ip:/data/share into /var/lib/libvirt/images/nfs, and it doesn't matter whether the NFS server / floating IP is local or on the other machine.

Except it does, because it's NFS.

This setup seems to work cleanly if I mount the NFS shares from a third machine.  Pacemaker can fail the NFSD/IP/DRBD from machine to machine while files are open on the client, and it all just works.  Lovely.  And no risk of split brain.

But I don't have three machines :-(  Only two.

If I try to mount an NFS share from the *same server* that's exporting it, I run into trouble when I try to failover - nfs-kernel-server seems to hold onto some file or other (lsof hangs, so I can't find out what).  umounting the filesystem sometimes works and sometimes doesn't; regardless, drbd still can't secondary itself, saying someone is holding the device open.

My idea was therefore to run the NFS server from inside a virtual machine, static to a single physical server.  Ugly, perhaps :-)  I thought it was weirdly elegant.

I believe there is a product called the gluster virtual appliance, which (I think) uses a similar technique (though using gluster, obviously, and not DRBD.)

I've had hassle after hassle trying to make pacemaker drive OCFS2 and O2CB using stock Ubuntu server installations - missing the ocf:ocfs2:o2cb resource agent, for example, and having no supported way of installing it.

I'm sure I'm missing something terribly obvious in all this.  I've been working on it for quite some man-hours now, and I'm probably not seeing the wood for the trees..

Any advice or words of encouragement warmly accepted! :-)

Cheers and beers,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20111021/e1d78cc6/attachment.htm>

More information about the drbd-user mailing list