<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 21 Oct 2011, at 07:22, Arnold Krille wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><blockquote type="cite">I have two physical servers, reasonably specced. I have a 250GB LVM volume<br></blockquote><blockquote type="cite">spare on each physical server (/dev/foo/storage). I would like to build a<br></blockquote><blockquote type="cite">KVM/QEMU virtual machine on each physical server, connect /dev/foo/storage<br></blockquote><blockquote type="cite">to the virtual machines, and run DRBD inside the two VM guests.<br></blockquote><br>Why would you want to do that?<br></div></span></blockquote></div><br><div>I have two servers; I would like to provide shared storage, and run virtual machines. I don't want to care which server the VM runs on.</div><div><br></div><div>I would prefer to use primary/secondary+nfs rather than dual-primary+gfs2/ocfs2, because of the hassle of split-brain. I mount nfs-floating-ip:/data/share into /var/lib/libvirt/images/nfs, and it doesn't matter whether the NFS server / floating IP is local or on the other machine.</div><div><br></div><div>Except it does, because it's NFS.</div><div><br></div><div>This setup seems to work cleanly if I mount the NFS shares from a third machine. Pacemaker can fail the NFSD/IP/DRBD from machine to machine while files are open on the client, and it all just works. Lovely. And no risk of split brain.</div><div><br></div><div>But I don't have three machines :-( Only two.</div><div><br></div><div>If I try to mount an NFS share from the *same server* that's exporting it, I run into trouble when I try to failover - nfs-kernel-server seems to hold onto some file or other (lsof hangs, so I can't find out what). umounting the filesystem sometimes works and sometimes doesn't; regardless, drbd still can't secondary itself, saying someone is holding the device open.</div><div><br></div><div>My idea was therefore to run the NFS server from inside a virtual machine, static to a single physical server. Ugly, perhaps :-) I thought it was weirdly elegant.</div><div><br></div><div>I believe there is a product called the gluster virtual appliance, which (I think) uses a similar technique (though using gluster, obviously, and not DRBD.)</div><div><br></div><div>I've had hassle after hassle trying to make pacemaker drive OCFS2 and O2CB using stock Ubuntu server installations - missing the ocf:ocfs2:o2cb resource agent, for example, and having no supported way of installing it.</div><div><br></div><div>I'm sure I'm missing something terribly obvious in all this. I've been working on it for quite some man-hours now, and I'm probably not seeing the wood for the trees..</div><div><br></div><div>Any advice or words of encouragement warmly accepted! :-)</div><div><br></div><div>Cheers and beers,</div><div>Nick</div></body></html>