<div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Gionatan,<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 27, 2017 at 12:14 AM, Gionatan Danti <span dir="ltr"><<a href="mailto:g.danti@assyoma.it" target="_blank">g.danti@assyoma.it</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi all,<br>
I have a possibly naive question about a dual primary setup involving LVM devices on top of DRBD.<br>
<br>
The main question is: using cLVM or native LVM locking, can I safely use a LV block device on the first node, *close it*, and reopen it on the second one? No filesystem is involved and no host is expected to concurrently use the same LV.<br>
<br>
Scenario: two CentOS 7 + DRBD 8.4 nodes with LVs on top of DRBD on top of a physical RAID array. Basically, DRBD replicate anything written to the specific hardware array.<br>
<br>
Goal: having a redundant virtual machine setup, where vms can be live migrated between the two hosts.<br>
<br>
Current setup: I currently run a single-primary, dual nodes setup, where the second host has no access at all to any LV. This setup worked very well in the past years, but it forbid using live migration (the secondary host has no access to the LV-based vdisk attached to the vms, so it is impossible to live migrate the running vms).<br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
I thought to use a dual-primary setup to have the LVs available on *both* nodes, using a lock manager to arbitrate access to them.<br>
<br>
How do you see such a solution? It is workable? Or would you recommend to use a clustered filesystem on top of the dual-primary DRBD device?<br><br></blockquote><div> </div></div><div style="font-size:small" class="gmail_default">I would recommend going through this lengthy post <a href="http://lists.linbit.com/pipermail/drbd-user/2011-January/015236.html">http://lists.linbit.com/pipermail/drbd-user/2011-January/015236.html</a> covering all pros and cons of several possible scenarios. <br><br>The easiest scenario for dual-primary DRBD would be a DRBD device per VM, so something like this RAID -> PV -> VG -> LV -> DRBD -> VM, where you don't even need LVM locking (since that layer is not even exposed to the user) and is great for dual-primary KVM clusters. You get live migration and also keep the resizing functionality too since you can grow the underlying LV and then the DRBD it self to increase the VM disk size lets say. The VM needs to be started on one node only of course so you (or your software) need to make sure this is always the case. One huge drawback of this approach though is the large number of DRBD device to maintain in case of hundreds of VM's! Although since you have already committed to different approach this might not be possible at this point. <br><br>Note: In this case though you don't even need dual primary since the DRBD for each VM can be independently promoted to primary on any node at any time. In case of single DRBD it is all-or-nothing so no possibility of migrating individual VMs. <br><br></div><div style="font-size:small" class="gmail_default">Now adding LV on top of DRBD is bit more complicated. I guess your current setup is something like this?<br><br></div><div style="font-size:small" class="gmail_default"> LV1 -> VM1<br></div><div style="font-size:small" class="gmail_default">RAID -> DRBD -> PV -> VG -> LV2 -> VM2</div><div style="font-size:small" class="gmail_default"> LV3 -> VM3<br> ...<br><br></div><div style="font-size:small" class="gmail_default">In this case when DRBD is in dual-primary the DLM/cLVM setup is imperative so the LVMs know who has the write access. But then on VM migration "something" needs to shutdown the VM on Node1 to release the LVM lock and start the VM on Node2. Same as above, as long as each VM is running on *only one* node you should be fine, the moment you start it on both you will probably corrupt your VM. Software like Proxmox should be able to help you on both points.<br><br></div><div style="font-size:small" class="gmail_default">Which brings me to an important question I should have asked at the very beginning actually: what do you use to manage the cluster?? (if anything)<br></div><div style="font-size:small" class="gmail_default"><br></div><div style="font-size:small" class="gmail_default">Regards,<br></div><div style="font-size:small" class="gmail_default">Igor<br></div><br></div></div>