[DRBD-user] DRBD, Xen, HVM and live migration

Jean Baptiste FAVRE drbd at jbfavre.org
Tue May 10 22:31:02 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hello,

On 10/05/2011 17:05, Bart Coninckx wrote:
> Hi,
> 
> I want to combine all the above mentioned technologies.
> 
> The Linbit pages warn not to use the drbd: VBD with HVM DomUs.

In fact, thay don't just warn not to use, they just advice you it's not
usable.

> This page however:
> http://publications.jbfavre.org/virtualisation/cluster-xen-corosync-pacemaker-drbd-ocfs2.en
Yep, I wrote it :)

> (thank you Jean), simply puts two DRBD devices in dual primary mode and
> starts Xen DomUs while pointing to the DRBD devices with phy: in the
> DomU config files.
> Live migration and HVM seem possible.
First, you're welcome.
Second, yes it works, but not "automagically". It's basically due to the
way Xen manage live migrations, and especially is related with VMs state.
Let me explain some things about live migration, even if this is not the
primary topic of this list. Xen live migration, as many other
virtualisation system like VMWare, work as follow:
When live migration is ordered, Xen will create a new domU on the other
node. This domU is started in paused state.

Then Xen will synchronize domU memory. This synchronization will be done
many times, at least to reduce the amount of data to synchronize.

When ready, Xen will pause domU on node A, unpause domU on node B and if
all is running fine, destroy domU on node A.

That's system point of vue. Now, with storage point of vue:
After memory synchornization, Xen needs to pause domU on node A (this
will stop write ops), promote DRBD resource on node A as secondary,
promote the same DRBD resource on node B as primary, unpause domU on
node B and destroy domU on node A.

Here is the point. To be able to promote DRBD resources, Xen needs to
know domU state. Xen knows PV domU state, but not HVM (at least without
specific drivers, but even with them I'm not sure it will be able to get
domU state).

For this reason, Xen can not deal with DRBD mode natively. That's why
you must not (it's not implemented) use "drbd:" syntax.
The solution I used was to have DRBD resources in primary/primary mode.

This way, no need to care about DRBD promotion and Xen can smoothly
handle live migration.

One or two things are almost important: if you have DRBD resource as
primary, you can start domU. What I want to tell is that you should
REALLY take care not to start the same domU on 2 differents nodes...
Really not, except if you want to troubleshot EXT FS internals to get
back your data :)


> Is it as simple as Jean describes it? What is the advantage then of the
> drbd: VBD? Just seeing to it that the dual primary is available (which
> is handled on Jean's Howto by Pacemaker)?
See above, drbd: VBD advantage is that it allows Xen to handle all
migration parts, DRBD promotion included. This way you can have a safe
primary/secondary mode.

And yes, my solution works and is "as simple" as I described :)

Hope this help,
Regards,
JB



More information about the drbd-user mailing list