[DRBD-user] dual primary DRBD, iSCSI and multipath possible?

Lars Ellenberg lars.ellenberg at linbit.com
Tue Oct 5 14:36:57 CEST 2010

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Mon, Oct 04, 2010 at 09:56:12PM +0200, Markus Hochholdinger wrote:
> Hello,
> Am 15.03.2010 um 15:47 Uhr schrieb Olivier LAMBERT 
> <lambert.olivier at gmail.com>:
> > :D
> > That's I'm using right now, but it's for Xen on the top.
> > I think this is the only good reason to do that.
> I'm also in the process of evaluating this (for a xen setup).
> My setup would be:
> On two nodes drbd with active/active (so xen live migration would work). On 
> each node export the drbd device with iscsi.

If you xen attaches to iSCSI,
why would you need anything else for live migration?

> On each other node import the iscsi devices of both drbd nodes and put 
> multipath over it.


> The tricky part now is how to handle failures. With this setup it is possible 
> that multipath switches between both drbd nodes. If we do this more than once 
> while we have a split brain, this would destroy our data!

With dual-primary DRBD, you currently still have to set it up so that at
least one node reboots if the replication link breaks, and the rebooted
node must not go primary again untill connection is re-established.

> So the goal would be to develope a good multipath strategy.
> How do you do this?

You don't.

> My idea is to say multipath to stick to one path and only switch on an error. 

Unfortunately you cannot control on what type of error the initiator will switch.

> Also you have to say multipath to NOT recover faulty paths autmatically to 
> prevent data loss in a split brain situation.

That's not your only problem there.

Please go for a failover setup.

You can have two targets, one active on box A, one active on box B,
both replicating to the respective other.

As the iSCSI target is not cluster aware, if you try to do multipath to
two _independent_ targets on an dual-primary DRBD, in general you will
break things.

DRBD only replicates the data changes.  To make that actually work, the
targets would have to be cluster aware, and replicate iSCSI state to
each other. All the non-data commands, unit attention, lun resets, cmd
aborts, not to speek of ordering or reservations.

It may appear to work as long as it is completely unloaded, you don't do
reservations, target and initiator are mostly idle, there are not much
scsi commands involved, and all you are doing is unplug an iSCSI cable.
But in general, if anything happens at all, you get high load on either
the network or the initiators or the targets or the replication link
or anything more interesting breaks, then I certainly don't want to be
the one cleaning up the mess.

I strongly recommend against it.

: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
please don't Cc me, but send to list   --   I'm subscribed

More information about the drbd-user mailing list