[Drbd-dev] changing the fencing policy on the fly

[Lars Ellenberg]

On Mon, Nov 30, 2009 at 01:51:02PM -0500, Petrakis, Peter wrote:
> Hi All,
>  
> I have a challenge with the use of fencing and the state of flux in our
> cluster. The system 
> will setup drbd devices in advance and use them independent of it's
> peer, which will mirror
> it's config once it joins the cluster. Since the fencing helper makes
> use of drbdadm and thus
> the config file we have a problem. If fencing were to be triggered, it
> would immediately fail
> with a bad exit status because the config file is inconsistent. We use a
> naming convention
> for the disks in our cluster so the only piece of information I don't
> know at this time is my
> peers ip address.
> 
> To work around this bug I set the fencing policy to 'dont-care' and then
> once the resource
> is connected and uptodate, detach the backing store and then reattach it
> with the desired
> fencing policy. This leaves us vulnerable however, should the link go
> down for any reason we're
> in big trouble.
> 
> I'm trying to eliminate this hole without detaching the backing store. I
> can't play tricks with
> aliasing the eth interface to some address because I'm not physically in
> control of the box and don't
> want to ruin someone's network in case they hook up the wires
> incorrectly. DRBD doesn't seem to support
> IPV6 link local addressing which would help in this case because the
> nodes are directly attached.

IPv6 is supposed to work.

> What I'd like to do is to change the fencing policy on demand and the
> mechanism I wish to use is sysfs.
> Do you guys see any pitfalls with this approach, locking especially? The
> reason I want to use sysfs is two
> fold. I want to get away from using drbdadm for the userspace helper and
> it'll make writing software to
> manage drbd more straight forward. Thanks in advance.

modinfo drbd
usermode_helper:string
	(default: /sbin/drbdadm)
	also available via /sys/modules/drbd/parameters/usermode_helper

You can add your own wrapper around drbdadm as "hanlder" helper.

But feel free to write a patch to replace the DRBD config interface
(currently netlink/connector resp. genetlink based) to
a sysfs based one, and we might consider it ;)

Use a "DRBD" bus, each sub dir being a single drbd,
expose all parameters we currently have,
see that you don't step on each others toes when configuring those.
you probably need a "live" and a "pending" config set,
so you can change paramters when configuring in sequence,
then "commit" them in one go.

Post on drbd-dev and possibly on lkml.

Converting single parameters to sysfs in some ad-hoc fashion,
just because you feel _this_ one should be there,
is not useful und unmaintainable.

Thanks,

	Lars