[DRBD-user] drbdadm attach - how to allow non-exclusive access ?

[TJ]

I've got a rather unique scenario and need to allow non-exclusive
read-only opening of the drbd device on the Secondary.

I've done some initial hacking on the code to avoid O_EXCL requirement
for 'attach' using "drbdsetup ... --force" not just 'primary' (or
'detach') but failed so far.

The scenario is two low-power devices, one (RasPi) on battery as the
Secondary, and the other (apu2) a headless Linux gateway/router/NAS that
can lose power, and has a LUKS encrypted file-system including at the
GRUB bootloader layer which requires GRUB unlocking via a serial console
connection.

The apu2 mirrors its /boot/ file-system to the Raspi Secondary. /boot/
is inside a LUKS encrypted container.

The aim is that if the apu2 loses power the RasPi notices the dropped
link and mounts the Secondary file-system to /srv/tftp/ and allows the
apu2 to do a PXE boot of GRUB core which in turn is able to load its
modules, Linux kernel and intrd from the RasPi.

The problem is that when the link drops and the RasPi promotes itself to
Primary and mounts /dev/drbd0 to /srv/tftp and then unmounts it and
returns to Secondary role once the apu2 is starting the RasPi's version
of the file-system in the drbd has had its metadata changed and thus the
two halves of the mirror cannot sync.

We first tried solving that with ext4 file-system mount flags but that
didn't solve it.

Then I thought to use a device-mapper COW snapshot - so the underlying
drbd device is never changed but the snapshot can be mounted at
/srv/tftp and any local changes won't affect the Primary.

Snapshot works great.

But this was blocked by drbdadm/setup not being able to open the drbd
device (or attach to it) when the device-mapper snapshot is open on it.

Being able to allow non-exclusive access in this case (using --force)
would solve the issue elegantly.

This is the patch I tried initially.

diff --git a/user/v84/config_flags.c b/user/v84/config_flags.c
index 81e732a..c304107 100644
--- a/user/v84/config_flags.c
+++ b/user/v84/config_flags.c
@@ -977,6 +977,7 @@ struct context_def attach_cmd_ctx = {
                /* { "*", STRING(backing_dev) }, */
                /* { "*", STRING(meta_dev) }, */
                /* { "*", NUMERIC(meta_dev_idx, MINOR_NUMBER) }, */
+               { "force", FLAG(force_attach) },
                { } },
 };

diff --git a/user/v84/linux/drbd_genl.h b/user/v84/linux/drbd_genl.h
index cf2ba40..b732b34 100644
--- a/user/v84/linux/drbd_genl.h
+++ b/user/v84/linux/drbd_genl.h
@@ -134,6 +134,7 @@ GENL_struct(DRBD_NLA_DISK_CONF, 3, disk_conf,
        __flg_field_def(23,     0 /* OPTIONAL */,       al_updates,
DRBD_AL_UPDATES_DEF)
        __flg_field_def(24,     0 /* OPTIONAL */,
discard_zeroes_if_aligned, DRBD_DISCARD_ZEROES_IF_ALIGNED_DEF)
        __flg_field_def(26,     0 /* OPTIONAL */,
disable_write_same, DRBD_DISABLE_WRITE_SAME_DEF)
+       __flg_field(27,         0 /* OPTIONAL */,       force_attach)
 )

 GENL_struct(DRBD_NLA_RESOURCE_OPTS, 4, res_opts,