[DRBD-user] up-to-date diskless secondary refuses down

[Pierre-Philipp Braun]

Hello, I've got a simple scenario here that doesn't quite behave as expected (or I am missing something).  Here's a three-node resource with dual-primaries enabled for easy XEN guest migrations, while having only a single primary in optimal state:

- node1 / diskless secondary
- node2 / primary
- node3 / secondary

Here's the status

root at NODE1:~# drbdadm status powerslack
powerslack role:Secondary
   disk:Diskless
   pmr2 role:Primary
     peer-disk:UpToDate
   pmr3 role:Secondary
     peer-disk:UpToDate

I can easily put node3 into maintenance, no problem there,

drbdadm down powerslack

and put that back into the farm, for that resource

drbdadm up powerslack

Now the problem happens on node1 which is diskless on that resource.  While putting node1 into maintenance, here's what I get.

root at NODE1:~# drbdadm down powerslack
powerslack: State change failed: (-10) State change was refused by peer node
additional info from kernel:
Declined by peer pmr2 (id: 2), see the kernel log there
Command 'drbdsetup down powerslack' terminated with exit code 11

and here are the logs from node2.

==> /var/log/messages <==
May 18 10:06:37 pmr2 kernel: drbd powerslack pmr1: Preparing remote state change 2530787709
May 18 10:06:37 pmr2 kernel: drbd powerslack pmr1: Aborting remote state change 2530787709

==> /var/log/syslog <==
May 18 10:06:37 pmr2 kernel: drbd powerslack: State change failed: Refusing to be Primary while peer is not outdated

(dmesg)
[560152.656733] drbd powerslack pmr1: Preparing remote state change 2530787709
[560152.656748] drbd powerslack: State change failed: Refusing to be Primary while peer is not outdated
[560152.656969] drbd powerslack pmr1: Aborting remote state change 2530787709

What is wrong?  Node1 is both up-to-date and secondary, and even diskless, why can't I take that resource out of the farm?

running on Linux 5.15.38

version: 9.1.7 (api:2/proto:110-121)
GIT-hash: bfd2450739e3e27cfd0a2eece2cde3d94ad993ae build by root at NODE3, 2022-05-11 22:21:08
Transports (api:18): tcp (9.1.7)


Thank you
-- 
Pierre-Philipp Braun
SMTP Health Campaign: enforce STARTTLS and verify MX certificates
<https://nethence.com/smtp/>