[DRBD-user] State change failed: (-12) Device is held open by someone

[Theodor Ratkin]

Hello everyone,

We have a Linstor cluster with HA managed by Pacemaker.

All DRBD resources are created as LVM thin volumes.

Like the documentation 
<https://linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-linstor_ha> 
recommends, the Linstor database is inside a DRBD resource managed by 
Linstor.

But something happened with the database resource, and it's now 
impossible to enable it on another node, because it refuses to become 
secondary, even when unmounted.

When trying to make it secondary with `drbdadm secondary` it fails with 
the following error:

# drbdadm secondary linstor_db
linstor_db: State change failed: (-12) Device is held open by someone
additional info from kernel:
/dev/drbd1000 opened by mount (pid 4007210) at 2022-02-27 17:49:02.233
Command 'drbdsetup secondary linstor_db' terminated with exit code 11

The resource is not mounted, lsof and fuser come out empty (expect for 
the drbd1000_submit process). The process 4007210 from the error text 
doesn't exist anymore.

# ls -l /dev/drbd/by-res/linstor_db/0
lrwxrwxrwx 1 root root 17 Mar 10 11:31 /dev/drbd/by-res/linstor_db/0 -> 
../../../drbd1000

# lsof 2>/dev/null | grep drbd1000
drbd1000_ 4006238   root  cwd       DIR  253,1   4096          2 /
drbd1000_ 4006238   root  rtd       DIR  253,12  4096          2 /
drbd1000_ 4006238   root  txt   unknown /proc/4006238/exe

# ps -fp 4006238
UID          PID    PPID  C STIME TTY          TIME CMD
root     4006238       2  0 Feb27 ?        00:00:00 [drbd1000_submit]

Could anyone help me find a solution for this problem that doesn't 
involve rebooting? Any help would be really appreciated!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20220310/fcd45e4d/attachment.htm>