[DRBD-user] drbdadm down failed (-12) - blocked by drbd_submit

[Radoslaw Garbacz]

Hi,


I have a problem, which (from what I found) has been discussed, however not
in the particular case, which I experienced, so I would be grateful for any
suggestions of how to deal with it.


I.
1. I get an error when demoting DRBD resource:
* drbdadm down data0

data0: State change failed: (-12) Device is held open by someone
additional info from kernel:
failed to demote
Command 'drbdsetup-84 down data0' terminated with exit code 11

2. The device is not mounted and not used by any LVM, so based on some
online discussions I checked the blocking process and it is "drbd0_submit"

* lsof | grep drbd0
drbd0_sub 16687         root  cwd       DIR              202,1
251         64 /
drbd0_sub 16687         root  rtd       DIR              202,1
251         64 /
drbd0_sub 16687         root  txt
unknown                                         /proc/16687/exe

* ps auxf
root     16687  0.0  0.0      0     0 ?        S<   15:19   0:00  \_
[drbd0_submit]

Is there a good way to deal with this case, as whether some DRBD step is
missing, which leaves the process or killing the process is the right way?

Thanks in advance,


II.
Some details:

1)
[root at ip-10-29-97-230 ~]# lsblk
xvdb        202:16   0   10G  0 disk
└─md0         9:0    0   20G  0 raid0
  ├─md0p1   259:0    0  9.9G  0 md
  │ └─drbd0 147:0    0  9.9G  0 disk
  └─md0p2   259:1    0 10.1G  0 md
xvdc        202:32   0   10G  0 disk
└─md0         9:0    0   20G  0 raid0
  ├─md0p1   259:0    0  9.9G  0 md
  │ └─drbd0 147:0    0  9.9G  0 disk
  └─md0p2   259:1    0 10.1G  0 md


2)
[root at ip-10-29-97-230 ~]# drbd-overview
NOTE: drbd-overview will be deprecated soon.
Please consider using drbdtop.

 0:data0/0  WFConnection Primary/Unknown UpToDate/DUnknown
 1:data1/0  Unconfigured .


[root at ip-10-29-101-160 ~]# drbd-overview
NOTE: drbd-overview will be deprecated soon.
Please consider using drbdtop.

 0:data0/0  Unconfigured . .
 1:data1/0  Unconfigured . .


3)
[root at ip-10-29-97-230 ~]# drbdadm dump data0
# resource data0 on ip-10-29-97-230.ec2.internal: not ignored, not stacked
# defined at /etc/drbd.d/data0.res:55
resource data0 {
    on ip-10-29-97-230.ec2.internal {
        device           /dev/drbd0 minor 0;
        disk             /dev/md0p1;
        meta-disk        internal;
        address          ipv4 10.29.97.230:7789;
    }
    on ip-10-29-101-160.ec2.internal {
        device           /dev/drbd0 minor 0;
        disk             /dev/md0p2;
        meta-disk        internal;
        address          ipv4 10.29.101.160:7790;
    }
    options {
        cpu-mask           0;
        on-no-data-accessible io-error;
    }
    net {
        protocol           C;
        sndbuf-size        0;
        rcvbuf-size        0;
        max-buffers      2048;
        max-epoch-size   2048;
        unplug-watermark 128;
        use-rle          yes;
        tcp-cork         yes;
        verify-alg        "";
        connect-int       10;
        ping-int          10;
        ping-timeout       8;
        timeout           60;
    }
    disk {
        on-io-error      call-local-io-error;
        resync-rate      1000M;
        c-plan-ahead       0;
        c-fill-target     1M;
        c-delay-target     1;
        c-min-rate         0;
        c-max-rate       1000M;
        disk-barrier      no;
        disk-flushes     yes;
        disk-drain       yes;
        al-extents       1237;
        read-balancing   prefer-local;
        md-flushes       yes;
        max-bio-bvecs      0;
    }
    startup {
        become-primary-on ip-10-29-97-230.ec2.internal;
    }
}

-- 
Best Regards,

Radoslaw Garbacz
XtremeData Incorporated
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20181002/9bbfc9db/attachment.htm>