[DRBD-user] Randomly crash drbdsetup

[Roland Kammerer]

On Wed, Jun 27, 2018 at 12:37:20PM +0200, Julien Escario wrote:
> Hello,
> We're experiencing a really strange situation.
> We often play with :
> drbdmanage peer-device-options --resource <ressource> --c-max-rate <rate>
> 
> especially when a node crash and need a (full) resync.
> 
> When doing this, sometimes (after 10 or 20 such commands), we end up with
> drbdmanage completely stuck and a drbdsetup that seems to block on an IO with
> returning.
> For example :
> drbdsetup disk-options 144 --set-defaults --read-balancing=prefer-local
> --al-extents=6481 --al-updates=no --md-flushes=no
> 
> drbdadm status display ressource up to this one then hangs on drbdsetup call.
> 
> drbdtop is still usable.
> 
> Right now, we didn't manage to find a solution without rebooting the node (sadly).
> 
> Do you experience such situation ?
> What can cause this ?

What version of DRBD9 is that (cat /proc/drbd)? drbdsetup hangs for a
reason, kernel related, not an actual bug in drbdsetup. "dmesg" at that
time would be interesting. Yes, I saw that, but not recently, only with
by now pretty old versions of DRBD9.

> Is there a way to unblock this process without rebooting ?

Depends, but as a rule of thumb: when that happens the kernel is already
in a state where you want to/have to reboot.

Regards, rck