[DRBD-user] Device is held open by someone

[Lars Ellenberg]

On Thu, Feb 28, 2013 at 11:00:44AM -0500, Phillips, Dan wrote:
> Seems this is a common problem!!! Can you point those of us experiencing " Device is held open by someone " to a DRBD resource (documentation) that gives an overview in this area for a better understanding of what may be going on? I will look at www.drbd.org for info.
> 
> 
> Thanks,
> 
> Dan Phillips
> 
> -----Original Message-----
> From: drbd-user-bounces at lists.linbit.com [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Andreas Kurz
> Sent: Thursday, February 28, 2013 5:07 AM
> To: drbd-user at lists.linbit.com
> Subject: Re: [DRBD-user] Device is held open by someone
> 
> On 2013-02-26 13:04, Felipe Gutierrez wrote:
> > Hi everyone,
> > 
> > I am trying to do a failover system only with drbd. When my primary 
> > node get out of the network, the secondary node became primary and I 
> > mount the filesystem.
> > secondary# drbdadm primary r7
> > secondary# mount /dev/drbd7 /mnt/drbd7/
> > 
> > Until that every thing is ok.
> > At this time, my old primary node has to became the secondary and I 
> > have to discard my changes.
> > primary# umount -l /mnt/drbd7
> > primary# drbdadm secondary r7
> > 7: State change failed: (-12) Device is held open by someone Command 
> > 'drbdsetup 7 secondary' terminated with exit code 11 primary# drbdadm 
> > -- --discard-my-data connect r7
> > 
> > Does anyone have a hint?
> 
> It's always worth checking device-mapper:
> 
> dmsetup ls --tree -o inverted

Yep, that's been one of my favorites.

It would "detect" forgotten kpartx mappings,
or still activated LVS (if using DRBD as PV).

Other things that cause "Device held open":

Mounts, of course: if you cannot umount the file system on DRBD,
DRBD would be still held open by that mounted file system;
you need to umount first.

No, lazy umount ("umount -l") DOES NOT help.

That only "unlinks" it from the name space (no new user can
find it/chdir into it), it does not close the underlying
device until it itself has no more users.

Because it has been mentioned at some point in this thread
as an "and I even tried that..." kind of thing, I reiterate here:

   Don't do umount -l; really, it does not help.
   Most of the time, it rather obfuscates things
   and causes further confusion.

If you  cannot umount, you need to first stop/terminate/kill
all its users.  Sometimes locks (not yet stopped/released nfs stuff)
can be hard to see; you still need to release those, too.

Relative unix domain socket paths would not show up in
fuser or lsof output, but would still hold references.


Finally, there may be a race condition with udev: sometimes, some udev
action triggeres just when you try to demote DRBD, and that short-lived
udev triggered probing process would cause that "held open" thing.

You then just need to retry and hope that
next time there really are no users anymore.


-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
__
please don't Cc me, but send to list   --   I'm subscribed