<br>T<span style="font-size: 10pt; color: navy;">Thanks Matt for your reply to the issue "<b>DRBD device held open by someone</b>".<br>
Today the below problem has been seen again. <br>
<br>
<br>
</span>
<p class="MsoNormal"><span style="background: silver none repeat scroll 0% 0%; font-size: 8pt; color: navy; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;">State change failed: (-12) Device is held open by someone</span></p>
<p class="MsoNormal"><span style="background: silver none repeat scroll 0% 0%; font-size: 8pt; color: navy; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;">Command '/sbin/drbdsetup /dev/drbd0 secondary' terminated with exit code 11</span></p>
<p class="MsoNormal"><span style="background: silver none repeat scroll 0% 0%; font-size: 8pt; color: navy; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial;">/sbin/drbdadm secondary drbd0: exit code 11, mapping to</span><span style="font-size: 10pt; color: navy;">
1</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; color: navy;"><br>
Then we have run the following commands after the above issue has been occurred:<br>
<br>
</span></p>
<p class="MsoNormal"><b><i><span style="font-size: 10pt; color: navy;">[root@abaco-a bin]# grep drbd /proc/mounts</span></i></b></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;"> </span></i></p>
<p class="MsoNormal"><b><i><span style="font-size: 10pt; color: navy;"> </span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size: 10pt; color: navy;">root@abaco-a bin]# lsof | grep drbd</span></i></b></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;">drbd0_wor 13993 root cwd DIR 9,2 1024 2 /</span></i></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;">drbd0_wor 13993 root rtd DIR 9,2 1024 2 /</span></i></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;">drbd0_wor 13993 root txt unknown /proc/13993/exe</span></i></p>
<p class="MsoNormal"><b><i><span style="font-size: 10pt; color: navy;"> </span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size: 10pt; color: navy;">root@abaco-a bin]# ps -eaf | grep drbd</span></i></b></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;">root 13993 1 0 Jun16 ? 00:00:00 [drbd0_worker]</span></i></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;">root 17368 11285 0 04:23 pts/0 00:00:00 grep drbd</span></i></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;">[root@abaco-a bin]#</span></i></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;"> </span></i></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; color: navy;"> </span></i></p>
<p class="MsoNormal"><span style="font-size: 10pt; color: navy;">Should we think that drbd0_worker should have been exited once this issue is occurred?</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; color: navy;"> </span></p>
<p class="MsoNormal"><span style="font-size: 10pt; color: navy;">Then we stopped the heartbeat service which caused system to reboot. After that the problem is gone away.</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; color: navy;"> </span></p>
<span style="font-size: 10pt; color: navy;">Please can anybody explain me what could be the problem at this moment.</span><br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<br>
<br>
Today's Topics:<br>
<br>
<br>
<br>
3. Re: RBD device held open by someone (Matt Graham)<br>
<br>
<br>
Message: 3<br>
Date: Thu, 16 Jun 2011 08:57:19 -0700<br>
From: "Matt Graham" <<a href="mailto:danceswithcrows@usa.net" target="_blank">danceswithcrows@usa.net</a>><br>
Subject: Re: [DRBD-user] RBD device held open by someone<br>
To: <<a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a>><br>
Message-ID: <<a href="mailto:606PFPP5T3168S04.1308239839@web04.cms.usa.net" target="_blank">606PFPP5T3168S04.1308239839@web04.cms.usa.net</a>><br>
Content-Type: text/plain; charset=ISO-8859-1<br>
<br>
Dileep Nayak wrote:<br>
> [*root@]# /etc/ha.d/resource.d/drbddisk drbd0 stop*<br>
> *State change failed: (-12) Device is held open by someone*<br>
> *'/sbin/drbdsetup /dev/drbd0 secondary' terminated with exit code 11<br>
<br>
> I checked the status of the LVM resource which is already stopped<br>
> and seems to be offline.<br>
<br>
> *[root@]# /etc/ha.d/resource.d/LVM vgdrbd0 status*<br>
> * Using volume group(s) on command line*<br>
> * Finding volume group "vgdrbd0"*<br>
> *2011/06/16_12:01:40 INFO: LVM Volume vgdrbd0 is offline*<br>
<br>
Some process somewhere has opened /dev/drbd0. You have to find what that<br>
process is and either tell it to close the DRBD device, or kill the process.<br>
Or /dev/drbd0 is mounted. Make sure that the device isn't mounted first, then<br>
use lsof or fuser to find out which process has the file open. Once you've<br>
figured out what's holding the file open, kill it or tell it to release the<br>
resource, and you should be able to set it to secondary.<br>
<br>
A while back, I had a problem like this, but it was related to bind mounts.<br>
/data was the DRBD resource, and it had been umounted. /data/stuff was<br>
bind-mounted on /somewhere/stuff , and that was preventing the resource from<br>
going secondary. Once I umounted all the bind mounts, stuff started working.<br>
This showed up in the output from mount, but not in lsof or fuser, so it took<br>
a bit longer to fix than I would've liked.<br>
<br>
--<br>
Matt G / Dances With Crows<br>
The Crow202 Blog: <a href="http://crow202.org/wordpress/" target="_blank">http://crow202.org/wordpress/</a><br>
There is no Darkness in Eternity/But only Light too dim for us to see<br>
<br>
<br>
<br>
-<br>
</blockquote></div>