[DRBD-user] DRBDmanage (re)initialization

[Julien Escario]

Le 12/06/2017 à 10:09, Robert Altnoeder a écrit :
> On 06/12/2017 09:39 AM, Julien Escario wrote:
> 
>> Finally, I've been able to fully restore vm4 and vm5 (drbdsetup and drbdmanage
>> working) but not vm7.
>>
>> I've done that by firewalling port 6999 (port used by .drbdctrl ressource) and
>> issuing a down/up on drbdctrl on vm4 and vm5.
>>
>> [...]
>>
>> It would be really nice to get back to normal without a reboot. Any advice ?
> Force-disconnecting and/or firewalling the ports of all DRBD resources
> sometimes unblocks stuck kernel threads, but if even that does not help,
> then there is virtually no way to recover from a stuck kernel without
> rebooting it (well, maybe with a kernel debugger there would be, but
> that's a rather theoretical possibility).

Nice idea ! Let's try this on vm7 :
# iptables -A INPUT -i eth1.1007 -p tcp --match multiport --dports 7000:7030 -j DROP
Same for outbound (to be sure) :
# iptables -A OUTPUT -o eth1.1007 -p tcp --match multiport --dports 7000:7030 -j
DROP

root at vm4:~# cat /var/lib/drbd.d/drbdmanage_vm-104-disk-1.res
	[...]
	   connection-mesh {
      		hosts vm7 vm4;
	   }
    on vm7 {
        node-id 1;
        address 10.152.12.60:7007;
	[...]

So, this ressource (vm104 disk 1) use port 7007.

Check :
root at vm4:~# nmap -p7007 vm7
PORT     STATE    SERVICE
7007/tcp filtered afs3-fileserver

TCP sessions were still opened so I killed them all with tcpkill :
root at vm4:~# tcpkill -i eth1.1007 portrange 7000-7030

All TCP sessions disapeard from netstat and my ressources are gone to connecting
state.

But situation isn't really better ... drbd processes are still blocked on an
undetermined I/O. I'll wait a few hours to see if something finally timeout on
the server and if it's not the case, I'll resignate and reboot the whole box.
(and UPGRADE !).

Thanks for your help !
Julien

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3705 bytes
Desc: Signature cryptographique S/MIME
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20170612/3468d783/attachment.bin>