<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1465467309;
mso-list-type:hybrid;
mso-list-template-ids:1116349258 567470382 68681731 68681733 68681729 68681731 68681733 68681729 68681731 68681733;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-font-family:Calibri;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="RO" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi folks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Thank you for your time and support in advance.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We have a problem since a while, and it consumed a lot of time to narrow down the reason.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Our final “catch” is that apparently a DRBD resource is stuck and that causes in cascade other problems.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The setup is:<o:p></o:p></span></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-US">RHEL cluster, Pacemaker, DRBD<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-US">httpd, httpd-fs, httpd-storage (configured as a drbd resource with 2 nodes and a diskless tiebreaker node)<o:p></o:p></span></li></ul>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Kernel stack (the problematic one):<o:p></o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffffc09ffeb3>] drbd_flush_workqueue+0x63/0x80 [drbd] (</span></b><b><span lang="EN-US" style="font-size:9.0pt;font-family:Wingdings">ç</span></b><b><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">
stuck)<o:p></o:p></span></b></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffffc09ed2e2>] conn_disconnect+0x102/0x450 [drbd]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffffc09ee438>] drbd_receiver+0x1f8/0x780 [drbd]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffffc09fc206>] drbd_thread_setup+0xb6/0x1f0 [drbd]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffff84ac61f1>] kthread+0xd1/0xe0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffff8518dd1d>] ret_from_fork_nospec_begin+0x7/0x21<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:Consolas">[<ffffffffffffffff>] 0xffffffffffffffff<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Software versions used:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">[root@dep-1 myusername]# rpm -qa | grep drbd<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">kmod-drbd90-9.0.20-1.el7_7.elrepo.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">drbd90-utils-9.12.2-1.el7.elrepo.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">[root@dep-1 myusername]# rpm -qa | grep pacema<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">pacemaker-libs-1.1.20-5.el7_7.2.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">pacemaker-cli-1.1.20-5.el7_7.2.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">pacemaker-1.1.20-5.el7_7.2.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">pacemaker-cluster-libs-1.1.20-5.el7_7.2.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">[root@dep-1 myusername]# rpm -qa | grep corosync<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">corosync-2.4.3-6.el7_7.1.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">corosync-qdevice-2.4.3-6.el7_7.1.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">corosynclib-2.4.3-6.el7_7.1.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">[root@dep-1 frqadmin]# cat /etc/redhat-release<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:8.0pt;font-family:Consolas">Red Hat Enterprise Linux Server release 7.7 (Maipo)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource config attached: drbd-resource-conf.txt.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Stack for all threads attached: drbd-resource-full-stack.txt.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Do you have any idea what could be wrong, what is causing the thread to stuck?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Best Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Szabi.</span><span lang="EN-GB" style="font-size:7.5pt;font-family:"Tahoma",sans-serif;color:dimgray;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</body>
</html>