<html><body><div style="color:#000; background-color:#fff; font-family:Courier New, courier, monaco, monospace, sans-serif;font-size:10pt">For reference: the DRBD resource configuration files...<br><br>> san1:~ # cat /etc/drbd.d/r*.res<br>> resource r0 {<br>> volume 0 {<br>> device /dev/drbd0 ;<br>> disk /dev/sda7 ;<br>> meta-disk internal ;<br>> }<br>> volume 1 {<br>> device /dev/drbd1 ;<br>> disk /dev/sda8 ;<br>> meta-disk internal ;<br>> }<br>> volume 2 {<br>> device /dev/drbd2 ;<br>> disk /dev/sda9 ;<br>>
meta-disk internal ;<br>> }<br>> volume 3 {<br>> device /dev/drbd3 ;<br>> disk /dev/sda10 ;<br>> meta-disk internal ;<br>> }<br>> on san1 {<br>> address 192.168.1.1:7789 ;<br>> }<br>> on san2 {<br>> address 192.168.1.2:7789 ;<br>> }<br>> }<br>> <br>> resource r1 {<br>> volume 0 {<br>> device /dev/drbd1000 ;<br>> disk /dev/sdb7 ;<br>> meta-disk internal ;<br>> }<br>>
volume 1 {<br>> device /dev/drbd1001 ;<br>> disk /dev/sdb8 ;<br>> meta-disk internal ;<br>> }<br>> volume 2 {<br>> device /dev/drbd1002 ;<br>> disk /dev/sdb9 ;<br>> meta-disk internal ;<br>> }<br>> volume 3 {<br>> device /dev/drbd1003 ;<br>> disk /dev/sdb10 ;<br>> meta-disk internal ;<br>> }<br>> on san1 {<br>> address 192.168.1.1:7790 ;<br>> }<br>> on san2 {<br>>
address 192.168.1.2:7790 ;<br>> }<br>> }<br><br><div><span>TIA,</span></div><div style="color: rgb(0, 0, 0); font-size: 13.3333px; font-family: Courier New,courier,monaco,monospace,sans-serif; background-color: transparent; font-style: normal;"><br></div><div class="yui_3_7_2_22_1358535454619_77" style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">Eric Pretorious</div><div class="yui_3_7_2_22_1358535454619_78" style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">Truckee, cA<br></div><br><div><br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; margin-top: 5px; padding-left: 5px;"> <div style="font-family: Courier New, courier, monaco, monospace,
sans-serif; font-size: 10pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font face="Arial" size="2"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Eric <epretorious@yahoo.com><br> <b><span style="font-weight: bold;">To:</span></b> "drbd-user@lists.linbit.com" <drbd-user@lists.linbit.com> <br> <b><span style="font-weight: bold;">Sent:</span></b> Sunday, January 20, 2013 10:24 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Diagnosing a Failed Resource<br> </font> </div> <br>
<div id="yiv1665203294"><div><div style="color:#000;background-color:#fff;font-family:Courier New, courier, monaco, monospace, sans-serif;font-size:10pt;"><div>I've configured corosync+pacemaker to managee a simple two-resource DRBD cluster:</div><div><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">> san1:~ # crm configure show | cat -<br>> node san1 \<br>> attributes standby="off"<br>> node san2 \<br>> attributes standby="off"<br>> primitive p_DRBD-r0 ocf:linbit:drbd \<br>> params drbd_resource="r0" \<br>> op monitor interval="60s"<br>> primitive p_DRBD-r1 ocf:linbit:drbd \<br>> params drbd_resource="r1" \<br>> op monitor interval="60s"<br>> primitive p_IP-1_253
ocf:heartbeat:IPaddr2 \<br>>
params ip="192.168.1.253" cidr_netmask="24" \<br>> op monitor interval="30s"<br>> primitive p_IP-1_254 ocf:heartbeat:IPaddr2 \<br>> params ip="192.168.1.254" cidr_netmask="24" \<br>> op monitor interval="30s"<br>> primitive p_iSCSI-san1 ocf:heartbeat:iSCSITarget \<br>> params iqn="<a target="_blank" href="http://iqn.2012-11.com.example.sa/">iqn.2012-11.com</a>.example.san1:sda" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san1_0 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san1:sda" lun="0" path="/dev/drbd0" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san1_1 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san1:sda" lun="1" path="/dev/drbd1" \<br>>
op monitor interval="10s"<br>> primitive
p_iSCSI-san1_2 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san1:sda" lun="2" path="/dev/drbd2" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san1_3 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san1:sda" lun="3" path="/dev/drbd3" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san2 ocf:heartbeat:iSCSITarget \<br>> params iqn="iqn.2012-11.com.example.san2:sda" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san2_0 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san2:sda" lun="0" path="/dev/drbd1000" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san2_1 ocf:heartbeat:iSCSILogicalUnit \<br>> params
target_iqn="iqn.2012-11.com.example.san2:sda" lun="1" path="/dev/drbd1001" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san2_2 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san2:sda" lun="2" path="/dev/drbd1002" \<br>> op monitor interval="10s"<br>> primitive p_iSCSI-san2_3 ocf:heartbeat:iSCSILogicalUnit \<br>> params target_iqn="iqn.2012-11.com.example.san2:sda" lun="3" path="/dev/drbd1003" \<br>> op monitor interval="10s"<br>> group g_iSCSI-san1 p_iSCSI-san1 p_iSCSI-san1_0 p_iSCSI-san1_1 p_iSCSI-san1_2 p_iSCSI-san1_3 p_IP-1_254<br>> group g_iSCSI-san2 p_iSCSI-san2 p_iSCSI-san2_0 p_iSCSI-san2_1 p_iSCSI-san2_2 p_iSCSI-san2_3 p_IP-1_253<br>> ms ms_DRBD-r0 p_DRBD-r0 \<br>> meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"<br>>
ms ms_DRBD-r1 p_DRBD-r1 \<br>> meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"<br>> location l_iSCSI-san1_and_DRBD-r0 p_IP-1_254 10240: san1<br>> location l_iSCSI-san2_and_DRBD-r1 p_IP-1_253 10240: san2<br>> colocation c_iSCSI_with_DRBD-r0 inf: g_iSCSI-san1 ms_DRBD-r0:Master<br>> colocation c_iSCSI_with_DRBD-r1 inf: g_iSCSI-san2 ms_DRBD-r1:Master<br>> order o_DRBD-r0_before_iSCSI-san1 inf: ms_DRBD-r0:promote g_iSCSI-san1:start<br>> order o_DRBD-r1_before_iSCSI-san2 inf: ms_DRBD-r1:promote g_iSCSI-san2:start<br>> property $id="cib-bootstrap-options" \<br>> dc-version="1.1.7-77eeb099a504ceda05d648ed161ef8b1582c7daf" \<br>> cluster-infrastructure="openais" \<br>> expected-quorum-votes="2" \<br>> stonith-enabled="false" \<br>> no-quorum-policy="ignore"<br></div><div
style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;"><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">The cluster appears to be functioning correctly:<br></div><div><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">> san1:~ # crm_mon -1<br>> ============<br>> Last updated: Sun Jan 20 22:20:17 2013<br>> Last change: Sun Jan 20 21:59:15 2013 by root via crm_attribute on san1<br>> Stack: openais<br>> Current DC: san1 - partition with quorum<br>> Version: 1.1.7-77eeb099a504ceda05d648ed161ef8b1582c7daf<br>> 2 Nodes configured, 2 expected votes<br>> 16 Resources
configured.<br>> ============<br>> <br>> Online: [ san1 san2 ]<br>> <br>> Master/Slave Set: ms_DRBD-r0 [p_DRBD-r0]<br>> Masters: [ san1 ]<br>> Slaves: [ san2 ]<br>> Resource Group: g_iSCSI-san1<br>> p_iSCSI-san1 (ocf::heartbeat:iSCSITarget): Started san1<br>> p_iSCSI-san1_0 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_iSCSI-san1_1 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_iSCSI-san1_2 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_iSCSI-san1_3 (ocf::heartbeat:iSCSILogicalUnit): Started
san1<br>> p_IP-1_254 (ocf::heartbeat:IPaddr2): Started san1<br>> Master/Slave Set: ms_DRBD-r1 [p_DRBD-r1]<br>> Masters: [ san2 ]<br>> Slaves: [ san1 ]<br>> Resource Group: g_iSCSI-san2<br>> p_iSCSI-san2 (ocf::heartbeat:iSCSITarget): Started san2<br>> p_iSCSI-san2_0 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_iSCSI-san2_1 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_iSCSI-san2_2 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_iSCSI-san2_3
(ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_IP-1_253 (ocf::heartbeat:IPaddr2): Started san2<br><br>> san2:~ # crm_mon -1<br>> ============<br>> Last updated: Sun Jan 20 22:20:17 2013<br>> Last change: Sun Jan 20 21:59:15 2013 by root via crm_attribute on san1<br>> Stack: openais<br>> Current DC: san1 - partition with quorum<br>> Version: 1.1.7-77eeb099a504ceda05d648ed161ef8b1582c7daf<br>> 2 Nodes configured, 2 expected votes<br>> 16 Resources configured.<br>> ============<br>> <br>> Online: [ san1 san2 ]<br>> <br>> Master/Slave Set: ms_DRBD-r0 [p_DRBD-r0]<br>> Masters: [ san1 ]<br>> Slaves: [ san2 ]<br>> Resource Group: g_iSCSI-san1<br>> p_iSCSI-san1
(ocf::heartbeat:iSCSITarget): Started san1<br>> p_iSCSI-san1_0 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_iSCSI-san1_1 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_iSCSI-san1_2 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_iSCSI-san1_3 (ocf::heartbeat:iSCSILogicalUnit): Started san1<br>> p_IP-1_254 (ocf::heartbeat:IPaddr2): Started san1<br>> Master/Slave Set: ms_DRBD-r1 [p_DRBD-r1]<br>> Masters: [ san2 ]<br>> Slaves: [ san1 ]<br>> Resource Group:
g_iSCSI-san2<br>> p_iSCSI-san2 (ocf::heartbeat:iSCSITarget): Started san2<br>> p_iSCSI-san2_0 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_iSCSI-san2_1 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_iSCSI-san2_2 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_iSCSI-san2_3 (ocf::heartbeat:iSCSILogicalUnit): Started san2<br>> p_IP-1_253 (ocf::heartbeat:IPaddr2): Started san2<br><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;
background-color:transparent;font-style:normal;">However, the two DRBD resources do not appear to be communicating:</div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;"><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">> san1:~ # cat /proc/drbd <br>> version: 8.4.1 (api:1/proto:86-100)<br>> GIT-hash: 91b4c048c1a0e06777b5f65d312b38d47abaea80 build by phil@fat-tyre, 2011-12-20 12:43:15<br>> 0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:3259080<br>> 1: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0
dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 2: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 3: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> <br>> 1000: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 1001: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 1002: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:0 al:0 bm:0
lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 1003: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r-----<br>> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0</div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;"><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">> san2:~ # cat /proc/drbd <br>> version: 8.4.1 (api:1/proto:86-100)<br>> GIT-hash: 91b4c048c1a0e06777b5f65d312b38d47abaea80 build by phil@fat-tyre, 2011-12-20 12:43:15<br>> 0: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:140<br>> 1:
cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 2: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 3: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> <br>> 1000: cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 1001: cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 1002: cs:WFConnection ro:Primary/Unknown
ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br>> 1003: cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown C r-----<br>> ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">How can I begin to troubleshoot this error?</div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;"><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">Eric Pretorious</div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:Courier
New, courier, monaco, monospace, sans-serif;background-color:transparent;font-style:normal;">Truckee, cA<br></div></div></div></div><br><br> </div> </div> </blockquote></div> </div></body></html>