<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style>
<!--
.EmailQuote
{margin-left:1pt;
padding-left:4pt;
border-left:#800000 2px solid}
-->
</style><style type="text/css" id="owaParaStyle">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;"><br>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div></div>
<div><font size="2" face="Calibri"><span style="font-size:11pt">
<div><font color="#0070C0">Hi</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">I have a basic 2 node active/passive cluster with Pacemaker (1.1.14 , pcs: 0.9.148) / CMAN (3.0.12.1) / Corosync (1.4.7) on RHEL 6.8.</font></div>
<div><font color="#0070C0">This cluster runs NFS on top of DRBD (8.4.4).</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Basically the system is working on both nodes and I can switch the resources from one node to the other.</font></div>
<div><font color="#0070C0">But switching resources to the other node does not work, if I try to move just one resource and have the others follow due to the location constraints.</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">From the logged messages I see that in this “failure case” there is NO attempt to demote/promote the DRBD clone resource.</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Here is my setup:</font></div>
<div><font color="#0070C0">==================================================================</font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Cluster Name: clst1</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Corosync Nodes:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">ventsi-clst1-sync ventsi-clst2-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Pacemaker Nodes:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">ventsi-clst1-sync ventsi-clst2-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resources:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resource: IPaddrNFS (class=ocf provider=heartbeat type=IPaddr2)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Attributes: ip=xxx.xxx.xxx.xxx cidr_netmask=24</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Operations: start interval=0s timeout=20s (IPaddrNFS-start-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> stop interval=0s timeout=20s (IPaddrNFS-stop-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> monitor interval=5s (IPaddrNFS-monitor-interval-5s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resource: NFSServer (class=ocf provider=heartbeat type=nfsserver)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Attributes: nfs_shared_infodir=/var/lib/nfsserversettings/ nfs_ip=xxx.xxx.xxx.xxx nfsd_args="-H xxx.xxx.xxx.xxx"</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Operations: start interval=0s timeout=40 (NFSServer-start-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> stop interval=0s timeout=20s (NFSServer-stop-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> monitor interval=10s timeout=20s (NFSServer-monitor-interval-10s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Master: DRBDClone</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Meta Attrs: master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Resource: DRBD (class=ocf provider=linbit type=drbd)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Attributes: drbd_resource=nfsdata</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Operations: start interval=0s timeout=240 (DRBD-start-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> promote interval=0s timeout=90 (DRBD-promote-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> demote interval=0s timeout=90 (DRBD-demote-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> stop interval=0s timeout=100 (DRBD-stop-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> monitor interval=1s timeout=5 (DRBD-monitor-interval-1s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resource: DRBD_global_clst (class=ocf provider=heartbeat type=Filesystem)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Attributes: device=/dev/drbd1 directory=/drbdmnts/global_clst fstype=ext4</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Operations: start interval=0s timeout=60 (DRBD_global_clst-start-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> stop interval=0s timeout=60 (DRBD_global_clst-stop-interval-0s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> monitor interval=20 timeout=40 (DRBD_global_clst-monitor-interval-20)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Stonith Devices:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resource: ipmi-fence-clst1 (class=stonith type=fence_ipmilan)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Attributes: lanplus=1 login=foo passwd=bar action=reboot ipaddr=yyy.yyy.yyy.yyy pcmk_host_check=static-list pcmk_host_list=ventsi-clst1-sync auth=password timeout=30 cipher=1</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Operations: monitor interval=60s (ipmi-fence-clst1-monitor-interval-60s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resource: ipmi-fence-clst2 (class=stonith type=fence_ipmilan)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Attributes: lanplus=1 login=foo passwd=bar action=reboot ipaddr=zzz.zzz.zzz.zzz pcmk_host_check=static-list pcmk_host_list=ventsi-clst2-sync auth=password timeout=30 cipher=1</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Operations: monitor interval=60s (ipmi-fence-clst2-monitor-interval-60s)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Fencing Levels:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Location Constraints:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Resource: ipmi-fence-clst1</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Disabled on: ventsi-clst1-sync (score:-INFINITY) (id:location-ipmi-fence-clst1-ventsi-clst1-sync--INFINITY)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Resource: ipmi-fence-clst2</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Disabled on: ventsi-clst2-sync (score:-INFINITY) (id:location-ipmi-fence-clst2-ventsi-clst2-sync--INFINITY)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Ordering Constraints:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> start IPaddrNFS then start NFSServer (kind:Mandatory) (id:order-IPaddrNFS-NFSServer-mandatory)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> promote DRBDClone then start DRBD_global_clst (kind:Mandatory) (id:order-DRBDClone-DRBD_global_clst-mandatory)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> start DRBD_global_clst then start IPaddrNFS (kind:Mandatory) (id:order-DRBD_global_clst-IPaddrNFS-mandatory)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Colocation Constraints:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> NFSServer with IPaddrNFS (score:INFINITY) (id:colocation-NFSServer-IPaddrNFS-INFINITY)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> DRBD_global_clst with DRBDClone (score:INFINITY) (id:colocation-DRBD_global_clst-DRBDClone-INFINITY)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> IPaddrNFS with DRBD_global_clst (score:INFINITY) (id:colocation-IPaddrNFS-DRBD_global_clst-INFINITY)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Resources Defaults:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">resource-stickiness: INFINITY</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Operations Defaults:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">timeout: 10s</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Cluster Properties:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">cluster-infrastructure: cman</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">dc-version: 1.1.14-8.el6-70404b0</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">have-watchdog: false</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">last-lrm-refresh: 1478277432</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">no-quorum-policy: ignore</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">stonith-enabled: true</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">symmetric-cluster: true</span></font></div>
<div><font color="#0070C0">==================================================================</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Initial state is e.g. this (all resources at node1):</font></div>
<div><font color="#0070C0"> </font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Online: [ ventsi-clst1-sync ventsi-clst2-sync ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Full list of resources:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">ipmi-fence-clst1 (stonith:fence_ipmilan): Started ventsi-clst2-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">ipmi-fence-clst2 (stonith:fence_ipmilan): Started ventsi-clst1-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">IPaddrNFS (ocf::heartbeat:IPaddr2): Started ventsi-clst1-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">NFSServer (ocf::heartbeat:nfsserver): Started ventsi-clst1-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Master/Slave Set: DRBDClone [DRBD]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Masters: [ ventsi-clst1-sync ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Slaves: [ ventsi-clst2-sync ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">DRBD_global_clst (ocf::heartbeat:Filesystem): Started ventsi-clst1-sync</span></font></div>
<div><font color="#0070C0">==================================================================</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">If I shutdown the cluster at node 1 (‘<font size="1" color="black" face="Courier New"><span style="font-size:8pt">pcs cluster stop</span></font>’) or if I move the DRBD clone resource (‘<font size="1" color="black" face="Courier New"><span style="font-size:8pt">pcs
resource move DRBDClone</span></font>’) all resources switch successfully to node2.</font></div>
<div><font color="#0070C0">I.e. the demote/promote of the DRBD clone resource is working in these cases.</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">But if I try to move any other resource (e.g. ‘<font size="1" color="black" face="Courier New"><span style="font-size:8pt">pcs resource move NFSServer</span></font>’) the resources
<font size="1" color="black" face="Courier New"><span style="font-size:8pt">NFSServer</span></font>,
<font size="1" color="black" face="Courier New"><span style="font-size:8pt">IPaddrNFS</span></font> and
<font size="1" color="black" face="Courier New"><span style="font-size:8pt">DRBD_global_clst</span></font> are stopped at node 1, but then already follows starting of the
<font size="1" color="black" face="Courier New"><span style="font-size:8pt">DRBD_global_clst</span></font> resource at node2, which fails due to the missing demote/promote.</font></div>
<div><font color="#0070C0">As far as I can see there is some follow-up attempt to repair things partially as the resources are started again at node1 exclusive the resource which I moved due to my move command.</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Final state is like this:</font></div>
<div><font color="#0070C0"> </font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Online: [ ventsi-clst1-sync ventsi-clst2-sync ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Full list of resources:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">ipmi-fence-clst1 (stonith:fence_ipmilan): Started ventsi-clst2-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">ipmi-fence-clst2 (stonith:fence_ipmilan): Started ventsi-clst1-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">IPaddrNFS (ocf::heartbeat:IPaddr2): Started ventsi-clst1-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">NFSServer (ocf::heartbeat:nfsserver): Stopped</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Master/Slave Set: DRBDClone [DRBD]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Masters: [ ventsi-clst1-sync ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> Slaves: [ ventsi-clst2-sync ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">DRBD_global_clst (ocf::heartbeat:Filesystem): Started ventsi-clst1-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> </span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">Failed Actions:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">* DRBD_global_clst_start_0 on ventsi-clst2-sync 'unknown error' (1): call=778, status=complete, exitreason='none',</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt"> last-rc-change='Fri Nov 4 19:32:56 2016', queued=0ms, exec=43ms</span></font></div>
<div><font color="#0070C0">==================================================================</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Here are the logged messages for this “failure case”:</font></div>
<div><font color="#0070C0"> </font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.163982+01:00 ventsi-clst1 crmd[6116]: notice: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.168100+01:00 ventsi-clst1 pengine[6115]: notice: On loss of CCM Quorum: Ignore</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.181252+01:00 ventsi-clst1 pengine[6115]: notice: Move IPaddrNFS#011(Started ventsi-clst1-sync -> ventsi-clst2-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.181260+01:00 ventsi-clst1 pengine[6115]: notice: Move NFSServer#011(Started ventsi-clst1-sync -> ventsi-clst2-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.181278+01:00 ventsi-clst1 pengine[6115]: notice: Move DRBD_global_clst#011(Started ventsi-clst1-sync -> ventsi-clst2-sync)
<font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt"><=== here no demote/promote is listed</span></font></span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.182385+01:00 ventsi-clst1 pengine[6115]: notice: Calculated Transition 202: /var/lib/pacemaker/pengine/pe-input-766.bz2</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.182998+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 15: stop NFSServer_stop_0 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.196265+01:00 ventsi-clst1 nfsserver(NFSServer)[15978]: INFO: Stopping NFS server ...</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.249137+01:00 ventsi-clst1 kernel: nfsd: last server has exited, flushing export cache</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.252241+01:00 ventsi-clst1 rpc.mountd[15282]: Caught signal 15, un-registering and exiting.</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.632708+01:00 ventsi-clst1 nfsserver(NFSServer)[15978]: INFO: Stopping sm-notify</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.650552+01:00 ventsi-clst1 nfsserver(NFSServer)[15978]: INFO: Stopping rpc.statd</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:55.666777+01:00 ventsi-clst1 rpc.statd[15243]: Caught signal 15, un-registering and exiting</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.692819+01:00 ventsi-clst1 nfsserver(NFSServer)[15978]: INFO: NFS server stopped</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.695523+01:00 ventsi-clst1 crmd[6116]: notice: Operation NFSServer_stop_0: ok (node=ventsi-clst1-sync, call=1220, rc=0, cib-update=1695, confirmed=true)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.696243+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 12: stop IPaddrNFS_stop_0 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.727882+01:00 ventsi-clst1 IPaddr2(IPaddrNFS)[16108]: INFO: IP status = ok, IP_CIP=</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.733383+01:00 ventsi-clst1 crmd[6116]: notice: Operation IPaddrNFS_stop_0: ok (node=ventsi-clst1-sync, call=1222, rc=0, cib-update=1696, confirmed=true)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.733917+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 48: stop DRBD_global_clst_stop_0 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.757181+01:00 ventsi-clst1 Filesystem(DRBD_global_clst)[16163]: INFO: Running stop for /dev/drbd1 on /drbdmnts/global_clst</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.764684+01:00 ventsi-clst1 Filesystem(DRBD_global_clst)[16163]: INFO: Trying to unmount /drbdmnts/global_clst</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.771260+01:00 ventsi-clst1 Filesystem(DRBD_global_clst)[16163]: INFO: unmounted /drbdmnts/global_clst successfully</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.776640+01:00 ventsi-clst1 crmd[6116]: notice: Operation DRBD_global_clst_stop_0: ok (node=ventsi-clst1-sync, call=1224, rc=0, cib-update=1697, confirmed=true)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.777140+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 49: start DRBD_global_clst_start_0 on ventsi-clst2-sync
<font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt"><=== here</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt"> is the
</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">attempt to start the filesystem at the other node, although
</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">DRBD has not yet been promoted</span></font></span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.840137+01:00 ventsi-clst1 crmd[6116]: warning: Action 49 (DRBD_global_clst_start_0) on ventsi-clst2-sync failed (target: 0 vs. rc: 1): Error</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.840158+01:00 ventsi-clst1 crmd[6116]: notice: Transition aborted by DRBD_global_clst_start_0 'modify' on ventsi-clst2-sync: Event failed (magic=0:1;49:202:0:b7941532-c74b-40cc-a8ad-27b5502b8fba,
cib=0.649.4, source=match_graph_event:381, 0)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.840232+01:00 ventsi-clst1 crmd[6116]: warning: Action 49 (DRBD_global_clst_start_0) on ventsi-clst2-sync failed (target: 0 vs. rc: 1): Error</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.840328+01:00 ventsi-clst1 crmd[6116]: notice: Transition 202 (Complete=5, Pending=0, Fired=0, Skipped=0, Incomplete=5, Source=/var/lib/pacemaker/pengine/pe-input-766.bz2):
Complete</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.843693+01:00 ventsi-clst1 pengine[6115]: notice: On loss of CCM Quorum: Ignore</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.844072+01:00 ventsi-clst1 pengine[6115]: warning: Processing failed op start for DRBD_global_clst on ventsi-clst2-sync: unknown error (1)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.844102+01:00 ventsi-clst1 pengine[6115]: warning: Processing failed op start for DRBD_global_clst on ventsi-clst2-sync: unknown error (1)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.845071+01:00 ventsi-clst1 pengine[6115]: notice: Start IPaddrNFS#011(ventsi-clst2-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.845078+01:00 ventsi-clst1 pengine[6115]: notice: Start NFSServer#011(ventsi-clst2-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.845081+01:00 ventsi-clst1 pengine[6115]: notice: Demote DRBD:0#011(Master -> Slave ventsi-clst1-sync)
<font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt"><=== he</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">re there would be the
</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">necessary</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">
</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">demote/promote … but it’s too late; the start of the filesystem already failed</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">
</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">…</span></font></span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.845083+01:00 ventsi-clst1 pengine[6115]: notice: Promote DRBD:1#011(Slave -> Master ventsi-clst2-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.845084+01:00 ventsi-clst1 pengine[6115]: notice: Recover DRBD_global_clst#011(Started ventsi-clst2-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.847986+01:00 ventsi-clst1 pengine[6115]: notice: Calculated Transition 203: /var/lib/pacemaker/pengine/pe-input-767.bz2
<font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt"><=== </span>
</font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt">… so the above transition gets caught by the</span></font><font size="2" color="#0070C0" face="Calibri"><span style="font-size:11pt"> following attempt to repair things partially</span></font></span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.867679+01:00 ventsi-clst1 pengine[6115]: notice: On loss of CCM Quorum: Ignore</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.868074+01:00 ventsi-clst1 pengine[6115]: warning: Processing failed op start for DRBD_global_clst on ventsi-clst2-sync: unknown error (1)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.868101+01:00 ventsi-clst1 pengine[6115]: warning: Processing failed op start for DRBD_global_clst on ventsi-clst2-sync: unknown error (1)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.868287+01:00 ventsi-clst1 pengine[6115]: warning: Forcing DRBD_global_clst away from ventsi-clst2-sync after 1000000 failures (max=1000000)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.869011+01:00 ventsi-clst1 pengine[6115]: notice: Start IPaddrNFS#011(ventsi-clst1-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.869023+01:00 ventsi-clst1 pengine[6115]: notice: Recover DRBD_global_clst#011(Started ventsi-clst2-sync -> ventsi-clst1-sync)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.869770+01:00 ventsi-clst1 pengine[6115]: notice: Calculated Transition 204: /var/lib/pacemaker/pengine/pe-input-768.bz2</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.870065+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 3: stop DRBD_global_clst_stop_0 on ventsi-clst2-sync</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.908075+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 42: start DRBD_global_clst_start_0 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.931072+01:00 ventsi-clst1 Filesystem(DRBD_global_clst)[16242]: INFO: Running start for /dev/drbd1 on /drbdmnts/global_clst</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.943250+01:00 ventsi-clst1 kernel: EXT4-fs (drbd1): warning: maximal mount count reached, running e2fsck is recommended</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.953253+01:00 ventsi-clst1 kernel: EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts:</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.964284+01:00 ventsi-clst1 crmd[6116]: notice: Operation DRBD_global_clst_start_0: ok (node=ventsi-clst1-sync, call=1225, rc=0, cib-update=1701, confirmed=true)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.965104+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 10: start IPaddrNFS_start_0 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.965325+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 43: monitor DRBD_global_clst_monitor_20000 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:56.996235+01:00 ventsi-clst1 IPaddr2(IPaddrNFS)[16308]: INFO: Adding inet address xxx.xxx.xxx.xxx/24 with broadcast address xxx.xxx.xxx.255 to device bond0</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:57.002059+01:00 ventsi-clst1 IPaddr2(IPaddrNFS)[16308]: INFO: Bringing device bond0 up</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:57.008128+01:00 ventsi-clst1 IPaddr2(IPaddrNFS)[16308]: INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-xxx.xxx.xxx.xxx bond0 xxx.xxx.xxx.xxx
auto not_used not_used</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:57.020159+01:00 ventsi-clst1 crmd[6116]: notice: Operation IPaddrNFS_start_0: ok (node=ventsi-clst1-sync, call=1226, rc=0, cib-update=1703, confirmed=true)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:57.020901+01:00 ventsi-clst1 crmd[6116]: notice: Initiating action 11: monitor IPaddrNFS_monitor_5000 on ventsi-clst1-sync (local)</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:57.052231+01:00 ventsi-clst1 crmd[6116]: notice: Transition 204 (Complete=6, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-768.bz2):
Complete</span></font></div>
<div><font size="1" face="Courier New"><span style="font-size:8pt">2016-11-04T19:32:57.052251+01:00 ventsi-clst1 crmd[6116]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]</span></font></div>
<div><font color="#0070C0">==================================================================</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Any ideas what could be the reason for this behavior?</font></div>
<div><font color="#0070C0">And how could this be fixed?</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">(I already found several articles on the internet with the recommendation to have two separately configured monitor operations for the DRBD resource configured one for the master role and another one for the slave role.</font></div>
<div><font color="#0070C0">Already tried this to no avail.)</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0">Regards</font></div>
<div><font color="#0070C0">Andi</font></div>
<div><font color="#0070C0"> </font></div>
<div><font color="#0070C0"> </font></div>
</span></font></div>
</div>
</div>
</body>
</html>