<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style>
<!--
@font-face
{font-family:Calibri}
@font-face
{font-family:Consolas}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
span.EmailStyle17
{font-family:"Calibri","sans-serif";
color:windowtext}
.MsoChpDefault
{font-family:"Calibri","sans-serif"}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.WordSection1
{}
-->
</style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I have managed to get this setup to work in a virtual environment, but am running into problems when running on physical hardware. My current setup is the latest debian linux with the approved packages for drbd and heartbeat</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">This is my heartbeat status when running.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><i>root@HPSATA01:~# crm_mon -1</i></p>
<p class="MsoNormal"><i>============</i></p>
<p class="MsoNormal"><i>Last updated: Tue Apr 2 11:14:32 2013</i></p>
<p class="MsoNormal"><i>Stack: Heartbeat</i></p>
<p class="MsoNormal"><i>Current DC: hpsata02 (9e01243d-9232-437e-81ad-e00aabffef8f) - partition with quorum</i></p>
<p class="MsoNormal"><i>Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b</i></p>
<p class="MsoNormal"><i>2 Nodes configured, unknown expected votes</i></p>
<p class="MsoNormal"><i>2 Resources configured.</i></p>
<p class="MsoNormal"><i>============</i></p>
<p class="MsoNormal"><i> </i></p>
<p class="MsoNormal"><i>Online: [ hpsata01 hpsata02 ]</i></p>
<p class="MsoNormal"><i> </i></p>
<p class="MsoNormal"><i>Master/Slave Set: ms_g_drbd</i></p>
<p class="MsoNormal"><i> Masters: [ hpsata01 ]</i></p>
<p class="MsoNormal"><i> Slaves: [ hpsata02 ]</i></p>
<p class="MsoNormal"><i>Resource Group: g_services</i></p>
<p class="MsoNormal"><i> MetaFS (ocf::heartbeat:Filesystem): Started hpsata01</i></p>
<p class="MsoNormal"><i> lvmdata (ocf::heartbeat:LVM): Started hpsata01</i></p>
<p class="MsoNormal"><i> ClusterIP (ocf::heartbeat:IPaddr2): Started hpsata01</i></p>
<p class="MsoNormal"><i> iscsi (lsb:iscsitarget): Started hpsata01</i></p>
<p class="MsoNormal"><i> </i></p>
<p class="MsoNormal"><i>Failed actions:</i></p>
<p class="MsoNormal"><i> iscsi_monitor_0 (node=hpsata01, call=7, rc=1, status=complete): unknown error</i></p>
<p class="MsoNormal"><i> iscsi_monitor_0 (node=hpsata02, call=105, rc=1, status=complete): unknown error</i></p>
<p class="MsoNormal"><i>root@HPSATA01:~#</i></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">And my DRBD status:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><i>root@HPSATA01:~# service drbd status</i></p>
<p class="MsoNormal"><i>drbd driver loaded OK; device status:</i></p>
<p class="MsoNormal"><i>version: 8.3.7 (api:88/proto:86-91)</i></p>
<p class="MsoNormal"><i>srcversion: EE47D8BF18AC166BE219757</i></p>
<p class="MsoNormal"><i>m:res cs ro ds p mounted fstype</i></p>
<p class="MsoNormal"><i>0:meta Connected Primary/Secondary UpToDate/UpToDate C /meta ext3</i></p>
<p class="MsoNormal"><i>1:data Connected Primary/Secondary UpToDate/UpToDate C</i></p>
<p class="MsoNormal"><i>root@HPSATA01:~#</i></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">So far so good. The problem arises when I put the primary node to standby. The same thing happens when I reboot, power off, or pull the plug.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><i>root@HPSATA01:~# crm_mon -1</i></p>
<p class="MsoNormal"><i>============</i></p>
<p class="MsoNormal"><i>Last updated: Tue Apr 2 12:54:56 2013</i></p>
<p class="MsoNormal"><i>Stack: Heartbeat</i></p>
<p class="MsoNormal"><i>Current DC: hpsata02 (9e01243d-9232-437e-81ad-e00aabffef8f) - partition with quorum</i></p>
<p class="MsoNormal"><i>Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b</i></p>
<p class="MsoNormal"><i>2 Nodes configured, unknown expected votes</i></p>
<p class="MsoNormal"><i>2 Resources configured.</i></p>
<p class="MsoNormal"><i>============</i></p>
<p class="MsoNormal"><i> </i></p>
<p class="MsoNormal"><i>Node hpsata01 (d60c3fcb-55e4-44a1-9e7a-2b03efb501e8): standby</i></p>
<p class="MsoNormal"><i>Online: [ hpsata02 ]</i></p>
<p class="MsoNormal"><i> </i></p>
<p class="MsoNormal"><i>Master/Slave Set: ms_g_drbd</i></p>
<p class="MsoNormal"><i> Slaves: [ hpsata02 ]</i></p>
<p class="MsoNormal"><i> Stopped: [ g_drbd:0 ]</i></p>
<p class="MsoNormal"><i> </i></p>
<p class="MsoNormal">On node 2 this is the DRBD status</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><i>root@HPSATA02:~# service drbd status</i></p>
<p class="MsoNormal"><i>drbd driver loaded OK; device status:</i></p>
<p class="MsoNormal"><i>version: 8.3.7 (api:88/proto:86-91)</i></p>
<p class="MsoNormal"><i>srcversion: EE47D8BF18AC166BE219757</i></p>
<p class="MsoNormal"><i>m:res cs ro ds p mounted fstype</i></p>
<p class="MsoNormal"><i>0:meta WFConnection Secondary/Unknown Outdated/DUnknown C</i></p>
<p class="MsoNormal"><i>1:data WFConnection Secondary/Unknown Outdated/DUnknown C</i></p>
<p class="MsoNormal"><i>root@HPSATA02:~#</i></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Here is some info from my ha-debug file:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size:8.0pt; font-family:"Courier New"">Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: unpack_config: On loss of CCM Quorum: Ignore<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: unpack_status: Node hpsata01 is in standby-mode<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: determine_online_status: Node hpsata01 is standby<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: WARN: unpack_rsc_op: Processing failed op iscsi_monitor_0 on hpsata01: unknown error (1)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: determine_online_status: Node hpsata02 is online<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: WARN: unpack_rsc_op: Processing failed op iscsi_monitor_0 on hpsata02: unknown error (1)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: clone_print: Master/Slave Set: ms_g_drbd<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: short_print: Slaves: [ hpsata02 ]<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: short_print: Stopped: [ g_drbd:0 ]<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: group_print: Resource Group: g_services<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: native_print: MetaFS (ocf::heartbeat:Filesystem): Stopped
<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: native_print: lvmdata (ocf::heartbeat:LVM): Stopped
<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: native_print: ClusterIP (ocf::heartbeat:IPaddr2): Stopped
<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: native_print: iscsi (lsb:iscsitarget): Stopped
<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: native_merge_weights: drbd_meta:0: Rolling back scores from drbd_data:0<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: native_color: Resource drbd_meta:0 cannot run anywhere<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: native_color: Resource drbd_data:0 cannot run anywhere<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: master_color: Promoting g_drbd:1 (Slave hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: master_color: ms_g_drbd: Promoted 1 instances of a possible 1 to master<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: master_color: Promoting g_drbd:1 (Slave hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: master_color: ms_g_drbd: Promoted 1 instances of a possible 1 to master<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: ERROR: create_notification_boundaries: Creating boundaries for ms_g_drbd<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: ERROR: create_notification_boundaries: Creating boundaries for ms_g_drbd<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: ERROR: create_notification_boundaries: Creating boundaries for ms_g_drbd<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: ERROR: create_notification_boundaries: Creating boundaries for ms_g_drbd<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: RecurringOp: Start recurring monitor (30s) for ClusterIP on hpsata02<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Leave resource drbd_meta:0 (Stopped)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Leave resource drbd_data:0 (Stopped)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Promote drbd_meta:1 (Slave -> Master hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Promote drbd_data:1 (Slave -> Master hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Start MetaFS (hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Start lvmdata (hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Start ClusterIP (hpsata02)<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: notice: LogActions: Start iscsi (hpsata02)<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: unpack_graph: Unpacked transition 51: 21 actions in 21 synapses<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: do_te_invoke: Processing graph 51 (ref=pe_calc-dc-1364928813-381) derived from /var/lib/pengine/pe-input-1669.bz2<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 43 fired and confirmed<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: te_rsc_command: Initiating action 84: notify drbd_meta:1_pre_notify_promote_0 on hpsata02 (local)<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: do_lrm_rsc_op: Performing key=84:51:0:d21de83f-ed1b-428e-845e-937210405b1c op=drbd_meta:1_notify_0 )<br>
Apr 02 12:53:33 HPSATA02 lrmd: [1883]: info: rsc:drbd_meta:1:143: notify<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: te_rsc_command: Initiating action 86: notify drbd_data:1_pre_notify_promote_0 on hpsata02 (local)<br>
Apr 02 12:53:33 HPSATA02 crmd: [1886]: info: do_lrm_rsc_op: Performing key=86:51:0:d21de83f-ed1b-428e-845e-937210405b1c op=drbd_data:1_notify_0 )<br>
Apr 02 12:53:33 HPSATA02 lrmd: [1883]: info: rsc:drbd_data:1:144: notify<br>
Apr 02 12:53:33 HPSATA02 pengine: [1892]: info: process_pe_message: Transition 51: PEngine Input stored in: /var/lib/pengine/pe-input-1669.bz2<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: process_lrm_event: LRM operation drbd_meta:1_notify_0 (call=143, rc=0, cib-update=254, confirmed=true) ok<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: process_lrm_event: LRM operation drbd_data:1_notify_0 (call=144, rc=0, cib-update=255, confirmed=true) ok<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: match_graph_event: Action drbd_meta:1_pre_notify_promote_0 (84) confirmed on hpsata02 (rc=0)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: match_graph_event: Action drbd_data:1_pre_notify_promote_0 (86) confirmed on hpsata02 (rc=0)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 44 fired and confirmed<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 41 fired and confirmed<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 27 fired and confirmed<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_rsc_command: Initiating action 16: promote drbd_meta:1_promote_0 on hpsata02 (local)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: do_lrm_rsc_op: Performing key=16:51:0:d21de83f-ed1b-428e-845e-937210405b1c op=drbd_meta:1_promote_0 )<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: rsc:drbd_meta:1:145: promote<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_meta:1:promote:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: process_lrm_event: LRM operation drbd_meta:1_promote_0 (call=145, rc=0, cib-update=256, confirmed=true) ok<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: match_graph_event: Action drbd_meta:1_promote_0 (16) confirmed on hpsata02 (rc=0)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_rsc_command: Initiating action 20: promote drbd_data:1_promote_0 on hpsata02 (local)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: do_lrm_rsc_op: Performing key=20:51:0:d21de83f-ed1b-428e-845e-937210405b1c op=drbd_data:1_promote_0 )<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: rsc:drbd_data:1:146: promote<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_data:1:promote:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: process_lrm_event: LRM operation drbd_data:1_promote_0 (call=146, rc=0, cib-update=257, confirmed=true) ok<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: match_graph_event: Action drbd_data:1_promote_0 (20) confirmed on hpsata02 (rc=0)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 28 fired and confirmed<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 42 fired and confirmed<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_pseudo_action: Pseudo action 45 fired and confirmed<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_rsc_command: Initiating action 85: notify drbd_meta:1_post_notify_promote_0 on hpsata02 (local)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: do_lrm_rsc_op: Performing key=85:51:0:d21de83f-ed1b-428e-845e-937210405b1c op=drbd_meta:1_notify_0 )<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: rsc:drbd_meta:1:147: notify<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: te_rsc_command: Initiating action 87: notify drbd_data:1_post_notify_promote_0 on hpsata02 (local)<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: do_lrm_rsc_op: Performing key=87:51:0:d21de83f-ed1b-428e-845e-937210405b1c op=drbd_data:1_notify_0 )<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: rsc:drbd_data:1:148: notify<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_meta:1:notify:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_data:1:notify:stdout)
<br>
<br>
drbd[2542]: 2013/04/02_12:53:34 ERROR: meta: Called drbdadm -c /etc/drbd.conf outdate meta<br>
drbd[2543]: 2013/04/02_12:53:34 ERROR: data: Called drbdadm -c /etc/drbd.conf outdate data<br>
drbd[2542]: 2013/04/02_12:53:34 ERROR: meta: Exit code 17<br>
drbd[2543]: 2013/04/02_12:53:34 ERROR: data: Exit code 17<br>
drbd[2542]: 2013/04/02_12:53:34 ERROR: meta: Command output: <br>
drbd[2543]: 2013/04/02_12:53:34 ERROR: data: Command output: <br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_meta:1:notify:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_data:1:notify:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 attrd: [1885]: info: attrd_trigger_update: Sending flush op to all hosts for: master-drbd_meta:1 (-INFINITY)<br>
Apr 02 12:53:34 HPSATA02 attrd: [1885]: info: attrd_perform_update: Sent update 137: master-drbd_meta:1=-INFINITY<br>
Apr 02 12:53:34 HPSATA02 attrd: [1885]: info: attrd_trigger_update: Sending flush op to all hosts for: master-drbd_data:1 (-INFINITY)<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_meta:1:notify:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 lrmd: [1883]: info: RA output: (drbd_data:1:notify:stdout)
<br>
<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: process_lrm_event: LRM operation drbd_meta:1_notify_0 (call=147, rc=0, cib-update=258, confirmed=true) ok<br>
Apr 02 12:53:34 HPSATA02 crmd: [1886]: info: process_lrm_event: LRM operation drbd_data:1_notify_0 (call=148, rc=0, cib-update=259, confirmed=true) ok</span><span style="font-size:10.5pt; font-family:Consolas"></span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Once I bring node 1 back up everything works fine again.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">If anyone has a clue as to what is going on I would appreciate the help.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Thanks,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Jared.</p>
</div>
<hr>
<table>
<tbody>
<tr>
<td>This email and any files transmitted with it are intended solely for the use of the individuals or entities to which they are addressed and are not to be disclosed to any other party. If you have received this email in error please return it to the sender,
and delete any copies thereof. Thank you for your cooperation.</td>
</tr>
</tbody>
</table>
</body>
</html>