<html>
<body>
Hi,<br><br>
I have a 2 node cluster with sles11sp1, with the latest patches.
Configured Pacemaker, dual primary drbd and xen.<br><br>
Here's the configuration:<br><br>
- drbd.conf<br>
global {<br>
usage-count yes;<br>
}<br>
common {<br>
protocol C;<br>
disk {<br>
on-io-error
detach;<br>
fencing resource-only;<br>
}<br>
syncer {<br>
rate
1G;<br>
al-extents 3389;<br>
}<br>
net {<br>
allow-two-primaries; # Enable this *after*
initial testing<br>
cram-hmac-alg sha1;<br>
shared-secret
"a6a0680c40bca2439dbe48343ddddcf4";<br>
after-sb-0pri discard-zero-changes;<br>
after-sb-1pri discard-secondary;<br>
after-sb-2pri disconnect;<br>
}<br>
startup {<br>
become-primary-on both;<br>
}<br>
handlers {<br>
fence-peer
"/usr/lib/drbd/crm-fence-peer.sh";<br>
after-resync-target
"/usr/lib/drbd/crm-unfence-peer.sh";<br><br>
}<br>
}<br>
resource vmsvn {<br>
device /dev/drbd0;<br>
disk
/dev/sdb;<br>
meta-disk internal;<br>
on xm01 {<br>
address 100.0.0.1:7788;<br>
}<br>
on xm02 {<br>
address 100.0.0.2:7788;<br>
}<br>
}<br><br>
resource srvsvn1 {<br>
protocol C;<br>
device /dev/drbd1;<br>
disk
/dev/sdc;<br>
meta-disk internal;<br>
on xm01 {<br>
address 100.0.0.1:7789;<br>
}<br>
on xm02 {<br>
address 100.0.0.2:7789;<br>
}<br>
}<br><br>
resource srvsvn2 {<br>
protocol C;<br>
device /dev/drbd2;<br>
disk
/dev/sdd;<br>
meta-disk internal;<br>
on xm01 {<br>
address 100.0.0.1:7790;<br>
}<br>
on xm02 {<br>
address 100.0.0.2:7790;<br>
}<br>
}<br><br>
resource vmconfig {<br>
protocol C;<br>
device /dev/drbd3;<br>
meta-disk internal;<br>
on xm01 {<br>
address 100.0.0.1:7791;<br>
disk
/dev/vg_xm01/lv_xm01_vmconfig;<br>
}<br>
on xm02 {<br>
address 100.0.0.2:7791;<br>
disk
/dev/vg_xm02/lv_xm02_vmconfig;<br>
}<br>
}<br><br>
================================<br><br>
- crm configuration:<br>
node xm01<br>
node xm02<br>
primitive VMSVN ocf:heartbeat:Xen \<br>
meta
target-role="Started" allow-migrate="true"
is-managed="true" resource-stickiness="0" \<br>
operations
$id="VMSVN-operations" \<br>
op monitor
interval="30" timeout="30" \<br>
op start
interval="0" timeout="60" \<br>
op stop interval="0"
timeout="60" \<br>
op migrate_to
interval="0" timeout="180" \<br>
params
xmfile="/etc/xen/vm/vmsvn"<br>
primitive clvm ocf:lvm2:clvmd \<br>
operations
$id="clvm-operations" \<br>
op monitor
interval="10" timeout="20"<br>
primitive dlm ocf:pacemaker:controld \<br>
operations
$id="dlm-operations" \<br>
op monitor
interval="10" timeout="20"
start-delay="0"<br>
primitive ipmi-stonith-xm01 stonith:external/ipmi \<br>
meta
target-role="Started" is-managed="true"
priority="10" \<br>
operations
$id="ipmi-stonith-xm01-operations" \<br>
op monitor
interval="15" timeout="15" start-delay="15"
\<br>
params
hostname="xm01" ipaddr="125.1.254.107"
userid="administrator" passwd="17xm45"
interface="lan"<br>
primitive ipmi-stonith-xm02 stonith:external/ipmi \<br>
meta
target-role="Started" is-managed="true"
priority="9" \<br>
operations
$id="ipmi-stonith-xm02-operations" \<br>
op monitor
interval="15" timeout="15" start-delay="15"
\<br>
params
hostname="xm02" ipaddr="125.1.254.248"
userid="administrator" passwd="17xm45"
interface="lan"<br>
primitive o2cb ocf:ocfs2:o2cb \<br>
operations
$id="o2cb-operations" \<br>
op monitor
interval="10" timeout="20"<br>
primitive srvsvn1-drbd ocf:linbit:drbd \<br>
params
drbd_resource="srvsvn1" \<br>
operations
$id="srvsvn1-drbd-operations" \<br>
op monitor
interval="20" role="Master" timeout="20"
\<br>
op monitor
interval="30" role="Slave" timeout="20"
\<br>
op start
interval="0" timeout="240" \<br>
op promote
interval="0" timeout="90" \<br>
op demote
interval="0" timeout="90" \<br>
op stop interval="0"
timeout="100" \<br>
meta
migration-threshold="10" failure-timeout="600"<br>
primitive srvsvn2-drbd ocf:linbit:drbd \<br>
params
drbd_resource="srvsvn2" \<br>
operations
$id="srvsvn2-drbd-operations" \<br>
op monitor
interval="20" role="Master" timeout="20"
\<br>
op monitor
interval="30" role="Slave" timeout="20"
\<br>
op start
interval="0" timeout="240" \<br>
op promote
interval="0" timeout="90" \<br>
op demote
interval="0" timeout="90" \<br>
op stop interval="0"
timeout="100" \<br>
meta
migration-threshold="10" failure-timeout="600"<br>
primitive vg_svn ocf:heartbeat:LVM \<br>
params
volgrpname="vg_svn"<br>
primitive vmconfig ocf:linbit:drbd \<br>
operations
$id="vmconfig-operations" \<br>
op monitor
interval="20" role="Master" timeout="120"
\<br>
op start
interval="0" timeout="240" \<br>
op promote
interval="0" timeout="90" \<br>
op demote
interval="0" timeout="90" \<br>
op stop interval="0"
timeout="100" \<br>
params
drbd_resource="vmconfig" \<br>
meta
migration-threshold="10" failure-timeout="600"<br>
primitive vmconfig-pri ocf:heartbeat:Filesystem \<br>
operations
$id="vmconfig-pri-operations" \<br>
op monitor
interval="20" timeout="40" \<br>
params
device="/dev/drbd3" directory="/vmconfig"
fstype="ocfs2"<br>
primitive vmsvn-drbd ocf:linbit:drbd \<br>
operations
$id="vmsvn-drbd-operations" \<br>
op monitor
interval="20" role="Master" timeout="20"
\<br>
op monitor
interval="30" role="Slave" timeout="20"
\<br>
op start
interval="0" timeout="240" \<br>
op promote
interval="0" timeout="90" \<br>
op demote
interval="0" timeout="90" \<br>
op stop interval="0"
timeout="100" \<br>
params
drbd_resource="vmsvn" \<br>
meta
is-managed="true" migration-threshold="10"
failure-timeout="600"<br>
group init-group dlm o2cb clvm<br>
group operaciones-group vg_svn<br>
ms ms_drbd_srvsvn1 srvsvn1-drbd \<br>
meta master-max="2"
notify="true" target-role="Started"
priority="6"<br>
ms ms_drbd_srvsvn2 srvsvn2-drbd \<br>
meta master-max="2"
notify="true" target-role="Started"
priority="5"<br>
ms ms_drbd_vmconfig vmconfig \<br>
meta master-max="2"
clone-max="2" notify="true" priority="8"
target-role="Started"<br>
ms ms_drbd_vmsvn vmsvn-drbd \<br>
meta
resource-stickiness="100" master-max="2"
notify="true" target-role="Started"
priority="7"<br>
clone init-clone init-group \<br>
meta
interleave="true" target-role="Started"
is-managed="true" priority="4"<br>
clone operaciones-clone operaciones-group \<br>
meta
target-role="Started" is-managed="true"
priority="2" interleave="true"<br>
clone vmconfig-clone vmconfig-pri \<br>
meta
target-role="Started" priority="3"
is-managed="true"<br>
location drbd-fence-by-handler-ms_drbd_srvsvn1 ms_drbd_srvsvn1 \<br>
rule
$id="drbd-fence-by-handler-rule-ms_drbd_srvsvn1"
$role="Master" -inf: #uname ne xm01<br>
location drbd-fence-by-handler-ms_drbd_srvsvn2 ms_drbd_srvsvn2 \<br>
rule
$id="drbd-fence-by-handler-rule-ms_drbd_srvsvn2"
$role="Master" -inf: #uname ne xm01<br>
location drbd-fence-by-handler-ms_drbd_vmconfig ms_drbd_vmconfig \<br>
rule
$id="drbd-fence-by-handler-rule-ms_drbd_vmconfig"
$role="Master" -inf: #uname ne xm01<br>
location drbd-fence-by-handler-ms_drbd_vmsvn ms_drbd_vmsvn \<br>
rule
$id="drbd-fence-by-handler-rule-ms_drbd_vmsvn"
$role="Master" -inf: #uname ne xm01<br>
location location-stonith-xm01 ipmi-stonith-xm01 -inf: xm01<br>
location location-stonith-xm02 ipmi-stonith-xm02 -inf: xm02<br>
colocation colocacion : init-clone operaciones-clone vmconfig-clone VMSVN
ms_drbd_vmconfig:Master ms_drbd_vmsvn:Master ms_drbd_srvsvn1:Master
ms_drbd_srvsvn2:Master<br>
order ordenamiento : ms_drbd_vmconfig:promote ms_drbd_vmsvn:promote
ms_drbd_srvsvn1:promote ms_drbd_srvsvn2:promote init-clone:start
operaciones-clone:start vmconfig-clone:start VMSVN:start<br>
property $id="cib-bootstrap-options" \<br>
dc-version="1.1.5-5bd2b9154d7d9f86d7f56fe0a74072a5a6590c60"
\<br>
cluster-infrastructure="openais" \<br>
expected-quorum-votes="2" \<br>
no-quorum-policy="ignore" \<br>
last-lrm-refresh="1330538418"<br><br>
=======================================<br><br>
Now, according to this page:
<a href="http://www.drbd.org/users-guide-8.3/s-pacemaker-fencing.html">
http://www.drbd.org/users-guide-8.3/s-pacemaker-fencing.html</a> <br>
the last paragraph, says: <br>
Thus, if the DRBD replication link becomes disconnected, the
crm-fence-peer.sh script contacts the cluster manager, determines the
Pacemaker Master/Slave resource associated with this DRBD resource, and
ensures that the Master/Slave resource no longer gets promoted on any
node other than the currently active one. Conversely, when the connection
is re-established and DRBD completes its synchronization process, then
that constraint is removed and the cluster manager is free to promote the
resource on any node again.<br><br>
Unfortunately, that is not happening in my configuration and I don't
understand why.<br>
Here's what I'm doing: <br>
1) rcnetwork stop on XM01<br>
2) XM02 stonith's XM01 (so far, so good)<br>
3) the VM migrates to XM02 (1 minute downtime which is more than
fine)<br>
4) XM01 comes back<br>
5) The DRBD resources appear as Master/Slave (on dual primary!)<br>
6) I can see some constraints generated by the handler in drbd.conf<br>
7) xm02:~ # rcdrbd status<br>
drbd driver loaded OK; device status:<br>
version: 8.3.11 (api:88/proto:86-96)<br>
GIT-hash: 0de839cee13a4160eed6037c4bddd066645e23c5 build by
phil@fat-tyre, 2011-06-29 11:37:11<br>
m:res
cs
ro
ds
p mounted fstype<br>
0:vmsvn Connected Secondary/Primary
UpToDate/UpToDate C<br>
1:srvsvn1 Connected Secondary/Primary
UpToDate/UpToDate C<br>
2:srvsvn2 Connected Secondary/Primary
UpToDate/UpToDate C<br>
3:vmconfig Connected Secondary/Primary
UpToDate/UpToDate C<br><br>
They are all UPTODATE!<br>
8) The constraints generated by the handler are still there. Waited a
lifetime, still there... <br>
9) Manually remove the constraints, the VM goes down for a little while
and the DRBD resources are back as Master/Master.<br><br>
is there anything wrong in my configuration? How can both nodes become
Master on a fail back?<br><br>
Thanks!<br>
Daniel</body>
</html>