<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>Jake,</span></div><div><span><br></span></div><div><span>Thank you! This really helped.</span></div><div><span><br></span></div><div><br></div><div><span>Thanks,</span></div><div><span><br></span></div><div><span>Bob</span></div><div><span><br></span></div><div><span><br></span></div><div>From: Jake Smith <<a ymailto="mailto:jsmith@argotec.com" href="mailto:jsmith@argotec.com">jsmith@argotec.com</a>><br></div><div style="font-size: 12pt; font-family: 'times new roman', 'new york', times, serif; "><div style="font-size: 12pt; font-family: 'times new roman', 'new york', times, serif; ">To: Bob Schatz <<a ymailto="mailto:bschatz@yahoo.com" href="mailto:bschatz@yahoo.com">bschatz@yahoo.com</a>><br>Cc: <a ymailto="mailto:drbd-user@lists.linbit.com"
href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>Sent: Wednesday, August 17, 2011 8:10 AM<br>Subject: Re: [DRBD-user] Fw: DRBD STONITH - how is Pacemaker constraint cleared?<br><br><br>Bob - <br>I noticed I don't currently have any colo or ordering "sets" (more than 2 resources) defined in my current config however I did have these two working lines prior to dropping a dual primary setup: <br><br>colocation c_fs_srv_on_ocfs2control_on_drbd_srv_master inf: cl_fs_srv cl_ocfs2control ms_drbd_srv:Master <br>order o_drbd_srv_master_before_ocfs2control_before_fs_srv 0: ms_drbd_srv:promote cl_ocfs2control:start cl_fs_srv:start <br><br>*****Correction****** Grouping and Ordering: <br>I got myself mixed up and gave you mis-information!! You're grouping was fine contrary to what I said in the last post! Sorry about that!<br>Groups start in the order written, stop in reverse order, and collocate<br>in reverse i.e. "group
my_group primitive_IP primitive_FS" is basically<br>writing order primitive_IP primitive_FS and collocation primitive_FS<br>primitive_IP in one step. If you go back to using groups (once you've got things working reliably) you can keep adding to them as you add dependent resources which makes things clearer as the config grows. The big gotchas are you can't put clones in groups and master/slaves basically equal clones so no ms in groups. . You can do the groups and ordering together like this: <br><br>group g-glance glance-fs-p glance-ip-p <br>order o-glance-drbd-before-g-glance inf: ms-glance-drbd:promote g-glance:start<br>colocation c-g-glance-with-glance-drbd inf: g-glance ms-glance-drbd:Master<br><br>The advantages are the scalability - for example here's the next step. Assume you've created a mysql resource glance-mysql-p:<br><br>group g-glance glance-fs-p glance-ip-p glance-mysql-p<br>order o-glance-drbd-before-g-glance inf:
ms-glance-drbd:promote g-glance:start<br>colocation c-g-glance-with-glance-drbd inf: g-glance ms-glance-drbd:Master<br><br>The only thing I had to change to accommodate mysql in ordering and colo was to add it to the existing group. Beautifully simple once you wrap your head around the logic of it! <br><br>(Tip) I don't know if you have used it but there is a property stop-all-resources which I will set to true and commit before making mass changes. It prevents all kinds of fails and weirdness. Then once the config is where you want it commit it and then commit property stop-all-resources=false and everything will start up per the new config. <br><br>(A) My config (I have ClusterMon off for right as it puts out a ton of info every time I test anything): <br><br>node Condor \<br> attributes standby="off"<br>node Vulture \<br> attributes standby="off"<br>primitive
p_crm_monitor ocf:pacemaker:ClusterMon \<br> params htmlfile="/tmp/ClusterMon_undef.html" extra_options="-n -r --mail-to <a ymailto="mailto:sysadmin@argotec.com" href="mailto:sysadmin@argotec.com">sysadmin@argotec.com</a> --mail-from <a ymailto="mailto:ha_cluster@argotec.com" href="mailto:ha_cluster@argotec.com">ha_cluster@argotec.com</a> --mail-host mail.argotec.com:25" \<br> op monitor interval="10" timeout="20" \<br> op start interval="0" timeout="90" \<br> op stop interval="0" timeout="100" \<br> meta target-role="Stopped"<br>primitive p_drbd_arpwatch ocf:linbit:drbd \<br> params drbd_resource="arpwatch" \<br> operations $id="p_drbd_arpwatch-operations"
\<br> op start interval="0" timeout="240" \<br> op stop interval="0" timeout="100" \<br> op monitor interval="30" role="Slave" \<br> op monitor interval="20" role="Master"<br>primitive p_drbd_bind ocf:linbit:drbd \<br> params drbd_resource="bind" \<br> operations $id="p_drbd_bind-operations" \<br> op start interval="0" timeout="240" \<br> op stop interval="0" timeout="100" \<br> op monitor interval="30" role="Slave" \<br> op monitor interval="20" role="Master"<br>primitive p_drbd_ldap ocf:linbit:drbd
\<br> params drbd_resource="ldap" \<br> operations $id="p_drbd_ldap-operations" \<br> op start interval="0" timeout="240" \<br> op stop interval="0" timeout="100" \<br> op monitor interval="30" role="Slave" \<br> op monitor interval="20" role="Master"<br>primitive p_drbd_postfix ocf:linbit:drbd \<br> params drbd_resource="postfix" \<br> operations $id="p_drbd_postfix-operations" \<br> op start interval="0" timeout="240" \<br> op stop interval="0" timeout="100" \<br> op monitor interval="30"
role="Slave" \<br> op monitor interval="20" role="Master"<br>primitive p_drbd_samba ocf:linbit:drbd \<br> params drbd_resource="samba" \<br> operations $id="p_drbd_samba-operations" \<br> op start interval="0" timeout="240" \<br> op stop interval="0" timeout="100" \<br> op monitor interval="30" role="Slave" \<br> op monitor interval="20" role="Master"<br>primitive p_fs_arpwatch ocf:heartbeat:Filesystem \<br> params device="/dev/drbd/by-res/arpwatch" directory="/srv/arpwatch" fstype="ext4" options="noatime" \<br> op monitor interval="120" timeout="40"
\<br> op start interval="0" timeout="60" \<br> op stop interval="0" timeout="60"<br>primitive p_fs_bind ocf:heartbeat:Filesystem \<br> params device="/dev/drbd/by-res/bind" directory="/srv/bind" fstype="ext4" options="noatime" \<br> op monitor interval="120" timeout="40" \<br> op start interval="0" timeout="60" \<br> op stop interval="0" timeout="60"<br>primitive p_fs_ldap ocf:heartbeat:Filesystem \<br> params device="/dev/drbd/by-res/ldap" directory="/srv/ldap" fstype="ext4" options="noatime" \<br> op monitor interval="120" timeout="40" \<br> op start interval="0" timeout="60"
\<br> op stop interval="0" timeout="60"<br>primitive p_fs_postfix ocf:heartbeat:Filesystem \<br> params device="/dev/drbd/by-res/postfix" directory="/srv/postfix" fstype="ext4" options="noatime" \<br> op monitor interval="120" timeout="40" \<br> op start interval="0" timeout="60" \<br> op stop interval="0" timeout="60"<br>primitive p_fs_samba ocf:heartbeat:Filesystem \<br> params device="/dev/drbd/by-res/samba" directory="/srv/samba" fstype="ext4" options="noatime" \<br> op monitor interval="120" timeout="40" \<br> op start interval="0" timeout="60" \<br> op stop interval="0"
timeout="60"<br>primitive p_ip_ntp ocf:heartbeat:IPaddr2 \<br> params ip="192.168.2.203" nic="bond0" cidr_netmask="32" broadcast="192.168.3.255" iflabel="NTP" \<br> operations $id="p_ip_ntp-operations" \<br> op start interval="0" timeout="20" \<br> op stop interval="0" timeout="20" \<br> op monitor interval="10" timeout="20" start-delay="0" \<br> meta target-role="Started"<br>primitive p_ipmilan_condor stonith:ipmilan \<br> params hostname="Condor" ipaddr="192.168.2.61" port="623" auth="md5" priv="admin" login="foo" password="bar" \<br> operations $id="p_ipmilan_condor-operations"
\<br> op start interval="0" timeout="60" \<br> op stop interval="0" timeout="60" \<br> op monitor interval="60" timeout="60" start-delay="0"<br>primitive p_ipmilan_vulture stonith:ipmilan \<br> params hostname="Vulture" ipaddr="192.168.2.62" port="623" auth="md5" priv="admin" login="foo" password="bar" \<br> operations $id="p_ipmilan_vulture-operations" \<br> op start interval="0" timeout="60" \<br> op stop interval="0" timeout="60" \<br> op monitor interval="60" timeout="60" start-delay="0"<br>primitive p_lsb_arpwatch lsb:arpwatch \<br> operations $id="p_lsb_arpwatch-operations"
\<br> op start interval="0" timeout="15" \<br> op stop interval="0" timeout="15" \<br> op monitor interval="15" timeout="15" start-delay="15"<br>primitive p_lsb_ntp lsb:ntp \<br> operations $id="p_lsb_ntp-operations" \<br> op start interval="0" timeout="15" \<br> op stop interval="0" timeout="15" \<br> op monitor interval="15" timeout="15" start-delay="15"<br>group g_arpwatch p_fs_arpwatch p_lsb_arpwatch \<br> meta target-role="Started"<br>ms ms_drbd_arpwatch p_drbd_arpwatch \<br> meta notify="true" master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"<br>ms
ms_drbd_bind p_drbd_bind \<br> meta notify="true" master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"<br>ms ms_drbd_ldap p_drbd_ldap \<br> meta notify="true" master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"<br>ms ms_drbd_postfix p_drbd_postfix \<br> meta notify="true" master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"<br>ms ms_drbd_samba p_drbd_samba \<br> meta notify="true" master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"<br>clone cl_ntp p_lsb_ntp \<br> meta clone-node-max="1" globally-unique="false"<br>location l_ipmilan_condor p_ipmilan_condor -inf: Condor<br>location l_ipmilan_vulture p_ipmilan_vulture -inf: Vulture<br>colocation c_fs_bind_on_drbd_bind_master inf: p_fs_bind
ms_drbd_bind:Master<br>colocation c_fs_ldap_on_drbd_ldap_master inf: p_fs_ldap ms_drbd_ldap:Master<br>colocation c_fs_postfix_on_drbd_postfix_master inf: p_fs_postfix ms_drbd_postfix:Master<br>colocation c_fs_samba_on_drbd_samba_master inf: p_fs_samba ms_drbd_samba:Master<br>colocation c_g_arpwatch_on_drbd_arpwatch_master inf: g_arpwatch ms_drbd_arpwatch:Master<br>colocation c_ip_ntp_on_cl_ntp inf: p_ip_ntp cl_ntp<br>order o_cl_ntp_before_ip_ntp inf: cl_ntp p_ip_ntp<br>order o_drbd_arpwatch_master_before_g_arpwatch inf: ms_drbd_arpwatch:promote g_arpwatch:start<br>order o_drbd_bind_master_before_fs_bind inf: ms_drbd_bind:promote p_fs_bind:start<br>order o_drbd_ldap_master_before_fs_ldap inf: ms_drbd_ldap:promote p_fs_ldap:start<br>order o_drbd_postfix_master_before_fs_postfix inf: ms_drbd_postfix:promote p_fs_postfix:start<br>order o_drbd_samba_master_before_fs_samba inf: ms_drbd_samba:promote p_fs_samba:start<br>property $id="cib-bootstrap-options"
\<br> dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \<br> cluster-infrastructure="openais" \<br> expected-quorum-votes="2" \<br> stop-all-resources="false" \<br> no-quorum-policy="ignore" \<br> stonith-enabled="true" \<br> last-lrm-refresh="1313505520"<br>rsc_defaults $id="rsc-options" \<br> resource-stickiness="200"<br><br>*snip*<br><br>HTH<br>Jake<br><br></div></div></div></body></html>