Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi again Sorry for missing the omission of the master role within the colocation constraint. I added it - but unfortunately still no success. (In the meantime I added 2 additional filesystem resources on top of the NFSServer, but that should not change anything regarding the root problem that I miss the demote of DRBDClone.) I again started with all resources located at ventsi-clst1 and issued a 'pcs resource move DRBD_global_clst' (the resource next collocated next to the DRBDClone). With that I end up with all primitive resources stopped and the DRBDClone resource still being master at ventsi-clst1. Here is what pacemaker pretends has to be done: ================================================================== [root at ventsi-clst2 ~]# crm_simulate -Ls Current cluster status: Online: [ ventsi-clst1-sync ventsi-clst2-sync ] ipmi-fence-clst1 (stonith:fence_ipmilan): Started ventsi-clst2-sync ipmi-fence-clst2 (stonith:fence_ipmilan): Started ventsi-clst1-sync IPaddrNFS (ocf::heartbeat:IPaddr2): Stopped NFSServer (ocf::heartbeat:nfsserver): Stopped Master/Slave Set: DRBDClone [DRBD] Masters: [ ventsi-clst1-sync ] <=== still not demoted Slaves: [ ventsi-clst2-sync ] DRBD_global_clst (ocf::heartbeat:Filesystem): Stopped NFS_global_clst (ocf::heartbeat:Filesystem): Stopped BIND_global_clst (ocf::heartbeat:Filesystem): Stopped Allocation scores: native_color: ipmi-fence-clst1 allocation score on ventsi-clst1-sync: -INFINITY native_color: ipmi-fence-clst1 allocation score on ventsi-clst2-sync: INFINITY native_color: ipmi-fence-clst2 allocation score on ventsi-clst1-sync: INFINITY native_color: ipmi-fence-clst2 allocation score on ventsi-clst2-sync: -INFINITY clone_color: DRBDClone allocation score on ventsi-clst1-sync: 0 clone_color: DRBDClone allocation score on ventsi-clst2-sync: 0 clone_color: DRBD:0 allocation score on ventsi-clst1-sync: INFINITY clone_color: DRBD:0 allocation score on ventsi-clst2-sync: 0 clone_color: DRBD:1 allocation score on ventsi-clst1-sync: 0 clone_color: DRBD:1 allocation score on ventsi-clst2-sync: INFINITY native_color: DRBD:0 allocation score on ventsi-clst1-sync: INFINITY native_color: DRBD:0 allocation score on ventsi-clst2-sync: 0 native_color: DRBD:1 allocation score on ventsi-clst1-sync: -INFINITY native_color: DRBD:1 allocation score on ventsi-clst2-sync: INFINITY DRBD:1 promotion score on ventsi-clst2-sync: 10000 DRBD:0 promotion score on ventsi-clst1-sync: 1 native_color: DRBD_global_clst allocation score on ventsi-clst1-sync: -INFINITY native_color: DRBD_global_clst allocation score on ventsi-clst2-sync: INFINITY native_color: IPaddrNFS allocation score on ventsi-clst1-sync: -INFINITY native_color: IPaddrNFS allocation score on ventsi-clst2-sync: 0 native_color: NFSServer allocation score on ventsi-clst1-sync: -INFINITY native_color: NFSServer allocation score on ventsi-clst2-sync: 0 native_color: NFS_global_clst allocation score on ventsi-clst1-sync: 0 native_color: NFS_global_clst allocation score on ventsi-clst2-sync: -INFINITY native_color: BIND_global_clst allocation score on ventsi-clst1-sync: -INFINITY native_color: BIND_global_clst allocation score on ventsi-clst2-sync: 0 Transition Summary: * Start IPaddrNFS (ventsi-clst2-sync) * Start NFSServer (ventsi-clst2-sync) * Demote DRBD:0 (Master -> Slave ventsi-clst1-sync) <=== this demote never happens * Promote DRBD:1 (Slave -> Master ventsi-clst2-sync) * Start DRBD_global_clst (ventsi-clst2-sync) * Start NFS_global_clst (ventsi-clst1-sync) * Start BIND_global_clst (ventsi-clst2-sync) And this is the executed transaction: ================================================================== [root at ventsi-clst2 ~]# crm_simulate --xml-file /var/lib/pacemaker/pengine/pe-input-1157.bz2 --save-graph problem5.graph --save-dotfile problem5.dot -V --simulate Using the original execution date of: 2016-11-09 17:54:10Z Current cluster status: Online: [ ventsi-clst1-sync ventsi-clst2-sync ] ipmi-fence-clst1 (stonith:fence_ipmilan): Started ventsi-clst2-sync ipmi-fence-clst2 (stonith:fence_ipmilan): Started ventsi-clst1-sync IPaddrNFS (ocf::heartbeat:IPaddr2): Started ventsi-clst1-sync NFSServer (ocf::heartbeat:nfsserver): Started ventsi-clst1-sync Master/Slave Set: DRBDClone [DRBD] Masters: [ ventsi-clst1-sync ] Slaves: [ ventsi-clst2-sync ] DRBD_global_clst (ocf::heartbeat:Filesystem): Started ventsi-clst1-sync NFS_global_clst (ocf::heartbeat:Filesystem): Started ventsi-clst2-sync BIND_global_clst (ocf::heartbeat:Filesystem): Started ventsi-clst1-sync Transition Summary: * Stop IPaddrNFS (ventsi-clst1-sync) * Stop NFSServer (ventsi-clst1-sync) * Stop DRBD_global_clst (ventsi-clst1-sync) * Stop NFS_global_clst (Started ventsi-clst2-sync) * Stop BIND_global_clst (ventsi-clst1-sync) Executing cluster transition: * Resource action: NFS_global_clst stop on ventsi-clst2-sync * Resource action: BIND_global_clst stop on ventsi-clst1-sync * Resource action: NFSServer stop on ventsi-clst1-sync * Resource action: IPaddrNFS stop on ventsi-clst1-sync * Resource action: DRBD_global_clst stop on ventsi-clst1-sync * Pseudo action: all_stopped <=== no demote Using the original execution date of: 2016-11-09 17:54:10Z Revised cluster status: Online: [ ventsi-clst1-sync ventsi-clst2-sync ] ipmi-fence-clst1 (stonith:fence_ipmilan): Started ventsi-clst2-sync ipmi-fence-clst2 (stonith:fence_ipmilan): Started ventsi-clst1-sync IPaddrNFS (ocf::heartbeat:IPaddr2): Stopped NFSServer (ocf::heartbeat:nfsserver): Stopped Master/Slave Set: DRBDClone [DRBD] Masters: [ ventsi-clst1-sync ] Slaves: [ ventsi-clst2-sync ] DRBD_global_clst (ocf::heartbeat:Filesystem): Stopped NFS_global_clst (ocf::heartbeat:Filesystem): Stopped BIND_global_clst (ocf::heartbeat:Filesystem): Stopped And finally here the updated config: ================================================================== [root at ventsi-clst1 ~]# pcs config Cluster Name: clst1 Corosync Nodes: ventsi-clst1-sync ventsi-clst2-sync Pacemaker Nodes: ventsi-clst1-sync ventsi-clst2-sync Resources: Resource: IPaddrNFS (class=ocf provider=heartbeat type=IPaddr2) Attributes: ip=xxx.xxx.xxx.xxx cidr_netmask=24 Operations: start interval=0 timeout=20 (IPaddrNFS-start-interval-0) stop interval=0 timeout=20 (IPaddrNFS-stop-interval-0) monitor interval=10 timeout=20 (IPaddrNFS-monitor-interval-10) Resource: NFSServer (class=ocf provider=heartbeat type=nfsserver) Attributes: nfs_shared_infodir=/drbdmnts/global_clst/nfsserversettings/ nfs_ip=xxx.xxx.xxx.xxx nfsd_args="-H xxx.xxx.xxx.xxx" Operations: start interval=0 timeout=40 (NFSServer-start-interval-0) stop interval=0 timeout=20 (NFSServer-stop-interval-0) monitor interval=10 timeout=20 (NFSServer-monitor-interval-10) Master: DRBDClone Meta Attrs: master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true Resource: DRBD (class=ocf provider=linbit type=drbd) Attributes: drbd_resource=nfsdata Operations: start interval=0 timeout=240 (DRBD-start-interval-0) promote interval=0 timeout=90 (DRBD-promote-interval-0) demote interval=0 timeout=90 (DRBD-demote-interval-0) stop interval=0 timeout=100 (DRBD-stop-interval-0) monitor interval=9 role=Master timeout=5 (DRBD-monitor-interval-9) monitor interval=10 role=Slave timeout=5 (DRBD-monitor-interval-10) Resource: DRBD_global_clst (class=ocf provider=heartbeat type=Filesystem) Attributes: device=/dev/drbd1 directory=/drbdmnts/global_clst fstype=ext4 Operations: start interval=0 timeout=60 (DRBD_global_clst-start-interval-0) stop interval=0 timeout=60 (DRBD_global_clst-stop-interval-0) monitor interval=20 timeout=40 (DRBD_global_clst-monitor-interval-20) Resource: NFS_global_clst (class=ocf provider=heartbeat type=Filesystem) Attributes: device=xxx.xxx.xxx.xxx:/drbdmnts/global_clst/nfs directory=/global/nfs fstype=nfs Operations: start interval=0 timeout=60 (NFS_global_clst-start-interval-0) stop interval=0 timeout=60 (NFS_global_clst-stop-interval-0) monitor interval=20 timeout=40 (NFS_global_clst-monitor-interval-20) Resource: BIND_global_clst (class=ocf provider=heartbeat type=Filesystem) Attributes: device=/drbdmnts/global_clst/nfs directory=/global/nfs fstype=none options=bind Operations: start interval=0 timeout=60 (BIND_global_clst-start-interval-0) stop interval=0 timeout=60 (BIND_global_clst-stop-interval-0) monitor interval=20 timeout=40 (BIND_global_clst-monitor-interval-20) Stonith Devices: Resource: ipmi-fence-clst1 (class=stonith type=fence_ipmilan) Attributes: lanplus=1 login=foo passwd=bar action=reboot ipaddr=yyy.yyy.yyy.yyy pcmk_host_check=static-list pcmk_host_list=ventsi-clst1-sync auth=password timeout=30 cipher=1 Operations: monitor interval=60 (ipmi-fence-clst1-monitor-interval-60) Resource: ipmi-fence-clst2 (class=stonith type=fence_ipmilan) Attributes: lanplus=1 login=foo passwd=bar action=reboot ipaddr=zzz.zzz.zzz.zzz pcmk_host_check=static-list pcmk_host_list=ventsi-clst2-sync auth=password timeout=30 cipher=1 Operations: monitor interval=60 (ipmi-fence-clst2-monitor-interval-60) Fencing Levels: Location Constraints: Resource: DRBD_global_clst Disabled on: ventsi-clst1-sync (score:-INFINITY) (role: Started) (id:cli-ban-DRBD_global_clst-on-ventsi-clst1-sync) Resource: ipmi-fence-clst1 Disabled on: ventsi-clst1-sync (score:-INFINITY) (id:location-ipmi-fence-clst1-ventsi-clst1-sync--INFINITY) Resource: ipmi-fence-clst2 Disabled on: ventsi-clst2-sync (score:-INFINITY) (id:location-ipmi-fence-clst2-ventsi-clst2-sync--INFINITY) Ordering Constraints: start IPaddrNFS then start NFSServer (kind:Mandatory) (id:order-IPaddrNFS-NFSServer-mandatory) promote DRBDClone then start DRBD_global_clst (kind:Mandatory) (id:order-DRBDClone-DRBD_global_clst-mandatory) start DRBD_global_clst then start IPaddrNFS (kind:Mandatory) (id:order-DRBD_global_clst-IPaddrNFS-mandatory) start NFSServer then start NFS_global_clst (kind:Mandatory) (id:order-NFSServer-NFS_global_clst-mandatory) start NFSServer then start BIND_global_clst (kind:Mandatory) (id:order-NFSServer-BIND_global_clst-mandatory) Colocation Constraints: NFSServer with IPaddrNFS (score:INFINITY) (id:colocation-NFSServer-IPaddrNFS-INFINITY) IPaddrNFS with DRBD_global_clst (score:INFINITY) (id:colocation-IPaddrNFS-DRBD_global_clst-INFINITY) NFS_global_clst with NFSServer (score:-INFINITY) (id:colocation-NFS_global_clst-NFSServer--INFINITY) BIND_global_clst with NFSServer (score:INFINITY) (id:colocation-BIND_global_clst-NFSServer-INFINITY) DRBD_global_clst with DRBDClone (score:INFINITY) (rsc-role:Started) (with-rsc-role:Master) (id:colocation-DRBD_global_clst-DRBDClone-INFINITY) Resources Defaults: resource-stickiness: INFINITY Operations Defaults: timeout: 10s Cluster Properties: cluster-infrastructure: cman dc-version: 1.1.14-8.el6-70404b0 have-watchdog: false last-lrm-refresh: 1478703150 no-quorum-policy: ignore stonith-enabled: true symmetric-cluster: true Node Attributes: ventsi-clst1-sync: PostgresSon-data-status=DISCONNECT ventsi-clst2-sync: PostgresSon-data-status=DISCONNECT Kind regards Andi -----Original Message----- > DRBD_global_clst with DRBDClone (score:INFINITY) > (id:colocation-DRBD_global_clst-DRBDClone-INFINITY) It took me a while to notice it, it's easily overlooked, but the above constraint is the problem. It says DRBD_global_clst must be located where DRBDClone is running ... not necessarily where DRBDClone is master. This constraint should be created like this: pcs constraint colocation add DRBD_global_clst with master DBRDClone