-------------------- Check the version of dopd and drbd-peer-outdater on node1 --------------------------------------------------------- debnode1:~# lsof /usr/lib/heartbeat/dopd COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME dopd 2778 hacluster txt REG 254,0 12744 676612 /usr/lib/heartbeat/dopd debnode1:~# md5sum /usr/lib/heartbeat/dopd 9ce67567ea50157bfd2f0e3f3623010d /usr/lib/heartbeat/dopd debnode1:~# lsof /usr/lib/heartbeat/drbd-peer-outdater debnode1:~# md5sum /usr/lib/heartbeat/drbd-peer-outdater b195f526bb6fa3659f4c63e8f23b1d99 /usr/lib/heartbeat/drbd-peer-outdater debnode1:/usr/lib/heartbeat# md5sum --check < MD5SUM dopd: OK drbd-peer-outdater: OK dopd.bz2: OK drbd-peer-outdater.bz2: OK Check the version of dopd and drbd-peer-outdater on node2 --------------------------------------------------------- debnode2:~# lsof /usr/lib/heartbeat/dopd COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME dopd 2811 hacluster txt REG 254,0 12744 676602 /usr/lib/heartbeat/dopd debnode2:~# md5sum /usr/lib/heartbeat/dopd 9ce67567ea50157bfd2f0e3f3623010d /usr/lib/heartbeat/dopd debnode2:~# lsof /usr/lib/heartbeat/drbd-peer-outdater debnode2:~# md5sum /usr/lib/heartbeat/drbd-peer-outdater b195f526bb6fa3659f4c63e8f23b1d99 /usr/lib/heartbeat/drbd-peer-outdater debnode2:/usr/lib/heartbeat# md5sum --check < MD5SUM dopd: OK drbd-peer-outdater: OK dopd.bz2: OK drbd-peer-outdater.bz2: OK Now test the failover with heartbeat ------------------------------------ debnode1:/usr/lib/heartbeat# cat /proc/drbd version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by root@debnode1, 2008-07-14 18:11:00 0: cs:Connected st:Primary/Secondary ds:UpToDate/UpToDate C r--- ns:4 nr:4 dw:8 dr:17 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 oos:0 ...power off debnode1 .. failover faield with loooping: Aug 18 11:26:57 debnode2 drbd-peer-outdater: [3216]: debug: message: outdater_rc, debnode2 Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: WARN: Cluster node: debnode1: status: dead Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Processed 1 messages Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: invoked: outdater Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Processed 0 messages Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: destroying connection: (null) Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Deleting outdater (0x805fe50) from mainloop Aug 18 11:26:57 debnode2 drbd-peer-outdater: [3219]: debug: drbd peer: debnode1 Aug 18 11:26:57 debnode2 drbd-peer-outdater: [3219]: debug: drbd resource: db Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Connecting channel Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Client outdater (0x805fe50) connected Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: invoked: outdater Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Processing msg from outdater Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Got message from (drbd-peer-outdater). (peer: debnode1, res :db) Aug 18 11:26:57 debnode2 /usr/lib/heartbeat/dopd: [2811]: debug: Starting node walk .. try manually debnode2:/usr/lib/heartbeat# drbdadm primary all /dev/drbd0: State change failed: (-7) Refusing to be Primary while peer is not outdated Command 'drbdsetup /dev/drbd0 primary' terminated with exit code 11 command exited with code 11 ... replace dopd and drbd-peer-outdater with the self compiled one and try again debnode2:/usr/lib/heartbeat# mv drbd-peer-outdater drbd-peer-outdater.NOT_WORKING debnode2:/usr/lib/heartbeat# mv dopd dopd.NOT_WORKING debnode2:/usr/lib/heartbeat# cp -a dopd.WORKING dopd debnode2:/usr/lib/heartbeat# cp -a drbd-peer-outdater.WORKING drbd-peer-outdater debnode2:/usr/lib/heartbeat# md5sum --check < MD5SUM dopd: FEHLSCHLAG drbd-peer-outdater: FEHLSCHLAG dopd.bz2: OK drbd-peer-outdater.bz2: OK md5sum: Warnung: 2 von 4 berechneten Prüfsumme passten NICHT debnode2:/usr/lib/heartbeat# md5sum dopd a91706c8c86cf3fdf3c83c0104862258 dopd debnode2:/usr/lib/heartbeat# md5sum drbd-peer-outdater 1fd162acce837e1a1f9f88718b065295 drbd-peer-outdater debnode2:/usr/lib/heartbeat# killall dopd debnode2:/usr/lib/heartbeat# cat /proc/drbd version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by root@debnode1, 2008-07-14 18:11:00 0: cs:WFConnection st:Secondary/Unknown ds:UpToDate/DUnknown C r--- ns:4 nr:4 dw:4 dr:4 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 oos:0 ... try again debnode2:/usr/lib/heartbeat# drbdadm primary all debnode2:/usr/lib/heartbeat# cat /proc/drbd version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by root@debnode1, 2008-07-14 18:11:00 0: cs:WFConnection st:Primary/Unknown ds:UpToDate/Outdated C r--- ns:4 nr:4 dw:4 dr:4 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 oos:0 --------------