Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Sorry, I accidentially sent the message again to this list. just ignore it.
2008/9/18 Christoph Eßer <cehser at gmail.com>:
> Hi there,
>
> I first came up with this question on Linux-HA mailing list. As they think
> it semms to be a DRBD related problem, I repeat it here:
>
> I am using Heartbeat 2.99,Pacemaker 0.6.6 and DRBD 8.0.13 on Debian Lenny.
>
> Anything works as expected if I just force the system to failover by
> unplugging the power supply of any of my two nodes. Shutting down the
> DRBD slave via "init 0" or "init 6" works fine as well.
>
> But whenever I restart my DRBD master via "init 6" the whole cluster
> crashes and fails to reassign the resources, even after the node started
> up again. Anyway, I found some strange DRBD related messages in
> /var/log/syslog on the second node while restarting the master node:
>
> Sep 16 13:48:44 viktor-01 drbd[4123]: [4135]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:48:44 viktor-01 drbd[4123]: [4151]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:49:14 viktor-01 drbd[4164]: [4176]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:49:14 viktor-01 drbd[4164]: [4192]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:49:36 viktor-01 drbd[4226]: [4239]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:49:36 viktor-01 drbd[4226]: [4255]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:49:41 viktor-01 drbd[4283]: [4296]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:49:41 viktor-01 drbd[4283]: [4313]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:49:54 viktor-01 drbd[4348]: [4360]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:49:54 viktor-01 drbd[4348]: [4376]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:49:54 viktor-01 drbd[4348]: [4386]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf primary r0
> Sep 16 13:49:54 viktor-01 kernel: [ 495.079064] drbd0: helper command:
> /sbin/drbdadm outdate-peer
> Sep 16 13:50:15 viktor-01 drbd[4405]: [4416]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf secondary r0
> Sep 16 13:50:16 viktor-01 drbd[4405]: [4424]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 secondary' terminated with exit code 20 drbdadm
> aborting
> Sep 16 13:50:16 viktor-01 lrmd: [2279]: info: RA output:
> (drbd:1:demote:stdout) No response from the DRBD driver! Is the module
> loaded? Command 'drbdsetup /dev/drbd0 secondary' terminated with exit
> code 20 drbdadm aborting
> Sep 16 13:50:16 viktor-01 drbd[4426]: [4439]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:50:17 viktor-01 drbd[4426]: [4447]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:17 viktor-01 drbd[4426]: [4455]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:50:17 viktor-01 drbd[4426]: [4463]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:17 viktor-01 drbd[4426]: [4464]: DEBUG: r0 status: No
> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not
> configured Not configured Unconfigured
> Sep 16 13:50:19 viktor-01 drbd[4482]: [4494]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:50:19 viktor-01 drbd[4482]: [4502]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:19 viktor-01 drbd[4482]: [4510]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:50:20 viktor-01 drbd[4482]: [4518]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:20 viktor-01 drbd[4482]: [4519]: DEBUG: r0 status: No
> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not
> configured Not configured Unconfigured
> Sep 16 13:50:21 viktor-01 drbd[4526]: [4538]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:50:21 viktor-01 drbd[4526]: [4546]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:21 viktor-01 drbd[4526]: [4554]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:50:22 viktor-01 drbd[4526]: [4562]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:22 viktor-01 drbd[4526]: [4563]: DEBUG: r0 status: No
> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not
> configured Not configured Unconfigured
> Sep 16 13:50:22 viktor-01 drbd[4526]: [4564]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf up r0
> Sep 16 13:50:22 viktor-01 drbd[4526]: [4572]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 disk /dev/cciss/c0d0p4 /dev/cciss/c0d0p4 internal
> --set-defaults --create-device --fencing=resource-only' terminated with
> exit code 20
> Sep 16 13:50:22 viktor-01 lrmd: [2279]: info: RA output:
> (drbd:0:start:stdout) No response from the DRBD driver! Is the module
> loaded? Command 'drbdsetup /dev/drbd0 disk /dev/cciss/c0d0p4
> /dev/cciss/c0d0p4 internal --set-defaults --create-device
> --fencing=resource-only' terminated with exit code 20
> Sep 16 13:50:22 viktor-01 drbd[4526]: [4574]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:50:23 viktor-01 drbd[4526]: [4582]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:23 viktor-01 drbd[4526]: [4590]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:50:23 viktor-01 drbd[4526]: [4598]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:23 viktor-01 drbd[4526]: [4599]: DEBUG: r0 status: No
> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not
> configured Not configured Unconfigured
> Sep 16 13:50:25 viktor-01 drbd[4613]: [4625]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:50:25 viktor-01 drbd[4613]: [4633]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:25 viktor-01 drbd[4613]: [4641]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:50:26 viktor-01 drbd[4613]: [4649]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:26 viktor-01 drbd[4613]: [4650]: DEBUG: r0 status: No
> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not
> configured Not configured Unconfigured
> Sep 16 13:50:26 viktor-01 drbd[4657]: [4669]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf state r0
> Sep 16 13:50:27 viktor-01 drbd[4657]: [4677]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 state' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:27 viktor-01 drbd[4657]: [4685]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf cstate r0
> Sep 16 13:50:27 viktor-01 drbd[4657]: [4693]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 cstate' terminated with exit code 20 drbdadm aborting
> Sep 16 13:50:27 viktor-01 drbd[4657]: [4694]: DEBUG: r0 status: No
> response from the DRBD driver! Is the module loaded? Command 'drbdsetup
> /dev/drbd0 state' terminated with exit code 20 drbdadm aborting Not
> configured Not configured Unconfigured
> Sep 16 13:50:27 viktor-01 drbd[4657]: [4695]: DEBUG: r0: Calling drbdadm
> -c /etc/drbd.conf up r0
> Sep 16 13:50:28 viktor-01 drbd[4657]: [4703]: DEBUG: r0: Command output:
> No response from the DRBD driver! Is the module loaded? Command
> 'drbdsetup /dev/drbd0 disk /dev/cciss/c0d0p4 /dev/cciss/c0d0p4 internal
> --set-defaults --create-device --fencing=resource-only' terminated with
> exit code 20
>
>
> I don't see any reason for these errors. I would appreciate anyone
> helping me to find a starting point for a solution.
>
> Regards
>
> Christoph
>
>
> resource r0 {
> protocol C;
> # incon-degr-cmd "killall heartbeat"; #"halt -f"; # killall heartbeat would
> be a good alternative :->
>
> syncer {
> rate 40M; # Note: 'M' is MegaBytes/s, not MegaBits/s; recommended is 30 %
> of maximum throughput (considering both network and hard disk access)
> }
>
>
> on viktor-01 {
> disk /dev/cciss/c0d0p4;
> device /dev/drbd0;
> meta-disk internal;
> address 172.16.1.1:7789;
> }
> on viktor-02 {
> disk /dev/cciss/c0d0p4;
> device /dev/drbd0;
> meta-disk internal;
> address 172.16.1.2:7789;
> }
> }
>
>
> ucast eth0 172.16.0.1
> ucast eth0 172.16.0.2
> ucast eth1 172.16.1.1
> ucast eth1 172.16.1.2
>
> node viktor-01 viktor-02
> crm on
> respawn hacluster /usr/lib/heartbeat/dopd
> apiauth dopd gid=haclient uid=hacluster
>
>
>