Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi I am a newcomer to HA/DRBD and had to setup a simplified 2 node VM cluster test envionment with OpenSuse 10.3/2.6.22 (i586) over the last few weeks to at least try to reproduce couple of serious problems which happened in a production environment, Unfortunately I was unable to solve the problems, partly because I don't understand why the ResourceManager seemingly invokes the members of the resource group twice. I have noticed that a normal failover/takeover cleanly stops or starts the resource group once only on the respective node, but when the primary node crashes, the resource group is acquired twice by the secondary node. Now here is the problem. The drbd device is encrypted with cryptsetup which opened the mapped device without an error the first time around, then when the ResourceManager started the member a second time, cryptsetup returned an error code 239 (~'device already opened'). A similar test with a corrupted key on the secondary node showed a similar pattern but the ResourceManager received error code 255 both times. I could never reproduce the specific problem on my test machines. As cryptsetup is started from a service, Crypto (one of the resource members), I thought of it returning a 0 value to the ResourceManager whenever cryptsetup tries to open the encrypted device a second time. Even it this could quickfix the problem, I still haven't got a clue as to why the ResourceManger behaves this way. Can anyone explain, please ? Best regards, Richard PS. Here is a short extract of the original and test message logs. The ha-logs are similar. The original environment uses servers with RAID1. Original Log Jun 25 11:43:24 sms2 kernel: drbd0: peer( Primary -> Secondary ) Jun 25 11:43:24 sms2 kernel: klogd 1.4.1, ---------- state change ---------- Jun 25 11:43:24 sms2 heartbeat: [3855]: info: Received shutdown notice from 'sms1'. Jun 25 11:43:24 sms2 heartbeat: [3855]: info: Resources being acquired from sms1. Jun 25 11:43:24 sms2 heartbeat: [3855]: debug: StartNextRemoteRscReq(): child count 1 Jun 25 11:43:24 sms2 heartbeat: [27727]: info: acquire all HA resources (standby). Jun 25 11:43:24 sms2 heartbeat: [27728]: info: No local resources [/usr/lib/heartbeat/ResourceManager listkeys sms2] to acquire. Jun 25 11:43:24 sms2 heartbeat: [3855]: debug: StartNextRemoteRscReq(): child count 1 *Jun 25 11:43:24 sms2 ResourceManager[27747]: info: Acquiring resource group: sms1 IPaddr::10.134.86.55 drbddisk::drbd0 Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Filesystem::/dev/mapper/cr_drbd0::/var/data::ext3::acl,user_xattr sms_ha* Jun 25 11:43:24 sms2 IPaddr[27771]: INFO: Resource is stopped Jun 25 11:43:24 sms2 ResourceManager[27747]: info: Running /etc/ha.d/resource.d/IPaddr 10.134.86.55 start Jun 25 11:43:24 sms2 ResourceManager[27747]: debug: Starting /etc/ha.d/resource.d/IPaddr 10.134.86.55 start Jun 25 11:43:25 sms2 IPaddr[27823]: INFO: Using calculated nic for 10.134.86.55: eth0 Jun 25 11:43:25 sms2 IPaddr[27823]: INFO: Using calculated netmask for 10.134.86.55: 255.255.255.0 Jun 25 11:43:25 sms2 IPaddr[27823]: INFO: Using calculated broadcast for 10.134.86.55: 10.134.86.255 Jun 25 11:43:25 sms2 avahi-daemon[3555]: Registering new address record for 10.134.86.55 on eth0.IPv4. Jun 25 11:43:25 sms2 avahi-daemon[3555]: Withdrawing address record for 10.134.86.55 on eth0. Jun 25 11:43:25 sms2 avahi-daemon[3555]: Registering new address record for 10.134.86.55 on eth0.IPv4. Jun 25 11:43:25 sms2 IPaddr[27823]: DEBUG: Sending Gratuitous Arp for 10.134.86.55 on eth0:0 [eth0] Jun 25 11:43:25 sms2 IPaddr[27814]: INFO: Success Jun 25 11:43:25 sms2 ResourceManager[27747]: debug: /etc/ha.d/resource.d/IPaddr 10.134.86.55 start done. RC=0 Jun 25 11:43:25 sms2 ResourceManager[27747]: info: Running /etc/ha.d/resource.d/drbddisk drbd0 start Jun 25 11:43:25 sms2 ResourceManager[27747]: debug: Starting /etc/ha.d/resource.d/drbddisk drbd0 start Jun 25 11:43:25 sms2 kernel: drbd0: role( Secondary -> Primary ) Jun 25 11:43:25 sms2 kernel: drbd0: Writing meta data super block now. Jun 25 11:43:25 sms2 ResourceManager[27747]: debug: /etc/ha.d/resource.d/drbddisk drbd0 start done. RC=0 *Jun 25 11:43:25 sms2 ResourceManager[27747]: info: Running /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start* *Jun 25 11:43:25 sms2 ResourceManager[27747]: debug: Starting /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start* *Jun 25 11:43:26 sms2 ResourceManager[27747]: debug: /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start done. RC=0* Jun 25 11:43:26 sms2 Filesystem[28000]: INFO: Resource is stopped Jun 25 11:43:26 sms2 ResourceManager[27747]: info: Running /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr start Jun 25 11:43:26 sms2 ResourceManager[27747]: debug: Starting /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr start Jun 25 11:43:26 sms2 Filesystem[28041]: INFO: Running start for /dev/mapper/cr_drbd0 on /var/data Jun 25 11:43:26 sms2 kernel: kjournald starting. Commit interval 5 seconds Jun 25 11:43:26 sms2 kernel: EXT3-fs warning: checktime reached, running e2fsck is recommended Jun 25 11:43:26 sms2 kernel: EXT3 FS on dm-0, internal journal Jun 25 11:43:26 sms2 kernel: EXT3-fs: mounted filesystem with ordered data mode. Jun 25 11:43:26 sms2 Filesystem[28038]: INFO: Success Jun 25 11:43:26 sms2 ResourceManager[27747]: debug: /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr start done. RC=0 Jun 25 11:43:26 sms2 logger: /etc/ha.d/resource.d/sms_ha status was called Jun 25 11:43:26 sms2 ResourceManager[27747]: info: Running /etc/ha.d/resource.d/sms_ha start Jun 25 11:43:26 sms2 ResourceManager[27747]: debug: Starting /etc/ha.d/resource.d/sms_ha start Jun 25 11:43:26 sms2 logger: /etc/ha.d/resource.d/sms_ha start was called Jun 25 11:43:26 sms2 su: (to lsvadb_3.0-2) root on none Jun 25 11:43:26 sms2 su: (to lsvadb_3.0-2) root on none Jun 25 11:43:27 sms2 kernel: drbd0: peer( Secondary -> Unknown ) conn( Connected -> TearDown ) pdsk( UpToDate -> DUnknown ) Jun 25 11:43:27 sms2 kernel: drbd0: Creating new current UUID Jun 25 11:43:27 sms2 kernel: drbd0: Writing meta data super block now. Jun 25 11:43:27 sms2 kernel: drbd0: meta connection shut down by peer. Jun 25 11:43:27 sms2 kernel: drbd0: asender terminated Jun 25 11:43:27 sms2 kernel: drbd0: tl_clear() Jun 25 11:43:27 sms2 kernel: drbd0: Connection closed Jun 25 11:43:27 sms2 kernel: drbd0: conn( TearDown -> Unconnected ) Jun 25 11:43:27 sms2 kernel: drbd0: receiver terminated Jun 25 11:43:27 sms2 kernel: drbd0: receiver (re)started Jun 25 11:43:27 sms2 kernel: drbd0: conn( Unconnected -> WFConnection ) Jun 25 11:43:27 sms2 kernel: JBD: barrier-based sync failed on dm-0 - disabling barriers Jun 25 11:43:29 sms2 su: (to lsva) root on none ... Jun 25 11:43:30 sms2 ResourceManager[27747]: debug: /etc/ha.d/resource.d/sms_ha start done. RC=0 Jun 25 11:43:30 sms2 heartbeat: [27727]: info: all HA resource acquisition completed (standby). Jun 25 11:43:30 sms2 heartbeat: [3855]: info: Standby resource acquisition done [all]. Jun 25 11:43:30 sms2 heartbeat: [28288]: debug: notify_world: setting SIGCHLD Handler to SIG_DFL Jun 25 11:43:30 sms2 harc[28288]: info: Running /etc/ha.d/rc.d/status status *Jun 25 11:43:30 sms2 mach_down[28307]: info: Taking over resource group IPaddr::10.134.86.55* *Jun 25 11:43:30 sms2 ResourceManager[28330]: info: Acquiring resource group: sms1 IPaddr::10.134.86.55 drbddisk::drbd0 Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Filesystem::/dev/mapper/cr_drbd0::/var/data::ext3::acl,user_xattr sms_ha* Jun 25 11:43:30 sms2 etnetclient[28225]: ALARMIERUNG: Ausfuehrung echo "SM Server gestartet." | mailx -s "SM SERVER NOTIFICATION" christoph.reichenbach at dtc-ag.ch erfolgreich Jun 25 11:43:30 sms2 IPaddr[28354]: INFO: Running OK *Jun 25 11:43:30 sms2 ResourceManager[28330]: info: Running /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start* *Jun 25 11:43:30 sms2 ResourceManager[28330]: debug: Starting /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start* *Jun 25 11:43:30 sms2 ResourceManager[28330]: debug: /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start done. RC=239* *Jun 25 11:43:30 sms2 ResourceManager[28330]: ERROR: Return code 239 from /etc/ha.d/resource.d/Crypto* *Jun 25 11:43:30 sms2 ResourceManager[28330]: CRIT: Giving up resources due to failure of Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto* Jun 25 11:43:30 sms2 ResourceManager[28330]: info: Releasing resource group: sms1 IPaddr::10.134.86.55 drbddisk::drbd0 Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Filesystem::/dev/mapper/cr_drbd0::/var/data::ext3::acl,user_xattr sms_ha Jun 25 11:43:30 sms2 ResourceManager[28330]: info: Running /etc/ha.d/resource.d/sms_ha stop Jun 25 11:43:30 sms2 ResourceManager[28330]: debug: Starting /etc/ha.d/resource.d/sms_ha stop Jun 25 11:43:31 sms2 logger: /etc/ha.d/resource.d/sms_ha stop was called *Test Log* * * Jul 14 06:08:45 sms1 IPaddr[5186]: DEBUG: Sending Gratuitous Arp for 192.168.1.200 on eth1:0 [eth1] Jul 14 06:08:45 sms1 IPaddr[5171]: INFO: Success Jul 14 06:08:45 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/IPaddr 192.168.1.200 start done. RC=0 Jul 14 06:08:45 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/drbddisk drbd0 start Jul 14 06:08:45 sms1 kernel: NET: Registered protocol family 17 Jul 14 06:08:45 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/drbddisk drbd0 start Jul 14 06:08:45 sms1 kernel: drbd0: role( Secondary -> Primary ) Jul 14 06:08:45 sms1 kernel: drbd0: Writing meta data super block now. Jul 14 06:08:45 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/drbddisk drbd0 start done. RC=0 Jul 14 06:08:45 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start Jul 14 06:08:45 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start Jul 14 06:08:45 sms1 avahi-daemon[3513]: Invalid query packet. Jul 14 06:08:46 sms1 syslog-ng[2562]: last message repeated 2 times Jul 14 06:08:46 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start done. RC=255 Jul 14 06:08:46 sms1 ResourceManager[5088]: ERROR: Return code 255 from /etc/ha.d/resource.d/Crypto Jul 14 06:08:46 sms1 ResourceManager[5088]: CRIT: Giving up resources due to failure of Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Jul 14 06:08:46 sms1 ResourceManager[5088]: info: Releasing resource group: sms1 IPaddr::192.168.1.200 drbddisk::drbd0 Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Filesystem::/dev/mapper/cr_drbd0::/var/data::ext3::acl,user_xattr sms_ha Jul 14 06:08:46 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/sms_ha stop Jul 14 06:08:46 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/sms_ha stop Jul 14 06:08:46 sms1 logger: /etc/ha.d/resource.d/sms_ha stop was called Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/sms_ha stop done. RC=0 Jul 14 06:08:48 sms1 avahi-daemon[3513]: Invalid query packet. Jul 14 06:08:48 sms1 kernel: drbd0: peer( Secondary -> Unknown ) conn( Connected -> TearDown ) pdsk( UpToDate -> DUnknown ) Jul 14 06:08:48 sms1 kernel: drbd0: Creating new current UUID Jul 14 06:08:48 sms1 kernel: drbd0: Writing meta data super block now. Jul 14 06:08:48 sms1 kernel: drbd0: meta connection shut down by peer. Jul 14 06:08:48 sms1 kernel: drbd0: asender terminated Jul 14 06:08:48 sms1 kernel: drbd0: tl_clear() Jul 14 06:08:48 sms1 kernel: drbd0: Connection closed Jul 14 06:08:48 sms1 kernel: drbd0: conn( TearDown -> Unconnected ) Jul 14 06:08:48 sms1 kernel: drbd0: receiver terminated Jul 14 06:08:48 sms1 kernel: drbd0: receiver (re)started Jul 14 06:08:48 sms1 kernel: drbd0: conn( Unconnected -> WFConnection ) Jul 14 06:08:48 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr stop Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr stop Jul 14 06:08:48 sms1 Filesystem[5481]: WARNING: Couldn't find device [/dev/mapper/cr_drbd0]. Expected /dev/??? to exist Jul 14 06:08:48 sms1 Filesystem[5481]: INFO: Running stop for /dev/mapper/cr_drbd0 on /var/data Jul 14 06:08:48 sms1 Filesystem[5478]: INFO: Success Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr stop done. RC=0 Jul 14 06:08:48 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto stop Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto stop Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto stop done. RC=0 Jul 14 06:08:48 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/drbddisk drbd0 stop Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/drbddisk drbd0 stop Jul 14 06:08:48 sms1 kernel: drbd0: role( Primary -> Secondary ) Jul 14 06:08:48 sms1 kernel: drbd0: Writing meta data super block now. Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/drbddisk drbd0 stop done. RC=0 Jul 14 06:08:48 sms1 ResourceManager[5088]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.1.200 stop Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: Starting /etc/ha.d/resource.d/IPaddr 192.168.1.200 stop Jul 14 06:08:48 sms1 avahi-daemon[3513]: Withdrawing address record for 192.168.1.200 on eth1. Jul 14 06:08:48 sms1 IPaddr[5582]: INFO: Success Jul 14 06:08:48 sms1 avahi-daemon[3513]: Invalid query packet. Jul 14 06:08:48 sms1 ResourceManager[5088]: debug: /etc/ha.d/resource.d/IPaddr 192.168.1.200 stop done. RC=0 Jul 14 06:08:48 sms1 heartbeat: [5073]: info: all HA resource acquisition completed (standby). Jul 14 06:08:48 sms1 heartbeat: [4374]: info: Standby resource acquisition done [all]. Jul 14 06:08:48 sms1 heartbeat: [5616]: debug: notify_world: setting SIGCHLD Handler to SIG_DFL Jul 14 06:08:48 sms1 harc[5616]: info: Running /etc/ha.d/rc.d/status status Jul 14 06:08:49 sms1 mach_down[5626]: info: /usr/lib/heartbeat/mach_down: nice_failback: foreign resources acquired Jul 14 06:08:49 sms1 mach_down[5626]: info: mach_down takeover complete for node sms2. Jul 14 06:08:49 sms1 heartbeat: [4374]: info: mach_down takeover complete. Jul 14 06:08:49 sms1 heartbeat: [5654]: debug: notify_world: setting SIGCHLD Handler to SIG_DFL Jul 14 06:08:49 sms1 harc[5654]: info: Running /etc/ha.d/rc.d/ip-request-resp ip-request-resp Jul 14 06:08:49 sms1 ip-request-resp[5654]: received ip-request-resp IPaddr::192.168.1.200 OK yes Jul 14 06:08:49 sms1 ResourceManager[5669]: info: Acquiring resource group: sms1 IPaddr::192.168.1.200 drbddisk::drbd0 Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Filesystem::/dev/mapper/cr_drbd0::/var/data::ext3::acl,user_xattr sms_ha Jul 14 06:08:49 sms1 IPaddr[5693]: INFO: Resource is stopped Jul 14 06:08:49 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.1.200 start Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/IPaddr 192.168.1.200 start Jul 14 06:08:49 sms1 IPaddr[5745]: INFO: Using calculated nic for 192.168.1.200: eth1 Jul 14 06:08:49 sms1 IPaddr[5745]: INFO: Using calculated netmask for 192.168.1.200: 255.255.255.0 Jul 14 06:08:49 sms1 IPaddr[5745]: INFO: Using calculated broadcast for 192.168.1.200: 192.168.1.255 Jul 14 06:08:49 sms1 avahi-daemon[3513]: Registering new address record for 192.168.1.200 on eth1.IPv4. Jul 14 06:08:49 sms1 IPaddr[5745]: DEBUG: Sending Gratuitous Arp for 192.168.1.200 on eth1:0 [eth1] Jul 14 06:08:49 sms1 IPaddr[5736]: INFO: Success Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: /etc/ha.d/resource.d/IPaddr 192.168.1.200 start done. RC=0 Jul 14 06:08:49 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/drbddisk drbd0 start Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/drbddisk drbd0 start Jul 14 06:08:49 sms1 kernel: drbd0: role( Secondary -> Primary ) Jul 14 06:08:49 sms1 kernel: drbd0: Writing meta data super block now. Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: /etc/ha.d/resource.d/drbddisk drbd0 start done. RC=0 Jul 14 06:08:49 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start Jul 14 06:08:49 sms1 avahi-daemon[3513]: Invalid query packet. Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto start done. RC=255 Jul 14 06:08:49 sms1 ResourceManager[5669]: ERROR: Return code 255 from /etc/ha.d/resource.d/Crypto Jul 14 06:08:49 sms1 ResourceManager[5669]: CRIT: Giving up resources due to failure of Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Jul 14 06:08:49 sms1 ResourceManager[5669]: info: Releasing resource group: sms1 IPaddr::192.168.1.200 drbddisk::drbd0 Crypto::cr_drbd0::/dev/drbd0::/etc/key.cr_drbd0::noauto Filesystem::/dev/mapper/cr_drbd0::/var/data::ext3::acl,user_xattr sms_ha Jul 14 06:08:49 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/sms_ha stop Jul 14 06:08:49 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/sms_ha stop Jul 14 06:08:49 sms1 logger: /etc/ha.d/resource.d/sms_ha stop was called Jul 14 06:08:51 sms1 ResourceManager[5669]: debug: /etc/ha.d/resource.d/sms_ha stop done. RC=0 Jul 14 06:08:51 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr stop Jul 14 06:08:51 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr stop Jul 14 06:08:51 sms1 Filesystem[5992]: WARNING: Couldn't find device [/dev/mapper/cr_drbd0]. Expected /dev/??? to exist Jul 14 06:08:51 sms1 Filesystem[5992]: INFO: Running stop for /dev/mapper/cr_drbd0 on /var/data Jul 14 06:08:51 sms1 Filesystem[5989]: INFO: Success Jul 14 06:08:51 sms1 ResourceManager[5669]: debug: /etc/ha.d/resource.d/Filesystem /dev/mapper/cr_drbd0 /var/data ext3 acl,user_xattr stop done. RC=0 Jul 14 06:08:51 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto stop Jul 14 06:08:51 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto stop Jul 14 06:08:51 sms1 ResourceManager[5669]: debug: /etc/ha.d/resource.d/Crypto cr_drbd0 /dev/drbd0 /etc/key.cr_drbd0 noauto stop done. RC=0 Jul 14 06:08:51 sms1 ResourceManager[5669]: info: Running /etc/ha.d/resource.d/drbddisk drbd0 stop Jul 14 06:08:51 sms1 ResourceManager[5669]: debug: Starting /etc/ha.d/resource.d/drbddisk drbd0 stop Jul 14 06:08:51 sms1 kernel: drbd0: role( Primary -> Secondary ) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20120724/e3b29b9f/attachment.htm>