[DRBD-user] Sometimes reinstall starts 'ld:Consistent' rather than the expected 'ld:Inconsistent'

[Jeremy Faith]

Hi,

I have recently started experimenting with drbd and heartbeat. I seem to have
the configuration and install method worked out OK, and generally everything
works fine.

But sometimes when reinstalling(to test my install process) I find that the
first time I run:-
   /etc/init.d/drbd start
cat /proc/drbd shows ld:Consistent rather the expected ld:Inconsistent.
And not surprisingly no sync occurs. This often occurs on both master and
standby machines.
The same thing seems to happen if I do 'modprobe drbd;drbdadm up all' rather
than drbd start.

On these occasions I have tried forcing the sync on the standby using
   drbdadm invalidate all
This forces the standby into an ld:Inconsistent state OK, and a full sync
occurs. But at this point the master has never been in an ld:Inconsistent state
and has never been forced to become primary.

After the full sync I start heartbeat and all seems ok. But when I fail the
system over the filesystem on the standby is messed up.

I'm guessing that drbd is starting in an 'ld:Consistent' state due to some
residual data from the previous install, and as such is not initialising the
meta-data.

First few lines of drbd ouput in /var/log/messages from a GOOD install:-

Mar 22 15:09:14 cdc0 kernel: drbd: initialised. Version: 0.7.17   (api:77/proto:74)
Mar 22 15:09:14 cdc0 kernel: drbd: SVN Revision: 2093 build by root at cdc0_sec,
2006-03-22 14:57:37
Mar 22 15:09:14 cdc0 kernel: drbd: registered as block device major 147
Mar 22 15:09:15 cdc0 kernel: drbd0: Creating state block
Mar 22 15:09:15 cdc0 kernel: klogd 1.4.1, ---------- state change ----------
Mar 22 15:09:15 cdc0 kernel: drbd0: resync bitmap: bits=737280 words=23040
Mar 22 15:09:15 cdc0 kernel: drbd0: size = 2880 MB (2949120 KB)


First few lines of drbd ouput in /var/log/messages from a BAD install:-

Mar 22 17:06:57 cdc0 kernel: drbd: initialised. Version: 0.7.17 (api:77/proto:74)
Mar 22 17:06:57 cdc0 kernel: drbd: SVN Revision: 2093 build by root at cdc0_sec,
2006-03-22 16:59:01
Mar 22 17:06:57 cdc0 kernel: drbd: registered as block device major 147
Mar 22 17:06:59 cdc0 kernel: drbd0: resync bitmap: bits=737280 words=23040
Mar 22 17:06:59 cdc0 kernel: drbd0: size = 2880 MB (2949120 KB)
Mar 22 17:06:59 cdc0 kernel: klogd 1.4.1, ---------- state change ----------
Mar 22 17:06:59 cdc0 kernel: drbd0: 0 KB marked out-of-sync by on disk bit-map


The BAD install is missing the line with:drbd0: Creating state block
So I think am correct about drbd picking up meta-data from a previous install.


What can I do to prevent this from occuring?

Before starting drbd is there some way of clearing the area where drbd stores
the meta-data(I am using meta-disk internal)?


Version info:-
   drbd-0.7.17
   heartbeat-2.0.4
   Fedora Core 4 Kernel 2.6.11-1.1369_FC4smp

I am running drbd on top ov lvm2.

drbd.conf:-

resource r0 {
   protocol C;
   incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep 60 ; halt -f";
   startup {
     wfc-timeout         0;  ## Infinite!
     degr-wfc-timeout  120;  ## 2 minutes.
   }
   disk {
     on-io-error detach;
   }
   net {
     # timeout           60;
     # connect-int       10;
     # ping-int          10;
     # max-buffers     2048;
     # max-epoch-size  2048;
   }
   syncer {
     rate   10M;
     group   1; # sync concurrently with r0
   }

   on cdc0_pri {
     device     /dev/drbd0;
     disk       /dev/VolGroup00/LogVol01;
     address    10.51.21.8:7789;
     meta-disk  internal;
   }

   on cdc0_sec {
     device     /dev/drbd0;
     disk       /dev/VolGroup00/LogVol01;
     address    10.51.21.10:7789;
     meta-disk  internal;
   }
}

Regards
Jeremy Faith