[DRBD-user] drbd turned on me, and bit my hand this morn

[Brad Barnett]

On Tue, 6 Dec 2005 19:09:47 +0100
Lars Ellenberg <Lars.Ellenberg at linbit.com> wrote:

> / 2005-12-06 11:56:52 -0500
> \ Brad Barnett:
> > 
> > 
> > 
> > Just to add to this.
> > 
> > When drbdadm is in this "locked" state, I can not kill it, or kill -9
> > it.  This means that I must reboot the box, instead of rmmod the
> > module and make my configuration changes.
> > 
> > Because of this, I do not think this is the expected behaviour.
> > 
> > I'm using 0.7.10-3 (The latest Debian stable version), and found the
> > same behaviour with 0.7.14 freshly installed from tarballs.
> 
> 
> how much storage?

1.2T

> 
> how long did you wait before rebooting the box?

I went through several reboots, probably about 10.  The longest I waited
was about 10 minutes.

At one point, I changed the config file to 10 seconds, for every time
listed there (including the wfc-timeout mentioned by Diego).  It didn't
seem to time out, and I waited for quite some time (at least 5 minutes).

> 
> what is in dmesg or kern.log?

dmesg showed drbd starting, ala:

Dec  6 10:14:23 files kernel: drbd: registered as block device major 147
Dec  6 10:14:29 files kernel: drbd0: resync bitmap: bits=292720365
words=9147512 
Dec  6 10:14:29 files kernel: drbd0: size = 1116 GB (1170881460 KB)
Dec  6 10:14:29 files kernel: drbd0: Assuming that all blocks are out of
sync (aka FullSync)


> 
> what does /proc/drbd look like at that time?

/proc/drbd showed this:


 0: cs:Unconfigured
 1: cs:Unconfigured


> 
> did you try to boot the box without drbd/heartbeat started by init,
> then start drbd by hand? if so, any difference?
> 

No difference.  After the initial trouble, I had to rename the startup
script.  I'm working remotely, and can not boot into single user mode.. I
need ssh and have no serial console.

Anyhow, starting drbd with the following:

drbdadm up all
drbdadm primary all
drbdadm whatever

Caused no difference.  I would modprobe the drbd module manually.

It is hard to go into great detail, as I was obviously quite pressed for
time, and it was paramount that the box returned.

If you want, I can take the box down at night, and do tests.  However, I
will need to wait for my new raid to finish rsyncing from the old drbd/now
mounted direct box.  Once done, I can resync and do tests.

With the resync speed, plus the rsync speed.. I guess I can do this
sometime Thursday...

I still don't understand why it always shows two drbd devices though. :/