Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Mon, Jun 20, 2011 at 09:31:03AM +0200, Ralf Gross wrote: > Hello, > > Debian Lenny AMD64, drbd 8.3.7 (active/passiv, 4 TB disk space), > heartbeat 2.1.3, Samba, Apache > > Nagios told me this morning that our cluster had some problems. At the > time I looked at the primary it had a load of ~100 and I couldn't even > stop the services or reboot properly. Not even kill -9 the processes. > In Cacti I see that the CPU load was constantly increasing since 23:00 > yesterday, at that time a script (durep) started which creates disk > usage reports. > > There was nothing drbd related written to the logfiles on the primary, > but dmesg output had some drbd lines. It seems that the primary already had > problems with sshd several hours before the first drbd kernel message appears. > > The first drbd message on primay is > > [9043241.932135] block drbd0: sock was shut down by peer > > > then after some more messages and a partial sync drbd decided that a > full sync was necessary. > > [9043258.995403] block drbd0: Writing the whole bitmap, full sync required after drbd_sync_handshake. > > > Any hints what happend (I guess network problems, but can't find any reason) > and why a full sync was started? At the time this happend the data hasn't > change much. Below are the dmesg and the log messages I have. There are various races when the connection is lost while starting, or finishing, a resync. Some of them led to "spurious full sync" on the next handshake. More recent drbd should handle these better, though we know we still have at least one corner case unsolved. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed