Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Wed, Mar 11, 2009 at 11:54:03AM -0700, Jeff Orr wrote: > /At Wed Mar 11 19:14:45 CET 2009, Lars Ellenberg said:/ > > I have no idea how exactly you got into this situation. > > > > but to recover from it, without doing a full sync, > > you need to low-level modify the DRBD meta data uuids. > > > > on the box where you want to throw away the data: > > stop (unconfigure) drbd. > > do > > drbdadm -- 0CE9A5964B14BF3A:0000000000000000:AC676DD67EC523BD set-gi $resourcename > > start (configure and connect) drbd. > > > > should then do a bitmap based sync. > > > > I would feel more comfortable with a full sync, though. > > to do so, I'd probably "start from scratch", > > and re-create drbd meta data on the "victim" node. > > > > or at least to an online verify afterwards (we have had various issues > > with online verify in the past, unfortunately, so use latest version). > > > > -- > > :/ Lars Ellenberg > > /:/ LINBIT | Your Way to High Availability > > /:/ DRBD/HA support and consulting http://www.linbit.com > > / > > Thanks for the reply, Lars. Here is what I just did on the secondary: > > [root at docidtxt04 ~]# drbdadm down all > [root at docidtxt04 ~]# drbdadm create-md all > md_offset 6999850872832 > al_offset 6999850840064 > bm_offset 6999637221376 > > Found xfs filesystem which uses 6835583224 kB > current configuration leaves usable 6835583224 kB > > ==> This might destroy existing data! <== > > Do you want to proceed? > [need to type 'yes' to confirm] yes > > You want me to create a v08 style flexible-size internal meta data block. > There apears to be a v08 flexible-size internal meta data block > already in place on /dev/sdc at byte offset 6999850872832 > Do you really want to overwrite the existing v08 meta-data? > [need to type 'yes' to confirm] yes > > Writing meta data... > initialising activity log > NOT initialized bitmap > New drbd meta data block sucessfully created. > [root at docidtxt04 ~]# cat /proc/drbd > version: 8.3.0 (api:88/proto:86-89) > GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by argus at docidtxt04, 2009-03-04 16:23:58 > 0: cs:Unconfigured > > I am going to bring the node's connect up with 'drbdadm connect all'. > Should this be safe to do? I assume that the 'victim' node will > immediately become a SyncTarget. "connect" will only configure the network. without a disk, you cannot become sync target ;) do a "drbdadm up". or "drbdadm adjust". > This would be easier to debug if I could have captured the kernel > panic from the 'victim' node when it crashed after becoming primary. maybe. or maybe not. you can grep -E -e "drbd0: (peer|self) " -e 'drbd0: .* -> ' your /var/log/messages (or wherever), on both nodes. hopefully the time stamps are in sync. go back until the "self" or "peer" lines start with AC676DD67EC523B. that should give an idea of what happened from the DRBD perspective, -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed