Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Sun, Oct 07, 2012 at 09:56:35AM -0700, ShockwaveCS wrote: > > Can I downgrade the secondary host back to 8.3.2 without any negative side > effects? -Or- Can I upgrade the 8.3.2 to 8.3.13 while the node is online, > serving data, and still Primary? Doesn't seem likely, but hey why not ask. I > want to do one of these because of the scenario below: > > Note: Split-Brain does not appear in logs on either node. > > I have an issue with our DRBD setup. I upgraded the Secondary host to 8.3.13 > during an upgrade of the host. When syncing the devices everything looks to > be OK and UpToDate. Upon a reboot of the system, 2 of the resources act like > split brain: > > Primary: > 0:r0 Connected Primary/Secondary UpToDate/UpToDate C > 1:r1 WFConnection Primary/Unknown UpToDate/DUnknown C > 2:r2 Connected Primary/Secondary UpToDate/UpToDate C > 3:r3 WFConnection Primary/Unknown UpToDate/DUnknown C > 4:r4 Connected Primary/Secondary UpToDate/UpToDate C > 5:r5 Connected Primary/Secondary UpToDate/UpToDate C > > > Secondary: > 0:r0 Connected Secondary/Primary UpToDate/UpToDate C > 1:r1 StandAlone Secondary/Unknown UpToDate/DUnknown r----- > 2:r2 Connected Secondary/Primary UpToDate/UpToDate C > 3:r3 StandAlone Secondary/Unknown UpToDate/DUnknown r----- > 4:r4 Connected Secondary/Primary UpToDate/UpToDate C > 5:r5 Connected Secondary/Primary UpToDate/UpToDate C > > in the logs I see: > Secondary: > Oct 6 02:32:53 kernel: block drbd1: self > F706B62542107DA8:0000000000000000:1C07F1C6CA998612:D56E6E590DC60A6B bits:0 > flags:0 > Oct 6 02:32:53 kernel: block drbd1: peer > F706B62542107DA9:1C07F1C6CA998613:D56E6E590DC60A6B:23CC2F9D13ACE404 > bits:492137 flags:0 > Oct 6 02:32:53 kernel: block drbd1: uuid_compare()=-1091 by rule 30 > Oct 6 02:32:53 kernel: block drbd1: To resolve this both sides have to > support at least protocol 91 > Oct 6 02:32:53 kernel: block drbd1: conn( WFReportParams -> Disconnecting > ) > Oct 6 02:32:53 kernel: block drbd1: error receiving ReportState, l: 4! > Oct 6 02:32:53 kernel: block drbd1: asender terminated > Oct 6 02:32:53 kernel: block drbd1: Terminating asender thread > Oct 6 02:32:53 kernel: block drbd1: Connection closed > Oct 6 02:32:53 kernel: block drbd1: conn( Disconnecting -> StandAlone ) > > > So it appears to not want to establish a link based on protocol version. > Fine. Let's roll back to Protocol 90? Hmm. Possible? You (well, your DRBD setup) managed to get into a situation where the UUID comparison during handshake gives "ambiguous" results, apparently. Which is a bug in older DRBD, and has been fixed with additional heuristics in more recent drbd. That's why you need the newer DRBD on both sides to automatically resolve this. Depending on additional context knowledge, you may be able to fix this by hand, and sync up anyways even with your current DRBD versions. If you are ok with a full resync, recreating the meta data on the to-be-sync-target side would be the easiest way out. If you need a more elegant solution, you know where to find us... -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed