[DRBD-user] possible split brain, neither node will promote to primary

[Lonni J Friedman]

On Thu, Oct 17, 2013 at 12:23 PM, Lonni J Friedman <netllama at gmail.com> wrote:
> Thanks for your prompt reply.  Comments/questions below....
>
> On Thu, Oct 17, 2013 at 12:18 PM, Digimer <lists at alteeve.ca> wrote:
>> On 17/10/13 15:08, Lonni J Friedman wrote:
>>> [ 1727.959118] block drbd0:   state = { cs:StandAlone
>>> ro:Secondary/Unknown ds:Inconsistent/DUnknown r----- }
>>> [ 1728.069711] block drbd0:  wanted = { cs:StandAlone
>>> ro:Primary/Unknown ds:Inconsistent/DUnknown r----- }
>>
>> This is your problem. I suspect that you invalidated the node that had
>> been UpToDate. This is a pretty dangerous state to be in. If you have
>> critical data on this cluster, I _strongly_ recommend calling LinBit and
>> getting professional guidance. It will be worth the money.
>
> Sadly the damage was done by a co-worker while I was out, and I'm
> stuck trying to fix the mess.  The data on this cluster doesn't change
> all that often, so the odds of anything being lost are quite slim.
>
>>
>> That said;
>>
>> The process will be a matter of deciding which node is actually UpToDate
>> and then forcing it into that role. Once done, connect should work and
>> the Inconsistent node should sync from UpToDate. You'll certainly want
>> to do an FS consistency check after.
>
> OK, but what's the actual command to force a node to be primary?
> There used to be a -f switch to force a node to become primary, but
> that options seems to no longer exist (at least in 8.4.x).


Nevermind, apparently what I needed was the --force option when
specifying the primary node.  Its finally doing the right thing, and
syncing.