<div dir="ltr">Hi,<div><br></div><div>I've encountered a problem with DRBD 8.4.2 when I try to enable --allow-two-primaries on the fly and immediately promoting the secondary to primary afterwards.</div><div>The problem doesn't occur always, and it seems like it is more likely to happen when there is more load on the device.</div>
<div><br></div><div>The exact command sequence is as follows:</div><div><br></div><div>Executed on primary and secondary node simultaneously (but also happens if only executed on secondary):</div><div><br></div><div>drbdsetup net-options ipv4:<loc_ip>:11001 ipv4:<rem_ip>:11001 --protocol C --after-sb-0pri discard-zero-changes --after-sb-1pri consensus --allow-two-primaries=yes --cram-hmac-alg md5 --shared-secret <secret></div>
<div>drbdsetup primary 1</div><div><br></div><div style>BTW, the only options which differs in regard to the previously issued drbdsetup connect command is --allow-two-primaries. The rest (protocol, secret, etc.) are just repeated.</div>
<div><br></div><div style>The outcome is that both nodes end up in the StandAlone state.</div><div style><br></div><div style>Their respective kernel log messages are:</div><div style><br></div><div style>(Old) primary:</div>
<div style><div>Apr 26 11:19:42 primary kernel: [181721.646750] block drbd0: peer( Secondary -> Primary ) </div><div>Apr 26 11:19:42 primary kernel: [181721.669870] block drbd1: peer( Secondary -> Primary ) </div><div>
Apr 26 11:19:42 primary kernel: [181722.057848] d-con resource1: sock was shut down by peer</div><div>Apr 26 11:19:42 primary kernel: [181722.057872] d-con resource1: peer( Primary -> Unknown ) conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown ) </div>
<div>Apr 26 11:19:42 primary kernel: [181722.057881] d-con resource1: short read (expected size 16)</div><div>Apr 26 11:19:42 primary kernel: [181722.057914] block drbd1: new current UUID DEEF411AB544C5D3:041691050BDB6491:FA1A2A8EC7D3D7CE:FA192A8EC7D3D7CF</div>
<div>Apr 26 11:19:42 primary kernel: [181722.057964] d-con resource1: asender terminated</div><div>Apr 26 11:19:42 primary kernel: [181722.057977] d-con resource1: Terminating asender thread</div><div>Apr 26 11:19:42 primary kernel: [181722.058485] d-con resource1: Connection closed</div>
<div>Apr 26 11:19:42 primary kernel: [181722.067019] d-con resource1: conn( BrokenPipe -> Unconnected ) </div><div>Apr 26 11:19:42 primary kernel: [181722.067027] d-con resource1: receiver terminated</div><div>Apr 26 11:19:42 primary kernel: [181722.067032] d-con resource1: Restarting receiver thread</div>
<div>Apr 26 11:19:42 primary kernel: [181722.067036] d-con resource1: receiver (re)started</div><div>Apr 26 11:19:42 primary kernel: [181722.067045] d-con resource1: conn( Unconnected -> WFConnection ) </div><div>Apr 26 11:19:43 primary kernel: [181722.558370] d-con resource1: Handshake successful: Agreed network protocol version 101</div>
<div>Apr 26 11:19:43 primary kernel: [181722.558702] d-con resource1: Peer authenticated using 16 bytes HMAC</div><div>Apr 26 11:19:43 primary kernel: [181722.558747] d-con resource1: conn( WFConnection -> WFReportParams ) </div>
<div>Apr 26 11:19:43 primary kernel: [181722.558754] d-con resource1: Starting asender thread (from drbd_r_resource [2039])</div><div>Apr 26 11:19:43 primary kernel: [181722.560436] block drbd1: drbd_sync_handshake:</div>
<div>Apr 26 11:19:43 primary kernel: [181722.560445] block drbd1: self DEEF411AB544C5D3:041691050BDB6491:FA1A2A8EC7D3D7CE:FA192A8EC7D3D7CF bits:3072 flags:0</div><div>Apr 26 11:19:43 primary kernel: [181722.560454] block drbd1: peer 9CE29D13EEB7B4B3:041691050BDB6491:FA1A2A8EC7D3D7CF:FA192A8EC7D3D7CF bits:0 flags:0</div>
<div>Apr 26 11:19:43 primary kernel: [181722.560466] block drbd1: uuid_compare()=100 by rule 90</div><div>Apr 26 11:19:43 primary kernel: [181722.560474] block drbd1: helper command: /bin/true initial-split-brain minor-1</div>
<div>Apr 26 11:19:43 primary kernel: [181722.565127] d-con resource1: conn( WFReportParams -> NetworkFailure ) </div><div>Apr 26 11:19:43 primary kernel: [181722.565134] d-con resource1: asender terminated</div><div>Apr 26 11:19:43 primary kernel: [181722.565138] d-con resource1: Terminating asender thread</div>
<div>Apr 26 11:19:43 primary kernel: [181722.570459] block drbd1: helper command: /bin/true initial-split-brain minor-1 exit code 0 (0x0)</div><div>Apr 26 11:19:43 primary kernel: [181722.570488] block drbd1: helper command: /bin/true split-brain minor-1</div>
<div>Apr 26 11:19:43 primary kernel: [181722.583047] block drbd1: helper command: /bin/true split-brain minor-1 exit code 0 (0x0)</div><div>Apr 26 11:19:43 primary kernel: [181722.583073] d-con resource1: conn( NetworkFailure -> Disconnecting ) </div>
<div>Apr 26 11:19:43 primary kernel: [181722.583143] d-con resource1: Connection closed</div><div>Apr 26 11:19:43 primary kernel: [181722.586237] d-con resource1: conn( Disconnecting -> StandAlone ) </div><div>Apr 26 11:19:43 primary kernel: [181722.586245] d-con resource1: receiver terminated</div>
<div>Apr 26 11:19:43 primary kernel: [181722.586249] d-con resource1: Terminating receiver thread</div><div>Apr 26 11:19:46 primary kernel: [181726.054479] br974: port 2(vif126.0) entering forwarding state</div><div>Apr 26 11:19:46 primary kernel: [181726.058824] br974: port 2(vif126.0) entering disabled state</div>
<div><br></div><div style>(Old) secondary:</div><div style><div>Apr 26 11:19:42 secondary kernel: [1809212.315376] block drbd0: role( Secondary -> Primary ) </div><div>Apr 26 11:19:42 secondary kernel: [1809212.338517] block drbd1: role( Secondary -> Primary ) </div>
<div>Apr 26 11:19:42 secondary kernel: [1809212.726247] d-con resource1: peer( Primary -> Unknown ) conn( Connected -> ProtocolError ) pdsk( UpToDate -> DUnknown ) </div><div>Apr 26 11:19:42 secondary kernel: [1809212.726278] block drbd1: new current UUID 9CE29D13EEB7B4B3:041691050BDB6491:FA1A2A8EC7D3D7CF:FA192A8EC7D3D7CF</div>
<div>Apr 26 11:19:42 secondary kernel: [1809212.726310] d-con resource1: asender terminated</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726340] d-con resource1: Terminating asender thread</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726719] d-con resource1: Connection closed</div>
<div>Apr 26 11:19:42 secondary kernel: [1809212.726749] d-con resource1: conn( ProtocolError -> Unconnected ) </div><div>Apr 26 11:19:42 secondary kernel: [1809212.726755] d-con resource1: receiver terminated</div><div>
Apr 26 11:19:42 secondary kernel: [1809212.726759] d-con resource1: Restarting receiver thread</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726763] d-con resource1: receiver (re)started</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726771] d-con resource1: conn( Unconnected -> WFConnection ) </div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.226864] d-con resource1: Handshake successful: Agreed network protocol version 101</div><div>Apr 26 11:19:43 secondary kernel: [1809213.227199] d-con resource1: Peer authenticated using 16 bytes HMAC</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.227238] d-con resource1: conn( WFConnection -> WFReportParams ) </div><div>Apr 26 11:19:43 secondary kernel: [1809213.227245] d-con resource1: Starting asender thread (from drbd_r_resource [20607])</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.231289] block drbd1: drbd_sync_handshake:</div><div>Apr 26 11:19:43 secondary kernel: [1809213.231297] block drbd1: self 9CE29D13EEB7B4B3:041691050BDB6491:FA1A2A8EC7D3D7CF:FA192A8EC7D3D7CF bits:0 flags:0</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.231306] block drbd1: peer DEEF411AB544C5D3:041691050BDB6491:FA1A2A8EC7D3D7CE:FA192A8EC7D3D7CF bits:3072 flags:0</div><div>Apr 26 11:19:43 secondary kernel: [1809213.231315] block drbd1: uuid_compare()=100 by rule 90</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.231322] block drbd1: helper command: /bin/true initial-split-brain minor-1</div><div>Apr 26 11:19:43 secondary kernel: [1809213.232460] block drbd1: helper command: /bin/true initial-split-brain minor-1 exit code 0 (0x0)</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.232494] block drbd1: helper command: /bin/true split-brain minor-1</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233512] block drbd1: helper command: /bin/true split-brain minor-1 exit code 0 (0x0)</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.233539] d-con resource1: conn( WFReportParams -> Disconnecting ) </div><div>Apr 26 11:19:43 secondary kernel: [1809213.233574] d-con resource1: asender terminated</div><div>
Apr 26 11:19:43 secondary kernel: [1809213.233579] d-con resource1: Terminating asender thread</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233631] d-con resource1: Connection closed</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233662] d-con resource1: conn( Disconnecting -> StandAlone ) </div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.233667] d-con resource1: receiver terminated</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233672] d-con resource1: Terminating receiver thread</div><div><br></div><div>
<br></div><div style>What am I doing wrong? Is there a requirement to wait for a sync/propagation of properties/random amount of time before promoting the secondary to primary? Is this a bug?</div><div style><br></div><div style>
Thanks,</div><div style>Thomas</div></div></div><div><div><br></div>-- <br><div dir="ltr"><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Thomas Thrainer |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(51,105,232);padding-top:2px;margin-top:2px"> Software Engineer |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,153,57);padding-top:2px;margin-top:2px"> <a href="mailto:thomasth@google.com" target="_blank">thomasth@google.com</a> |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"> </span><br>
<div><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"><br></span></div><div>
<span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"><div style="font-size:13px;line-height:normal;color:rgb(34,34,34);font-family:arial,sans-serif">
<span style="font-family:Arial,Verdana,sans-serif">Google Germany GmbH</span></div><div style="font-size:13px;line-height:normal;color:rgb(34,34,34);font-family:arial,sans-serif"><div style="font-family:Arial,Verdana,sans-serif">
Dienerstr. 12</div><div style="font-family:Arial,Verdana,sans-serif">80331 München<br><br>Registergericht und -nummer: Hamburg, HRB 86891<br>Sitz der Gesellschaft: <font face="arial, sans-serif">Hamburg</font><br>Geschäftsführer: Graham Law, Katherine Stephens</div>
</div></span></div></div>
</div></div>