<div dir="ltr">Hi Luca,<div><br></div><div>(CC'd drbd-user, I guess that might be helpful for others as well)</div><div><br></div><div style>We're not using drbdadm but drbdsetup directly.</div><div style><br></div>
<div style>I tried `drbdsetup net-options ipv4:<local_ip>:11001 ipv4:<remote_ip>:11001 --protocol C --allow-two-</div><div>primaries=yes` (i.e. I stripped the repeated options), but the result is still the same.</div>
<div style>Note however, that the problem occurs only every now and then, and primarily when there is load on the disk(s).</div><div style><br></div><div style>BTW, I actually do set two disks to dual-primary mode at the same time (using different connections/resources tough), and one disk normally works while the other fails (is't not deterministic which of disk fails).</div>
<div style><br></div><div style>Cheers,</div><div style>Thomas</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Apr 26, 2013 at 3:58 PM, Luca Fornasari <span dir="ltr"><<a href="mailto:luca.fornasari@gmail.com" target="_blank">luca.fornasari@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Thomas,<div><br></div><div>Just execute the following on one node only:</div><div><br></div><div>drbdadm net-options --protocol=C --allow-two-primaries r0</div>
<div><br></div><div>
I guess that the command you are issuing just try to restart an already running resource.</div><div><br></div><div>Cheers,</div><div>Luca</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">
On Fri, Apr 26, 2013 at 2:27 PM, Thomas Thrainer <span dir="ltr"><<a href="mailto:thomasth@google.com" target="_blank">thomasth@google.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5">
<div dir="ltr">Hi,<div><br></div><div>I've encountered a problem with DRBD 8.4.2 when I try to enable --allow-two-primaries on the fly and immediately promoting the secondary to primary afterwards.</div><div>The problem doesn't occur always, and it seems like it is more likely to happen when there is more load on the device.</div>
<div><br></div><div>The exact command sequence is as follows:</div><div><br></div><div>Executed on primary and secondary node simultaneously (but also happens if only executed on secondary):</div><div><br></div><div>drbdsetup net-options ipv4:<loc_ip>:11001 ipv4:<rem_ip>:11001 --protocol C --after-sb-0pri discard-zero-changes --after-sb-1pri consensus --allow-two-primaries=yes --cram-hmac-alg md5 --shared-secret <secret></div>
<div>drbdsetup primary 1</div><div><br></div><div>BTW, the only options which differs in regard to the previously issued drbdsetup connect command is --allow-two-primaries. The rest (protocol, secret, etc.) are just repeated.</div>
<div><br></div><div>The outcome is that both nodes end up in the StandAlone state.</div><div><br></div><div>Their respective kernel log messages are:</div><div><br></div><div>(Old) primary:</div>
<div><div>Apr 26 11:19:42 primary kernel: [181721.646750] block drbd0: peer( Secondary -> Primary ) </div><div>Apr 26 11:19:42 primary kernel: [181721.669870] block drbd1: peer( Secondary -> Primary ) </div><div>
Apr 26 11:19:42 primary kernel: [181722.057848] d-con resource1: sock was shut down by peer</div><div>Apr 26 11:19:42 primary kernel: [181722.057872] d-con resource1: peer( Primary -> Unknown ) conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown ) </div>
<div>Apr 26 11:19:42 primary kernel: [181722.057881] d-con resource1: short read (expected size 16)</div><div>Apr 26 11:19:42 primary kernel: [181722.057914] block drbd1: new current UUID DEEF411AB544C5D3:041691050BDB6491:FA1A2A8EC7D3D7CE:FA192A8EC7D3D7CF</div>
<div>Apr 26 11:19:42 primary kernel: [181722.057964] d-con resource1: asender terminated</div><div>Apr 26 11:19:42 primary kernel: [181722.057977] d-con resource1: Terminating asender thread</div><div>Apr 26 11:19:42 primary kernel: [181722.058485] d-con resource1: Connection closed</div>
<div>Apr 26 11:19:42 primary kernel: [181722.067019] d-con resource1: conn( BrokenPipe -> Unconnected ) </div><div>Apr 26 11:19:42 primary kernel: [181722.067027] d-con resource1: receiver terminated</div><div>Apr 26 11:19:42 primary kernel: [181722.067032] d-con resource1: Restarting receiver thread</div>
<div>Apr 26 11:19:42 primary kernel: [181722.067036] d-con resource1: receiver (re)started</div><div>Apr 26 11:19:42 primary kernel: [181722.067045] d-con resource1: conn( Unconnected -> WFConnection ) </div><div>Apr 26 11:19:43 primary kernel: [181722.558370] d-con resource1: Handshake successful: Agreed network protocol version 101</div>
<div>Apr 26 11:19:43 primary kernel: [181722.558702] d-con resource1: Peer authenticated using 16 bytes HMAC</div><div>Apr 26 11:19:43 primary kernel: [181722.558747] d-con resource1: conn( WFConnection -> WFReportParams ) </div>
<div>Apr 26 11:19:43 primary kernel: [181722.558754] d-con resource1: Starting asender thread (from drbd_r_resource [2039])</div><div>Apr 26 11:19:43 primary kernel: [181722.560436] block drbd1: drbd_sync_handshake:</div>
<div>Apr 26 11:19:43 primary kernel: [181722.560445] block drbd1: self DEEF411AB544C5D3:041691050BDB6491:FA1A2A8EC7D3D7CE:FA192A8EC7D3D7CF bits:3072 flags:0</div><div>Apr 26 11:19:43 primary kernel: [181722.560454] block drbd1: peer 9CE29D13EEB7B4B3:041691050BDB6491:FA1A2A8EC7D3D7CF:FA192A8EC7D3D7CF bits:0 flags:0</div>
<div>Apr 26 11:19:43 primary kernel: [181722.560466] block drbd1: uuid_compare()=100 by rule 90</div><div>Apr 26 11:19:43 primary kernel: [181722.560474] block drbd1: helper command: /bin/true initial-split-brain minor-1</div>
<div>Apr 26 11:19:43 primary kernel: [181722.565127] d-con resource1: conn( WFReportParams -> NetworkFailure ) </div><div>Apr 26 11:19:43 primary kernel: [181722.565134] d-con resource1: asender terminated</div><div>Apr 26 11:19:43 primary kernel: [181722.565138] d-con resource1: Terminating asender thread</div>
<div>Apr 26 11:19:43 primary kernel: [181722.570459] block drbd1: helper command: /bin/true initial-split-brain minor-1 exit code 0 (0x0)</div><div>Apr 26 11:19:43 primary kernel: [181722.570488] block drbd1: helper command: /bin/true split-brain minor-1</div>
<div>Apr 26 11:19:43 primary kernel: [181722.583047] block drbd1: helper command: /bin/true split-brain minor-1 exit code 0 (0x0)</div><div>Apr 26 11:19:43 primary kernel: [181722.583073] d-con resource1: conn( NetworkFailure -> Disconnecting ) </div>
<div>Apr 26 11:19:43 primary kernel: [181722.583143] d-con resource1: Connection closed</div><div>Apr 26 11:19:43 primary kernel: [181722.586237] d-con resource1: conn( Disconnecting -> StandAlone ) </div><div>Apr 26 11:19:43 primary kernel: [181722.586245] d-con resource1: receiver terminated</div>
<div>Apr 26 11:19:43 primary kernel: [181722.586249] d-con resource1: Terminating receiver thread</div><div>Apr 26 11:19:46 primary kernel: [181726.054479] br974: port 2(vif126.0) entering forwarding state</div><div>Apr 26 11:19:46 primary kernel: [181726.058824] br974: port 2(vif126.0) entering disabled state</div>
<div><br></div><div>(Old) secondary:</div><div><div>Apr 26 11:19:42 secondary kernel: [1809212.315376] block drbd0: role( Secondary -> Primary ) </div><div>Apr 26 11:19:42 secondary kernel: [1809212.338517] block drbd1: role( Secondary -> Primary ) </div>
<div>Apr 26 11:19:42 secondary kernel: [1809212.726247] d-con resource1: peer( Primary -> Unknown ) conn( Connected -> ProtocolError ) pdsk( UpToDate -> DUnknown ) </div><div>Apr 26 11:19:42 secondary kernel: [1809212.726278] block drbd1: new current UUID 9CE29D13EEB7B4B3:041691050BDB6491:FA1A2A8EC7D3D7CF:FA192A8EC7D3D7CF</div>
<div>Apr 26 11:19:42 secondary kernel: [1809212.726310] d-con resource1: asender terminated</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726340] d-con resource1: Terminating asender thread</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726719] d-con resource1: Connection closed</div>
<div>Apr 26 11:19:42 secondary kernel: [1809212.726749] d-con resource1: conn( ProtocolError -> Unconnected ) </div><div>Apr 26 11:19:42 secondary kernel: [1809212.726755] d-con resource1: receiver terminated</div><div>
Apr 26 11:19:42 secondary kernel: [1809212.726759] d-con resource1: Restarting receiver thread</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726763] d-con resource1: receiver (re)started</div><div>Apr 26 11:19:42 secondary kernel: [1809212.726771] d-con resource1: conn( Unconnected -> WFConnection ) </div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.226864] d-con resource1: Handshake successful: Agreed network protocol version 101</div><div>Apr 26 11:19:43 secondary kernel: [1809213.227199] d-con resource1: Peer authenticated using 16 bytes HMAC</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.227238] d-con resource1: conn( WFConnection -> WFReportParams ) </div><div>Apr 26 11:19:43 secondary kernel: [1809213.227245] d-con resource1: Starting asender thread (from drbd_r_resource [20607])</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.231289] block drbd1: drbd_sync_handshake:</div><div>Apr 26 11:19:43 secondary kernel: [1809213.231297] block drbd1: self 9CE29D13EEB7B4B3:041691050BDB6491:FA1A2A8EC7D3D7CF:FA192A8EC7D3D7CF bits:0 flags:0</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.231306] block drbd1: peer DEEF411AB544C5D3:041691050BDB6491:FA1A2A8EC7D3D7CE:FA192A8EC7D3D7CF bits:3072 flags:0</div><div>Apr 26 11:19:43 secondary kernel: [1809213.231315] block drbd1: uuid_compare()=100 by rule 90</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.231322] block drbd1: helper command: /bin/true initial-split-brain minor-1</div><div>Apr 26 11:19:43 secondary kernel: [1809213.232460] block drbd1: helper command: /bin/true initial-split-brain minor-1 exit code 0 (0x0)</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.232494] block drbd1: helper command: /bin/true split-brain minor-1</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233512] block drbd1: helper command: /bin/true split-brain minor-1 exit code 0 (0x0)</div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.233539] d-con resource1: conn( WFReportParams -> Disconnecting ) </div><div>Apr 26 11:19:43 secondary kernel: [1809213.233574] d-con resource1: asender terminated</div><div>
Apr 26 11:19:43 secondary kernel: [1809213.233579] d-con resource1: Terminating asender thread</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233631] d-con resource1: Connection closed</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233662] d-con resource1: conn( Disconnecting -> StandAlone ) </div>
<div>Apr 26 11:19:43 secondary kernel: [1809213.233667] d-con resource1: receiver terminated</div><div>Apr 26 11:19:43 secondary kernel: [1809213.233672] d-con resource1: Terminating receiver thread</div><div><br></div><div>
<br></div><div>What am I doing wrong? Is there a requirement to wait for a sync/propagation of properties/random amount of time before promoting the secondary to primary? Is this a bug?</div><div><br></div><div>
Thanks,</div><div>Thomas</div></div></div><span><font color="#888888"><div><div><br></div>-- <br><div dir="ltr"><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Thomas Thrainer |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(51,105,232);padding-top:2px;margin-top:2px"> Software Engineer |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,153,57);padding-top:2px;margin-top:2px"> <a href="mailto:thomasth@google.com" target="_blank">thomasth@google.com</a> |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"> </span><br>
<div><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"><br></span></div><div>
<span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"><div style="font-size:13px;line-height:normal;color:rgb(34,34,34);font-family:arial,sans-serif">
<span style="font-family:Arial,Verdana,sans-serif">Google Germany GmbH</span></div><div style="font-size:13px;line-height:normal;color:rgb(34,34,34);font-family:arial,sans-serif"><div style="font-family:Arial,Verdana,sans-serif">
Dienerstr. 12</div><div style="font-family:Arial,Verdana,sans-serif">80331 München<br><br>Registergericht und -nummer: Hamburg, HRB 86891<br>Sitz der Gesellschaft: <font face="arial, sans-serif">Hamburg</font><br>Geschäftsführer: Graham Law, Katherine Stephens</div>
</div></span></div></div>
</div></font></span></div>
<br></div></div>_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Thomas Thrainer |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(51,105,232);padding-top:2px;margin-top:2px"> Software Engineer |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,153,57);padding-top:2px;margin-top:2px"> <a href="mailto:thomasth@google.com" target="_blank">thomasth@google.com</a> |</span><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"> </span><br>
<div><span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"><br></span></div><div>
<span style="color:rgb(85,85,85);font-family:sans-serif;font-size:small;line-height:1.5em;border-width:2px 0px 0px;border-style:solid;border-color:rgb(238,178,17);padding-top:2px;margin-top:2px"><div style="font-size:13px;line-height:normal;color:rgb(34,34,34);font-family:arial,sans-serif">
<span style="font-family:Arial,Verdana,sans-serif">Google Germany GmbH</span></div><div style="font-size:13px;line-height:normal;color:rgb(34,34,34);font-family:arial,sans-serif"><div style="font-family:Arial,Verdana,sans-serif">
Dienerstr. 12</div><div style="font-family:Arial,Verdana,sans-serif">80331 München<br><br>Registergericht und -nummer: Hamburg, HRB 86891<br>Sitz der Gesellschaft: <font face="arial, sans-serif">Hamburg</font><br>Geschäftsführer: Graham Law, Katherine Stephens</div>
</div></span></div></div>
</div>