<font face="trebuchet ms,sans-serif">Hi, <br clear="all"></font><div><font face="'trebuchet ms', sans-serif"><br></font></div><div><font face="'trebuchet ms', sans-serif">I have a small issue with one of my DRBD setup. When my backup is running (-see lower for setup and backup details), i`m getting those errors:</font></div>
<div><font face="'trebuchet ms', sans-serif"><div><br></div><div>Aug 27 10:24:18 pig-two -- MARK --</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: peer( Secondary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown )</div>
</font></div><div><font face="'trebuchet ms', sans-serif"><div>Aug 27 10:27:26 pig-two kernel: drbd0: asender terminated</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: Terminating asender thread</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: sock was reset by peer</div>
<div>Aug 27 10:27:26 pig-two kernel: drbd0: _drbd_send_page: size=4096 len=3064 sent=-32</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: Creating new current UUID</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: Writing meta data super block now.</div>
<div>Aug 27 10:27:26 pig-two kernel: drbd0: tl_clear()</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: Connection closed</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: conn( NetworkFailure -> Unconnected )</div><div>
Aug 27 10:27:26 pig-two kernel: drbd0: receiver terminated</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: receiver (re)started</div><div>Aug 27 10:27:26 pig-two kernel: drbd0: conn( Unconnected -> WFConnection )</div>
<div>Aug 27 10:27:27 pig-two kernel: drbd0: Handshake successful: Agreed network protocol version 88</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: Peer authenticated using 20 bytes of 'sha1' HMAC</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: conn( WFConnection -> WFReportParams )</div>
<div>Aug 27 10:27:27 pig-two kernel: drbd0: Starting asender thread (from drbd0_receiver [3066])</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: data-integrity-alg: md5</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> UpToDate )</div>
<div>Aug 27 10:27:27 pig-two kernel: drbd0: Writing meta data super block now.</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: conn( WFBitMapS -> SyncSource ) pdsk( UpToDate -> Inconsistent )</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: Began resync as SyncSource (will sync 2160 KB [540 bits set]).</div>
<div>Aug 27 10:27:27 pig-two kernel: drbd0: Writing meta data super block now.</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: Resync done (total 1 sec; paused 0 sec; 2160 K/sec)</div><div>Aug 27 10:27:27 pig-two kernel: drbd0: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )</div>
<div>Aug 27 10:27:27 pig-two kernel: drbd0: Writing meta data super block now.</div><div><div>Aug 27 10:44:19 pig-two -- MARK --</div></div><div><br></div><div>and</div><div><br></div><div><div>Aug 27 11:04:19 pig-two -- MARK --</div>
<div>Aug 27 11:20:36 pig-two kernel: drbd0: _drbd_send_page: size=4096 len=4096 sent=-104</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: peer( Secondary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown )</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: Creating new current UUID</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Writing meta data super block now.</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: asender terminated</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: Terminating asender thread</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: sock was shut down by peer</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: tl_clear()</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Connection closed</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: conn( NetworkFailure -> Unconnected )</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: receiver terminated</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: receiver (re)started</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: conn( Unconnected -> WFConnection )</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Handshake successful: Agreed network protocol version 88</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Peer authenticated using 20 bytes of 'sha1' HMAC</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: conn( WFConnection -> WFReportParams )</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Starting asender thread (from drbd0_receiver [3066])</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: data-integrity-alg: md5</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> UpToDate )</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Writing meta data super block now.</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: conn( WFBitMapS -> SyncSource ) pdsk( UpToDate -> Inconsistent )</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Began resync as SyncSource (will sync 5788 KB [1447 bits set]).</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: Writing meta data super block now.</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: Resync done (total 1 sec; paused 0 sec; 5788 K/sec)</div><div>Aug 27 11:20:37 pig-two kernel: drbd0: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )</div>
<div>Aug 27 11:20:37 pig-two kernel: drbd0: Writing meta data super block now.</div><div>Aug 27 11:44:19 pig-two -- MARK --</div></div><div><br></div><div>Analysis: it look like the network is failing, then everything - under a second - re-connect, resync and work again. There are no impact on the 'production'. Anyone got some kind of idea, why ? Is it an error in my setup/design (see lower).</div>
<div><br></div><div><br></div><div><b>Some background on the setup: </b></div><div><div><br></div><div>It's an old version. Very old in fact - roadmap to upgrade has been drafted and submitted to client - I`m just wondering about the specific issue here... I want to be sure it's not an infrastructure design problem.</div>
<div>pig-two:~# cat /proc/drbd</div><div>version: 8.2.6 (api:88/proto:86-88)</div><div>GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by root@pig-two, 2008-08-19 15:02:28</div><div> 0: cs:Connected st:Primary/Secondary ds:UpToDate/UpToDate C r---</div>
<div> ns:650469968 nr:0 dw:648856776 dr:16725553 al:5463958 bm:22571 lo:0 pe:0 ua:0 ap:0 oos:0</div></div><div><br></div><div>We are speaking, of:</div><div> - 4x SAS 15k drives in a hardware raid-5 array (DELL Perc5)... presented to the OS as /dev/sda. </div>
<div> - /dev/sda is the back-end device for DRBD... presented to the OS as /dev/drbd0</div><div> - /dev/drbd0 is a lone "physical volume" in a volume group (called SAN) from which Logical Volume are created. Those are NOT locally mounted.</div>
<div> - those logical volumes are exported with vblade (AoE protocol, layer 2) to some other physical system (Xen dom0) where they are used as backend device (/dev/etherd/e0.1) for root volume of virtual system</div><div>
<br></div><div>Everything work fine, but when I do backup, I follow this process: </div><div> - mount a CIFS exported share over the network</div><div> - take a LV snapshot, mount it, and copy everything to the CIFS share. </div>
<div> - unmount snapshot, delete it... do for all LV.</div><div> - unmount network share</div><div><br></div><div>The backup are consistent and valid (tested)... What have I missed ? Should I move away from AoE to a Linux based iSCSI ? </div>
<div><br></div><div>P.</div><div><br></div><div>--</div></font></div><div><font face="'trebuchet ms', sans-serif">Pascal Charest - <i>Cutting-edge technology consultant</i><br><a href="https://www.labsphoenix.com" target="_blank">https://www.labsphoenix.com</a> </font><br>
</div>