<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_default"><div class="gmail_default"><font face="arial, helvetica, sans-serif">Here's some more info.</font></div><div class="gmail_default"><font face="arial, helvetica, sans-serif">Dmesg shows some suspicious looking log message, such as:<br><div class="gmail_default"><br></div></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">1) FIXME drbd_s_vm-117-s[2830] op clear, bitmap locked for 'receive bitmap' by drbd_r_vm-117-s[5038]</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">2) Wrong magic value 0xffff0007 in protocol version 114</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">3) peer request with dagtag 399201392 not found</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">got_peer_ack [drbd] failed</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">4) Rejecting concurrent remote state change 2226202936 because of state change 2923939731</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">Ignoring P_TWOPC_ABORT packet 2226202936.</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">5) drbd_r_vm-117-s[5038] going to 'detect_finished_resyncs()' but bitmap already locked for 'write from resync_finished' by drbd_w_vm-117-s[2812]</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">md_sync_timer expired! Worker calls drbd_md_sync().</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">6) incompatible discard-my-data settings</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">conn( Connecting -> Disconnecting )</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">error receiving P_PROTOCOL, e: -5 l: 7!</font></div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif">Two of the four nodes have DRBD 9.0.15-1 and two have 9.0.16-1. All of them API v 16:</div><div><br><div class="gmail_default"><font face="monospace, monospace" size="1">== mox-a ==</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">version: 9.0.15-1 (api:2/proto:86-114)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">GIT-hash: c46d27900f471ea0f5ba587592439a9ddde1d08b build by root@mox-a, 2018-10-28 03:08:58</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">Transports (api:16): tcp (9.0.15-1)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">== mox-b ==</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">version: 9.0.15-1 (api:2/proto:86-114)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">GIT-hash: c46d27900f471ea0f5ba587592439a9ddde1d08b build by root@mox-b, 2018-10-10 17:50:25</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">Transports (api:16): tcp (9.0.15-1)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">== mox-c ==</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">version: 9.0.16-1 (api:2/proto:86-114)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">GIT-hash: ab9777dfeaf9d619acc9a5201bfcae8103e9529c build by root@mox-c, 2018-10-28 05:45:05</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">Transports (api:16): tcp (9.0.16-1)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1"><br></font></div><div class="gmail_default"><font face="monospace, monospace" size="1">== mox-d ==</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">version: 9.0.16-1 (api:2/proto:86-114)</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">GIT-hash: ab9777dfeaf9d619acc9a5201bfcae8103e9529c build by root@mox-d, 2018-10-29 00:22:23</font></div><div class="gmail_default"><font face="monospace, monospace" size="1">Transports (api:16): tcp (9.0.16-1)</font></div><br class="gmail-Apple-interchange-newline"></div><div style="font-family:arial,helvetica,sans-serif">Running Proxmox (5.2-2) as can you'd guess from host names. DRBD resources being managed by LINSTOR.</div></div><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr">On Thu, 1 Nov 2018 at 17:32, Jarno Elonen <<a href="mailto:elonen@iki.fi">elonen@iki.fi</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif">Okay, today one of these resources got a sudden, severe filesystem corruption on the primary.</div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif">On the other hand, the secondaries (that showed 8ZiB out-of-sync) were still mountable after I disconnected the corrupted primary. No idea how current data the secondaries had, but drbdtop still showed them as connected and 8Zib out-of-sync.</div><div style="font-family:arial,helvetica,sans-serif"><br></div><div><font face="arial, helvetica, sans-serif">This is getting quite worrisome. Is anyone else experiencing this with DRBD 9? Is it something really wrong in my setup, or are there perhaps some known instabilities in DRBD 9.0.15-1?</font></div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif">-Jarno</div><div style="font-family:arial,helvetica,sans-serif"><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 31 Oct 2018 at 20:46, Jarno Elonen <<a href="mailto:elonen@iki.fi" target="_blank">elonen@iki.fi</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><font face="arial, helvetica, sans-serif">I've got several DRBD 9 resource that constantly show <b>UpToDate</b> with 9223372036854774304 bytes (exactly 8ZiB) of OutOfDate data.</font></div><div><span style="font-family:arial,helvetica,sans-serif"><br></span></div><div><span style="font-family:arial,helvetica,sans-serif">Any idea what might cause this and how to fix it?</span><br></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Example:</font><br></div><div><br></div><div><font face="monospace, monospace" size="1"><div># drbdsetup status --verbose --statistics vm-106-disk-1</div><div>vm-106-disk-1 node-id:0 role:Primary suspended:no</div><div> write-ordering:flush</div><div> volume:0 minor:1003 disk:UpToDate quorum:yes</div><div> size:16777688 read:215779 written:22369564 al-writes:89 bm-writes:0 upper-pending:0</div><div> lower-pending:0 al-suspended:no blocked:no</div><div> mox-a node-id:1 connection:Connected role:Secondary congested:no ap-in-flight:0</div><div> rs-in-flight:18446744073709549808</div><div> volume:0 replication:Established peer-disk:UpToDate resync-suspended:no</div><div> received:215116 sent:22368903 out-of-sync:9223372036854774304 pending:0 unacked:0</div><div> mox-c node-id:2 connection:Connected role:Secondary congested:no ap-in-flight:0</div><div> rs-in-flight:18446744073709549808</div><div> volume:0 replication:Established peer-disk:UpToDate resync-suspended:no</div><div> received:1188 sent:19884428 out-of-sync:0 pending:0 unacked:0</div></font></div><div><br></div><div>Version info:</div><div><div><font face="monospace, monospace" size="1">version: 9.0.15-1 (api:2/proto:86-114)</font></div><div><font face="monospace, monospace" size="1">GIT-hash: c46d27900f471ea0f5ba587592439a9ddde1d08b build by root@mox-b, 2018-10-10 17:50:25</font></div><div><font face="monospace, monospace" size="1">Transports (api:16): tcp (9.0.15-1)</font></div></div><div><br></div><div><font face="arial, helvetica, sans-serif">-Jarno</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div></div></div></div></div>
</blockquote></div>
</blockquote></div></div></div></div></div>