<!DOCTYPE html><html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body><div data-html-editor-font-wrapper="true" style="font-family: arial, sans-serif; font-size: 13px;">Thanks a lot for your answer.<br><br>I will check if I can update the drbd module on debian.<br><br>For the logs saying "Connected -> TearDown", it appears Ganeti did it.<br>It's when someone is moving a virtual machine from one node to another. It explains the tapX cards going up/down.<br><br>For the pingAck I need to check ;)<br><br>Thanks ;)<br><br><signature>Nicolas</signature><br><br>30 août 2018 10:46 "Lars Ellenberg" <<a target="_blank" tabindex="-1" href="mailto:lars.ellenberg@linbit.com?to=%22Lars%20Ellenberg%22%20<lars.ellenberg@linbit.com>">lars.ellenberg@linbit.com</a>> a écrit:<br> <blockquote> <pre>On Wed, Aug 29, 2018 at 11:33:26AM +0000, Nicolas wrote: </pre> <blockquote>Hello<br><br>Sorry for the misunderstanding of utils version.<br><br>I'm using the kernel : 4.9.88-1+deb9u1 (4.9.0-6-amd64 debian).<br>And the module version v8.4.7.<br>srcversion: 0904DF2CCF7283ACE07D07A</blockquote> <br>Not that I think it has anything to do with this particular issue,<br>but I'd suggest you upgrade to 8.4.11 anyways.<br><br><br> <blockquote>For example when a node says:<br><br>[Tue Aug 28 14:32:38 2018] drbd resource10: peer( Primary -> Unknown ) conn( Connected -> Disconnecting ) pdsk( UpToDate -> DUnknown )<br>[Tue Aug 28 14:32:38 2018] drbd resource10: ack_receiver terminated<br>[Tue Aug 28 14:32:38 2018] drbd resource10: Terminating drbd_a_resource<br>[Tue Aug 28 14:32:38 2018] drbd resource10: Connection closed<br>[Tue Aug 28 14:32:38 2018] drbd resource10: conn( Disconnecting -> StandAlone )<br>[Tue Aug 28 14:32:38 2018] drbd resource10: receiver terminated<br>[Tue Aug 28 14:32:38 2018] drbd resource10: Terminating drbd_r_resource<br>[Tue Aug 28 14:32:38 2018] block drbd10: disk( UpToDate -> Failed )<br>[Tue Aug 28 14:32:38 2018] block drbd10: 0 KB (0 bits) marked out-of-sync by on disk bit-map.<br>[Tue Aug 28 14:32:38 2018] block drbd10: disk( Failed -> Diskless )<br>[Tue Aug 28 14:32:38 2018] drbd resource10: Terminating drbd_w_resource<br>[Tue Aug 28 14:32:40 2018] drbd resource10: Starting worker thread (from drbdsetup-84 [10222])</blockquote> <br>Okay. So this is "someone or something" doing a "drbdadm down ; drbdadm up"<br> <blockquote>The second says:<br><br>[Tue Aug 28 14:35:33 2018] br0: port 8(tap6) entered disabled state<br>[Tue Aug 28 14:35:33 2018] device tap6 left promiscuous mode</blockquote> <br>Uhm, time stamps do not match the excerpt above.<br><br><br> <blockquote>[Tue Aug 28 14:35:33 2018] br0: port 8(tap6) entered disabled state<br>[Tue Aug 28 14:35:37 2018] drbd resource10: peer( Secondary -> Unknown ) conn( Connected -> TearDown ) pdsk( UpToDate -> DUnknown )<br>[Tue Aug 28 14:35:37 2018] drbd resource10: ack_receiver terminated<br>[Tue Aug 28 14:35:37 2018] drbd resource10: Terminating drbd_a_resource<br>[Tue Aug 28 14:35:37 2018] block drbd10: new current UUID 629F1036CD6CA2AF:0748EE11C429D3B5:FDAEFCD2E8D9890B:FDADFCD2E8D9890B<br>[Tue Aug 28 14:35:37 2018] drbd resource10: Connection closed<br>[Tue Aug 28 14:35:37 2018] drbd resource10: conn( TearDown -> Unconnected )<br>[Tue Aug 28 14:35:37 2018] drbd resource10: receiver terminated<br>[Tue Aug 28 14:35:37 2018] drbd resource10: Restarting receiver thread<br>[Tue Aug 28 14:35:37 2018] drbd resource10: receiver (re)started<br>[Tue Aug 28 14:35:37 2018] drbd resource10: conn( Unconnected -> WFConnection )</blockquote> <br>This is "peer node disconnected for some reason".<br><br><br> <blockquote>[Tue Aug 28 14:35:38 2018] block drbd10: role( Primary -> Secondary )<br>[Tue Aug 28 14:35:38 2018] block drbd10: 0 KB (0 bits) marked out-of-sync by on disk bit-map.<br>[Tue Aug 28 14:35:38 2018] drbd resource10: conn( WFConnection -> Disconnecting )<br>[Tue Aug 28 14:35:38 2018] drbd resource10: Discarding network configuration.<br>[Tue Aug 28 14:35:38 2018] drbd resource10: Connection closed<br>[Tue Aug 28 14:35:38 2018] drbd resource10: conn( Disconnecting -> StandAlone )<br>[Tue Aug 28 14:35:38 2018] drbd resource10: receiver terminated<br>[Tue Aug 28 14:35:38 2018] drbd resource10: Terminating drbd_r_resource<br>[Tue Aug 28 14:35:38 2018] block drbd10: disk( UpToDate -> Failed )<br>[Tue Aug 28 14:35:38 2018] block drbd10: 0 KB (0 bits) marked out-of-sync by on disk bit-map.<br>[Tue Aug 28 14:35:38 2018] block drbd10: disk( Failed -> Diskless )<br>[Tue Aug 28 14:35:38 2018] drbd resource10: Terminating drbd_w_resource</blockquote> <br>And again, this is a "drbdadm down ; drbdadm up"<br><br><br><br> <blockquote>And it seems for this example the second node was the origin of this.<br>This night I got another error, saying network failure, but I'm sure there was no network issue:<br><br>First node:<br><br>[Wed Aug 29 01:39:48 2018] drbd resource0: meta connection shut down by peer.<br>[Wed Aug 29 01:39:48 2018] drbd resource0: peer( Primary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown )</blockquote> <br>...<br><br>peer node shut down the connection,<br>and as a result this node goes through a state called NetworkFailure,<br>then all the motions,<br>then reconnects,<br>and syncs up.<br> <blockquote>Second node:<br><br>[Wed Aug 29 01:42:48 2018] drbd resource0: PingAck did not arrive in time.</blockquote> <br>Again, time stamps do not match up.<br>But there is your reason for this incident: "PingAck did not arrive in time".<br><br>Find out why, or simply increase the ping ack timeout.<br><br><br> <blockquote>-------- Message transféré -------<br>De: "Lars Ellenberg" <<a target="_blank" rel="noopener noreferrer" href="mailto:lars.ellenberg@linbit.com">lars.ellenberg@linbit.com</a> (mailto:lars.ellenberg@linbit.com?to=%22Lars%20Ellenberg%22%20<<a target="_blank" rel="noopener noreferrer" href="mailto:lars.ellenberg@linbit.com">lars.ellenberg@linbit.com</a>>)><br>À: <a target="_blank" rel="noopener noreferrer" href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a> (mailto:<a target="_blank" rel="noopener noreferrer" href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a>)<br>Envoyé: 29 août 2018 12:09<br>Objet: Re: [DRBD-user] drbd issue?<br><br>On Tue, Aug 28, 2018 at 02:43:47PM +0000, Nicolas wrote: Hi<br><br>I'm using some servers on debian with ganeti and drbd.<br><br>Since I've upgraded them to debian 9, and drbd 8.9.10-2 (from debian repo).<br>"drbd 8.9.10" is the *utils* version<br>(drbdadm, drbdsetup, drbdmeta, various scripts ...)<br><br>drbd utils version is meanwhile at 9.5.0, btw. And no, that has not<br>much to do with what DRBD kernel module driver version you are using,<br>since we ship the "unified utils" for both "drbd 8" and "drbd 9",<br>which started years ago already, the utils version is decoupled from<br>the module versions.<br><br>What kernel version,<br>and what DRBD module version?<br><br>Maybe you want to make sure you use the latest 8.4 version (8.4.11<br>currently), and not whatever "shipts with the debian kernel"?<br>I got a lot of issue with my drbd resources, I got randomly on my dmesg some resources disconnected:<br><br>today for example:<br><br>[Tue Aug 28 14:32:38 2018] drbd resource10: peer( Primary -> Unknown ) conn( Connected -> Disconnecting ) pdsk( UpToDate -> DUnknown )<br>Well, what does the other node say?<br>Hit some timeouts?<br>Some strangeness with the new NIC drivers?<br>A bug in the "shipped with the debian kernel" DRBD version?</blockquote> <br>--<br>: Lars Ellenberg<br>: LINBIT | Keeping the Digital World Running<br>: DRBD -- Heartbeat -- Corosync -- Pacemaker<br><br>DRBD® and LINBIT® are registered trademarks of LINBIT<br>__<br>please don't Cc me, but send to list -- I'm subscribed<br>_______________________________________________<br>drbd-user mailing list<br><a target="_blank" rel="noopener noreferrer" href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br><a target="_blank" rel="noopener noreferrer" href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a> </blockquote> </div></body></html>