<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>I’ll try not to speculate in an effort to save face and not appear [too] foolish.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I stood up drbd 8.3.8 on 2 Gentoo VM’s. Replication was working between the nodes & everything was super quick. During a ~100GB transfer to the primary I stopped the service on the secondary to see how quickly it would catch up. After a few minutes I restarted the drbd service on the secondary node and it replicated just fine. Sometime after that (after it caught up and wasn’t oos) I again stopped the service on the secondary node but this time left it off until the job completed (a couple of hours). When I restarted it, they wouldn’t sync. The connection states would change but the secondary wasn’t ‘catching up.’<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I tried collecting as much information as possible but it’s too much to share here; dare I say too much to be useful…? Instead, if that’s ok, I’d like to refer to my pastebin where I’ve stored this information: <a href="http://pastebin.com/VEs6YYgR">http://pastebin.com/VEs6YYgR</a><o:p></o:p></p><p class=MsoNormal>It includes /var/log/messages, the config file dmesg, /proc/drbd and other output.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>DRBD wasn't built as a module; its baked into the kernel; lsmod doesn't show the drbd module loaded. (i.e.: find /lib/modules -name "drbd*" and find /lib/modules/2.6.*gentoo-*/ -type f -iname '*.o' -or -iname '*.ko' | grep drbd return no results)<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Linux primary 2.6.38-gentoo-r6 #1 SMP Mon Jun 27 07:35:00 EDT 2011 x86_64 Intel(R) Xeon(R) CPU E5504 @ 2.00GHz GenuineIntel GNU/Linux<o:p></o:p></p><p class=MsoNormal>Linux secondary 2.6.38-gentoo-r6 #1 SMP Sun Jun 26 15:57:21 EDT 2011 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Using 'sys-cluster/drbd' emerge build (v 8.3.8.1) on both nodes but I did not emerge 'sys-cluster/drbd-kernel' (v 8.0.16).<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Other steps performed:<o:p></o:p></p><p class=MsoNormal>restarted services on both nodes<o:p></o:p></p><p class=MsoNormal>`drbdadm {disconnect,connect} resource` on both nodes<o:p></o:p></p><p class=MsoNormal>`drbdadm role resource` on both nodes and primary reports primary, secondary reports secondary<o:p></o:p></p><p class=MsoNormal>`drbdadm -- --discard-my-data connect resource` on the secondary<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I tried trolling through a number of forums & mailing lists, and I feel it’s very similar to what was outlined here: http://copilotco.com/mail-archives/drbd.2009/msg00449.html. Instead of a bad link, or link with latency I simply stopped the DRBD service on the secondary to simulate an outage or disconnect scenario. I left it off for hours just to get an idea of how well it replicated & how long it would take but to my dismay it hasn't reconnected.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I’m not sure what I may have done wrong. I feel I’ve run into this issue before and I couldn’t get the boxes to sync unless I restarted them both. This time I would prefer to troubleshoot this a bit more while I have the problem to see if I can learn from this & avoid these types of problems in the future.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Many thanks to all for reviewing this message as well as your time & effort.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Cheers<o:p></o:p></p><p class=MsoNormal>J<o:p></o:p></p></div></body></html>