<div dir="ltr">Lars, Thanks for the MTU suggestion! It ended up being an issue on our 10gb switches. While we could ping, it wasn't correctly sending jumbo frames. Nothing showed up in the logs except that it would keep retrying and retrying but I started doing some ping tests with different sizes and found the issue quickly after that.<div><br></div><div>Regards,</div><div><br></div><div>Josh Hunholz</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, May 7, 2015 at 3:34 AM, Lars Ellenberg <span dir="ltr"><<a href="mailto:lars.ellenberg@linbit.com" target="_blank">lars.ellenberg@linbit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Thu, Apr 30, 2015 at 11:16:00AM -0500, Josh Hunholz wrote:<br>
> Hey Folks,<br>
><br>
> I posted this issue on the Ganeti mailing list as well but they suggested<br>
> that you might be able to help me out to more quickly figure out what is<br>
> going on. So we are setting up a new Ganeti cluster with version 2.12.1<br>
> running on Ubuntu 14.04. It is using DRBD version 8.4.3. The disk starts<br>
> to create but never completes syncing. On the primary, it looks like this:<br>
><br>
> root@icnodeb-03:~# cat /proc/drbd<br>
> version: 8.4.3 (api:1/proto:86-101)<br>
> srcversion: F97798065516C94BE0F27DC<br>
> 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---n-<br>
> ns:576 nr:0 dw:0 dr:15304 al:0 bm:0 lo:0 pe:1 ua:7 ap:0 ep:1 wo:f<br>
> oos:10485760<br>
> [>....................] sync'ed: 0.1% (10240/10240)Mfinish: 283:59:21<br>
> speed: 0 (0) K/sec<br>
><br>
> And on the secondary it looks like this:<br>
><br>
> root@icnodeb-04:~# cat /proc/drbd<br>
> version: 8.4.3 (api:1/proto:86-101)<br>
> srcversion: F97798065516C94BE0F27DC<br>
> 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----<br>
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:7 ua:0 ap:0 ep:1 wo:f oos:10485760<br>
> [>....................] sync'ed: 0.1% (10240/10240)Mfinish: 123:47:24<br>
> speed: 0 (0) want: 0 K/sec<br>
><br>
> It stays that way and never gets past 0.1%.<br>
<br>
</div></div>Network problem?<br>
MTU mismatch?<br>
Are you able to write to the Primary while it is in that state?<br>
<span class=""><br>
> Is there anything else I can<br>
> check to try and figure out what is going on?<br>
<br>
</span>Check the kernel logs.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
: Lars Ellenberg<br>
: <a href="http://www.LINBIT.com" target="_blank">http://www.LINBIT.com</a> | Your Way to High Availability<br>
: DRBD, Linux-HA and Pacemaker support and consulting<br>
<br>
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.<br>
__<br>
please don't Cc me, but send to list -- I'm subscribed<br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
</font></span></blockquote></div><br></div>