<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><o:SmartTagType
namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="place"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
p
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=blue>
<div class=Section1>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>I would suggest looking at the disk
hardware. What sort of RAID card are you using in the servers? If you are
using an HP Smart Array there is some tuning to do to get some descent
performance out of those as they have a rather deep buffer.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>-Morey<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 face=Tahoma><span style='font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font size=2
face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'>
drbd-user-bounces@lists.linbit.com [mailto:drbd-user-bounces@lists.linbit.com] <b><span
style='font-weight:bold'>On Behalf Of </span></b>Matt Baer<br>
<b><span style='font-weight:bold'>Sent:</span></b> Friday, August 05, 2011 9:26
AM<br>
<b><span style='font-weight:bold'>To:</span></b> Dan Barker<br>
<b><span style='font-weight:bold'>Cc:</span></b> drbd-user@lists.linbit.com<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [DRBD-user] DRBD
won't take 1G syncer rate</span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>Well we're the perfect
supplements for each other because, as you can see, I don't know DRBD from a
hole in the wall.<br>
<br>
Yes, it's a brand new resource. I don't think I would say it's full of
zeros, as it has a clean ext3 file system on it.<br>
<br>
Yes, I would LOVE to skip the sync, I've been dealing with this for weeks now
and right when I was about to go live, I tested the failover and it didn't work
because of a service heartbeat wanted to start wasn't going all that
well. While troubleshooting, I lost my 100% perfectly live server and
have to start from scratch. Problem is I only have two days to do it and
the thing has to sync 1.8TB at 12MB/s. I have no idea where the
bottleneck could be. Two servers, a cable connecting eth1 to eth1, both
are auto-negotiated at 1gbps on their own /30 subnet. The only thing
there would be garbage NIC cards, possible, but not probable, or the cable,
more likely, but I've never had an issue with it until now. Freshly
constructed servers, too.<br>
<br>
I tried the drbdsetup /dev/drbd0 syncer -r 120M, been running like that for
about 5 minutes now and it hasn't changed at all.<br>
<br>
<o:p></o:p></span></font></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'>On Fri, Aug 5, 2011 at 10:13 AM, Dan Barker <<a
href="mailto:dbarker@visioncomm.net">dbarker@visioncomm.net</a>> wrote:<o:p></o:p></span></font></p>
<div link=blue vlink=purple>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>Is this a brand new resource? Why are you doing a full sync? If
it’s brand new (full of zeros), you can skip the sync. Instructions upon
request.</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>Btw, I don’t know why you are getting 12% of your requested
syncer rate. I’m not a hot-shot linux performace analyzer, but there is a
bottleneck somewhere. I get 25M routinely here on GB nics. I have my Syncer set
to 25M. It drops to about 14M (each) if 2 are syncing.</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>To change sync rate without stop/start drbd: drbdsetup
/dev/drbd1 syncer -r 120M</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>AL Extents seems a bit low. I use 1801 (big prime number that
felt about right).</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>Dan</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><font
size=2 face="Times New Roman"><span style='font-size:10.0pt;font-weight:bold'>From:</span></font></b><font
size=2><span style='font-size:10.0pt'> Matt Baer [mailto:<a
href="mailto:mbaer@lrnet1.com" target="_blank">mbaer@lrnet1.com</a>] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Friday, August 05, 2011
11:03 AM<br>
<b><span style='font-weight:bold'>To:</span></b> Dan Barker</span></font><o:p></o:p></p>
<div>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [DRBD-user] DRBD
won't take 1G syncer rate<o:p></o:p></span></font></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'>Ok, revised
/etc/drbd.conf and restarted DRBD with the following<o:p></o:p></span></font></p>
<div style='margin-left:30.0pt'>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'>common { syncer {
rate 100M; al-extents 257; } }<o:p></o:p></span></font></p>
</div>
<p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'><br>
And I'm getting from /proc/drbd:<o:p></o:p></span></font></p>
<div style='margin-left:30.0pt'>
<p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'>GIT-hash:
3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by buildsvn@c5-x8664-build,
2008-10-03 11:30:17<br>
0: cs:SyncTarget st:Secondary/Primary ds:Inconsistent/UpToDate C r---<br>
ns:0 nr:281056 dw:272864 dr:0 al:0 bm:16 lo:257 pe:1969
ua:256 ap:0 oos:1308428488<br>
[>....................]
sync'ed: 0.1% (1277762/1278028)M<br>
finish: 25:57:39 speed: 13,904
(12,400) K/sec<o:p></o:p></span></font></p>
</div>
<p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'>And I only have
one resource, r0. All it's syncing right now is the post mkfs.ext3 /dev/drbd0<br>
<br>
<o:p></o:p></span></font></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'>On Fri, Aug 5,
2011 at 9:51 AM, Dan Barker <<a href="mailto:dbarker@visioncomm.net"
target="_blank">dbarker@visioncomm.net</a>> wrote:<o:p></o:p></span></font></p>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>You are still mixing megabits and megabytes. Your 1000 megabit
pipe won’t take a 600 megabyte stream, or a 150 megabyte stream. The maximum is
about 125 MBps.</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>DRBD talks (and is documented to talk) bytes. Most everyone else
talks bits.</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>You don’t mention the speed you are getting.</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>Also, if you have 3 resources syncing, each will try for the
syncer limit. So, to use 50% of your capacity to sync 3 resources, you’d
specify the rate as 21M. Note: you can change the rate on the fly, during a
sync.</span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'>Dan </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color="#1f497d" face="Times New Roman"><span style='font-size:11.0pt;
color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><font
size=2 face="Times New Roman"><span style='font-size:10.0pt;font-weight:bold'>From:</span></font></b><font
size=2><span style='font-size:10.0pt'> <a
href="mailto:drbd-user-bounces@lists.linbit.com" target="_blank">drbd-user-bounces@lists.linbit.com</a>
[mailto:<a href="mailto:drbd-user-bounces@lists.linbit.com" target="_blank">drbd-user-bounces@lists.linbit.com</a>]
<b><span style='font-weight:bold'>On Behalf Of </span></b>Matt Baer<br>
<b><span style='font-weight:bold'>Sent:</span></b> Friday, August 05, 2011 8:51
AM<br>
<b><span style='font-weight:bold'>To:</span></b> Caspar Smit<br>
<b><span style='font-weight:bold'>Cc:</span></b> <a
href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [DRBD-user] DRBD
won't take 1G syncer rate</span></font><o:p></o:p></p>
<div>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<p><font size=3 face="Times New Roman"><span style='font-size:12.0pt'>I was
playing with the settings yesterday and it let me set it at 600M and it didn't
make a difference in the sync speed at all. I then tried it with your
suggestion, dropped it to 150M just to be safe. Still no
difference. I wonder what the deal is. Could it be that this is the
initial sync?<o:p></o:p></span></font></p>
<p><font size=3 face="Times New Roman"><span style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'>On Aug 5, 2011 1:29
AM, "Caspar Smit" <<a href="mailto:c.smit@truebit.nl"
target="_blank">c.smit@truebit.nl</a>> wrote:<br>
> Hi Matt,<br>
> <br>
> 1000M means 1000 Mb/s NOT 1000mbps. To reach 1000M you should have at
least<br>
> one (probably two) 10gbit interface(s). Since you have two 1gbit
interfaces<br>
> (bonded with balance-rr?) a value between 100M and around 170M would be
more<br>
> appropiate.<br>
> <br>
> Kind regards,<br>
> Caspar<br>
> Op 5 aug. 2011 08:21 schreef "Matt Baer" <<a
href="mailto:mbaer@lrnet1.com" target="_blank">mbaer@lrnet1.com</a>> het volgende:<br>
>> When setting the syncer rate in drbd.conf to 1G, it won't start,
citing<br>
> that<br>
>> 1G is invalid. Get the same thing with 1000M. Any clue as to why? It<br>
>> explicitly states that <<a href="mailto:mbaer@lrnet1.com"
target="_blank">mbaer@lrnet1.com</a>>1G is acceptable in the docs.<br>
> I've<br>
>> triple checked and both interfaces are auto-negotiated at 1000mbps
full<br>
>> duplex.<o:p></o:p></span></font></p>
</div>
</div>
</div>
</div>
</div>
<p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'><br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><o:p></o:p></span></font></p>
</div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'><br>
<br clear=all>
<br>
-- <br>
Thanks,<br>
Matt Baer<br>
Network Manager<br>
Leaf River Telephone Company<br>
102 W Second Street<br>
<st1:place w:st="on">Leaf River</st1:place>, IL <br>
<a href="tel:815.738.2211" target="_blank" value="+18157382211">815.738.2211</a>
- Main Office<br>
<a href="tel:866.738.2211" target="_blank" value="+18667382211">866.738.2211</a>
- Toll Free<br>
<a href="mailto:mbaer@lrnet1.com" target="_blank">mbaer@lrnet1.com</a><o:p></o:p></span></font></p>
</div>
</div>
</div>
</div>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><br>
<br clear=all>
<br>
-- <br>
Thanks,<br>
Matt Baer<br>
Network Manager<br>
Leaf River Telephone Company<br>
102 W Second Street<br>
<st1:place w:st="on">Leaf River</st1:place>, IL <br>
815.738.2211 - Main Office<br>
866.738.2211 - Toll Free<br>
<a href="mailto:mbaer@lrnet1.com" target="_blank">mbaer@lrnet1.com</a><o:p></o:p></span></font></p>
</div>
</body>
</html>