<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:D="DAV:" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>So I’ve got bonded 1G Ethernet going and I simplified it by
getting rid of bonding. Doing simple SCPs between the two hosts I can easily
hit 115MB/sec. I reduced my sync rates to 110M but still all my writes are
limited to a max of 18MB/sec. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Thanks,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Sylvain<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
dwight.hubbard@gmail.com [mailto:dwight.hubbard@gmail.com] <b>On Behalf Of </b>Dwight
Hubbard<br>
<b>Sent:</b> Wednesday, June 03, 2009 7:32 AM<br>
<b>To:</b> Sylvain Niles<br>
<b>Cc:</b> drbd-user@lists.linbit.com<br>
<b>Subject:</b> Re: [DRBD-user] Unable to adjust syncer rate.<o:p></o:p></span></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'>I've noticed in my usage of
DRBD if the rate exceeds the performance of the network between the nodes the
sync rate sucks. Since rate is in bytes per second instead of bits per
second a rate of 500MB/s would need 10GB ethernet and even 100MB/s would exceed
the performance of GB Ethernet cards plugged into a standard PCI bus.<br>
<br>
You might want to run iperf with the -f M parameter to get the actual
performance of the sync network and use it to tune the rate values.<o:p></o:p></p>
<div>
<p class=MsoNormal>On Tue, Jun 2, 2009 at 4:56 PM, Sylvain Niles <<a
href="mailto:sniles@slingmedia.com">sniles@slingmedia.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p>I have tried everything under the sun and I can’t get syncer to go faster
that about 15Mbit/sec. Any hints would be greatly appreciated as these devices
are greater than 300G and initial sync is taking forever at ~10-15M/sec. <o:p></o:p></p>
<p> <o:p></o:p></p>
<p>I’m running drbd82 and kmod-drbd82 (v8.2.6) on CentOS 5. Here’s the cleansed
version of my drbd.conf:<o:p></o:p></p>
<p> <o:p></o:p></p>
<p>global { <o:p></o:p></p>
<p> usage-count no; <o:p></o:p></p>
<p>}<o:p></o:p></p>
<p>common {<o:p></o:p></p>
<p> protocol C;<o:p></o:p></p>
<p>}<o:p></o:p></p>
<p>resource mysql-01 {<o:p></o:p></p>
<p> syncer {<o:p></o:p></p>
<p> al-extents 3833;<o:p></o:p></p>
<p> rate 500M;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> net {<o:p></o:p></p>
<p>
max-buffers 8192;<o:p></o:p></p>
<p>
}<o:p></o:p></p>
<p> disk {<o:p></o:p></p>
<p> no-disk-flushes;<o:p></o:p></p>
<p> no-md-flushes;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> on mysql1 {<o:p></o:p></p>
<p> device /dev/drbd1;<o:p></o:p></p>
<p> disk /dev/data1/mysql-01;<o:p></o:p></p>
<p> address <IP>;<o:p></o:p></p>
<p> meta-disk internal;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> on mysql2 {<o:p></o:p></p>
<p> device /dev/drbd1;<o:p></o:p></p>
<p> disk /dev/data1/mysql-01;<o:p></o:p></p>
<p> address <IP>;<o:p></o:p></p>
<p> meta-disk internal;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p>}<o:p></o:p></p>
<p>resource mysql -02 {<o:p></o:p></p>
<p> syncer {<o:p></o:p></p>
<p> al-extents 3833;<o:p></o:p></p>
<p> rate 100M;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> net {<o:p></o:p></p>
<p>
max-buffers 8192;<o:p></o:p></p>
<p>
}<o:p></o:p></p>
<p> disk {<o:p></o:p></p>
<p> no-disk-flushes;<o:p></o:p></p>
<p> no-md-flushes;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> on mysql1 {<o:p></o:p></p>
<p> device /dev/drbd2;<o:p></o:p></p>
<p> disk /dev/data1/mysql-02;<o:p></o:p></p>
<p> address <IP>;<o:p></o:p></p>
<p> meta-disk internal;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> on mysql2 {<o:p></o:p></p>
<p> device /dev/drbd2;<o:p></o:p></p>
<p> disk /dev/data1/mysql-02;<o:p></o:p></p>
<p> address <IP>;<o:p></o:p></p>
<p> meta-disk internal;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p>}<o:p></o:p></p>
<p>resource mysql3 {<o:p></o:p></p>
<p> syncer {<o:p></o:p></p>
<p> al-extents 3833;<o:p></o:p></p>
<p> rate 100M;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> net {<o:p></o:p></p>
<p>
max-buffers 8192;<o:p></o:p></p>
<p>
}<o:p></o:p></p>
<p> disk {<o:p></o:p></p>
<p> no-disk-flushes;<o:p></o:p></p>
<p> no-md-flushes;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> on mysql1 {<o:p></o:p></p>
<p>
device
/dev/drbd3;<o:p></o:p></p>
<p> disk /dev/data1/mysql-3;<o:p></o:p></p>
<p> address
<IP>;<o:p></o:p></p>
<p> meta-disk internal;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p> on mysql2 {<o:p></o:p></p>
<p>
device
/dev/drbd3;<o:p></o:p></p>
<p> disk /dev/data1/mysql-3;<o:p></o:p></p>
<p>
address
<IP>;<o:p></o:p></p>
<p> meta-disk internal;<o:p></o:p></p>
<p> }<o:p></o:p></p>
<p>}<o:p></o:p></p>
<p> <o:p></o:p></p>
<p>Thanks,<o:p></o:p></p>
<p>Sylvain Niles<o:p></o:p></p>
<p> <o:p></o:p></p>
</div>
</div>
<p class=MsoNormal style='margin-bottom:12.0pt'><br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><o:p></o:p></p>
</div>
<p class=MsoNormal><br>
<br clear=all>
<br>
-- <br>
Dwight Hubbard<br>
Owner Effective Automation Solutions<br>
Website: <a href="http://effectiveautomationsolutions.com">http://effectiveautomationsolutions.com</a><br>
Blog: <a href="http://computing.dwighthubbard.info">http://computing.dwighthubbard.info</a><br>
Email: <a href="mailto:dwight@dwighthubbard.com">dwight@dwighthubbard.com</a><br>
Phone: (503) 616-4493<br>
Redhat Certified Engineer - RHCE #804007137224095<br>
VMware Certified Professional - VCP #18529<o:p></o:p></p>
</div>
</body>
</html>