Sylvain,<br><br>I am experiencing something like what you are experiencing :)<br><br>I'm new to DRBD, but thought I'd share how I got slightly "better" performance after understanding a few things - the old guys in this list will give you much better advice, though, I'm sure.<br>
<br>There are many other factors which can cause "slow sync" - some things you might want to check out or test are:<br><br>- Local RAID or disk write speed - you can test it using dd, with "oflag=direct" - write in 1G chunks - this webinar shows quite abit of info on how local disk read/write can affect DRBD's performance <a href="http://www.linbit.com/en/education/on-demand-webinars/drbd-performance-tuning/">http://www.linbit.com/en/education/on-demand-webinars/drbd-performance-tuning/</a> - and is your raid array Quick Init'ed or Fully Initialized?<br>
<br>- Write caching? Read caching? Once again, refer to that tutorial above<br><br>- Write scheduler? google for stuff like "deadline" - i think its also inside the tutorial<br><br>- *Actual* network throughout - you can test using "iperf" tool - it's much more accurate then transferring files via ftp or scp since you can easily run like loads of tests<br>
<br>- Meta disk - you might want to play around with external meta disks. For me, i get severely impacted when using internal meta disks, but external meta disks gives me alot better performance<br><br><br>The Syncer rate in drbd.conf is not really meant to "accelerate", rather, its meant to "cap" speeds, incase your drbd does a full resync and eat up too much bandwidth within your network - somewhere in the docs I belief there's mention of that.<br>
<br>After you understand some of these, you might further be able to pinpoint exactly how "fast" or rather, where the bottleneck is in your setup.<br><br>Hope this helps.<br><br>Good luck!<br><br>Regards,<br><br>
Kelvin<br><br><div class="gmail_quote">On Thu, Jun 4, 2009 at 2:09 AM, Sylvain Niles <span dir="ltr"><<a href="mailto:sniles@slingmedia.com">sniles@slingmedia.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">So I’ve got bonded 1G Ethernet going and I simplified it by
getting rid of bonding. Doing simple SCPs between the two hosts I can easily
hit 115MB/sec. I reduced my sync rates to 110M but still all my writes are
limited to a max of 18MB/sec. </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Thanks,</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Sylvain</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<div style="border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;">
<a href="mailto:dwight.hubbard@gmail.com" target="_blank">dwight.hubbard@gmail.com</a> [mailto:<a href="mailto:dwight.hubbard@gmail.com" target="_blank">dwight.hubbard@gmail.com</a>] <b>On Behalf Of </b>Dwight
Hubbard<br>
<b>Sent:</b> Wednesday, June 03, 2009 7:32 AM<br>
<b>To:</b> Sylvain Niles<br>
<b>Cc:</b> <a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<b>Subject:</b> Re: [DRBD-user] Unable to adjust syncer rate.</span></p>
</div><div><div></div><div class="h5">
<p> </p>
<p style="margin-bottom: 12pt;">I've noticed in my usage of
DRBD if the rate exceeds the performance of the network between the nodes the
sync rate sucks. Since rate is in bytes per second instead of bits per
second a rate of 500MB/s would need 10GB ethernet and even 100MB/s would exceed
the performance of GB Ethernet cards plugged into a standard PCI bus.<br>
<br>
You might want to run iperf with the -f M parameter to get the actual
performance of the sync network and use it to tune the rate values.</p>
<div>
<p>On Tue, Jun 2, 2009 at 4:56 PM, Sylvain Niles <<a href="mailto:sniles@slingmedia.com" target="_blank">sniles@slingmedia.com</a>> wrote:</p>
<div>
<div>
<p>I have tried everything under the sun and I can’t get syncer to go faster
that about 15Mbit/sec. Any hints would be greatly appreciated as these devices
are greater than 300G and initial sync is taking forever at ~10-15M/sec. </p>
<p> </p>
<p>I’m running drbd82 and kmod-drbd82 (v8.2.6) on CentOS 5. Here’s the cleansed
version of my drbd.conf:</p>
<p> </p>
<p>global { </p>
<p> usage-count no; </p>
<p>}</p>
<p>common {</p>
<p> protocol C;</p>
<p>}</p>
<p>resource mysql-01 {</p>
<p> syncer {</p>
<p> al-extents 3833;</p>
<p> rate 500M;</p>
<p> }</p>
<p> net {</p>
<p>
max-buffers 8192;</p>
<p>
}</p>
<p> disk {</p>
<p> no-disk-flushes;</p>
<p> no-md-flushes;</p>
<p> }</p>
<p> on mysql1 {</p>
<p> device /dev/drbd1;</p>
<p> disk /dev/data1/mysql-01;</p>
<p> address <IP>;</p>
<p> meta-disk internal;</p>
<p> }</p>
<p> on mysql2 {</p>
<p> device /dev/drbd1;</p>
<p> disk /dev/data1/mysql-01;</p>
<p> address <IP>;</p>
<p> meta-disk internal;</p>
<p> }</p>
<p>}</p>
<p>resource mysql -02 {</p>
<p> syncer {</p>
<p> al-extents 3833;</p>
<p> rate 100M;</p>
<p> }</p>
<p> net {</p>
<p>
max-buffers 8192;</p>
<p>
}</p>
<p> disk {</p>
<p> no-disk-flushes;</p>
<p> no-md-flushes;</p>
<p> }</p>
<p> on mysql1 {</p>
<p> device /dev/drbd2;</p>
<p> disk /dev/data1/mysql-02;</p>
<p> address <IP>;</p>
<p> meta-disk internal;</p>
<p> }</p>
<p> on mysql2 {</p>
<p> device /dev/drbd2;</p>
<p> disk /dev/data1/mysql-02;</p>
<p> address <IP>;</p>
<p> meta-disk internal;</p>
<p> }</p>
<p>}</p>
<p>resource mysql3 {</p>
<p> syncer {</p>
<p> al-extents 3833;</p>
<p> rate 100M;</p>
<p> }</p>
<p> net {</p>
<p>
max-buffers 8192;</p>
<p>
}</p>
<p> disk {</p>
<p> no-disk-flushes;</p>
<p> no-md-flushes;</p>
<p> }</p>
<p> on mysql1 {</p>
<p>
device
/dev/drbd3;</p>
<p> disk /dev/data1/mysql-3;</p>
<p> address
<IP>;</p>
<p> meta-disk internal;</p>
<p> }</p>
<p> on mysql2 {</p>
<p>
device
/dev/drbd3;</p>
<p> disk /dev/data1/mysql-3;</p>
<p>
address
<IP>;</p>
<p> meta-disk internal;</p>
<p> }</p>
<p>}</p>
<p> </p>
<p>Thanks,</p>
<p>Sylvain Niles</p>
<p> </p>
</div>
</div>
<p style="margin-bottom: 12pt;"><br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a></p>
</div>
<p><br>
<br clear="all">
<br>
-- <br>
Dwight Hubbard<br>
Owner Effective Automation Solutions<br>
Website: <a href="http://effectiveautomationsolutions.com" target="_blank">http://effectiveautomationsolutions.com</a><br>
Blog: <a href="http://computing.dwighthubbard.info" target="_blank">http://computing.dwighthubbard.info</a><br>
Email: <a href="mailto:dwight@dwighthubbard.com" target="_blank">dwight@dwighthubbard.com</a><br>
Phone: (503) 616-4493<br>
Redhat Certified Engineer - RHCE #804007137224095<br>
VMware Certified Professional - VCP #18529</p>
</div></div></div>
</div>
<br>_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
<br></blockquote></div><br>