Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Sylvain,
I am experiencing something like what you are experiencing :)
I'm new to DRBD, but thought I'd share how I got slightly "better"
performance after understanding a few things - the old guys in this list
will give you much better advice, though, I'm sure.
There are many other factors which can cause "slow sync" - some things you
might want to check out or test are:
- Local RAID or disk write speed - you can test it using dd, with
"oflag=direct" - write in 1G chunks - this webinar shows quite abit of info
on how local disk read/write can affect DRBD's performance
http://www.linbit.com/en/education/on-demand-webinars/drbd-performance-tuning/-
and is your raid array Quick Init'ed or Fully Initialized?
- Write caching? Read caching? Once again, refer to that tutorial above
- Write scheduler? google for stuff like "deadline" - i think its also
inside the tutorial
- *Actual* network throughout - you can test using "iperf" tool - it's much
more accurate then transferring files via ftp or scp since you can easily
run like loads of tests
- Meta disk - you might want to play around with external meta disks. For
me, i get severely impacted when using internal meta disks, but external
meta disks gives me alot better performance
The Syncer rate in drbd.conf is not really meant to "accelerate", rather,
its meant to "cap" speeds, incase your drbd does a full resync and eat up
too much bandwidth within your network - somewhere in the docs I belief
there's mention of that.
After you understand some of these, you might further be able to pinpoint
exactly how "fast" or rather, where the bottleneck is in your setup.
Hope this helps.
Good luck!
Regards,
Kelvin
On Thu, Jun 4, 2009 at 2:09 AM, Sylvain Niles <sniles at slingmedia.com> wrote:
> So I’ve got bonded 1G Ethernet going and I simplified it by getting rid
> of bonding. Doing simple SCPs between the two hosts I can easily hit
> 115MB/sec. I reduced my sync rates to 110M but still all my writes are
> limited to a max of 18MB/sec.
>
>
>
> Thanks,
>
> Sylvain
>
>
>
> *From:* dwight.hubbard at gmail.com [mailto:dwight.hubbard at gmail.com] *On
> Behalf Of *Dwight Hubbard
> *Sent:* Wednesday, June 03, 2009 7:32 AM
> *To:* Sylvain Niles
> *Cc:* drbd-user at lists.linbit.com
> *Subject:* Re: [DRBD-user] Unable to adjust syncer rate.
>
>
>
> I've noticed in my usage of DRBD if the rate exceeds the performance of the
> network between the nodes the sync rate sucks. Since rate is in bytes per
> second instead of bits per second a rate of 500MB/s would need 10GB ethernet
> and even 100MB/s would exceed the performance of GB Ethernet cards plugged
> into a standard PCI bus.
>
> You might want to run iperf with the -f M parameter to get the actual
> performance of the sync network and use it to tune the rate values.
>
> On Tue, Jun 2, 2009 at 4:56 PM, Sylvain Niles <sniles at slingmedia.com>
> wrote:
>
> I have tried everything under the sun and I can’t get syncer to go faster
> that about 15Mbit/sec. Any hints would be greatly appreciated as these
> devices are greater than 300G and initial sync is taking forever at
> ~10-15M/sec.
>
>
>
> I’m running drbd82 and kmod-drbd82 (v8.2.6) on CentOS 5. Here’s the
> cleansed version of my drbd.conf:
>
>
>
> global {
>
> usage-count no;
>
> }
>
> common {
>
> protocol C;
>
> }
>
> resource mysql-01 {
>
> syncer {
>
> al-extents 3833;
>
> rate 500M;
>
> }
>
> net {
>
> max-buffers 8192;
>
> }
>
> disk {
>
> no-disk-flushes;
>
> no-md-flushes;
>
> }
>
> on mysql1 {
>
> device /dev/drbd1;
>
> disk /dev/data1/mysql-01;
>
> address <IP>;
>
> meta-disk internal;
>
> }
>
> on mysql2 {
>
> device /dev/drbd1;
>
> disk /dev/data1/mysql-01;
>
> address <IP>;
>
> meta-disk internal;
>
> }
>
> }
>
> resource mysql -02 {
>
> syncer {
>
> al-extents 3833;
>
> rate 100M;
>
> }
>
> net {
>
> max-buffers 8192;
>
> }
>
> disk {
>
> no-disk-flushes;
>
> no-md-flushes;
>
> }
>
> on mysql1 {
>
> device /dev/drbd2;
>
> disk /dev/data1/mysql-02;
>
> address <IP>;
>
> meta-disk internal;
>
> }
>
> on mysql2 {
>
> device /dev/drbd2;
>
> disk /dev/data1/mysql-02;
>
> address <IP>;
>
> meta-disk internal;
>
> }
>
> }
>
> resource mysql3 {
>
> syncer {
>
> al-extents 3833;
>
> rate 100M;
>
> }
>
> net {
>
> max-buffers 8192;
>
> }
>
> disk {
>
> no-disk-flushes;
>
> no-md-flushes;
>
> }
>
> on mysql1 {
>
> device /dev/drbd3;
>
> disk /dev/data1/mysql-3;
>
> address <IP>;
>
> meta-disk internal;
>
> }
>
> on mysql2 {
>
> device /dev/drbd3;
>
> disk /dev/data1/mysql-3;
>
> address <IP>;
>
> meta-disk internal;
>
> }
>
> }
>
>
>
> Thanks,
>
> Sylvain Niles
>
>
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
>
>
> --
> Dwight Hubbard
> Owner Effective Automation Solutions
> Website: http://effectiveautomationsolutions.com
> Blog: http://computing.dwighthubbard.info
> Email: dwight at dwighthubbard.com
> Phone: (503) 616-4493
> Redhat Certified Engineer - RHCE #804007137224095
> VMware Certified Professional - VCP #18529
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20090604/042e51aa/attachment.htm>