[DRBD-user] Sync its so slow

Igor Neves igor at 3gnt.net
Wed Oct 24 18:03:51 CEST 2007


Hi,

Here is more information about my hardware:

08:0e.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS

Driver: megaraid_sas

Partitions on megaraid controller:

major minor  #blocks  name

   8     0   35155968 sda
   8     1     104391 sda1
   8     2   35045797 sda2
   8    16  570310656 sdb
   8    17  570307468 sdb1
 147     0  570290024 drbd0

On Wed, 24 Oct 2007 09:04:11 -0500
Todd Denniston <Todd.Denniston at ssa.crane.navy.mil> wrote:

> Igor Neves wrote, On 10/24/2007 04:21 AM:
> > Hi,
> > 
> > On Tue, 23 Oct 2007 14:52:33 -0400 (EDT)
> > Nate Seif <seifn at rstechnicalservices.com> wrote:
> > 
> >> Identify where the bottleneck is to [possibly] get faster sync
> >> speeds: (1) Check to make sure your kernel has proper [driver]
> >> support for your chipset. Run 'lspci' to get info about what
> >> chipset you have if you don't already know.
> > 
> > I have support, and i have full support for the device.
> > 
> >> (2) Make sure you have DMA enabled for your hard drives. Research
> >> first, then run 'hdparm' to test write speeds to see if hard drives
> >> are the bottleneck.
> > 
> > DMA in my case, have nothing to do with it, it's SAS device.
> > 
> >> (3) Check your network: can you get faster transfer speeds with
> >> other app's? Make sure you've got the proper drivers for your
> >> NIC's...
> > 
> > I get much faster transfer on my NIC's, yes, i have tested with
> > iperf.
> > 
> 
> Have you seen the speed tricks from the list?
> 
> http://www.google.com/search?hl=en&q=speed+gigabit.ethernet+site%3Alists.linbit.com%2Fpipermail%2Fdrbd-user%2F&btnG=Google+Search
> 
> Oh, and this next guy _usually_ knows what he is talking about,
> though he was using an older drbd and you may need to figure out how
> some of the settings map to current versions: :)
> http://lists.linbit.com/pipermail/drbd-user/2004-November/002241.html
> 
> And another recent speed thread:
> http://lists.linbit.com/pipermail/drbd-user/2007-June/007010.html
> http://lists.linbit.com/pipermail/drbd-user/2007-June/007033.html
> 
> 
> Also, are you running multiple DRBD's?
> If so do you have them all on one spindle or Array?
> Are you using Sync groups?
> 
> What protocol (A,B,C) are you using (should not matter with initial
> sync, just being pedantic)?

All this questions, in the attach drbd.conf, :).

> 
> a few more comments below
> 
> >> Good luck.
> > 
> > Thanks.
> > 
> >> Nate
> >>
> >>
> >> On Tue, 23 Oct 2007, Igor Neves wrote:
> >>
> >>> Hi,
> >>>
> >>> I'm having problems with sync, using drbd 8.0.4, in centos 5.0.
> >>>
> >>> When i have to do one full sync, of the disc, it takes me 12 Hours
> >>> (500G).
> 
> so you are saying ...
> echo -en "scale=2\n(500*1024*1024)/(12*60*60)\n"|bc
> .. you are seeing ~12MBytes/second going through?

Yes, exactly 12MBytes, which is ridiculous for gigabit ethernet and this
hardware, and it's exactly how it takes, about 10~12 hours to finish
the 500G.

> 
> Looks suspiciously like the 100Mbit network max.
> 
> try giving us `cat /proc/drbd` output so we are not making up numbers
> from yours. :)
> 
> >>>
> >>> The 2 nodes are connected between Gigabit Ethernet, my question
> >>> it's, what do i have to do, to make it faster? This time is
> >>> insane! :)
> >>>
> >>> I know my disk does not write at Gigabit ethernet capacity, but
> >>> indeed, lets say it writes at least at 50MBytes/sec.
> >>>
> >>> If we write data, at 50MBytes/sec it should take theoretically
> >>> about 1.66 hours, its about 1h and 40 minutes or so.
> >>>
> >>> Of course the disk writes faster than that, at least in this case
> >>> i have one SAS controller with 15k rpm disk's.
> >>>
> >>> I have in drbd.conf configured 'rate 100M'.
> >>>
> >>> I have drbd over the disk, and ext3 over it, but i think this is
> >>> transparent for drbd, this should not take the drbd to slow.
> >>>
> >>> When it is doing the full sync, i see in '/proc/drbd' and it says
> >>> it's copying at about 20K/sec, and since it is in bits, it is
> >>> pretty slow.
> >>>
> 
> IIRC, the data in /proc/drbd is in KBytes, so you are seeing
> 20KBytes/second (or if you did a little math wrong for us,
> 20MBytes/second). The following is indicating ~15MBytes/seconds:
> finish: 4:20:22h speed: 15,290 (15,374) K/sec

This is not easy since i dont want to sync for 12 hours again! :)

But yes, its that ~12-15Mb i get.

> 
> 
> >>> This is bug? Or I am the bug? :) Since I'm new to drbd, i think
> >>> the second option will be more likely then the first one! :D
> >>>
> >>> Gigabit Ethernet = 125MBytes/Sec
> >>>
> >>> Thanks for all the help.
> 
> 

Thanks for all your help guys.
---------
Igor Neves <igor.neves at 3gnt.net>
3GNTW - Tecnologias de Informação, Lda
 
 sip igor at 3gnt.net 	jid igor at jabber.3gnt.org
 icq 249075444		tlm 00351914503611

-------------- next part --------------
A non-text attachment was scrubbed...
Name: drbd.conf
Type: application/octet-stream
Size: 446 bytes
Desc: not available
Url : http://lists.linbit.com/pipermail/drbd-user/attachments/20071024/e4fbc1bf/attachment.obj 


More information about the drbd-user mailing list