[DRBD-user] Sync its so slow
igor at 3gnt.net
Wed Oct 24 18:03:51 CEST 2007
Here is more information about my hardware:
08:0e.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS
Partitions on megaraid controller:
major minor #blocks name
8 0 35155968 sda
8 1 104391 sda1
8 2 35045797 sda2
8 16 570310656 sdb
8 17 570307468 sdb1
147 0 570290024 drbd0
On Wed, 24 Oct 2007 09:04:11 -0500
Todd Denniston <Todd.Denniston at ssa.crane.navy.mil> wrote:
> Igor Neves wrote, On 10/24/2007 04:21 AM:
> > Hi,
> > On Tue, 23 Oct 2007 14:52:33 -0400 (EDT)
> > Nate Seif <seifn at rstechnicalservices.com> wrote:
> >> Identify where the bottleneck is to [possibly] get faster sync
> >> speeds: (1) Check to make sure your kernel has proper [driver]
> >> support for your chipset. Run 'lspci' to get info about what
> >> chipset you have if you don't already know.
> > I have support, and i have full support for the device.
> >> (2) Make sure you have DMA enabled for your hard drives. Research
> >> first, then run 'hdparm' to test write speeds to see if hard drives
> >> are the bottleneck.
> > DMA in my case, have nothing to do with it, it's SAS device.
> >> (3) Check your network: can you get faster transfer speeds with
> >> other app's? Make sure you've got the proper drivers for your
> >> NIC's...
> > I get much faster transfer on my NIC's, yes, i have tested with
> > iperf.
> Have you seen the speed tricks from the list?
> Oh, and this next guy _usually_ knows what he is talking about,
> though he was using an older drbd and you may need to figure out how
> some of the settings map to current versions: :)
> And another recent speed thread:
> Also, are you running multiple DRBD's?
> If so do you have them all on one spindle or Array?
> Are you using Sync groups?
> What protocol (A,B,C) are you using (should not matter with initial
> sync, just being pedantic)?
All this questions, in the attach drbd.conf, :).
> a few more comments below
> >> Good luck.
> > Thanks.
> >> Nate
> >> On Tue, 23 Oct 2007, Igor Neves wrote:
> >>> Hi,
> >>> I'm having problems with sync, using drbd 8.0.4, in centos 5.0.
> >>> When i have to do one full sync, of the disc, it takes me 12 Hours
> >>> (500G).
> so you are saying ...
> echo -en "scale=2\n(500*1024*1024)/(12*60*60)\n"|bc
> .. you are seeing ~12MBytes/second going through?
Yes, exactly 12MBytes, which is ridiculous for gigabit ethernet and this
hardware, and it's exactly how it takes, about 10~12 hours to finish
> Looks suspiciously like the 100Mbit network max.
> try giving us `cat /proc/drbd` output so we are not making up numbers
> from yours. :)
> >>> The 2 nodes are connected between Gigabit Ethernet, my question
> >>> it's, what do i have to do, to make it faster? This time is
> >>> insane! :)
> >>> I know my disk does not write at Gigabit ethernet capacity, but
> >>> indeed, lets say it writes at least at 50MBytes/sec.
> >>> If we write data, at 50MBytes/sec it should take theoretically
> >>> about 1.66 hours, its about 1h and 40 minutes or so.
> >>> Of course the disk writes faster than that, at least in this case
> >>> i have one SAS controller with 15k rpm disk's.
> >>> I have in drbd.conf configured 'rate 100M'.
> >>> I have drbd over the disk, and ext3 over it, but i think this is
> >>> transparent for drbd, this should not take the drbd to slow.
> >>> When it is doing the full sync, i see in '/proc/drbd' and it says
> >>> it's copying at about 20K/sec, and since it is in bits, it is
> >>> pretty slow.
> IIRC, the data in /proc/drbd is in KBytes, so you are seeing
> 20KBytes/second (or if you did a little math wrong for us,
> 20MBytes/second). The following is indicating ~15MBytes/seconds:
> finish: 4:20:22h speed: 15,290 (15,374) K/sec
This is not easy since i dont want to sync for 12 hours again! :)
But yes, its that ~12-15Mb i get.
> >>> This is bug? Or I am the bug? :) Since I'm new to drbd, i think
> >>> the second option will be more likely then the first one! :D
> >>> Gigabit Ethernet = 125MBytes/Sec
> >>> Thanks for all the help.
Thanks for all your help guys.
Igor Neves <igor.neves at 3gnt.net>
3GNTW - Tecnologias de Informação, Lda
sip igor at 3gnt.net jid igor at jabber.3gnt.org
icq 249075444 tlm 00351914503611
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 446 bytes
Desc: not available
Url : http://lists.linbit.com/pipermail/drbd-user/attachments/20071024/e4fbc1bf/attachment.obj
More information about the drbd-user