[DRBD-user] Sync its so slow

Todd Denniston Todd.Denniston at ssa.crane.navy.mil
Wed Oct 24 16:04:11 CEST 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Igor Neves wrote, On 10/24/2007 04:21 AM:
> Hi,
> 
> On Tue, 23 Oct 2007 14:52:33 -0400 (EDT)
> Nate Seif <seifn at rstechnicalservices.com> wrote:
> 
>> Identify where the bottleneck is to [possibly] get faster sync speeds:
>> (1) Check to make sure your kernel has proper [driver] support for
>> your chipset. Run 'lspci' to get info about what chipset you have if
>> you don't already know.
> 
> I have support, and i have full support for the device.
> 
>> (2) Make sure you have DMA enabled for your hard drives. Research
>> first, then run 'hdparm' to test write speeds to see if hard drives
>> are the bottleneck.
> 
> DMA in my case, have nothing to do with it, it's SAS device.
> 
>> (3) Check your network: can you get faster transfer speeds with other 
>> app's? Make sure you've got the proper drivers for your NIC's...
> 
> I get much faster transfer on my NIC's, yes, i have tested with iperf.
> 

Have you seen the speed tricks from the list?

http://www.google.com/search?hl=en&q=speed+gigabit.ethernet+site%3Alists.linbit.com%2Fpipermail%2Fdrbd-user%2F&btnG=Google+Search

Oh, and this next guy _usually_ knows what he is talking about, though he was 
using an older drbd and you may need to figure out how some of the settings 
map to current versions: :)
http://lists.linbit.com/pipermail/drbd-user/2004-November/002241.html

And another recent speed thread:
http://lists.linbit.com/pipermail/drbd-user/2007-June/007010.html
http://lists.linbit.com/pipermail/drbd-user/2007-June/007033.html


Also, are you running multiple DRBD's?
If so do you have them all on one spindle or Array?
Are you using Sync groups?

What protocol (A,B,C) are you using (should not matter with initial sync, just 
being pedantic)?

a few more comments below

>> Good luck.
> 
> Thanks.
> 
>> Nate
>>
>>
>> On Tue, 23 Oct 2007, Igor Neves wrote:
>>
>>> Hi,
>>>
>>> I'm having problems with sync, using drbd 8.0.4, in centos 5.0.
>>>
>>> When i have to do one full sync, of the disc, it takes me 12 Hours
>>> (500G).

so you are saying ...
echo -en "scale=2\n(500*1024*1024)/(12*60*60)\n"|bc
.. you are seeing ~12MBytes/second going through?

Looks suspiciously like the 100Mbit network max.

try giving us `cat /proc/drbd` output so we are not making up numbers from 
yours. :)

>>>
>>> The 2 nodes are connected between Gigabit Ethernet, my question
>>> it's, what do i have to do, to make it faster? This time is
>>> insane! :)
>>>
>>> I know my disk does not write at Gigabit ethernet capacity, but
>>> indeed, lets say it writes at least at 50MBytes/sec.
>>>
>>> If we write data, at 50MBytes/sec it should take theoretically about
>>> 1.66 hours, its about 1h and 40 minutes or so.
>>>
>>> Of course the disk writes faster than that, at least in this case i
>>> have one SAS controller with 15k rpm disk's.
>>>
>>> I have in drbd.conf configured 'rate 100M'.
>>>
>>> I have drbd over the disk, and ext3 over it, but i think this is
>>> transparent for drbd, this should not take the drbd to slow.
>>>
>>> When it is doing the full sync, i see in '/proc/drbd' and it says
>>> it's copying at about 20K/sec, and since it is in bits, it is
>>> pretty slow.
>>>

IIRC, the data in /proc/drbd is in KBytes, so you are seeing 20KBytes/second 
(or if you did a little math wrong for us, 20MBytes/second).
The following is indicating ~15MBytes/seconds:
finish: 4:20:22h speed: 15,290 (15,374) K/sec


>>> This is bug? Or I am the bug? :) Since I'm new to drbd, i think the
>>> second option will be more likely then the first one! :D
>>>
>>> Gigabit Ethernet = 125MBytes/Sec
>>>
>>> Thanks for all the help.


-- 
Todd Denniston
Crane Division, Naval Surface Warfare Center (NSWC Crane)
Harnessing the Power of Technology for the Warfighter
http://www.crane.navy.mil/custfeedback



More information about the drbd-user mailing list