[DRBD-user] Slow initial full sync

[aurelien panizza]

Hi,

Thank you for your help.

Here are the results of ls -1 /sys/block/*/queue/max_hw_sectors_kb | xargs
-i bash -c 'echo -n "{} " ; cat {}'

Server 1
/sys/block/dm-0/queue/max_hw_sectors_kb 32767
/sys/block/dm-1/queue/max_hw_sectors_kb 32767
/sys/block/dm-2/queue/max_hw_sectors_kb 32767
*/sys/block/drbd0/queue/max_hw_sectors_kb 1024*
/sys/block/loop0/queue/max_hw_sectors_kb 127
/sys/block/loop1/queue/max_hw_sectors_kb 127
/sys/block/loop2/queue/max_hw_sectors_kb 127
/sys/block/loop3/queue/max_hw_sectors_kb 127
/sys/block/loop4/queue/max_hw_sectors_kb 127
/sys/block/loop5/queue/max_hw_sectors_kb 127
/sys/block/loop6/queue/max_hw_sectors_kb 127
/sys/block/loop7/queue/max_hw_sectors_kb 127
/sys/block/md0/queue/max_hw_sectors_kb 32767
/sys/block/md1/queue/max_hw_sectors_kb 32767
/sys/block/ram0/queue/max_hw_sectors_kb 512
/sys/block/ram10/queue/max_hw_sectors_kb 512
/sys/block/ram11/queue/max_hw_sectors_kb 512
/sys/block/ram12/queue/max_hw_sectors_kb 512
/sys/block/ram13/queue/max_hw_sectors_kb 512
/sys/block/ram14/queue/max_hw_sectors_kb 512
/sys/block/ram15/queue/max_hw_sectors_kb 512
/sys/block/ram1/queue/max_hw_sectors_kb 512
/sys/block/ram2/queue/max_hw_sectors_kb 512
/sys/block/ram3/queue/max_hw_sectors_kb 512
/sys/block/ram4/queue/max_hw_sectors_kb 512
/sys/block/ram5/queue/max_hw_sectors_kb 512
/sys/block/ram6/queue/max_hw_sectors_kb 512
/sys/block/ram7/queue/max_hw_sectors_kb 512
/sys/block/ram8/queue/max_hw_sectors_kb 512
/sys/block/ram9/queue/max_hw_sectors_kb 512
/sys/block/sda/queue/max_hw_sectors_kb 32767
/sys/block/sdb/queue/max_hw_sectors_kb 32767
/sys/block/sr0/queue/max_hw_sectors_kb 128

Server 2
/sys/block/dm-0/queue/max_hw_sectors_kb 32767
/sys/block/dm-1/queue/max_hw_sectors_kb 32767
/sys/block/dm-2/queue/max_hw_sectors_kb 32767
*/sys/block/drbd0/queue/max_hw_sectors_kb 1024*
/sys/block/loop0/queue/max_hw_sectors_kb 127
/sys/block/loop1/queue/max_hw_sectors_kb 127
/sys/block/loop2/queue/max_hw_sectors_kb 127
/sys/block/loop3/queue/max_hw_sectors_kb 127
/sys/block/loop4/queue/max_hw_sectors_kb 127
/sys/block/loop5/queue/max_hw_sectors_kb 127
/sys/block/loop6/queue/max_hw_sectors_kb 127
/sys/block/loop7/queue/max_hw_sectors_kb 127
/sys/block/md0/queue/max_hw_sectors_kb 32767
/sys/block/md1/queue/max_hw_sectors_kb 32767
/sys/block/ram0/queue/max_hw_sectors_kb 512
/sys/block/ram10/queue/max_hw_sectors_kb 512
/sys/block/ram11/queue/max_hw_sectors_kb 512
/sys/block/ram12/queue/max_hw_sectors_kb 512
/sys/block/ram13/queue/max_hw_sectors_kb 512
/sys/block/ram14/queue/max_hw_sectors_kb 512
/sys/block/ram15/queue/max_hw_sectors_kb 512
/sys/block/ram1/queue/max_hw_sectors_kb 512
/sys/block/ram2/queue/max_hw_sectors_kb 512
/sys/block/ram3/queue/max_hw_sectors_kb 512
/sys/block/ram4/queue/max_hw_sectors_kb 512
/sys/block/ram5/queue/max_hw_sectors_kb 512
/sys/block/ram6/queue/max_hw_sectors_kb 512
/sys/block/ram7/queue/max_hw_sectors_kb 512
/sys/block/ram8/queue/max_hw_sectors_kb 512
/sys/block/ram9/queue/max_hw_sectors_kb 512
/sys/block/sda/queue/max_hw_sectors_kb 32767
/sys/block/sdb/queue/max_hw_sectors_kb 32767
/sys/block/sr0/queue/max_hw_sectors_kb 128

This is the same value as my other DRBD cluster. Is this an expected value ?


I tried to run again the disk performance using oflag=direct (ran multiple
times)

dd if=/dev/zero of=/home/oracle/output oflag=direct bs=384k count=1k; rm -f
1024+0 records in
1024+0 records out
402653184 bytes (403 MB) copied, 2.26735 s, 178 MB/s

Any other thoughts about my really slow sync ?


Regards

2014-10-29 21:33 GMT+11:00 Lutz Vieweg <lvml at 5t9.de>:

> On 10/29/2014 05:34 AM, aurelien panizza wrote:
>
>> I've got two servers connected via a switch with a dedicated 1Gb NIC. We
>> have a SSD raid 1 on both
>> servers (software RAID on Redhat 6.5).
>>
>
>  Full sync between the primary and the secondary is from 3MB/s to 12MB/s
>>
>
>  iostat report 100%util on /dev/drbd0 on the secondary.
>>
>
> Check whether there's a block device with a low max_hw_sectors_kb value
> in your device stack:
>
>  ls -1 /sys/block/*/queue/max_hw_sectors_kb | xargs -i bash -c 'echo -n
>> "{} " ; cat {}'
>>
>
> If you see small values for max_hw_sectors_kb in one of these, chances
> are that just inefficient smalls blocks are transferred - you can
> also find a discussion on this in the DRBD mailing list archive, just
> search for max_hw_sectors_kb.
>
> Regards,
>
> Lutz Vieweg
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20141103/0e14faef/attachment.htm>