[DRBD-user] slow sync speed of 12TB over 10GE

Robert Dunkley Robert at saq.co.uk
Mon Jan 19 11:11:34 CET 2009

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Hi Lars,

What are you using to interconnect the two DRBD systems? Loss of half of performance seems excessive. I usually get about 95% of the arrays sustained write performance and multiple arrays can synch at once without penalty.


-----Original Message-----
From: drbd-user-bounces at lists.linbit.com [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Tomasz Chmielewski
Sent: 19 January 2009 09:54
To: Lars Täuber
Cc: drbd-user at lists.linbit.com
Subject: Re: [DRBD-user] slow sync speed of 12TB over 10GE

Lars Täuber schrieb:

> 6553600000 bytes (6.6 GB) copied, 25.4397 s, 258 MB/s

> md4           72164,00         0,00    288656,00          0     288656
> The numbers are floating between ~18900 and ~23400.
> Should I really mount the device with a filesystem on it? The last tests were made without a filesystem too.

Linear writes to a raw device are different to writes to a filesystem 
- you don't have to update metadata, journal etc., so you can write faster
- raw device can not be fragmented, where a filesystem can (heads in 
your HDD won't jump here and there)

Technically speaking, such writes:

          [==============>.....] sync'ed: 76.4% (242356/1023804)K
          finish: 0:00:01 speed: 130,240 (130,240) K/sec

can be also fragmented as well - some area can be out of sync here, some 
area there - the head will jump.

Also, note that at the same time there can be real writes to that device 
(i.e., something writes to your /dev/drbdXY device) - it will impact the 
speed of that sync because those writes have to be synchronized as well; 
also HDD head will jump more often.

It's hard to tell if such difference (258 MB/s dd write speed versus 130 
MB/s DRBD sync) can be explained by only those items I mentioned.

Tomasz Chmielewski
drbd-user mailing list
drbd-user at lists.linbit.com

The SAQ Group

Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ
SAQ is the trading name of SEMTEC Limited. Registered in England & Wales
Company Number: 06481952

http://www.saqnet.co.uk AS29219

SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business.

Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support.

ISPA Member

Find us in http://www.thebestof.co.uk/petersfield

More information about the drbd-user mailing list