[DRBD-user] The need for Speed 2

Philipp Reisner philipp.reisner at linbit.com
Thu Jan 13 11:50:30 CET 2005

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Hardware this time:
2 Sun Fire v20z, which are dual Opteron (250) machines with 4GB Ram.

The Raid-controller identifies itself as LSI Logic / Symbios Logic 53c1030 
PCI-X Fusion-MPT Dual Ultra320 SCSI, and some nice SCSI disks. The
disks set up as RAID1 array.

For the courious I am using a inofficial Debian Sarge port to amd64
as distribution. LVM2 is used as a middle layer, DRBD on lop of LVs.

Raw write performance to the LVs:
59.69 MB/sec (3221225472 B / 00:51.468961)
60.54 MB/sec (3221225472 B / 00:50.743192)
60.71 MB/sec (3221225472 B / 00:50.603030)

Once again I was curious how inhomogen the performance on the
disks is, and here are the results from an other (later allocated)
LV are:
56.07 MB/sec (3221225472 B / 00:54.784013)
56.60 MB/sec (3221225472 B / 00:54.279036)
57.06 MB/sec (3221225472 B / 00:53.838054)

Resync done (total 59 sec; paused 0 sec; 53316 K/sec)
DRBD's syncer ran on average in the area of >50MB/sec, afterwards
I limited it to 15MB/sec for production purposis.

Here are the values for mirrored writes. Protocol C
with an MTU of 1500.
38.88 MB/sec (3221225472 B / 01:19.004518)
38.12 MB/sec (3221225472 B / 01:20.589348)
38.39 MB/sec (3221225472 B / 01:20.020814)

Then I changed the MTU of the networking cards to 6000,
CPU ussage on the secondary side dropped from about 18 %
to 9% . Performance is abot the same:
39.64 MB/sec (3221225472 B / 01:17.488087)
38.48 MB/sec (3221225472 B / 01:19.824027)
38.41 MB/sec (3221225472 B / 01:20.020814)

These values were quite lower than what I expected, since
on the last pair of machines I did test on, DRBD's overhead
was lower than the inhomogenity of the disks.
On this pair of machines we get with DRBD only 65% of the
performance we have without DRBD.

A hint where we loose are the numbers from writing on DRBD
in disconnected mode:
41.45 MB/sec (3221225472 B / 01:14.113481)
41.05 MB/sec (3221225472 B / 01:14.829909)
41.09 MB/sec (3221225472 B / 01:14.763764)

DRBD instructs the IO subsystem to submit 4KB at most in a
single BIO, while LVM most probabely allows bigger BIOs. Maybe
this is an area where we will optimize/improve DRBD in 
the future...
OTOH it is interesting that the syncer was able to do 
more than 41MB/sec ...

Interesting stuff, unforunately these machines go to some
data center by the end of next week, then I have to use my
rather old development machines again....


: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :

More information about the drbd-user mailing list