[DRBD-user] Having trouble with performance on hardware RAID + 10gbps

Digimer lists at alteeve.ca
Thu Jan 9 07:51:28 CET 2014

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On 09/01/14 01:39 AM, Digimer wrote:
> Hi all,
>    I've been trying to do some tuning with DRBD. The network interface
> sustains 1.225 GB/sec tested with iperf (multiple tests, hf and fd). The
> disks themselves sustain 1.2 GB/sec sequential writes (975 GB write via
> 'dd if=/dev/zero of=/dev/sda5 bs=4M), tested repeatedly and on both
> nodes. The storage array is an 8-disk (146 GB 15krpm 2.5" SAS disks on
> an LSI RAID controller with a 2208 ASIC).

Answering some of my own questions; sharing what I've done/tested;

I saw (http://www.drbd.org/users-guide/ch-benchmark.html):

For freshly enabled DRBD devices, it is normal to see significantly 
reduced performance on the first dd run. This is due to the Activity Log 
being "cold"

So I am going to let the initial sync finish and try again.

I also noticed that I had enabled 'data-integrity-alg' and re-enabled 
'disk-flushes no;' and 'md-flushes no;'. I've disabled these and it's 
already dd'ing at ~650 MB/sec. This is still half of what the 
storage/network can handle, but it's a mighty scene better than before. :)

I'll post more as I experiment/learn. In the meantime, I would very much 
love to hear feedback, specially from folks with experience tuning for 
LSI-based controllers and 10 gbps ethernet networks.


Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without 
access to education?

More information about the drbd-user mailing list