Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello Ingard,
On Fri, 04 Apr 2008 11:29:46 +0200 Ingard Mevåg wrote:
> I've tried to tune the parameters of the config file and i achieve
> somewhat different results:
>
> The best read speeds i've seen is with the default config, I got close
> to 60000K/sec vs bare metal ~100000K/sec
>
> The best write speeds i've seen is with the following config:
> common {
> protocol C;
> syncer {
> rate 160M;
> al-extents 1801;
> }
> }
>
> net {
> allow-two-primaries;
> sndbuf-size 2M;
> max-buffers 16000;
> max-epoch-size 16000;
> unplug-watermark 16000;
>
These are (except for the 2 primaries) nearly exactly the same settings I
came up with after a lot of testing.
In my case it's a dual quad core box with 24GB RAM and 8 1TB SATA drives
in a RAID-5 that give me about 120MB/s writes on the "bare" MD device.
The first try with default values on DRBD gave me about 41MB/s writes and
a big "WTH?" feeling.
After tuning all the above parameters I wound up with about 60MB/s.
Taking the advice from this ML I moved the meta-data from internal to an
external RAID-10 partition (since writes on RAID-5 are sucky by nature)
and got to a much more satisfying 75MB/s. This is also roughly the speed
I see for syncs.
Since you are already on a RAID-10, no need for that. Of course if you
have additional drives not in that raid, putting the meta-data there
might help.
In addition to that I turned on "use-bmbv" since nobody here actually
managed to give me a good reason (other than cargo-cult quoting the manual)
not to with my particular setup (identical disks and all on Linux MD) and
it made about a 5MB/s difference in writes...
> where i've gotten ~ 65000K/sec as opposed to ~ 75000K/sec bare metal.
>
> Anyone have any suggestions to what parameter settings I should try in
> order to achieve a better read performance from the drbd resource?
>
I don't think there is much more you can do here. I would reckon that
being able to crank up the buffers even more (like doubling them) might
help, but the max value of 20k for unplug-watermark puts a stop on that.
I have not seen anybody quoting higher numbers combined with actual setups
and configs here either.
My personal conclusion is that if one wants to build a high-speed
(writes) DRBD setup where speed is definitely more important than
storage capacity, go for a RAID-10 (preferably the Linux MD RAID-10
with far or offset replication) with as many and as fast drives you
can fit. For extra oomph, consider a battery backed ramdisk or solid
state drive to hold the meta-data.
Regards,
Christian
--
Christian Balzer Network/Systems Engineer NOC
chibi at gol.com Global OnLine Japan/Fusion Network Services
http://www.gol.com/