Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Tue, 8 Sep 2009 13:21:50 +0900, Christian Balzer <chibi at gol.com> wrote:
> common {
> syncer {
> rate 200M;
> al-extents 3833;
> }
> disk {
> no-disk-barrier;
> no-disk-flushes;
> no-md-flushes;
> use-bmbv;
> }
> net {
> sndbuf-size 0;
> rcvbuf-size 0;
> max-buffers 16384;
> unplug-watermark 16384;
> max-epoch-size 16384;
> }
> ---
>
> Note the 3833 for extents, the website/manpage say 3843 but DRBD
insisted
> on 10 less. ^_-
> The initial sync hovered around 58MB/s while I had rcvbuf-size
unspecified
> in the config and thus at the default. I tried various variations of
> sndbuf-size including 0 (auto-tune), but they had at best minor effects.
>
> After changing both of them to 0, speeds went up 113MB/s (both sync and
> later filesystem tests). I guess this should be a major hint to people
> using modern kernels, the defaults and manual tuning made things
actually
> a lot worse.
>
> I'm still missing about 80MB/s performance wise and wonder where they
> could be hiding. And yes, I did change the scheduler to deadline, though
> I expect CFQ on this kernel to be doing well enough, too.
> TCP_Cork on or off had no measurable effects either, btw.
>
> Will experiment more, but after the surprise speed doubling by just
> letting the kernel handle the net buffer section I'm wondering what else
> is there to poke that could yield a similar improvement.
Christian,
If nobody else has yet, I wanted to thank you for your post. I had become
increasingly frustrated with the performance of my DRBD system. Only
52-55MB/s on hardware that easily supported 180-200MB/s was frustrating
and
aggravating. The fact that so many others were experiencing the same
problem with no "fix" on the list was even more frustrating. After all
the
misdirection on what could be the problem, I was contemplating a new disk
subsystem on my secondary. After setting the sndbuf-size and rcvbuf-size
to 0, tests using "dd" on the filesystem jumped to 122MB/s. Not too shabby
with only a single Gig cross-connect. Now I'll be looking into bonded
interfaces to see how close to the actual hardware speed I can get.
Let us know if you find any other hidden gems that Linbit wants to keep
hidden from us.
Thanks!
Ken