[DRBD-user] MySQL-over-DRBD Performance

Art Age Software artagesw at gmail.com
Wed Dec 19 18:57:22 CET 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Thanks for looking at this...

> you have a very interesting maximum and a huge deviation there.

Does this reflect a problem with my TCP stack? What might be causing
the huge max?

> but, lets use the 0.180 ms average rtt of 4k packets.
> averages from the dd commands below are
>  drbd disconnected: 0.310 ms per 4k request
>  drbd connected     1.170 ms per 4k request
>  non-drbd           0.300 ms per 4k request
> I've also already seen non-drbd be slower than
> drbd-unconnected on the same hardware,
> there are funny effects in play.
> but they are close within 3%, this is expected.

Hmmm, it looks to me like non-drbd is **faster** than
drbd-disconnected from my numbers, which I would expect. Am I

> however your drbd-connected seems bad.
> from ping rtt and non-drbd numbers we'd expect that
> latency of drbd connected should be ~ 0.480 ms.
> your measurement indicates it is worse than this
> expectation by a factor of 2.5.

Yes, this is the crux of the problem I am experiencing - now confirmed
with MySQL out of the equation.

> in all setups I have tuned so far,
> the actual (measured) latency of drbd,
> and the rough estimate given by said ping and dd commands
> are very close.
> so I suspect your secondaries ("node2") io subsystem is slower.
> please verify.

The 2 nodes are identical - right down to the io subsystem (identical
RAID-10 hardware with battery-backed write-back cache enabled and
identical model hard drives).

> other than that, pinning of drbd related threads to one CPU,
> preferably the same where you pinned the NIC driver irq to,
> could help to reduce latency.

I have not pinned NIC driver IRQs. (I don't know how.) I have pinned
the DRBD-related threads to a single CPU core and the test results
reflect that configuration.

I'm really at a loss here. Do you have any other suggestions for
getting to the bottom of this?

Should I disable irqbalance daemon? (I tried it and it seemed to make
no difference).

Should I disable SELinux?



More information about the drbd-user mailing list