[DRBD-user] DRBD terrible sync performance on 10GigE

Digimer lists at alteeve.ca
Fri Dec 11 16:30:21 CET 2015

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On 11/12/15 03:04 AM, Harald Dunkel wrote:
> Hi Lars,
> On 12/04/2015 05:04 PM, Lars Ellenberg wrote:
>> You are not supposed to disable the resync controller,
>> you are supposed to correctly use it.
>> https://blogs.linbit.com/p/443/drbd-sync-rate-controller-2/
>> ... you should:
>>     * set c-plan-ahead to 20 (default with 8.4),
>> 	or more if there’s a lot of latency on the connection (WAN
>> 	link with protocol A); or less, if you want to have it
>> 	react faster to changes
>>     * leave the fixed resync rate (the initial guess for the controller)
>>       at about 30% or less of what your hardware can handle;
>>     * set c-max-rate to 100% (or slightly more) of
>>       what your hardware can handle;
>>       (the default is 100M, which was the effective limitation in this case)
>>     * set c-fill-target to the minimum (just as high as necessary)
>>       that gets your hardware saturated, if the system is otherwise idle.
>> Respectively, figure out the maximum possible resync rate in your
>> setup while the system is idle, then set c-fill-target to the
>> minimum setting that still reaches that rate.
>> And finally, while checking application request
>> latency/responsiveness, tune c-min-rate to the maximum that still
>> allows for acceptable responsiveness.
>> You may need to adjust max-buffers and/or tcp send/receive buffer
>> sizes as well.
> Sorry to say, but this is way too complex for an ordinary user
> (speaking just for myself, of course). It gets even more complex
> if you have several drbd resources. The man page doesn't help in
> this case, either.

It's an unfortunate side effect of HA. As much as the devs try, there is
an unavoidable base line of complexity.

This is also true with performance tuning pretty much anything in Linux.
The default values try to aim for the best level of trade off, but work
loads and hardware vary so much that it's impossible to have a default
"best config". To tune, you need to understand your hardware, understand
your workload and understand what you get and lose when you adjust
various tunables.

Please grab a $drink, settle in and take some time to read. As you see
here, the community will do its best to help, but you will need to do
the heavy lifting because you understand your environment.

> How about some recommendations with real numbers?
> BTW, the man page for version 8.9.4 mentions that the default for
> c-plan-ahead is 0. If I got this correctly this means that the
> dynamic resync speed controller is disabled by default. Is this
> correct?
> Regards
> Harri
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user

Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?

More information about the drbd-user mailing list