[DRBD-user] Speeding up sync rate on fast links and storage

Gennadiy Nerubayev parakie at gmail.com
Tue Jan 13 19:06:20 CET 2009

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Fri, Dec 19, 2008 at 1:39 PM, Lars Ellenberg
<lars.ellenberg at linbit.com>wrote:

> On Fri, Dec 19, 2008 at 09:24:32AM -0500, Gennadiy Nerubayev wrote:
> > On Thu, Dec 18, 2008 at 1:50 PM, Lars Ellenberg <
> lars.ellenberg at linbit.com>
> > wrote:
> uh. oh.
> I have to admit that this was probably not really realistical.
> sort of only writing 500MB (odirect, "synchronously"), as that
> was what fit into the controller cache...
> and it finished subsecond. that's where the number comes from
>  ;)
> don't have a real storage backend in the lab capable of sustained
> writes in that performance range. (yet.)
> but, I think nothing special, actually,
> it was jumbo frames, disabling flow control, and huge max-buffers
> and the like, that did the trick, mostly, as well as allowing more than
> one core (as one single cpu was maxed out sometimes).

Small update:

500MB/s makes sense if it's a single burst. What I'm finding is that during
a long sync, the speed fluctuates wildly, even though neither the network
link nor the storage exhibit such fluctuations on their own. I made a graph
showing this effect during a sync lasting ~40 minutes. A script ran cat
/proc/drbd ran every second, taking the first speed value. The average after
the first minute or two stabilized at ~385MB/s:

http://www.panix.com/~jasonm/drbdspeed-big.jpg (note: large image)

There's a definite pattern, so the question is why is the sync speed
(mis)behaving like that..


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20090113/6e490991/attachment.htm>

More information about the drbd-user mailing list