[DRBD-user] LVM (not cLVM) on pri/pri?

Lars Ellenberg lars.ellenberg at linbit.com
Wed Sep 19 16:12:23 CEST 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Wed, Sep 19, 2007 at 05:33:09AM -0700, Kelly Byrd wrote:
> >
> > if it is static, anyways,
> > why not have drbd on top of lvs,
> > and run "classical" Primary/Secondary per drbd/lv?
> > 
> I would like this bug I have concerns about scaling 
> to 24 drbds, see below.
> So let's say I was talking about"
> storage -> 1 drbd -> 24 LVs.
> I belive you're suggesting.
> storage -> 24 LVs -> 24 drbds (on per LV)
> I asked about this on the list a few days ago and got no response. I'll
> ask the same questions here:
> - Are there problems with this many drbds? What's the overhead of all those
> drbd worker threads and all those TCP connections vs. just one or two?

if you have SMP (you have), doing independend tcp connections will
likely improve "felt" responsiveness for single lvs.
there has been some patch by some hosting provider allowing for hundreds
of drbd, with current drbd you are limited to the 255 minors, provided
without module parameter "minor_count" you are limited to the default of
32 minors.
I don't think having hundreds of drbd is very useful.
but aparently it did the job for them...

you may be the first one to trigger some undiscovered race conditions
only occuring when using exactly 24 drbd,
when configuring in reverse order on the second full moon of the year.
or some such.
but I'd expect it to "just work".

> - Working out the sync groups would be kind of tough. My underlying storage
> is a single big RAIDed mass of disks. Would I be better off to let all sync
> serially or randomly pick groups of three to keep the RAID busy?

I'd try that out.
DRBD resync should be able to saturate it going one at a time,
so I would probably serialize all of them.

watch out for the memory foot print of in-flight io on the receiving
side (Secondary), you can tune this with max-buffers.
this is per drbd, and counted in "pages", so max-buffers 2048 means
possibly 8MB in-flight io, 24*8 is 200 MB... if you tune max-buffers up
to achieve better throughput, this can increase quickly if all lvs
suddenly do streamin io (drbd resync) at the same time...

> - Is there any way to specify a global sync rate so that if only one drbd
> is syncing it'll take the full sync rate, but 6 or 7 drbds syncing
> will collectively share that same sync rate? 

no, we don't have that.
yet.  go, request that feature at sales :)

: Lars Ellenberg                           http://www.linbit.com :
: DRBD/HA support and consulting             sales at linbit.com :
: LINBIT Information Technologies GmbH      Tel +43-1-8178292-0  :
: Vivenotgasse 48, A-1120 Vienna/Europe     Fax +43-1-8178292-82 :
please use the "List-Reply" function of your email client.

More information about the drbd-user mailing list