Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Wed, Sep 19, 2007 at 05:33:09AM -0700, Kelly Byrd wrote: > > > > > if it is static, anyways, > > why not have drbd on top of lvs, > > and run "classical" Primary/Secondary per drbd/lv? > > > I would like this bug I have concerns about scaling > to 24 drbds, see below. > > > So let's say I was talking about" > storage -> 1 drbd -> 24 LVs. > > I belive you're suggesting. > > storage -> 24 LVs -> 24 drbds (on per LV) > > > I asked about this on the list a few days ago and got no response. I'll > ask the same questions here: > > - Are there problems with this many drbds? What's the overhead of all those > drbd worker threads and all those TCP connections vs. just one or two? if you have SMP (you have), doing independend tcp connections will likely improve "felt" responsiveness for single lvs. there has been some patch by some hosting provider allowing for hundreds of drbd, with current drbd you are limited to the 255 minors, provided without module parameter "minor_count" you are limited to the default of 32 minors. I don't think having hundreds of drbd is very useful. but aparently it did the job for them... you may be the first one to trigger some undiscovered race conditions only occuring when using exactly 24 drbd, when configuring in reverse order on the second full moon of the year. or some such. but I'd expect it to "just work". > - Working out the sync groups would be kind of tough. My underlying storage > is a single big RAIDed mass of disks. Would I be better off to let all sync > serially or randomly pick groups of three to keep the RAID busy? I'd try that out. DRBD resync should be able to saturate it going one at a time, so I would probably serialize all of them. watch out for the memory foot print of in-flight io on the receiving side (Secondary), you can tune this with max-buffers. this is per drbd, and counted in "pages", so max-buffers 2048 means possibly 8MB in-flight io, 24*8 is 200 MB... if you tune max-buffers up to achieve better throughput, this can increase quickly if all lvs suddenly do streamin io (drbd resync) at the same time... > - Is there any way to specify a global sync rate so that if only one drbd > is syncing it'll take the full sync rate, but 6 or 7 drbds syncing > will collectively share that same sync rate? no, we don't have that. yet. go, request that feature at sales :) -- : Lars Ellenberg http://www.linbit.com : : DRBD/HA support and consulting sales at linbit.com : : LINBIT Information Technologies GmbH Tel +43-1-8178292-0 : : Vivenotgasse 48, A-1120 Vienna/Europe Fax +43-1-8178292-82 : __ please use the "List-Reply" function of your email client.