[DRBD-user] 2.6.33 panic on resync

Lars Ellenberg lars.ellenberg at linbit.com
Sat Mar 6 17:40:56 CET 2010

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Sat, Mar 06, 2010 at 05:12:33PM +0100, Lars Ellenberg wrote:
> On Sat, Mar 06, 2010 at 04:42:20PM +0100, Florian Haas wrote:
> > On 03/06/2010 03:31 PM, Matteo Tescione wrote:
> > > Hello,
> > > 
> > > I succesfully compiled drbd-8.3.7 against linux-2.6.33 with builtin drbd module running centos5.3 x86_64.
> > 
> > What do you mean you compiled 8.3.7 "against" 2.6.33? Did you do an
> > out-of-tree module build from current git? If so, why? Or did you just
> > roll your kernel with DRBD enabled?
> > 
> > > However i'm experiencing kernel panic few seconds after a full resync starts.
> > > Attached is a screnshot of the panic.
> > 
> > Even though the picture isn't exactly of prime quality,
> 
> Pictures of text should be png, if possible.
> 
> > the message is
> > plain English: "out of memory and no killable processes". Fix your
> > memory pressure?
> 
> 
> Let's do an example.
> I've got some "embeded router" thingy, with 32 Meg RAM,
> which is plenty for what it usually does.
> 
> It sports a USB "storage link", and "hacked up" linux firmware.
> So I could, in theory, add DRBD to it,
> and hook up a 1 TB external usb drive.
> 
> In practice, as DRBD uses a fixed 4k storage -> 1 bit granularity,
> and still holds the full bitmap in memory,
> we need (1 ** 40 byte / 1 ** 12 byte ) bits,
> which is (1 ** 25 byte): 32 Megs of unswappable in core memory.
> 
> As Homer puts it: D'oh.
> 
> We used to do a sanity check for available memory in kernel,
> but the necessary functions have been made unavailable or changed,
> so we dropped that again.
> 
> We can easily add a sanity check to drbdsetup, which is
> what will do short term, to prevent you from killing
> your box that easily.
> 
> We also currently rework our meta data layout and handling,
> to make the bitmap granularity as well as the in-core
> amount of pages configurable. But that won't happen overnight.
> 
> So: this is not a "real bug". Just a "usage bug".

Oh, I just realized: you said it was breaking during resync...

Well, probably the bitmap was "just fitting",
and the additional buffers allocated for resync traffic
killed your box then.

See "Summarizing DRBD memory usage"
http://marc.info/?l=drbd-user&m=121689625213486

And "memory requirements for DRBD"
http://marc.info/?l=drbd-user&m=123273268107703

You should know, you are with us since almost three years, iirc!
Thanks for that, btw ;-)

PS we also do consulting...

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
__
please don't Cc me, but send to list   --   I'm subscribed



More information about the drbd-user mailing list