[DRBD-user] Large drbd devices vs vmalloc workarounds

Lars Ellenberg Lars.Ellenberg at linbit.com
Mon Apr 25 15:10:53 CEST 2005

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


/ 2005-04-24 23:32:22 +0200
\ Leroy van Logchem:
> Hi,
> 
> I have two drbd devices working fine. Each 1.4 tB running on rhes4
> 2.6.9-5.0.5.ELsmp using drbd-0.7.10. Adding a thirth device hangs the
> module:
> 
> /allocation failed: out of vmalloc space

if you need that much storage, use a 64bit architecture,
and whoops, your problems (should) have vanished.

> /After some research I changed the grub and kernel config like this:
> 
> <grub.conf>
> title Red Hat Enterprise Linux ES (2.6.9-5.0.5.ELsmp)
>         root (hd0,0)
>         uppermem 524288
>         kernel /boot/vmlinuz-2.6.9-5.0.5.ELsmp ro root=LABEL=/ rhgb quiet vmalloc=192m
>         initrd /boot/initrd-2.6.9-5.0.5.ELsmp.img
> 
> The servers have 1GB memory but grub is told to load the initrd *past* the vmalloc by the use of uppermem (512 * 1024).
> After this the vmalloc=192m option doesnt hangup the loader anymore and /proc/meminfo VmallocTotal increased from 106488 kB to 
> 172024 kB.

> Q: What are the memory requirements for this configuration?

for each device, drbd will (try to) allocate X MB of bitmap,
plus some constant amount (<1MB).
X = storage_size_in_GB/32, so 1 TB storage -> 32 MB bitmap

> Q: Will the memory footprint increase during production use?
no.

but again, use a 64bit architecture if you need so much storage.
you will be much cheerier. really.

	Lars



More information about the drbd-user mailing list