[DRBD-user] problem with 3x 1,5TB partitions

Ralf Gross Ralf-Lists at ralfgross.de
Mon Nov 13 11:05:03 CET 2006

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Me said:
>> * parted output:
>>
>> Disk geometry for /dev/sdf: 0kB - 4500GB
>> Disk label type: gpt
>> Number  Start   End     Size    File system  Name
>> Flags
>> 1       17kB    1500GB  1500GB
>> 2       1500GB  3000GB  1500GB
>> 3       3000GB  4500GB  1500GB
>
> Ok, I just checked  if it works with a smaller 600GB partition at the
> end.
>
> Number  Start   End     Size    File system  Name Flags
> 1       17kB    1500GB  1500GB
> 2       1500GB  3000GB  1500GB
> 3       3000GB  3600GB  600GB
>
> /etc/init.d/drbd start
> Starting DRBD resources:    [ d0 d1 d2 s0 s1 s2 n0 n1 n2 ].
> ..........
>
> I'm not sure about that. Am I running in a drbd limitation? I thought
> the limit of a _single_ drbd device would be ~3,8TB.

Next step: I used the functionality of the RAID Controller to create <2TB
volumes. Thus I now have 3 disks (sde/sdf/sdg), each with 1,5TB.

drbd still fails to start r2.

Starting DRBD resources:    [ d0 d1 d2 ioctl(,SET_DISK_CONFIG,) failed:
Cannot allocate memory

cmd /sbin/drbdsetup /dev/drbd2 disk /dev/sdg1 internal -1
--on-io-error=detach  failed!

/var/log/messages is a bit more verbose:

42949598.710000] allocation failed: out of vmalloc space - use
vmalloc=<size> to increase size.
[42949598.710000] drbd2: OUT OF MEMORY! Could not allocate bitmap! Set
device size => 0

This seems to be an FAQ:

http://www.linux-ha.org/DRBD/FAQ#head-c6586035cbdd5cdae726b02406b838ee2fa56eae

| What is warning: ''out of vmalloc space''
|
| For each device, drbd will (try to) allocate X MB of bitmap, plus some
| constant amount (<1MB). X = storage_size_in_GB/32, so 1 TB storage -> 32
| MB bitmap.
|
| By default Linux allocates 128MB to Vmalloc. For systems using more than
| 4TB, this may cause an issue.
|
| If you get the following error message in /var/log/messages, Try a Linux
| 2.6 hugemem kernel.
|
| kernel: allocation failed: out of vmalloc space - use vmalloc=<size> to
| increase size.

I tried this two kernel images that are available for ubuntu 6.06.

2.6.15-26-server
2.6.15-27-686

I can't find anything about hugemem support on the ubuntu web pages or the
ubuntu wiki.

Anyone using ubunto or debian? Do I have to compile a custom kernel to get
this working?

Ralf




More information about the drbd-user mailing list