[DRBD-user] Summarizing DRBD memory usage

Lars Ellenberg lars.ellenberg at linbit.com
Thu Jul 24 12:43:52 CEST 2008

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Thu, Jul 24, 2008 at 11:02:16AM +0200, Iustin Pop wrote:
> There have been lately some very good threads about DRBD memory usage.
> I'd like to ask if, after this, my understanding about the total memory
> usage is correct.
> 
> If I understand correctly, we have:
>   - per-TB 32MB of memory used, statically, independent of over how many
>     DRBD minors it is spread
>   - a per-minor usage of up to:
>     - sndbuf-size for the TCP connection (can be ignored)
>     - PAGE_SIZE * max_buffers
>     - probably some other small values (kernel structures or such) which
>       can be ignored for normal cases
> 
> This means, a 1TB device with the defaults of sndbuf-size 132KB and
> max_buffers 2048 will use (at max, e.g. during resync):
>   - 32MB for bitmap
>   - 132KB for TCP connection
>   - 8MB for the buffers
> 
> I.e. this is ~40MB.
> 
> On the other hand, a 1TB device split over 8 x 128MB, will have 32MB +
> 8x8MB =~ 96MB.

the max-buffer setting limits the abount of in-flight data.
if nothing is in-flight, no such buffers are allocated.
if something is in-flight, these are allocated on the receiving side
to be able to submit it to the block device.

so "up to" max-buffers on the receiving side,
which on the sending side corresponds to
that many dirty (write-out) pages,
or to pages read in by the resync process.

we also have a small number of mempools, and a page pool emergency
reserve to not stall during write-out caused by hard memory pressure.

-- 
: Lars Ellenberg                           http://www.linbit.com :
: DRBD/HA support and consulting             sales at linbit.com :
: LINBIT Information Technologies GmbH      Tel +43-1-8178292-0  :
: Vivenotgasse 48, A-1120 Vienna/Europe     Fax +43-1-8178292-82 :
__
please don't Cc me, but send to list -- I'm subscribed



More information about the drbd-user mailing list