[DRBD-user] DRBD 84 resyncer runs out of memory

freebird linbit at gelin.co.uk
Fri Feb 12 17:10:14 CET 2016

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hi, I have a memory usage problem with the resync sender on a single resource.  If the total size of the out of sync blocks exceeds the memory size when drbd connects, the sync progresses until it exhausts the memory leading to the OOM killer kicking in (too slowly to make any difference) followed by a kernel panic.  Everything else including replication of very large files that exceed the memory size, works fine.

I've tested with out of sync sizes up to just short of the total memory when the resync succeeds and observed that the memory used increases in line with sync'ed blocks i.e. the resync process seems to be allocating memory per block and doesn't release or reuse it ... only when the resync completes does the memory get released.  The strange thing is that I can't find which specific process is retaining the memory ... nothing shows up in top, slabtop or the process table.

Platform
--------
RHEL: 6.7
kernel: 2.6.32-573.18.1.el6.x86_64
kmod-drbd84: 8.4.7-1_1
drbd84-utils: 8.9.5-1
cores: 20
ram: 32GB
dedicated gigabit sync link
resource size: 25TB

Some (what's relevant I think) config options
---------------------------------------------
minor-count 12;
protocol C;
max-buffers 4096;
max-epoch-size 4096;
al-extents 257;
resync-rate 32M;

Nige

-- 
Experience is something you don't get until just after you need it.



More information about the drbd-user mailing list