[DRBD-user] Page allocation failure (IPOIB, Infiniband, connected mode)
chibi at gol.com
Wed Mar 14 07:48:12 CET 2012
This is basically a repeat of:
32GB RAM, Debian Squeeze, 3.2 (debian backport) kernel, 8.3.12 DRBD,
IPOIB in connected mode with a 64k MTU. Just 2 DRBD resources.
After encountering this for the first time (never showed up in two weeks
of stress testing, which only goes to prove that real life just can't be
simulated) I found the above article and changed the following sysctls:
vm/min_free_kbytes = 262144
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 65536 87380 16777216
net.ipv4.tcp_wmem = 65536 65536 16777216
net/ipv4/tcp_mem is by default at "775464 1033954 1550928" which should be
After the changes I disconnected and reconnected the resources, thus hopefully applying those changes to the actual links.
And initially I couldn't reproduce the problem, but it showed up again last
Lars hinted at "atomic reserves" in his reply, which particular parameters
are we talking about here?
See this for the pertinent kernel log entry in my case:
Christian Balzer Network/Systems Engineer
chibi at gol.com Global OnLine Japan/Fusion Communications
More information about the drbd-user