Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
I've been running DRBD set up for 3 months, and recently noticed high
kernel memory usage on the secondary machines.
The secondary machine runs very light user applications, but total memory
usage reaches as much as 60G.
Is there a known issue with kernel message leak? Attaching top, slabtop and
meminfo from my backup machine. You can see that the processes from *top* show
very light RES size, but the system memory usage reaches 63 G. Can you
identify something obvious?
top:
last pid: 5836; load avg: 0.11, 0.17, 0.12; up 250+16:35:23
19:12:32
326 processes: 1 running, 325 sleeping
CPU states: 0.0% user, 0.0% nice, 0.0% system, 100% idle, 0.0% iowait
Kernel: 106 ctxsw, 1019 intr
Memory: 63G used, 426M free, 52G buffers, 179M cached
Swap: 116K used, 8000M free
PID USERNAME THR PRI NICE SIZE RES SHR STATE TIME CPU COMMAND
25785 ganglia 1 15 0 140M 8460K 3476K sleep 88:42 0.00% gmond
4730 root 1 16 0 88M 3368K 2636K sleep 0:00 0.00% sshd
4732 lin 1 15 0 88M 1836K 1084K sleep 0:00 0.00% sshd
4733 lin 1 16 0 65M 1596K 1272K sleep 0:00 0.00% bash
7523 root 1 15 0 65M 1596K 1272K sleep 0:00 0.00% bash
5500 root 1 15 0 61M 1208K 644K sleep 2:20 0.00% sshd
5834 root 1 15 0 61M 848K 336K sleep 0:00 0.00% crond
8785 root 1 16 0 61M 1024K 516K sleep 0:03 0.00% crond
7493 root 1 15 0 51M 1372K 1036K sleep 0:00 0.00% login
5066 root 3 20 0 28M 576K 448K sleep 0:00 0.00%
brcm_iscsiuio
8886 root 1 15 0 23M 1984K 1464K sleep 0:00 0.00% ntpd
1798 root 1 11 -4 12M 776K 456K sleep 0:00 0.00% udevd
5072 root 1 5 -10 12M 4452K 3164K sleep 0:00 0.00% iscsid
5071 root 1 18 0 12M 652K 416K sleep 0:00 0.00% iscsid
5718 lin 1 15 0 11M 1152K 848K run 0:00 0.00% top
12349 root 1 15 0 11M 1532K 612K sleep 2:54 0.00% syslogd
1 root 1 15 0 10M 752K 632K sleep 4:17 0.00% init
5835 root 1 19 0 8688K 1072K 924K sleep 0:00 0.00% sh
7301 root 1 19 0 3808K 532K 448K sleep 0:00 0.00% mingetty
7300 root 1 18 0 3808K 532K 448K sleep 0:00 0.00% mingetty
7299 root 1 17 0 3808K 532K 448K sleep 0:00 0.00% mingetty
7298 root 1 16 0 3808K 532K 448K sleep 0:00 0.00% mingetty
7302 root 1 18 0 3808K 528K 448K sleep 0:00 0.00% mingetty
7303 root 1 18 0 3808K 528K 448K sleep 0:00 0.00% mingetty
5836 root 1 19 0 3808K 484K 408K sleep 0:00 0.00% sleep
1744 root 1 10 -5 0K 0K 0K sleep 649:36 0.00%
md2_raid10
6586 root 1 15 0 0K 0K 0K sleep 227:57 0.00%
drbd1_receiver
6587 root 1 -3 0 0K 0K 0K sleep 72:20 0.00%
drbd1_asender
1740 root 1 10 -5 0K 0K 0K sleep 64:18 0.00%
md1_raid10
1750 root 1 10 -5 0K 0K 0K sleep 16:02 0.00% kjournald
slabtop:
Active / Total Objects (% used) : 108378294 / 108636165 (99.8%)
Active / Total Slabs (% used) : 2746709 / 2746710 (100.0%)
Active / Total Caches (% used) : 100 / 150 (66.7%)
Active / Total Size (% used) : 10273556.84K / 10298936.27K (99.8%)
Minimum / Average / Maximum Object : 0.02K / 0.09K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
108327280 108091540 20% 0.09K 2708182 40 10832728K buffer_head
228606 228226 99% 0.52K 32658 7 130632K radix_tree_node
9856 9832 99% 0.09K 224 44 896K sysfs_dir_cache
7847 3871 49% 0.06K 133 59 532K size-64
7596 5889 77% 0.21K 422 18 1688K dentry_cache
6300 5208 82% 0.12K 210 30 840K size-128
4368 3794 86% 0.03K 39 112 156K size-32
3150 2793 88% 0.25K 210 15 840K size-256
3068 2563 83% 0.06K 52 59 208K Acpi-Operand
2904 1253 43% 0.17K 132 22 528K vm_area_struct
2376 2342 98% 1.00K 594 4 2376K size-1024
2304 380 16% 0.02K 16 144 64K anon_vma
2256 1852 82% 0.08K 47 48 188K selinux_inode_security
2121 1943 91% 0.55K 303 7 1212K inode_cache
1776 1463 82% 0.50K 222 8 888K size-512
1710 705 41% 0.25K 114 15 456K filp
1698 1642 96% 0.58K 283 6 1132K proc_inode_cache
1632 1606 98% 2.00K 816 2 3264K size-2048
1590 1147 72% 0.25K 106 15 424K skbuff_head_cache
1584 324 20% 0.02K 11 144 44K numa_policy
1180 359 30% 0.06K 20 59 80K delayacct_cache
1140 1101 96% 0.74K 228 5 912K ext3_inode_cache
1080 1049 97% 0.09K 27 40 108K drbd_ee
1054 1024 97% 0.11K 31 34 124K drbd_req
1010 339 33% 0.02K 5 202 20K biovec-1
1008 888 88% 0.03K 9 112 36K Acpi-Namespace
944 335 35% 0.06K 16 59 64K pid
650 514 79% 0.75K 130 5 520K shmem_inode_cache
630 542 86% 0.12K 21 30 84K bio
558 353 63% 0.81K 62 9 496K signal_cache
496 496 100% 4.00K 496 1 1984K size-4096
410 351 85% 1.84K 205 2 820K task_struct
399 355 88% 2.06K 133 3 1064K sighand_cache
354 54 15% 0.06K 6 59 24K fs_cache
memtop:
MemTotal: 65996216 kB
MemFree: 436188 kB
Buffers: 54272396 kB
Cached: 183784 kB
SwapCached: 0 kB
Active: 324660 kB
Inactive: 54143868 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 65996216 kB
LowFree: 436188 kB
SwapTotal: 8192504 kB
SwapFree: 8192388 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 12320 kB
Mapped: 8312 kB
Slab: 10988324 kB
PageTables: 1584 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 41190612 kB
Committed_AS: 44772 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 267000 kB
VmallocChunk: 34359471059 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 2048 kB
--
Lin Zhao
Project Lead of Messagebus
https://wiki.groupondev.com/Message_Bus
3101 Park Blvd, Palo Alto, CA 94306
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130521/43bd99d9/attachment.htm>