Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi all, we are seeing some strange page allocation failure on our target hosts running custom kernel 2.6.38.5 with drbd-8.3.11 userspace/modules. Link is across IPoIB. Is it a low memory situation or increasing the RAM size (currently we have 4GB) cannot help? is it safe to continue operations in such condition? below is a snippet: __alloc_pages_slowpath: 396 callbacks suppressed drbd0_asender: page allocation failure. order:1, mode:0x4020 Pid: 25123, comm: drbd0_asender Not tainted 2.6.38 #3 Call Trace: <IRQ> [<ffffffff8106ed74>] ? __alloc_pages_nodemask+0x17c/0x586 [<ffffffff8109596b>] ? alloc_pages_current+0x9e/0xa7 [<ffffffff81098c70>] ? new_slab+0xa4/0x226 [<ffffffff810999bd>] ? __slab_alloc+0x1da/0x2d7 [<ffffffff81253977>] ? dev_alloc_skb+0x16/0x2c [<ffffffff8127a675>] ? ip_local_deliver_finish+0x78/0xc7 [<ffffffff81253977>] ? dev_alloc_skb+0x16/0x2c [<ffffffff8109b1ca>] ? __kmalloc_node_track_caller+0x80/0xba [<ffffffff81253591>] ? __alloc_skb+0x71/0x136 [<ffffffff81253977>] ? dev_alloc_skb+0x16/0x2c [<ffffffffa020b131>] ? ipoib_cm_alloc_rx_skb+0x2c/0x37a [ib_ipoib] [<ffffffffa020cf21>] ? ipoib_cm_handle_rx_wc+0x409/0x633 [ib_ipoib] [<ffffffffa02071a1>] ? ipoib_ib_handle_rx_wc+0x219/0x241 [ib_ipoib] [<ffffffffa0208457>] ? ipoib_poll+0x86/0x11f [ib_ipoib] [<ffffffff8125d561>] ? net_rx_action+0xa6/0x156 [<ffffffff810346a4>] ? __do_softirq+0x90/0x11e [<ffffffff81002d8c>] ? call_softirq+0x1c/0x28 [<ffffffff81004889>] ? do_softirq+0x33/0x6a [<ffffffff81034612>] ? irq_exit+0x36/0x38 [<ffffffff81003f2c>] ? do_IRQ+0x9b/0xb2 [<ffffffff812d9353>] ? ret_from_intr+0x0/0xe <EOI> [<ffffffffa02d4706>] ? page_chain_tail+0x9/0x25 [drbd] [<ffffffffa02d8c3f>] ? drbd_pp_free+0x71/0x10e [drbd] [<ffffffffa02d8d10>] ? drbd_free_some_ee+0x34/0xa6 [drbd] [<ffffffffa02d8e87>] ? drbd_process_done_ee+0x105/0x13c [drbd] [<ffffffffa02da9a6>] ? drbd_asender+0x143/0x56b [drbd] [<ffffffffa02ed11b>] ? drbd_thread_setup+0x138/0x1e1 [drbd] [<ffffffff81002c94>] ? kernel_thread_helper+0x4/0x10 [<ffffffffa02ecfe3>] ? drbd_thread_setup+0x0/0x1e1 [drbd] [<ffffffff81002c90>] ? kernel_thread_helper+0x0/0x10 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 CPU 1: hi: 0, btch: 1 usd: 0 CPU 2: hi: 0, btch: 1 usd: 0 CPU 3: hi: 0, btch: 1 usd: 0 Node 0 DMA32 per-cpu: CPU 0: hi: 186, btch: 31 usd: 165 CPU 1: hi: 186, btch: 31 usd: 107 CPU 2: hi: 186, btch: 31 usd: 56 CPU 3: hi: 186, btch: 31 usd: 165 Node 0 Normal per-cpu: CPU 0: hi: 186, btch: 31 usd: 11 CPU 1: hi: 186, btch: 31 usd: 134 CPU 2: hi: 186, btch: 31 usd: 87 CPU 3: hi: 186, btch: 31 usd: 152 active_anon:5791 inactive_anon:5350 isolated_anon:0 active_file:337540 inactive_file:494676 isolated_file:0 unevictable:6035 dirty:3506 writeback:291 unstable:0 free:11306 slab_reclaimable:46370 slab_unreclaimable:8605 mapped:2743 shmem:51 pagetables:1359 bounce:0 Node 0 DMA free:15728kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:120kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15628kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:4kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 2162 3926 3926 Node 0 DMA32 free:21580kB min:4408kB low:5508kB high:6612kB active_anon:800kB inactive_anon:7856kB active_file:739044kB inactive_file:1275184kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:2214460kB mlocked:0kB dirty:8040kB writeback:396kB mapped:276kB shmem:12kB slab_reclaimable:145284kB slab_unreclaimable:11732kB kernel_stack:472kB pagetables:552kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 1764 1764 Node 0 Normal free:9696kB min:3596kB low:4492kB high:5392kB active_anon:22364kB inactive_anon:13544kB active_file:610996kB inactive_file:703520kB unevictable:24140kB isolated(anon):0kB isolated(file):0kB present:1806336kB mlocked:24140kB dirty:5984kB writeback:768kB mapped:10696kB shmem:192kB slab_reclaimable:40192kB slab_unreclaimable:19104kB kernel_stack:2808kB pagetables:4884kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 Node 0 DMA: 2*4kB 1*8kB 2*16kB 2*32kB 2*64kB 1*128kB 2*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15728kB Node 0 DMA32: 1187*4kB 1039*8kB 363*16kB 237*32kB 12*64kB 2*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 27476kB Node 0 Normal: 1064*4kB 201*8kB 88*16kB 17*32kB 11*64kB 4*128kB 2*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 9544kB 834441 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 2104504kB Total swap = 2104504kB 1048560 pages RAM 50556 pages reserved 849683 pages shared 146502 pages non-shared SLUB: Unable to allocate memory on node -1 (gfp=0x20) cache: kmalloc-8192, object size: 8192, buffer size: 8192, default order: 3, min order: 1 node 0: slabs: 313, objs: 604, free: 40 Regards, -- Matteo Tescione