Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi!
Look to this commit e0645836e870346cafe688cbdd8ec29092f6cdb5 (Tue Nov 8
11:43:09 2016) and this d9aea72bb66bb27f815de082d5b347fcddfc9c1b (Thu Nov 10
14:48:33 2016) in http://git.linbit.com/drbd-8.4.git/
I am not sure if this solves your particular problem, but you could use the
newest version 8.4.9-2 and test if it is gone, when you have a test scenario
where it happens easily.
BR,
Jasmin
*******************************************************************************
On 02/07/2017 06:28 PM, Tadashi Abe wrote:
> Hi,
>
> I'm using DRBD 8.4.8-1 with linux-2.6.32 kernel, on 2 nodes.
>
> # cat /proc/drbd
> version: 8.4.8-1 (api:1/proto:86-101)
> GIT-hash: 22b4c802192646e433d3f7399d578ec7fecc6272
>
> When running a kind of system test (detach/attach loop in high system load),
> NULL pointer deref occurs on a node at drbd_submit_peer_request.
> One thing I notice is the following 2 assertion failure about the same drbd
> device (drbd6) is seen many times
> before NULL deref occurs.
>
> Jan 18 12:19:29 HOSTA_101 kernel: : [161157.608191] block drbd6: ASSERT( i >= 0
> ) in drivers/block/drbd/drbd_int.h:2270
> Jan 18 12:19:29 HOSTA_101 kernel: : [161158.332840] block drbd6: ASSERT(
> atomic_read(&device->local_cnt) ) in drivers/block/drbd/drbd_actlog.c:691
>
> These makes me the suspicion that device->local_cnt is invalid count.
> (the former assertion failure occurs in put_ldev() in drbd code I'm using).
>
> Here's syslog snippet of BUG.
>
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.600846] BUG: unable to handle
> kernel NULL pointer dereference at (null)
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.601807] IP: [<ffffffffa010f07d>]
> drbd_submit_peer_request+0x8d/0x4c0 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.610078] PGD 765c6067 PUD 7645d067
> PMD 0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.610078] Oops: 0000 [#1] PREEMPT SMP
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.610078] last sysfs file:
> /sys/devices/virtual/block/drbd7/removable
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.644817] CPU 4
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.660713] Modules linked in: e1000
> igb mlx4_core mlx4_en virtio_net virtio_balloon ipmi_msghandler ipmi_watchdog
> kplugdr libcrc32c crc32c drbd scsi_transport_iscsi libiscsi libiscsi_tcp
> iscsi_tcp [last unloaded: ipmi_msghandler]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.692781] Pid: 25503, comm: drbd_r_r4
> Tainted: G W 2.6.32.59.cge 60-A64-N1.07 KVM
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.710234] RIP:
> 0010:[<ffffffffa010f07d>] [<ffffffffa010f07d>]
> drbd_submit_peer_request+0x8d/0x4c0 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.714138] RSP: 0018:ffff88004fc09d90
> EFLAGS: 00010286
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.725343] RAX: 0000000000000000 RBX:
> ffff88004fc09e60 RCX: ffff8800376bf8c0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.725343] RDX: ffff88007b863800 RSI:
> 0000000000011200 RDI: 0000000000000246
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.738464] RBP: ffff88004fc09df0 R08:
> 0000000000000000 R09: 0000000000000000
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.738464] R10: 0000000000000000 R11:
> 0000000000000000 R12: ffffea0001a1b550
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.757103] R13: 0000000000064000 R14:
> 0000000000043400 R15: 0000000000000064
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.777606] FS: 0000000000000000(0000)
> GS:ffff880001d00000(0000) knlGS:0000000000000000
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.783633] CS: 0010 DS: 0018 ES: 0018
> CR0: 000000008005003b
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.796394] CR2: 0000000000000000 CR3:
> 000000004f345000 CR4: 00000000000406e0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.804852] DR0: 0000000000000000 DR1:
> 0000000000000000 DR2: 0000000000000000
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.809335] DR3: 0000000000000000 DR6:
> 00000000ffff0ff0 DR7: 0000000000000400
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.809335] Process drbd_r_r4 (pid:
> 25503, threadinfo ffff88004fc08000, task ffff88007b6bb2a0)
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.838090] Stack:
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.838090] 0000000000000000
> 0000000000000001 0000000000000002 ffff88007b863800
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.838090] <0> ffff88004ed31810
> ffff8800376bf8c0 ffff88004fc09df0 ffff88004fc09e60
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.838090] <0> ffff88007b863800
> ffff88004ed31810 ffff88007b863800 0000000000043400
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.852971] Call Trace:
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.856040] [<ffffffffa01126ab>]
> receive_RSDataReply+0x13b/0x490 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.856040] [<ffffffffa0110880>]
> drbd_receiver+0x100/0x2e0 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.856040] [<ffffffffa0124740>] ?
> drbd_thread_setup+0x0/0x110 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.856040] [<ffffffffa012476d>]
> drbd_thread_setup+0x2d/0x110 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.856040] [<ffffffffa0124740>] ?
> drbd_thread_setup+0x0/0x110 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.871833] [<ffffffff811251b6>]
> kthread+0x96/0xa0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.871833] [<ffffffff81125120>] ?
> kthread+0x0/0xa0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.871833] [<ffffffff810b445a>]
> child_rip+0xa/0x20
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.880486] [<ffffffff81125120>] ?
> kthread+0x0/0xa0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.880486] [<ffffffff81125120>] ?
> kthread+0x0/0xa0
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.885895] [<ffffffff810b4450>] ?
> child_rip+0x0/0x20
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.887926] Code: 00 00 00 e8 76 45 0f
> e1 48 85 c0 48 89 45 c8 4c 8b 4d a0 0f 84 f6 03 00 00 48 8b 4d c8 4c 89 31 48
> 8b 55 b8 48 8b 82 80 00 00 00 <48> 8b 00 48 89 41 10 48 8b 45 a8 48 89 41 20 48
> 8b 55 c0 48 c7
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.894609] RIP [<ffffffffa010f07d>]
> drbd_submit_peer_request+0x8d/0x4c0 [drbd]
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.894609] RSP <ffff88004fc09d90>
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.894609] CR2: 0000000000000000
> Jan 18 12:20:38 HOSTA_101 kernel: : [161228.924389] ---[ end trace
> d6d8759a31519f4d ]---
>
> Any help really appreciated. Thanks a lot.
>
> Regards,
> Tadashi
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>