Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
I've got a pair of HA servers I'm trying to get into production.
Here are some specs :
Xeon X3210 Quad Core (aka Core 2 Quad) 2.13Ghz (four logical
processors, no Hyper Threading)
4GB memory
Hardware (3ware) Raid 1 mirror, 2 x Seagate 750GB SATA2
650GB DRBD partition run on top of an LVM2 partition.
CentOS 5.2 2.6.18-92.1.6.el5.centos.plus
DRBD 8.2 (drbd82-8.2.6-1.el5.centos)
Kernel Module kmod-drbd82-8.2.6-1.2.6.18_92.1.6.el5.centos.plus
I've been trying to rsync data from a remote server and it's crashed
a couple of times now. It does not happen immediately, but over
time. I connected a serial console and got the below panic message.
The last file copied was ~1GB in size, but previous files up to 4GB
had been copied. I do not have kernel core dumping enabled, but
that's a possibility if needed. Not sure if this is a bug or is
caused by something I've done. This isn't my first DRBD install
(although first on top of LVM) and I believe I've gotten everything
setup correctly. I did have a full sync rate (110M) enabled over
Gbe, if that's relevant. Thoughts?
Regards,
Chris
[root at haws2 ~]# pvscan
PV /dev/sda2 VG VolGroup00 lvm2 [698.28 GB / 0 free]
Total: 1 [698.28 GB] / in use: 1 [698.28 GB] / in no VG: 0 [0 ]
[root at haws2 ~]# lvscan
ACTIVE '/dev/VolGroup00/LogVol00' [39.06 GB] inherit
ACTIVE '/dev/VolGroup00/LogVol02' [658.72 GB] inherit
ACTIVE '/dev/VolGroup00/LogVol01' [512.00 MB] inherit
[root at haws2 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
39676508 1938880 35689628 6% /
/dev/sda1 194442 23650 160753 13% /boot
tmpfs 1684156 0 1684156 0% /dev/shm
/dev/drbd0 679824572 113321224 531970212 18% /home
[root at haws1 ~]# BUG: unable to handle kernel paging request at
virtual address c
printing eip:
c04e9291
*pde = 00000000
Oops: 0000 [#1]
SMP
last sysfs file: /devices/pci0000:00/0000:00:00.0/irq
Modules linked in: softdog drbd(U) autofs4 hidp rfcomm l2cap
bluetooth sunrpc id
CPU: 0
EIP: 0060:[<c04e9291>] Tainted: G VLI
EFLAGS: 00010046 (2.6.18-92.1.6.el5.centos.plus #1)
EIP is at list_del+0x25/0x5c
eax: fe187128 ebx: f04a6ab8 ecx: f04a6a8c edx: f04a6a8c
esi: fe187128 edi: f4e355a0 ebp: f426c800 esp: f385df3c
ds: 007b es: 007b ss: 0068
Process drbd0_asender (pid: 2900, ti=f385d000 task=f4932000
task.ti=f385d000)
Stack: 000000e6 f8d1953b 00000000 f04a6a8c 000000e6 00000001
ee187b14 00000046
f49e7bc0 f04a6ab8 f04a6a8c f426c800 fe187128 f4e355a0
0000349f f8d24805
00000800 f426c800 f426c800 00000008 f426c9f4 f8d14d47
f385dfbc f8d15fbc
Call Trace:
[<f8d1953b>] _req_may_be_done+0x4ea/0x710 [drbd]
[<f8d24805>] tl_release+0x35/0x172 [drbd]
[<f8d14d47>] got_BarrierAck+0x10/0x6b [drbd]
[<f8d15fbc>] drbd_asender+0x3b1/0x4e7 [drbd]
[<f8d24a53>] drbd_thread_setup+0x0/0x14e [drbd]
[<f8d24adb>] drbd_thread_setup+0x88/0x14e [drbd]
[<f8d24a53>] drbd_thread_setup+0x0/0x14e [drbd]
[<c0405c3b>] kernel_thread_helper+0x7/0x10
=======================
Code: 89 c3 eb eb 90 90 53 89 c3 8b 40 04 8b 00 39 d8 74 17 50 53 68
9b 9a 63 c
EIP: [<c04e9291>] list_del+0x25/0x5c SS:ESP 0068:f385df3c
<0>Kernel panic - not syncing: Fatal exception
BUG: warning at arch/i386/kernel/smp.c:550/smp_call_function()
(Tainted: G )
[<c0417ae0>] stop_this_cpu+0x0/0x33
[<c04178cf>] smp_call_function+0x57/0xc3
[<c0426682>] printk+0x18/0x8e
[<c041794e>] smp_send_stop+0x13/0x1c
[<c0425c53>] panic+0x4c/0x16d
[<c04064dd>] die+0x25d/0x291
[<c060c48b>] do_page_fault+0x3ea/0x4b8
[<c060c0a1>] do_page_fault+0x0/0x4b8
[<c0405a71>] error_code+0x39/0x40
[<c04e9291>] list_del+0x25/0x5c
[<f8d1953b>] _req_may_be_done+0x4ea/0x710 [drbd]
[<f8d24805>] tl_release+0x35/0x172 [drbd]
[<f8d14d47>] got_BarrierAck+0x10/0x6b [drbd]
[<f8d15fbc>] drbd_asender+0x3b1/0x4e7 [drbd]
[<f8d24a53>] drbd_thread_setup+0x0/0x14e [drbd]
[<f8d24adb>] drbd_thread_setup+0x88/0x14e [drbd]
[<f8d24a53>] drbd_thread_setup+0x0/0x14e [drbd]
[<c0405c3b>] kernel_thread_helper+0x7/0x10
=======================