Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Tue, 2005-04-05 at 13:34 +0200, Philipp Reisner wrote:
> As it is now, the report is next to be useless...
>
> How does the machine locks up ?
>
> Does it respond to pings?
> Does it respond to keystrokes?
> Is it still possible to loggle the keyboard leds like shift-lock?
>
> In case your kernel has "CONFIG_MAGIC_SYSRQ=y" in /proc/config.gz
> press ALT-'Print Screen'-t and try to get the output to me and/or
> the list.
> [ Nb. It might be necessary to do "echo 1 > /proc/sys/kernel/sysrq"
> first. ]
>
> -Phil
After hooking up the console server to log the error messages it took
quite some time until the error reproduced itself but here it finally
is.
As for your questions, there is no keyboard interaction possible after
the crash, there are no ping responses either.
MAGIC_SYSRQ is not enabled on the machine.
I will also post some of the hardware specs and the dmesg.
Please let me know if I can provide you with any more/other information.
Best regards
Sebastian Schmitzdorff
<errorlog>
smsc2:~# Unable to handle kernel NULL pointer dereference at virtual
address 0000002c
printing eip:
c011d1c6
*pde = 00000000
Oops: 0000 [#1]
Modules linked in: drbd e100
CPU: 0
EIP: 0060:[<c011d1c6>] Not tainted VLI
EFLAGS: 00010096 (2.6.10)
EIP is at recalc_sigpending_tsk+0x16/0x70
eax: c15e3000 ebx: 00000000 ecx: c15e3000 edx: 7ffbfeff
esi: 00000001 edi: df825000 ebp: 00000018 esp: d88b14fc
ds: 007b es: 007b ss: 0068
Process ^Á´1^Áeth1 (pid: 6338, threadinfo=d88b1000 task=c15e3000)
Stack: 00000202 00000001 df825000 e08dfc60 df825000 c13d90a0 00000000
00001000
00008000 00000000 7ffbfeff ffffffff 67027483 10100000 00000000
50d3dd00
dc08c12c 00000000 c029afe0 c15b108c c15b1030 df825000 00000001
dc08c12c
Call Trace:
[<e08dfc60>] drbd_send_dblock+0x260/0x3b0 [drbd]
[<c029afe0>] blk_plug_device+0x50/0x80
[<e08da840>] drbd_make_request_common+0x570/0x8b0 [drbd]
[<e08dac66>] drbd_make_request_26+0xe6/0x29d [drbd]
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c029caed>] generic_make_request+0x17d/0x220
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c0133623>] mempool_alloc+0x73/0x130
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c029cbed>] submit_bio+0x5d/0x100
[<c0130650>] find_or_create_page+0x30/0xd0
[<c0151624>] bio_add_page+0x34/0x40
[<c0247130>] _pagebuf_ioapply+0x1f0/0x2f0
[<c0111990>] default_wake_function+0x0/0x20
[<c0111990>] default_wake_function+0x0/0x20
[<c02472b8>] pagebuf_iorequest+0x88/0x170
[<c0111990>] default_wake_function+0x0/0x20
[<c0111990>] default_wake_function+0x0/0x20
[<c0246847>] pagebuf_associate_memory+0x67/0x190
[<c02288df>] xlog_bdstrat_cb+0x1f/0x70
[<c02291f2>] xlog_sync+0x222/0x460
[<c022a896>] xlog_state_release_iclog+0x26/0xe0
[<c022ad59>] xlog_state_sync+0x159/0x210
[<c01e0ffd>] xfs_alloc_ag_vextent_near+0xcd/0xc30
[<c0227e34>] xfs_log_force+0x74/0x80
[<c01e3b29>] xfs_alloc_search_busy+0x99/0x9b
[<c01e0cc6>] xfs_alloc_ag_vextent+0xe6/0x130
[<c01e377d>] xfs_alloc_vextent+0x3fd/0x500
[<c01f3688>] xfs_bmap_alloc+0xa88/0x18e0
[<c01f7b38>] xfs_bmapi+0x5b8/0x1780
[<c0246082>] _pagebuf_lookup_pages+0x222/0x340
[<c0237e5e>] xfs_trans_read_buf+0x1ce/0x350
[<c021e1d4>] xfs_itobp+0x114/0x260
[<c0208133>] xfs_dir2_grow_inode+0x103/0x460
[<c021d035>] xfs_iget_core+0x2d5/0x500
[<c022182e>] xfs_idata_realloc+0x4e/0x180
[<c0243c09>] kmem_alloc+0x59/0xe0
[<c02388bd>] xfs_trans_log_inode+0x2d/0x60
[<c0209fe3>] xfs_dir2_sf_to_block+0xd3/0x6c0
[<c023873b>] xfs_trans_iget+0xfb/0x180
[<c0222dda>] xfs_ichgtime+0x6a/0x11b
[<c024e3ab>] vfs_init_vnode+0x4b/0x50
[<c0210d2a>] xfs_dir2_sf_addname+0xaa/0x140
[<c02079b7>] xfs_dir2_createname+0x177/0x180
[<c0239241>] xfs_dir_ialloc+0x91/0x2e0
[<c02387f5>] xfs_trans_ijoin+0x35/0x90
[<c023ec4b>] xfs_create+0x46b/0x780
[<c024aaeb>] linvfs_mknod+0x1db/0x240
[<c0207b0c>] xfs_dir2_lookup+0x14c/0x160
[<c0159a0b>] link_path_walk+0x8eb/0xb70
[<c02390cc>] xfs_dir_lookup_int+0x4c/0x130
[<c0158c15>] permission+0x35/0x60
[<c015a349>] vfs_create+0x79/0xe0
[<c015ab56>] open_namei+0x5a6/0x600
[<c014b8ce>] filp_open+0x3e/0x70
[<c014bb0e>] get_unused_fd+0x3e/0xd0
[<c014bc39>] sys_open+0x49/0xb0
[<c01024b3>] syscall_call+0x7/0xb
=======================
Code: 1f 7f f7 b8 00 00 43 08 d3 e8 a8 01 74 ec ba 01 00 00 00 eb e5 83
ec 0c 89 c1 89 1c 24 89 74 24 04 89 7c 24 08 8b 98 6c 04 00 00 <8b> 43
2c 85 c0 7e 17 8b 41 04 0f ba 68 08 02 8b 1c 24 8b 74 24
<1>Unable to handle kernel paging request at virtual address 3a300008
printing eip:
c0111657
*pde = 00000000
Oops: 0002 [#2]
Modules linked in: drbd e100
CPU: 0
EIP: 0060:[<c0111657>] Not tainted VLI
EFLAGS: 00010003 (2.6.10)
EIP is at scheduler_tick+0x107/0x440
eax: 3a300000 ebx: c15e3000 ecx: 00000000 edx: 00000000
esi: 00000001 edi: c0473aa0 ebp: c046bf7c esp: c046bf5c
ds: 007b es: 007b ss: 0068
Process ^Á´1^Áeth1 (pid: 6338, threadinfo=c046b000 task=c15e3000)
Stack: dcec1a20 00000000 00000000 c1507218 195e4ddd 00000000 00000001
d88b1384
00000000 c011c7a4 00000000 00000001 00000001 00000000 d88b1384
00000000
c01064b4 00000000 df42b740 c03c8900 00000000 d88b1384 c012f110
00000000
Call Trace:
[<c011c7a4>] update_process_times+0x44/0x50
[<c01064b4>] timer_interrupt+0x44/0xf0
[<c012f110>] handle_IRQ_event+0x30/0x70
[<c012f1fd>] __do_IRQ+0xad/0xf0
[<c0103f4e>] do_IRQ+0x3e/0x60
=======================
[<c0102622>] common_interrupt+0x1a/0x20
[<c0102d2e>] die+0xee/0x150
[<c0114ad7>] printk+0x17/0x20
[<c010d02a>] do_page_fault+0x2ca/0x5c5
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c02ed1d6>] kernel_sendmsg+0x46/0x60
[<e08dffe5>] drbd_send+0xb5/0x230 [drbd]
[<c010cd60>] do_page_fault+0x0/0x5c5
[<c010265b>] error_code+0x2b/0x30
[<c011d1c6>] recalc_sigpending_tsk+0x16/0x70
[<e08dfc60>] drbd_send_dblock+0x260/0x3b0 [drbd]
[<c029afe0>] blk_plug_device+0x50/0x80
[<e08da840>] drbd_make_request_common+0x570/0x8b0 [drbd]
[<e08dac66>] drbd_make_request_26+0xe6/0x29d [drbd]
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c029caed>] generic_make_request+0x17d/0x220
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c0133623>] mempool_alloc+0x73/0x130
[<c0126e10>] autoremove_wake_function+0x0/0x60
[<c029cbed>] submit_bio+0x5d/0x100
[<c0130650>] find_or_create_page+0x30/0xd0
[<c0151624>] bio_add_page+0x34/0x40
[<c0247130>] _pagebuf_ioapply+0x1f0/0x2f0
[<c0111990>] default_wake_function+0x0/0x20
[<c0111990>] default_wake_function+0x0/0x20
[<c02472b8>] pagebuf_iorequest+0x88/0x170
[<c0111990>] default_wake_function+0x0/0x20
[<c0111990>] default_wake_function+0x0/0x20
[<c0246847>] pagebuf_associate_memory+0x67/0x190
[<c02288df>] xlog_bdstrat_cb+0x1f/0x70
[<c02291f2>] xlog_sync+0x222/0x460
[<c022a896>] xlog_state_release_iclog+0x26/0xe0
[<c022ad59>] xlog_state_sync+0x159/0x210
[<c01e0ffd>] xfs_alloc_ag_vextent_near+0xcd/0xc30
[<c0227e34>] xfs_log_force+0x74/0x80
[<c01e3b29>] xfs_alloc_search_busy+0x99/0x9b
[<c01e0cc6>] xfs_alloc_ag_vextent+0xe6/0x130
[<c01e377d>] xfs_alloc_vextent+0x3fd/0x500
[<c01f3688>] xfs_bmap_alloc+0xa88/0x18e0
[<c01f7b38>] xfs_bmapi+0x5b8/0x1780
[<c0246082>] _pagebuf_lookup_pages+0x222/0x340
[<c0237e5e>] xfs_trans_read_buf+0x1ce/0x350
[<c021e1d4>] xfs_itobp+0x114/0x260
[<c0208133>] xfs_dir2_grow_inode+0x103/0x460
[<c021d035>] xfs_iget_core+0x2d5/0x500
[<c022182e>] xfs_idata_realloc+0x4e/0x180
[<c0243c09>] kmem_alloc+0x59/0xe0
[<c02388bd>] xfs_trans_log_inode+0x2d/0x60
[<c0209fe3>] xfs_dir2_sf_to_block+0xd3/0x6c0
[<c023873b>] xfs_trans_iget+0xfb/0x180
[<c0222dda>] xfs_ichgtime+0x6a/0x11b
[<c024e3ab>] vfs_init_vnode+0x4b/0x50
[<c0210d2a>] xfs_dir2_sf_addname+0xaa/0x140
[<c02079b7>] xfs_dir2_createname+0x177/0x180
[<c0239241>] xfs_dir_ialloc+0x91/0x2e0
[<c02387f5>] xfs_trans_ijoin+0x35/0x90
[<c023ec4b>] xfs_create+0x46b/0x780
[<c024aaeb>] linvfs_mknod+0x1db/0x240
[<c0207b0c>] xfs_dir2_lookup+0x14c/0x160
[<c0159a0b>] link_path_walk+0x8eb/0xb70
[<c02390cc>] xfs_dir_lookup_int+0x4c/0x130
[<c0158c15>] permission+0x35/0x60
[<c015a349>] vfs_create+0x79/0xe0
[<c015ab56>] open_namei+0x5a6/0x600
[<c014b8ce>] filp_open+0x3e/0x70
[<c014bb0e>] get_unused_fd+0x3e/0xd0
[<c014bc39>] sys_open+0x49/0xb0
[<c01024b3>] syscall_call+0x7/0xb
=======================
Code: 8b 55 ec 89 47 08 89 57 0c 89 f0 99 03 05 b0 3a 47 c0 13 15 b4 3a
47 c0 89 47 10 89 57 14 a1 48 3b 47 c0 39 43 28 74 10 8b 43 04 <0f> ba
68 08 03 83 c4 14 5b 5e 5f 5d c3 8b 7b 18 83 ff 63 0f 8e
<0>Kernel panic - not syncing: Fatal exception in interrupt
</errorlog>
<hardware>
IBM xSeries 330
IBM PCI ServeRAID 7.10.18 Build 731 <ServeRAID 4Lx>
2 x IBM 18.2GB U160
RAID 1 setup
512MB ECC Ram
</hardware>
<dmesg>
smsc2:~# dmesg
Linux version 2.6.10 (root at smsc2) (gcc version 3.3.4 (Debian
1:3.3.4-6sarge1)) #3 Tue Apr 5 14:09:01 CEST 2005
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009d000 (usable)
BIOS-e820: 000000000009d000 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000001ffec340 (usable)
BIOS-e820: 000000001ffec340 - 000000001fff0000 (ACPI data)
BIOS-e820: 000000001fff0000 - 0000000020000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
511MB LOWMEM available.
On node 0 totalpages: 131052
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 126956 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
DMI 2.3 present.
Built 1 zonelists
Kernel command line: auto BOOT_IMAGE=serial rw root=802 console=tty0
console=ttyS0,115200n8
Initializing CPU#0
CPU 0 irqstacks, hard=c046b000 soft=c046a000
PID hash table entries: 2048 (order: 11, 32768 bytes)
Detected 1130.345 MHz processor.
Using tsc for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 515060k/524208k available (2490k kernel code, 8612k reserved,
848k data, 132k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode...
Ok.
Calibrating delay loop... 2228.22 BogoMIPS (lpj=1114112)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
CPU: After generic identify, caps: 0383fbff 00000000 00000000 00000000
CPU: After vendor identify, caps: 0383fbff 00000000 00000000 00000000
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
CPU: After all inits, caps: 0383fbff 00000000 00000000 00000040
CPU: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
NET: Registered protocol family 16
PCI: PCI BIOS revision 2.10 entry at 0xfd61c, last bus=1
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
SCSI subsystem initialized
PCI: Probing PCI hardware
PCI: Probing PCI hardware (bus 00)
PCI: Discovered peer bus 01
audit: initializing netlink socket (disabled)
audit(1112788861.667:0): initialized
Installing knfsd (copyright (C) 1996 okir at monad.swb.de).
SGI XFS with large block numbers, no debug enabled
Initializing Cryptographic API
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing
disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
elevator: using anticipatory as default io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
Probing IDE interface ide0...
hda: CD-224E, ATAPI CD/DVD-ROM drive
Probing IDE interface ide1...
Probing IDE interface ide2...
ide2: Wait for ready failed before probe !
Probing IDE interface ide3...
ide3: Wait for ready failed before probe !
Probing IDE interface ide4...
ide4: Wait for ready failed before probe !
Probing IDE interface ide5...
ide5: Wait for ready failed before probe !
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: ATAPI 24X CD-ROM drive, 128kB Cache
Uniform CD-ROM driver Revision: 3.20
ips 0000:01:05.0: Warning ! ! ! ServeRAID Version Mismatch
ips 0000:01:05.0: Bios = 4.70.17, Firmware = 4.70.17, Device Driver =
7.10.18
ips 0000:01:05.0: These levels should match to avoid possible
compatibility problems.
scsi0 : IBM PCI ServeRAID 7.10.18 Build 731 <ServeRAID 4Lx>
Vendor: IBM Model: SERVERAID Rev: 1.00
Type: Direct-Access ANSI SCSI revision: 02
Vendor: IBM Model: SERVERAID Rev: 1.00
Type: Processor ANSI SCSI revision: 02
Vendor: IBM Model: FTlV1 S2 Rev: 0
Type: Processor ANSI SCSI revision: 02
SCSI device sda: 35547136 512-byte hdwr sectors (18200 MB)
SCSI device sda: drive cache: write through
SCSI device sda: 35547136 512-byte hdwr sectors (18200 MB)
SCSI device sda: drive cache: write through
sda: sda1 sda2 sda3 < sda5 sda6 > sda4
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0, type 0
Attached scsi generic sg1 at scsi0, channel 0, id 15, lun 0, type 3
Attached scsi generic sg2 at scsi0, channel 1, id 8, lun 0, type 3
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard on isa0060/serio0
input: ImPS/2 Generic Wheel Mouse on isa0060/serio1
NET: Registered protocol family 2
IP: routing cache hash table of 4096 buckets, 32Kbytes
TCP: Hash tables configured (established 32768 bind 65536)
ip_tables: (C) 2000-2002 Netfilter core team
ipt_recent v0.3.1: Stephen Frost <sfrost at snowman.net>.
http://snowman.net/projects/ipt_recent/
arp_tables: (C) 2002 David S. Miller
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
UDF-fs: No VRS found
XFS mounting filesystem sda2
Starting XFS recovery on filesystem: sda2 (dev: sda2)
Ending XFS recovery on filesystem: sda2 (dev: sda2)
VFS: Mounted root (xfs filesystem).
Freeing unused kernel memory: 132k freed
Adding 1064092k swap on /dev/sda6. Priority:-1 extents:1
e100: Intel(R) PRO/100 Network Driver, 3.2.3-k2-NAPI
e100: Copyright(c) 1999-2004 Intel Corporation
e100: eth0: e100_probe: addr 0xfeb7f000, irq 10, MAC addr
00:02:55:C7:C2:17
e100: eth1: e100_probe: addr 0xfeb7e000, irq 10, MAC addr
00:02:55:C7:C2:18
drbd: initialised. Version: 0.7.7 (api:77/proto:74)
drbd: SVN Revision: 1680 build by root at smsc2, 2005-01-18 16:52:58
drbd: registered as block device major 147
e100: eth1: e100_watchdog: link up, 100Mbps, full-duplex
e100: eth0: e100_watchdog: link up, 10Mbps, half-duplex
drbd0: resync bitmap: bits=3629681 words=113428
drbd0: size = 13 GB (14518724 KB)
drbd0: 0 KB marked out-of-sync by on disk bit-map.
drbd0: Found 6 transactions (324 active extents) in activity log.
drbd0: Marked additional 128 MB as out-of-sync based on AL.
drbd0: drbdsetup [1082]: cstate Unconfigured --> StandAlone
drbd0: drbdsetup [1095]: cstate StandAlone --> Unconnected
drbd0: drbd0_receiver [1096]: cstate Unconnected --> WFConnection
drbd0: Secondary/Unknown --> Primary/Unknown
XFS mounting filesystem drbd0
Starting XFS recovery on filesystem: drbd0 (dev: drbd0)
Ending XFS recovery on filesystem: drbd0 (dev: drbd0)
</dmesg>