Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
>>
>>
>> Most recent crash. Appears to have started on gt6, but gt5
>> by the time I got paged both boxes had crashed and rebooted.
>>
>> Apr 16 12:05:57 gt6 kernel: BUG: unable to handle kernel paging request at
>> virtual address d6ef9000
>> Apr 16 12:05:57 gt6 kernel: printing eip:
>> Apr 16 12:05:57 gt6 kernel: ee20d19a
>> Apr 16 12:05:57 gt6 kernel: 10414000 -> *pde = 00000002:700e1001
>> Apr 16 12:05:57 gt6 kernel: 0fc1b000 -> *pme = 00000000:3c0ae067
>> Apr 16 12:05:57 gt6 kernel: 000ae000 -> *pte = 00000000:00000000
>> Apr 16 12:05:57 gt6 kernel: Oops: 0000 [#1]
>> (this would be a crash...)
>
> Simon Graham explained an interesting effect when using scatter gather
> and DRBD and xen...
> you could try and tell drbd to no longer use zero copy send using
sendpage,
> but always do an actual data copy to the socket buffer, which should
> avoid the described problem. easiest way to do so: use DRBD protocol A,
> and see if these crashes still occur.
Given that I seem to have a reproducable test case (see below), that
should be easy enough to try.
YUP, tried it. I have NOT exhaustively tested it, but given that
typing sync without any significant I/O preceeding it (and I'm in
the guest in single user mode), is enough to crash dom0 (proto C),
and I can't crash the box at all with proto A, I think you know the
core cause.
> up to now, you are the only one reporting this kind of behaviour,
> and I don't think you are the only one using DRBD + Xen.
> so this may also be something specific to your setup.
>
> but as long as I don't have any backtraces, I can only speculate.
> we would have to be able to reproduce this to actually debug it.
I just did. I fortunately have two machines at the colo that are spare.
Pulling the plug on on a drbd link isn't sufficient. It appears to need to
be XEN guest using the drbd device... e.g.
disk = [ 'phy:/dev/drbd/by-res/bk4-sys,sda1,w' ]
Unfortunately the dump (which came through a serial console) is mostly
greek to me... hopefully it make some sense to you. I have included the
(re)boot messaging in case there's anything in there relevent... but I
suspect it'll all be in the trace...
Thanks for looking at this Lars!
[root at gt3 ~]#
[root at gt3 ~]#
[root at gt3 ~]# drbd0: PingAck did not arrive in time.
drbd0: short read expecting header on sock: r=-512
BUG: unable to handle kernel paging request at virtual address ec3f7000
printing eip:
c04e695a
2b0af000 -> *pde = 00000000:94c30001
2b230000 -> *pme = 00000000:3c158067
00158000 -> *pte = 00000000:00000000
Oops: 0000 [#1]
SMP
last sysfs file: /module/drbd/parameters/cn_idx
Modules linked in: xt_physdev ipt_MASQUERADE iptable_nat ip_nat xt_state ip_conntrack nfnetlink ipt_REJECT netloop netbk blktap blkbk bridge xt_tcpudp iptable_filter ip_tables x_tables drbd(U) ipv6 xfrm_nalgo crypto_api w83792d dm_multipath scsi_dh video hwmon backlight sbs i2c_ec button battery asus_acpi ac parport_pc lp parport floppy sg pcspkr ide_cd cdrom i2c_i801 i2c_core tg3 libphy serial_core dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod usb_storage sata_mv ata_piix ahci libata sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
CPU: 1
EIP: 0061:[<c04e695a>] Tainted: G VLI
EFLAGS: 00010282 (2.6.18-128.1.6.el5xen #1)
EIP is at csum_partial+0xe2/0x120
eax: 00000000 ebx: c04e695a ecx: 0000000b edx: 00000588
esi: ec3f7008 edi: 00000588 ebp: 00000054 esp: c0725de4
ds: 007b es: 007b ss: 0069
Process swapper (pid: 0, ti=c0725000 task=c0d99550 task.ti=c0d94000)
Stack: ec3f7000 00000054 c05af786 ec3f7000 00000588 00000000 00000018 eb217eac
00000001 00000040 000005dc c666ab1c c05b0699 c666aa00 00000588 eb217eac
eb2ea6cc eb2ea6e0 c0725ef4 c05b4098 99ab4a4a eb56c948 00000003 c0725ef4
Call Trace:
[<c05af786>] skb_checksum+0x111/0x27b
[<c05b0699>] pskb_expand_head+0xd6/0x11a
[<c05b4098>] skb_checksum_help+0x64/0xb3
[<ee4572ee>] ip_nat_fn+0x42/0x185 [iptable_nat]
[<ee457628>] ip_nat_local_fn+0x34/0xa4 [iptable_nat]
[<c05d0cb4>] dst_output+0x0/0x7
[<c05c9744>] nf_iterate+0x30/0x61
[<c05d0cb4>] dst_output+0x0/0x7
[<c05c986a>] nf_hook_slow+0x3a/0x90
[<c05d0cb4>] dst_output+0x0/0x7
[<c05d2fb0>] ip_queue_xmit+0x3ba/0x40b
[<c05d0cb4>] dst_output+0x0/0x7
[<c05c2588>] __qdisc_run+0x66/0x1d3
[<c0424128>] local_bh_enable+0x5/0x81
[<c05b61e4>] dev_queue_xmit+0x2ca/0x2ea
[<ee442d48>] br_dev_queue_push_xmit+0x15b/0x17e [bridge]
[<c05e0b8c>] tcp_transmit_skb+0x5c7/0x5f5
[<c05e18ee>] tcp_retransmit_skb+0x4c0/0x59e
[<c05daaab>] tcp_enter_loss+0x1a2/0x1ff
[<c05e39dc>] tcp_write_timer+0x0/0x5dc
[<c05e3dda>] tcp_write_timer+0x3fe/0x5dc
[<c0427942>] run_timer_softirq+0x101/0x15c
[<c04243c3>] __do_softirq+0x8b/0x11c
[<c0406e5d>] do_softirq+0x56/0xae
[<c04463c8>] __do_IRQ+0x0/0xd6
[<c0406f6a>] do_IRQ+0xb5/0xc3
[<c054efc7>] evtchn_do_upcall+0xfa/0x191
[<c04055d9>] hypervisor_callback+0x3d/0x48
[<c0408664>] raw_safe_halt+0x8c/0xaf
[<c040321a>] xen_idle+0x22/0x2e
[<c0403339>] cpu_idle+0x91/0xab
=======================
Code: bc 13 46 c0 13 46 c4 13 46 c8 13 46 cc 13 46 d0 13 46 d4 13 46 d8 13 46 dc 13 46 e0 13 46 e4 13 46 e8 13 46 ec 13 46 f0 13 46 f4 <13> 46 f8 13 46 fc 8d b6 80 00 00 00 83 d0 00 49 7d 94 89 d1 83
EIP: [<c04e695a>] csum_partial+0xe2/0x120 SS:ESP 0069:c0725de4
<0>Kernel panic - not syncing: Fatal exception in interrupt
BUG: warning at arch/i386/kernel/smp-xen.c:529/smp_call_function() (Tainted: G )
[<c04109cf>] smp_call_function+0x59/0xfe
[<c0410a87>] smp_send_stop+0x13/0x1e
[<c041f6d7>] panic+0x4c/0x171
[<c04060a5>] die+0x262/0x296
[<c0611449>] do_page_fault+0xa85/0xbf9
[<c04e695a>] csum_partial+0xe2/0x120
[<c06109c4>] do_page_fault+0x0/0xbf9
[<c0405597>] error_code+0x2b/0x30
[<c04e695a>] csum_partial+0xe2/0x120
[<c04e695a>] csum_partial+0xe2/0x120
[<c05af786>] skb_checksum+0x111/0x27b
[<c05b0699>] pskb_expand_head+0xd6/0x11a
[<c05b4098>] skb_checksum_help+0x64/0xb3
[<ee4572ee>] ip_nat_fn+0x42/0x185 [iptable_nat]
[<ee457628>] ip_nat_local_fn+0x34/0xa4 [iptable_nat]
[<c05d0cb4>] dst_output+0x0/0x7
[<c05c9744>] nf_iterate+0x30/0x61
[<c05d0cb4>] dst_output+0x0/0x7
[<c05c986a>] nf_hook_slow+0x3a/0x90
[<c05d0cb4>] dst_output+0x0/0x7
[<c05d2fb0>] ip_queue_xmit+0x3ba/0x40b
[<c05d0cb4>] dst_output+0x0/0x7
[<c05c2588>] __qdisc_run+0x66/0x1d3
[<c0424128>] local_bh_enable+0x5/0x81
[<c05b61e4>] dev_queue_xmit+0x2ca/0x2ea
[<ee442d48>] br_dev_queue_push_xmit+0x15b/0x17e [bridge]
[<c05e0b8c>] tcp_transmit_skb+0x5c7/0x5f5
[<c05e18ee>] tcp_retransmit_skb+0x4c0/0x59e
[<c05daaab>] tcp_enter_loss+0x1a2/0x1ff
[<c05e39dc>] tcp_write_timer+0x0/0x5dc
[<c05e3dda>] tcp_write_timer+0x3fe/0x5dc
[<c0427942>] run_timer_softirq+0x101/0x15c
[<c04243c3>] __do_softirq+0x8b/0x11c
[<c0406e5d>] do_softirq+0x56/0xae
[<c04463c8>] __do_IRQ+0x0/0xd6
[<c0406f6a>] do_IRQ+0xb5/0xc3
[<c054efc7>] evtchn_do_upcall+0xfa/0x191
[<c04055d9>] hypervisor_callback+0x3d/0x48
[<c0408664>] raw_safe_halt+0x8c/0xaf
[<c040321a>] xen_idle+0x22/0x2e
[<c0403339>] cpu_idle+0x91/0xab
=======================
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
Booting 'CentOS (2.6.18-128.1.6.el5xen)'
root (hd0,4)
Filesystem type is ext2fs, partition type 0x83
kernel /boot/xen.gz-2.6.18-128.1.6.el5 com1=115200,8n1 console=com1
[Multiboot-elf, <0x100000:0xae4c4:0x51b3c>, shtab=0x200078, entry=0x100000]
module /boot/vmlinuz-2.6.18-128.1.6.el5xen ro root=/dev/sda5 console=tty0 conso
le=ttyS0,115200
[Multiboot-module @ 0x201000, 0x47e0a0 bytes]
module /boot/initrd-2.6.18-128.1.6.el5xen.img
[Multiboot-module @ 0x680000, 0x5b0400 bytes]
__ __ _____ _ ____ _ ____ ___ _ __ _ ____
\ \/ /___ _ __ |___ / / | |___ \ / |___ \( _ ) / | / /_ ___| | ___|
\ // _ \ '_ \ |_ \ | | __) |__| | __) / _ \ | || '_ \ / _ \ |___ \
/ \ __/ | | | ___) || |_ / __/|__| |/ __/ (_) || || (_) | __/ |___) |
/_/\_\___|_| |_| |____(_)_(_)_____| |_|_____\___(_)_(_)___(_)___|_|____/
http://www.cl.cam.ac.uk/netos/xen
University of Cambridge Computer Laboratory
Xen version 3.1.2-128.1.6.el5 (mockbuild at centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) Wed Apr 1 09:11:34 EDT 2009
Latest ChangeSet: unavailable
(XEN) Command line: com1=115200,8n1 console=com1
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN) Found 4 MBR signatures
(XEN) Found 4 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f400 (usable)
(XEN) 000000000009f400 - 00000000000a0000 (reserved)
(XEN) 00000000000f0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bf6e0000 (usable)
(XEN) 00000000bf6e0000 - 00000000bf6e3000 (ACPI NVS)
(XEN) 00000000bf6e3000 - 00000000bf6f0000 (ACPI data)
(XEN) 00000000bf6f0000 - 00000000bf700000 (reserved)
(XEN) 00000000e0000000 - 00000000f0000000 (reserved)
(XEN) 00000000fec00000 - 0000000100000000 (reserved)
(XEN) System RAM: 3062MB (3135996kB)
(XEN) Xen heap: 9MB (10136kB)
(XEN) Domain heap initialised: DMA width 32 bits
(XEN) PAE enabled, limit: 16 GB
(XEN) Processor #0 15:4 APIC version 20
(XEN) Processor #1 15:4 APIC version 20
(XEN) IOAPIC[0]: apic_id 4, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 5, version 32, address 0xfec84400, GSI 24-47
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3200.316 MHz processor.
(XEN) CPU0: Intel(R) Pentium(R) 4 CPU 3.20GHz stepping 03
(XEN) Booting processor 1/1 eip 90000
(XEN) CPU1: Intel(R) Pentium(R) 4 CPU 3.20GHz stepping 03
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) Platform timer overflows in 2 jiffies.
(XEN) Platform timer is 1.193MHz PIT
(XEN) Brought up 2 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0xc0400000 memsz=0x2756cc
(XEN) elf_parse_binary: phdr: paddr=0xc0676000 memsz=0x161000
(XEN) elf_parse_binary: memory: 0xc0400000 -> 0xc07d7000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xc0000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0xc0000000
(XEN) elf_xen_parse_note: ENTRY = 0xc0400000
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xc0401000
(XEN) elf_xen_parse_note: FEATURES = "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel"
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_addr_calc_check: addresses:
(XEN) virt_base = 0xc0000000
(XEN) elf_paddr_offset = 0xc0000000
(XEN) virt_offset = 0x0
(XEN) virt_kstart = 0xc0400000
(XEN) virt_kend = 0xc07d7000
(XEN) virt_entry = 0xc0400000
(XEN) Xen kernel: 32-bit, PAE, lsb
(XEN) Dom0 kernel: 32-bit, PAE, lsb, paddr 0xc0400000 -> 0xc07d7000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 000000003c000000->000000003e000000 (670288 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: c0400000->c07d7000
(XEN) Init. ramdisk: c07d7000->c0d87400
(XEN) Phys-Mach map: c0d88000->c101e940
(XEN) Start info: c101f000->c101f46c
(XEN) Page tables: c1020000->c102f000
(XEN) Boot stack: c102f000->c1030000
(XEN) TOTAL: c0000000->c1400000
(XEN) ENTRY ADDRESS: c0400000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xc0400000 -> 0xc06756cc
(XEN) elf_load_binary: phdr 1 at 0xc0676000 -> 0xc071fc64
(XEN) Initrd len 0x5b0400, start at 0xc07d7000
(XEN) Scrubbing Free RAM: ...done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
(XEN) Freed 96kB init memory.
Linux version 2.6.18-128.1.6.el5xen (mockbuild at builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Apr 1 10:38:05 EDT 2009
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 00000000a6250000 (usable)
1930MB HIGHMEM available.
727MB LOWMEM available.
NX (Execute Disable) protection: active
found SMP MP-table at 000f3a60
DMI 2.3 present.
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 4, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x05] address[0xfec84400] gsi_base[24])
IOAPIC[1]: apic_id 5, version 32, address 0xfec84400, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Enabling APIC mode: Flat. Using 2 I/O APICs
Using ACPI (MADT) for SMP configuration information
Built 1 zonelists. Total pages: 680528
Kernel command line: ro root=/dev/sda5 console=tty0 console=ttyS0,115200
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c0744000 soft=c0724000
PID hash table entries: 4096 (order: 12, 16384 bytes)
Xen reported: 3200.316 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Software IO TLB enabled:
Aperture: 64 megabytes
Kernel range: 0x00000000c25ca000 - 0x00000000c65ca000
vmalloc area: ee000000-f4ffe000, maxmem 2d7fe000
Memory: 2610828k/2722112k available (2125k kernel code, 102104k reserved, 876k data, 176k init, 1976648k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 8006.93 BogoMIPS (lpj=16013879)
Security Framework v1.0.0 initialized
SELinux: Initializing.
selinux_register_security: Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 512
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
Checking 'hlt' instruction... OK.
SMP alternatives: switching to UP code
ACPI: Core revision 20060707
CPU 1 irqstacks, hard=c0745000 soft=c0725000
ENABLING IO-APIC IRQs
SMP alternatives: switching to SMP code
Brought up 2 CPUs
Initializing CPU#1
migration_cost=7
checking if image is initramfs... it is
Freeing initrd memory: 5825k freed
Grant table initialized
NET: Registered protocol family 16
No dock devices found.
ACPI: bus type pci registered
PCI: BIOS Bug: MCFG area at e0000000 is not E820-reserved
PCI: Not using MMCONFIG.
PCI: Using configuration type 1
Setting up standard PCI resources
Allocating PCI resources starting at c0000000 (gap: bf700000:20900000)
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI quirk: region 0400-047f claimed by ICH6 ACPI/GPIO/TCO
PCI quirk: region 0480-04bf claimed by ICH6 GPIO
PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 7 9 10 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 9 10 *12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 *9 10 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 9 *10 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 7 9 10 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 7 9 10 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK0] (IRQs 3 4 5 7 9 10 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK1] (IRQs 3 4 5 *7 9 10 12 14 15)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
(XEN) ioapic_guest_write: apic=0, pin=4, old_irq=4, new_irq=4
(XEN) ioapic_guest_write: old_entry=000009f1, new_entry=000109f1
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
pnp: PnP ACPI: found 13 devices
xen_mem: Initialising balloon driver.
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
pnp: 00:09: ioport range 0x400-0x4bf could not be reserved
PCI: Ignore bogus resource 6 [0:0] of 0000:00:02.0
PCI: Bridge: 0000:01:00.0
IO window: d000-dfff
MEM window: disabled.
PREFETCH window: disabled.
PCI: Bridge: 0000:00:01.0
IO window: d000-dfff
MEM window: d0000000-d00fffff
PREFETCH window: disabled.
PCI: Bridge: 0000:00:1c.0
IO window: disabled.
MEM window: d0100000-d01fffff
PREFETCH window: disabled.
PCI: Bridge: 0000:00:1c.1
IO window: disabled.
MEM window: d0200000-d02fffff
PREFETCH window: disabled.
PCI: Bridge: 0000:00:1e.0
IO window: disabled.
MEM window: disabled.
PREFETCH window: disabled.
ACPI: PCI Interrupt 0000:00:01.0[A] -> GSI 16 (level, low) -> IRQ 16
ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 16 (level, low) -> IRQ 16
ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI 17 (level, low) -> IRQ 17
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
audit: initializing netlink socket (disabled)
type=2000 audit(1239924769.500:1): initialized
highmem bounce pool size: 64 pages
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
alg: No test for crc32c (crc32c-generic)
ksign: Installing public key data
Loading keyring
- Added public key 88A2B9299D40C296
- User ID: CentOS (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
assign_interrupt_mode Found MSI capability
assign_interrupt_mode Found MSI capability
assign_interrupt_mode Found MSI capability
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
ACPI (exconfig-0456): Dynamic SSDT Load - OemId [ PmRef] OemTableId [ Cpu1Ist] [20060707]
Real Time Clock Driver v1.12ac
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
agpgart: Detected an Intel E7221 (i915) Chipset.
agpgart: Detected 7932K stolen memory.
agpgart: AGP aperture is 256M @ 0xc0000000
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
Xen virtual console successfully installed as ttyS0
Event-channel device installed.
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ICH6: IDE controller at PCI slot 0000:00:1f.1
ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI 18 (level, low) -> IRQ 18
ICH6: chipset revision 3
ICH6: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0xf008-0xf00f, BIOS settings: hdc:DMA, hdd:DMA
hdb: CD-224E, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
Using IPI No-Shortcut mode
Freeing unused kernel memory: 176k freed
Write protecting the kernel read-only data: 383k
Red Hat nash version 5.1.19.6 starting
Mounting proc filesystem
Mounting sysfs filesystem
Creating /dev
Creating initial device nodes
Setting up hotplug.
Creating block device nodes.
Loading ehci-hcd.ko module
PCI: Enabling device 0000:00:1d.7 (0000 -> 0002)
ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 23 (level, low) -> IRQ 19
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1d.7: irq 19, io mem 0xd03c0000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 8 ports detected
Loading ohci-hcd.ko module
Loading uhci-hcd.ko module
USB Universal Host Controller Interface driver v3.0
Loading jbd.ko module
Loading ext3.ko module
Loading scsi_mod.ko module
SCSI subsystem initialized
Loading sd_mod.ko module
Loading libata.ko module
Loading ahci.ko module
ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 19 (level, low) -> IRQ 20
ahci 0000:00:1f.2: AHCI 0001.0000 32 slots 4 ports 1.5 Gbps 0xf impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq pm led slum part
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
ata1: SATA max UDMA/133 abar m1024 at 0xd03c3000 port 0xd03c3100 irq 20
ata2: SATA max UDMA/133 abar m1024 at 0xd03c3000 port 0xd03c3180 irq 20
ata3: SATA max UDMA/133 abar m1024 at 0xd03c3000 port 0xd03c3200 irq 20
ata4: SATA max UDMA/133 abar m1024 at 0xd03c3000 port 0xd03c3280 irq 20
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: ATA-7: Maxtor 6B200S0, BANC1G10, max UDMA/133
ata1.00: 398297088 sectors, multi 0: LBA48 NCQ (not used)
ata1.00: configured for UDMA/133
ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata2.00: ATA-7: Maxtor 6B200S0, BANC1E00, max UDMA/133
ata2.00: 398297088 sectors, multi 0: LBA48 NCQ (not used)
ata2.00: configured for UDMA/133
ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata3.00: ATA-7: Maxtor 6L250S0, BACE1G10, max UDMA/133
ata3.00: 490234752 sectors, multi 0: LBA48 NCQ (depth 31/32)
ata3.00: configured for UDMA/133
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata4.00: ATA-7: Maxtor 6L250S0, BACE1G10, max UDMA/133
ata4.00: 490234752 sectors, multi 0: LBA48 NCQ (depth 31/32)
ata4.00: configured for UDMA/133
Vendor: ATA Model: Maxtor 6B200S0 Rev: BANC
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sda: 398297088 512-byte hdwr sectors (203928 MB)
sda: Write Protect is off
SCSI device sda: drive cache: write back
SCSI device sda: 398297088 512-byte hdwr sectors (203928 MB)
sda: Write Protect is off
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda4 < sda5 sda6 >
sd 0:0:0:0: Attached scsi disk sda
Vendor: ATA Model: Maxtor 6B200S0 Rev: BANC
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sdb: 398297088 512-byte hdwr sectors (203928 MB)
sdb: Write Protect is off
SCSI device sdb: drive cache: write back
SCSI device sdb: 398297088 512-byte hdwr sectors (203928 MB)
sdb: Write Protect is off
SCSI device sdb: drive cache: write back
sdb: unknown partition table
sd 1:0:0:0: Attached scsi disk sdb
Vendor: ATA Model: Maxtor 6L250S0 Rev: BACE
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sdc: 490234752 512-byte hdwr sectors (251000 MB)
sdc: Write Protect is off
SCSI device sdc: drive cache: write back
SCSI device sdc: 490234752 512-byte hdwr sectors (251000 MB)
sdc: Write Protect is off
SCSI device sdc: drive cache: write back
sdc: unknown partition table
sd 2:0:0:0: Attached scsi disk sdc
Vendor: ATA Model: Maxtor 6L250S0 Rev: BACE
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sdd: 490234752 512-byte hdwr sectors (251000 MB)
sdd: Write Protect is off
SCSI device sdd: drive cache: write back
SCSI device sdd: 490234752 512-byte hdwr sectors (251000 MB)
sdd: Write Protect is off
SCSI device sdd: drive cache: write back
sdd: unknown partition table
sd 3:0:0:0: Attached scsi disk sdd
Loading ata_piix.ko module
Loading sata_mv.ko module
Loading usb-storage.ko module
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
Waiting for driver initialization.
Loading dm-mod.ko module
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel at redhat.com
Loading dm-log.ko module
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Loading dm-mem-cache.ko module
Loading dm-region_hash.ko module
Loading dm-message.ko module
Loading dm-raid45.ko module
device-mapper: dm-raid45: initialized v0.2429
Waiting for driver initialization.
Scanning and configuring dmraid supported devices
Creating root device.
Mounting root filesystem.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
Setting up other filesystems.
Setting up new root fs
no fstab.sys, mounting internal defaults
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
type=1404 audit(1239924804.198:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
type=1403 audit(1239924804.442:3): policy loaded auid=4294967295 ses=4294967295
INIT: version 2.86 booting
Welcome to CentOS release 5.3 (Final)
Press 'I' to enter interactive startup.
Setting clock (utc): Thu Apr 16 16:33:26 PDT 2009 [ OK ]
Starting udev: [ OK ]
Loading default keymap (us): [ OK ]
Setting hostname gt3.baremetal.com: [ OK ]
No devices found
Setting up Logical Volume Management: 4 logical volume(s) in volume group "vg1" now active
[ OK ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/sda5
/dev/sda5: clean, 75650/1224000 files, 801841/2443880 blocks
[ OK ]
Remounting root filesystem in read-write mode: [ OK ]
Mounting local filesystems: [ OK ]
Enabling local filesystem quotas: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
INIT: Entering runlevel: 3
Entering non-interactive startup
Applying Intel CPU microcode update: [ OK ]
Starting monitoring for VG vg1: 4 logical volume(s) in volume group "vg1" monitored
[ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: [ OK ]
Starting auditd: [ OK ]
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Starting irqbalance: [ OK ]
Mounting other filesystems: [ OK ]
Starting lm_sensors: [ OK ]
Starting sshd: [ OK ]
Starting sendmail: