[kvm-devel] Problem with SMP boot

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi,

I am only able to bring up a single cpu on my guest even when I
specify a 4 cpu smp config.  My test machine 
is a two socket dual core woodcrest with 4GB RAM. I am
using a stock RHEL5 kernel on my guest and 2.6.20-rc5 
on my host. 

The command I used is:
qemu-kvm -hda vdisk2.img -smp 4 -boot c -m 1024 -no-acpi

I've tried also without -no-acpi with the same result.

Now when I switch to qemu-system-x86_64 without kvm, the guest boots
with all 4 cpus being specified.  

>From the boot log, it looks like it is APIC related.
Is this a known problem?

Thanks.

Tim Chen

------------Boot log on guest below----------------

boot log for command
qemu-kvm -hda vdisk2.img -smp 4 -boot c -m 1024 -no-acpi

Linux version 2.6.18-1.3002.el5 (bre...@hs...) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-51)) #1 SMP Tue Jan 9 21:19:58 EST 2007
Command line: ro root=LABEL=/ rhgb quiet
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000040000000 (usable)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
DMI not present or invalid.
ACPI: Unable to locate RSDP
No NUMA configuration found
Faking a node at 0000000000000000-0000000040000000
Bootmem setup node 0 0000000000000000-0000000040000000
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: QEMUCPU  Product ID: 0.1          APIC at: 0xFEE00000
Processor #0 6:0 APIC version 17
Processor #1 6:0 APIC version 17
Processor #2 6:0 APIC version 17
Processor #3 6:0 APIC version 17
I/O APIC #4 Version 17 at 0xFEC00000.
Setting APIC routing to physical flat
Processors: 4
Nosave address range: 000000000009f000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000e8000
Nosave address range: 00000000000e8000 - 0000000000100000
Allocating PCI resources starting at 50000000 (gap: 40000000:bffc0000)
SMP: Allowing 4 CPUs, 0 hotplug CPUs
Built 1 zonelists.  Total pages: 257255
Kernel command line: ro root=LABEL=/ rhgb quiet
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
Checking aperture...
Memory: 1025624k/1048576k available (2397k kernel code, 22564k reserved, 1222k data, 196k init)
Calibrating delay using timer specific routine.. 5372.61 BogoMIPS (lpj=2686305)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 0/0 -> Node 0
SMP alternatives: switching to UP code
Using IO-APIC 4
activating NMI Watchdog ... done.
Using local APIC timer interrupts.
result 4636414
Detected 4.636 MHz APIC timer.
SMP alternatives: switching to SMP code
Booting processor 1/4 APIC 0x1
Not responding.
Inquiring remote APIC #1...
... APIC #1 ID: failed
... APIC #1 VERSION: failed
... APIC #1 SPIV: failed
SMP alternatives: switching to SMP code
Booting processor 2/3 APIC 0x2
Not responding.
Inquiring remote APIC #2...
... APIC #2 ID: failed
... APIC #2 VERSION: failed
... APIC #2 SPIV: failed
SMP alternatives: switching to SMP code
Booting processor 3/2 APIC 0x3
Not responding.
Inquiring remote APIC #3...
... APIC #3 ID: failed
... APIC #3 VERSION: failed
... APIC #3 SPIV: failed
Brought up 1 CPUs
testing NMI watchdog ... CPU#0: NMI appears to be stuck (0->0)!

-----------------------------------------------------
Boot log for sucessfule brought up of 4 cpus with command:
qemu-system-x86_64 -hda vdisk2.img -smp 4 -boot c -m 1024 -no-acpi

Linux version 2.6.18-1.3002.el5 (bre...@hs...) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-51)) #1 SMP Tue Jan 9 21:19:58 EST 2007
Command line: ro root=LABEL=/ rhgb acpi=off
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000040000000 (usable)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
DMI not present or invalid.
No NUMA configuration found
Faking a node at 0000000000000000-0000000040000000
Bootmem setup node 0 0000000000000000-0000000040000000
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: QEMUCPU  Product ID: 0.1          APIC at: 0xFEE00000
Processor #0 6:0 APIC version 17
Processor #1 6:0 APIC version 17
Processor #2 6:0 APIC version 17
Processor #3 6:0 APIC version 17
I/O APIC #4 Version 17 at 0xFEC00000.
Setting APIC routing to physical flat
Processors: 4
Nosave address range: 000000000009f000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000e8000
Nosave address range: 00000000000e8000 - 0000000000100000
Allocating PCI resources starting at 50000000 (gap: 40000000:bffc0000)
SMP: Allowing 4 CPUs, 0 hotplug CPUs
Built 1 zonelists.  Total pages: 257255
Kernel command line: ro root=LABEL=/ rhgb acpi=off
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
Checking aperture...
Memory: 1025624k/1048576k available (2397k kernel code, 22564k reserved, 1222k data, 196k init)
Calibrating delay using timer specific routine.. 5481.21 BogoMIPS (lpj=2740609)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
 starting audispd
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 0/0 -> Node 0
SMP alternatives: switching to UP code
Using IO-APIC 4
activating NMI Watchdog ... done.
Using local APIC timer interrupts.
result 62501278
Detected 62.501 MHz APIC timer.
SMP alternatives: switching to SMP code
Booting processor 1/4 APIC 0x1
Initializing CPU#1
calibrate_delay_direct() failed to get a good estimate for loops_per_jiffy.
Probably due to long platform interrupts. Consider using "lpj=" boot option.
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 1/0 -> Node 0
QEMU Virtual CPU version 0.8.2 stepping 03
CPU 1: Syncing TSC to CPU 0.
CPU 1: synchronized TSC with CPU 0 (last diff -108 cycles, maxerr 5205480 cycles)
SMP alternatives: switching to SMP code
Booting processor 2/4 APIC 0x2
Initializing CPU#2
calibrate_delay_direct() failed to get a good estimate for loops_per_jiffy.
Probably due to long platform interrupts. Consider using "lpj=" boot option.
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 2/0 -> Node 0
QEMU Virtual CPU version 0.8.2 stepping 03
CPU 2: Syncing TSC to CPU 0.
CPU 2: synchronized TSC with CPU 0 (last diff 1312276 cycles, maxerr 7810648 cycles)
SMP alternatives: switching to SMP code
Booting processor 3/4 APIC 0x3
Initializing CPU#3
calibrate_delay_direct() failed to get a good estimate for loops_per_jiffy.
Probably due to long platform interrupts. Consider using "lpj=" boot option.
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 3/0 -> Node 0
QEMU Virtual CPU version 0.8.2 stepping 03
CPU 3: Syncing TSC to CPU 0.
CPU 3: synchronized TSC with CPU 0 (last diff 2613452 cycles, maxerr 10414152 cycles)
Brought up 4 CPUs