kvm-devel

[kvm-devel] kvm-67: kernel panic while booting debian-40r3-i386-businesscard.iso

[Jan L. <jl...@la...>]

Hi!

I'm preparing kvm-67 for debian. While testing i noticed a problem:

When booting the debian installer from the official CD [1] this problem:

CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 2048K
Compat vDSO mapped to ffffe000.
CPU: Intel QEMU Virtual CPU version 0.9.1 stepping 03
Checking 'hlt' instruction... OK.
ACPI: Core revision 20060707
invalid opcode: 0000 [#1]
Modules linked in:
CPU:    0                           
EIP:    0060:[<c01467be>]    Not tainted VLI
EFLAGS: 00010202   (2.6.18-6-486 #1)
EIP is at kmem_cache_zalloc+0x2a0x53
eax: 0000000a   ebx: c7fe75c0   ecx: c7fe9e00   edx: 000000d0
esi: c02c50c0   edi: 00000202   ebp: c036bd20   esp: c030ff80
ds: 007b   es: 007b   ss: 0068    
Process swapper (pid: 0, ti=c030e000 task=c02bd7a0 task.ti=c030e000)
Stack: 00000004 c028f968 c029c49a c0146d5b 00000004 00000000 00000014
c029c499
       00000046 c030ffc4 00000046 00000046 00000000 00000000 00039100
c0302800
       003a7007 c01c7e90 00000000 00000000 00000000 c01db3b5 c0378ce8
c01dcf07
Call Trace:                     
 [<c0146d5b>] kmem_cache_create+0x15e0x410
 [<c01c7e90>] acpi_os_create_cache+0x100x1c
 [<c01db3b5>] acpi_ut_create_caches+0x190x93
 [<c01dcf07>] acpi_ut_init_globals+0x50x1de
 [<c01dc5d1>] acpi_initialize_subsystem+0x1b0x56
 [<c0323a73>] acpi_early_init+0x450xfe
 [<c03105f5>] start_kernel+0x26b0x272
Code: c3 57 56 53 89 c6 9c 5f fa 8b 08 83 39 00 74 12 c7 41 0c 01 00 00
00 8b 01
 48 89 01 8b 5c 81 10 eb 07 e8 a5 fb ff ff 89 c3 57 9d <0f> 0d 0b 90 85
db 74 1b
 8b 56 10 31 c0 89 d1 c1 e9 02 89 df f3
EIP: [<c01467be>] kmem_cache_zalloc+0x2a0x53 SS:ESP 0068:c030ff80
 <0>Kernel panic - not syncing: Attempted to kill the idle task!

Similar errors in acpi-functions also happen with all other linux
kernels i have here (2.6.22-25). -no-acpi makes no difference.

Userspace 66 works (with -66 and -67 kernel module), userspace 67 has
this problem (with -66 and -67 kernel module).

Host is an ThinkPad X60 (Core Duo), kernel 2.6.25.

I've tried to bisect the userspace, but the bug seems to be introduced
with the merge from qemu-cvs. Any tips on how to bisect the individual
commits on the qemu-cvs branch?

Thanks,
Jan Lübbe

[1]: http:cdimage.debian.orgdebian-cd4.0_r3i386iso-cddebian-40r3-i386-businesscard.iso

Re: [kvm-devel] kvm-67: kernel panic while booting debian-40r3-i386-businesscard.iso

[Joerg R. <jo...@8b...>]

On Mon, Apr 28, 2008 at 06:50:22PM +0200, Jan Luebbe wrote:
> Hi!
> 
> I'm preparing kvm-67 for debian. While testing i noticed a problem:
> 
> When booting the debian installer from the official CD [1] this problem:
> 
> CPU: L1 I cache: 32K, L1 D cache: 32K
> CPU: L2 cache: 2048K
> Compat vDSO mapped to ffffe000.
> CPU: Intel QEMU Virtual CPU version 0.9.1 stepping 03
> Checking 'hlt' instruction... OK.
> ACPI: Core revision 20060707
> invalid opcode: 0000 [#1]
> Modules linked in:
> CPU:    0                           
> EIP:    0060:[<c01467be>]    Not tainted VLI
> EFLAGS: 00010202   (2.6.18-6-486 #1)
> EIP is at kmem_cache_zalloc+0x2a/0x53
> eax: 0000000a   ebx: c7fe75c0   ecx: c7fe9e00   edx: 000000d0
> esi: c02c50c0   edi: 00000202   ebp: c036bd20   esp: c030ff80
> ds: 007b   es: 007b   ss: 0068    
> Process swapper (pid: 0, ti=c030e000 task=c02bd7a0 task.ti=c030e000)
> Stack: 00000004 c028f968 c029c49a c0146d5b 00000004 00000000 00000014
> c029c499
>        00000046 c030ffc4 00000046 00000046 00000000 00000000 00039100
> c0302800
>        003a7007 c01c7e90 00000000 00000000 00000000 c01db3b5 c0378ce8
> c01dcf07
> Call Trace:                     
>  [<c0146d5b>] kmem_cache_create+0x15e/0x410
>  [<c01c7e90>] acpi_os_create_cache+0x10/0x1c
>  [<c01db3b5>] acpi_ut_create_caches+0x19/0x93
>  [<c01dcf07>] acpi_ut_init_globals+0x5/0x1de
>  [<c01dc5d1>] acpi_initialize_subsystem+0x1b/0x56
>  [<c0323a73>] acpi_early_init+0x45/0xfe
>  [<c03105f5>] start_kernel+0x26b/0x272
> Code: c3 57 56 53 89 c6 9c 5f fa 8b 08 83 39 00 74 12 c7 41 0c 01 00 00
> 00 8b 01
>  48 89 01 8b 5c 81 10 eb 07 e8 a5 fb ff ff 89 c3 57 9d <0f> 0d 0b 90 85
> db 74 1b
>  8b 56 10 31 c0 89 d1 c1 e9 02 89 df f3
> EIP: [<c01467be>] kmem_cache_zalloc+0x2a/0x53 SS:ESP 0068:c030ff80
>  <0>Kernel panic - not syncing: Attempted to kill the idle task!

I tried to reproduce this on an AMD system with no success. But when
looking into the code of kmem_cache_zalloc this looks like a guest state
corruption. The guest disables interrupts and the hypervisor reenables
them which triggers the BUG() macro. Maybe kvmtrace can give a hint
which intercept causes this.

Joerg

Re: [kvm-devel] kvm-67: kernel panic while booting debian-40r3-i386-businesscard.iso

[Avi K. <av...@qu...>]

Jan Luebbe wrote:
> Hi!
>
> I'm preparing kvm-67 for debian. While testing i noticed a problem:
>
> When booting the debian installer from the official CD [1] this problem:
>   

> Call Trace:                     
>  [<c0146d5b>] kmem_cache_create+0x15e/0x410
> Code: c3 57 56 53 89 c6 9c 5f fa 8b 08 83 39 00 74 12 c7 41 0c 01 00 00
> 00 8b 01
>  48 89 01 8b 5c 81 10 eb 07 e8 a5 fb ff ff 89 c3 57 9d <0f> 0d 0b 90 85
> db 74 1b
>  8b 56 10 31 c0 89 d1 c1 e9 02 89 df f3
> EIP: [<c01467be>] kmem_cache_zalloc+0x2a/0x53 SS:ESP 0068:c030ff80
>  <0>Kernel panic - not syncing: Attempted to kill the idle task!
>   
0f 0d 0b                prefetchw (%ebx)

This is an AMD 3Dnow! instruction, which is not supported on Intel 
processors.  I guess the 3Dnow! cpuid bit leaked in via the qemu merge.

I guess two fixes are needed:
- remove the 3Dnow! bit
- add emulation for prefetchw (easy, as it doesn't need to do anything) 
to support live migration from AMD to Intel

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [kvm-devel] kvm-67: kernel panic while booting debian-40r3-i386-businesscard.iso

[Jan L. <jl...@la...>]

On Fri, 2008-05-02 at 13:12 +0300, Avi Kivity wrote:
> > Call Trace:                     
> >  [<c0146d5b>] kmem_cache_create+0x15e/0x410
> > Code: c3 57 56 53 89 c6 9c 5f fa 8b 08 83 39 00 74 12 c7 41 0c 01 00 00
> > 00 8b 01
> >  48 89 01 8b 5c 81 10 eb 07 e8 a5 fb ff ff 89 c3 57 9d <0f> 0d 0b 90 85
> > db 74 1b
> >  8b 56 10 31 c0 89 d1 c1 e9 02 89 df f3
> > EIP: [<c01467be>] kmem_cache_zalloc+0x2a/0x53 SS:ESP 0068:c030ff80
> >  <0>Kernel panic - not syncing: Attempted to kill the idle task!
> >   
> 0f 0d 0b                prefetchw (%ebx)
> 
> This is an AMD 3Dnow! instruction, which is not supported on Intel 
> processors.  I guess the 3Dnow! cpuid bit leaked in via the qemu merge.
> 
> I guess two fixes are needed:
> - remove the 3Dnow! bit
> - add emulation for prefetchw (easy, as it doesn't need to do anything) 
> to support live migration from AMD to Intel

This problem still occours with kvm-68. Which CPUs will be affected by
this (is it only the Core Duo)?
I'm currently delaying the upload of a new kvm package to debian because
of this.

Thanks,
Jan

Re: [kvm-devel] kvm-67: kernel panic while booting debian-40r3-i386-businesscard.iso

[Avi K. <av...@qu...>]

Jan Luebbe wrote:
>> 0f 0d 0b                prefetchw (%ebx)
>>
>> This is an AMD 3Dnow! instruction, which is not supported on Intel 
>> processors.  I guess the 3Dnow! cpuid bit leaked in via the qemu merge.
>>
>> I guess two fixes are needed:
>> - remove the 3Dnow! bit
>> - add emulation for prefetchw (easy, as it doesn't need to do anything) 
>> to support live migration from AMD to Intel
>>     
>
> This problem still occours with kvm-68. Which CPUs will be affected by
> this (is it only the Core Duo)?
>   

All Intels.

> I'm currently delaying the upload of a new kvm package to debian because
> of this.
>   

I've fixed it for kvm-69.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.