kvm-devel — kernel virtual machine development

Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3)

[Avi K. <av...@qu...>]

David S. Ahern wrote:
> kvm_stat -1 is practically impossible to time correctly to get a good snippet.
>
> kvmtrace is a fascinating tool. I captured trace data that encompassed one
> intense period where the VM appeared to freeze (no terminal response for a few
> seconds).
>
> After converting to text I examined an arbitrary section in time (how do you
> correlate tsc to unix epoch?), and it shows vcpu0 hammered with interrupts and
> vcpu1 hammered with page faults. (I put the representative data below; I can
> send the binary or text files if you really want to see them.) All toll over
> about a 10-12 second time period the trace text files contain 8426221 lines and
> 2051344 of them are PAGE_FAULTs (that's 24% of the text lines which seems really
> high).
>
> david
>   

>
> vcpu1 data:
>
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db0 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db4 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db0 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000009, virt = 0x00000000 fffb6d28 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db4 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db0 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db4 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000003, virt = 0x00000000 c0009db0 ]
> 0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
> 0x00000009, virt = 0x00000000 fffb6d30 ]
>
>
>   

The pattern here is  c0009db4, c0009db0, fffb6xxx, c0009db0.  Setting a 
pte at c0009db0, accessing the page mapped by the pte, unmapping the 
pte.  Note that c0009db0 (bits 3:11) == 0x1b6 == fffb6xxx (bits 12:20).  
That's a kmap_atomic() + access +kunmap_atomic() sequence.

The expensive accesses ~50K cycles) seem to be the onces at fffb6xxx.  
Now theses shouldn't show up at all -- the kvm_mmu_pte_write() ought to 
have set up the ptes correctly.

Can you add a trace at mmu_guess_page_from_pte_write(), right before "if 
(is_present_pte(gpte))"?  I'm interested in gpa and gpte.  Also a trace 
at kvm_mmu_pte_write(), where it sets flooded = 1 (hmm, try to increase 
the 3 to 4 in the line right above that, maybe the fork detector is 
misfiring).


---------------------------------

vcpu0 data:

0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400020536 (+    1712)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400096784 (+   76248)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7a ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400098576 (+    1792)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400114528 (+   15952)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7a ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400116328 (+    1800)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400137216 (+   20888)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7a ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400138840 (+    1624)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400209344 (+   70504)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7c ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400211056 (+    1712)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400226312 (+   15256)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7c ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400228040 (+    1728)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400248688 (+   20648)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7c ]




Those are probably IPIs due to the kmaps above.

>
> Avi Kivity wrote:
>   
>> David S. Ahern wrote:
>>     
>>> I have been looking at RHEL3 based guests lately, and to say the least
>>> the
>>> performance is horrible. Rather than write a long tome on what I've
>>> done and
>>> observed, I'd like to find out if anyone has some insights or known
>>> problem
>>> areas running 2.4 guests. The short of it is that % system time spikes
>>> from time
>>> to time (e.g., on exec of a new process such as running /bin/true).
>>>
>>> I do not see the problem running RHEL3 on ESX, and an equivalent VM
>>> running
>>> RHEL4 runs fine. That suggests that the 2.4 kernel is doing something
>>> in a way
>>> that is not handled efficiently by kvm.
>>>
>>> Can someone shed some light on it?
>>>   
>>>       
>> It's not something that I test regularly.  If you're running a 32-bit
>> kernel, I'd suspect kmap(), or perhaps false positives from the fork
>> detector.
>>
>> kvmtrace will probably give enough info to tell exactly what's going on;
>> 'kvmstat -1' while the badness is happening may also help.
>>
>>     


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

[kvm-devel] (no subject)

[钟文辉 <she...@12...>]

                 


               各位老总：您们好！
                
                   诚祝：您们在2008年里；有鼠不尽的快乐！鼠不尽的收获！鼠不尽的钞票！ 
　
               鼠不尽的幸福！鼠不尽的美满生活！愿：您们阖家欢乐！幸福安康！
    
                   我是（深圳市珊湖岛进出口有限公司）的负责人；可以提供：出口报关单，
　
               核销单等等一系列手续；代理：出口报关，商检，境内外运输......等等；还可

               以代办：出口欧盟许可证，欧盟产地证；并且还有（广州国际贸易交易会）的摊

               位可以转让；有意者请来邮件或来电联系。
　
                                             电话：0755-81153047。
　
                                             传真：0755-81172940。
　
                                             手机：15817477278。
　
                                             联系人：钟文辉。
　
                         此致：
                                                                 

                                                                          敬礼！
　

Re: [kvm-devel] kvm-65/66 bug with Solaris 10 U4 ?

[Ian K. <bl...@bl...>]

Avi Kivity wrote:

> > Yes, that is very easy isn't it. Oops to my stupidity. I've got it built
> > and will give it a go tomorrow and report back on each test case.
>
> Please don't flame on kvm-devel, even if the flames are self-directed.

Er, OK...

[kvm-devel] (no subject)

[钟文辉 <she...@12...>]

                 


               各位老总：您们好！
                
                   诚祝：您们在2008年里；有鼠不尽的快乐！鼠不尽的收获！鼠不尽的钞票！ 
　
               鼠不尽的幸福！鼠不尽的美满生活！愿：您们阖家欢乐！幸福安康！
    
                   我是（深圳市珊湖岛进出口有限公司）的负责人；可以提供：出口报关单，
　
               核销单等等一系列手续；代理：出口报关，商检，境内外运输......等等；还可

               以代办：出口欧盟许可证，欧盟产地证；并且还有（广州国际贸易交易会）的摊

               位可以转让；有意者请来邮件或来电联系。
　
                                             电话：0755-81153047。
　
                                             传真：0755-81172940。
　
                                             手机：15817477278。
　
                                             联系人：钟文辉。
　
                         此致：
                                                                 

                                                                          敬礼！
　

Re: [kvm-devel] kvm-65/66 bug with Solaris 10 U4 ?

[Avi K. <av...@qu...>]

Ian Kirk wrote:
> Avi Kivity wrote:
>
>   
>> I do this regularly, basically you need to install kernel-devel and
>> that's it.
>>     
>
> Yes, that is very easy isn't it. Oops to my stupidity. I've got it built
> and will give it a go tomorrow and report back on each test case.
>   

Please don't flame on kvm-devel, even if the flames are self-directed.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [kvm-devel] kvm-65/66 bug with Solaris 10 U4 ?

[Ian K. <bl...@bl...>]

Avi Kivity wrote:

> Actually kvm is affected by pae: it enables nx support.  Please try
> (separately)
>
>  1.  Boot with 'noexec=off' on the host kernel command line

2.6.24.4-64.fc8PAE noexec=off:

Using normal F8 modules
qemu-kvm dies in the same way

>  2.  Loading the kernel modules that come with kvm-66

Against 2.6.24.4-64.fc8 it works.

I can't compile them against 2.6.24.4-64.fc8PAE as the module magic name
mismatches, and I don't know how to change kernel-devel to know it's PAE.

Probably won't be able to do any more tests that require a reboot till
Tuesday now, but feel free to leave me some things to try.

Ian

Re: [kvm-devel] [kvm-ppc-devel] [PATCH 1/5]Add some trace markers and exposeinterfaces in kernel for tracing

[Christian E. <ehr...@li...>]

Liu, Eric E wrote:
> Hollis Blanchard wrote:
>> On Wednesday 16 April 2008 01:45:34 Liu, Eric E wrote:
[...]
>> Actually... we could have kvmtrace itself insert the metadata, so
>> there would be no chance of it being overwritten in the kernel
>> buffers. The header could be written in tip_open_output(), and update
>> fs_size accordingly. 
>>
> Yes, let kvmtrace insert the metadata is more reasonable.
> 

I wanted to note that the kvmtrace tool should, but not need to know everything about the data format.
I think of e.g. changing kernel implementations that change endianess or even flags we don't yet know, but we might need in the future.

What about adding another debugfs entry the kernel can use to expose the "kvmtrace-metadata" defined by the kernel implementation.
The kvmtrace tool could then use that to build up the record by using one entry for kernel defined metadata and another to add any metadata that would be defined by kvmtrace tool itself.

what about that one:
	struct metadata {
		u32 kmagic; /* stores kernel defined metadata read from debugfs entry */
		u32 umagic; /* stores userspace tool defined metadata */
		u32 extra;  /* it is redundant, only use to fit into record. */
	} 

That should give us the flexibility to keep the format if we get more metadata requirements in the future.

-- 

Grüsse / regards, 
Christian Ehrhardt
IBM Linux Technology Center, Open Virtualization

Re: [kvm-devel] [PATCH 1/1] QEMU/KVM: Support for PCI Passthrough

[Amit S. <ami...@qu...>]

* On Monday 14 Apr 2008 06:01:07 Samuel Masham wrote:
> On Sun, Apr 13, 2008 at 9:49 PM, Dor Laor <dor...@qu...> wrote:
> >  On Thu, 2008-04-10 at 11:48 +0300, Amit Shah wrote:
> >  > If kvm uses the in-kernel irqchip, interrupts are routed to
> >  > the guest via the kvm module (accompanied kernel changes are
> >  > necessar). If -no-kvm-irqchip is used, the 'irqhook' module, also
> >  > included here, is to be used.
> >
> >  IMO we can drop the pci pass through support in userspace and only keep
> >  the in-kenel chip path. Normally we do want the option of running w/o
> >  kernel devices but it's only to test for regressions against userspace.
> >  In this case it's all new code so there is no point to compare, it's
> >  just lots of complicated code (userspace apic, irqhook module) that only
> >  helpful for qemu. Since it's not likely qemu will merge it, let's stick
> >  with the main path.
> >  Comments?
>
> Please keep the userspace support alive.
>
> I am particularly interested in using the pci-passthough to qemu
> running non x86 system emulation
> (at the moment mips)
>
> My hope is that the pci - passthough could help with developing
> drivers and testing across architectures...

OK; keeping support around won't be too much of a hassle, though the current 
support for pci-passthrough and the irqhook module are developed with x86 in 
mind (and only tested on x86).

Since it's not tested on any other architecture, I've marked it TARGET_I386 
and TARGET_X86_64 for now. Feel free to extend it to other architectures.

Amit.

Re: [kvm-devel] [kvm-ppc-devel] [PATCH 1/5]Add some trace markers and exposeinterfaces in kernel for tracing

[Liu, E. E <eri...@in...>]

Hollis Blanchard wrote:
> On Wednesday 16 April 2008 01:45:34 Liu, Eric E wrote:
>> 
>>> Hmm, while we're on the subject, I'm not sure what the best way to
>>> automatically byteswap will be. It probably isn't worth it to
>>> convert all trace data to a standard ordering (which would add
>>> overhead to tracing), but I suppose there is no metadata in the
>>> trace log? A command line switch might be inconvenient but
>>> inevitable. 
>> 
>> A tricky approach is that we insert medadata to the trace file before
> reading the trace log, so that the analysis tool can look at the
> medadata to check whether we need to convert byte order?
> 
> Actually, can't we lose trace records? It would be unfortunate if the
> magic metadata record was overwritten by the trace data. Perhaps a
> 1-byte "flags" variable at the beginning of each record could
> indicate the data's endianness?
> 
> Another option would be to have the "kvmtrace" tool transcribe the
> data itself. I think that would be fine, since kvmtrace must run on
> the target anyways, but your kvmtrace implementation doesn't actually
> understand the format of the data.
> 
> Actually... we could have kvmtrace itself insert the metadata, so
> there would be no chance of it being overwritten in the kernel
> buffers. The header could be written in tip_open_output(), and update
> fs_size accordingly. 
> 
Yes, let kvmtrace insert the metadata is more reasonable.

> Do you have any suggestions for the format of the metadata? I'm not
> sure how it should fit into the record format expected by
> kvmtrace_format. 
I think maybe it can like this: 
	struct metadata {
		u32 magic;
		u64 extra;  /* it is redundant, only use to fit into
record. */
	} 
kvmtrace_format can check magic to judge whethe we need to convert byte
order.

[kvm-devel] QEMU/KVM: clear HF_HALTED mask at vcpu startup time

[Marcelo T. <mto...@re...>]

Now that threads are spinned up before machine->init(), clearing
of HF_HALTED_MASK for irqchip in kernel case needs to be moved
to actual vcpu startup.

Signed-off-by: Marcelo Tosatti <mto...@re...>

diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
index f7a217a..50589a7 100644
--- a/qemu/qemu-kvm.c
+++ b/qemu/qemu-kvm.c
@@ -325,6 +325,8 @@ static int kvm_main_loop_cpu(CPUState *env)
 
     setup_kernel_sigmask(env);
     pthread_mutex_lock(&qemu_mutex);
+    if (kvm_irqchip_in_kernel(kvm_context))
+	env->hflags &= ~HF_HALTED_MASK;
 
     kvm_qemu_init_env(env);
     env->ready_for_interrupt_injection = 1;
@@ -368,8 +370,6 @@ static void *ap_main_loop(void *_env)
     sigprocmask(SIG_BLOCK, &signals, NULL);
     kvm_create_vcpu(kvm_context, env->cpu_index);
     kvm_qemu_init_env(env);
-    if (kvm_irqchip_in_kernel(kvm_context))
-	env->hflags &= ~HF_HALTED_MASK;
     kvm_main_loop_cpu(env);
     return NULL;
 }

Re: [kvm-devel] [PATCH 1 of 9] Lock the entire mm to prevent any mmu related operation to happen

[Andrea A. <an...@qu...>]

On Thu, Apr 17, 2008 at 12:10:52PM -0700, Christoph Lameter wrote:
> EMM was/is using a single linked list which allows atomic updates. Looked 
> cleaner to me since doubly linked list must update two pointers.

Cleaner would be if it would provide an abstraction in list.h. The
important is the memory taken by the head for this usage.

> I have not seen docs on the locking so not sure why you use rcu 
> operations here? Isnt the requirement to have either rmap locks or 
> mmap_sem held enough to guarantee the consistency of the doubly linked list?

Yes, exactly, I'm not using rcu anymore.

Re: [kvm-devel] [kvm-ppc-devel] [PATCH 1/5]Add some trace markers and exposeinterfaces in kernel for tracing

[Hollis B. <ho...@us...>]

On Wednesday 16 April 2008 01:45:34 Liu, Eric E wrote:
> 
> > Hmm, while we're on the subject, I'm not sure what the best way to
> > automatically byteswap will be. It probably isn't worth it to convert
> > all trace data to a standard ordering (which would add overhead to
> > tracing), but I suppose there is no metadata in the trace log? A
> > command line switch might be inconvenient but inevitable.
> 
> A tricky approach is that we insert medadata to the trace file before 
reading the trace log, so that the analysis tool can look at the medadata to 
check whether we need to convert byte order?  

Actually, can't we lose trace records? It would be unfortunate if the magic 
metadata record was overwritten by the trace data. Perhaps a 1-byte "flags" 
variable at the beginning of each record could indicate the data's 
endianness?

Another option would be to have the "kvmtrace" tool transcribe the data 
itself. I think that would be fine, since kvmtrace must run on the target 
anyways, but your kvmtrace implementation doesn't actually understand the 
format of the data.

Actually... we could have kvmtrace itself insert the metadata, so there would 
be no chance of it being overwritten in the kernel buffers. The header could 
be written in tip_open_output(), and update fs_size accordingly.

Do you have any suggestions for the format of the metadata? I'm not sure how 
it should fit into the record format expected by kvmtrace_format.

-- 
Hollis Blanchard
IBM Linux Technology Center

[kvm-devel] [PATCH] Fix missing decleration for kvm_enabled() in qemu for target-ppc/helper.c

[Jerone Y. <jy...@us...>]

1 file changed, 1 insertion(+)
qemu/target-ppc/helper.c |    1 +


Recent change now requires target-ppc/helper.c to now include "qemu-kvm.h" to get the definition for kvm_enabled(). This fixes it so things now compile again.

Signed-off-by: Jerone Young <jy...@us...>

diff --git a/qemu/target-ppc/helper.c b/qemu/target-ppc/helper.c
--- a/qemu/target-ppc/helper.c
+++ b/qemu/target-ppc/helper.c
@@ -29,6 +29,7 @@
 #include "exec-all.h"
 #include "helper_regs.h"
 #include "qemu-common.h"
+#include "qemu-kvm.h"
 
 //#define DEBUG_MMU
 //#define DEBUG_BATS

Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3)

[David S. A. <da...@ci...>]

kvm_stat -1 is practically impossible to time correctly to get a good snippet.

kvmtrace is a fascinating tool. I captured trace data that encompassed one
intense period where the VM appeared to freeze (no terminal response for a few
seconds).

After converting to text I examined an arbitrary section in time (how do you
correlate tsc to unix epoch?), and it shows vcpu0 hammered with interrupts and
vcpu1 hammered with page faults. (I put the representative data below; I can
send the binary or text files if you really want to see them.) All toll over
about a 10-12 second time period the trace text files contain 8426221 lines and
2051344 of them are PAGE_FAULTs (that's 24% of the text lines which seems really
high).

david

---------------------------------

vcpu0 data:

0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400020536 (+    1712)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400096784 (+   76248)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7a ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400098576 (+    1792)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400114528 (+   15952)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7a ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400116328 (+    1800)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400137216 (+   20888)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7a ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400138840 (+    1624)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400209344 (+   70504)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7c ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400211056 (+    1712)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400226312 (+   15256)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7c ]
0 (+       0)  INTR          vcpu = 0x00000001  pid = 0x000011ea [ vector = 0x00 ]
9968400228040 (+    1728)  VMENTRY       vcpu = 0x00000001  pid = 0x000011ea
9968400248688 (+   20648)  VMEXIT        vcpu = 0x00000001  pid = 0x000011ea [
exitcode = 0x00000001, rip = 0x00000000 c0154d7c ]




vcpu1 data:

9968400002032 (+    3808)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c016127f ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db0 ]
9968400005448 (+    3416)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400009832 (+    4384)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c016104a ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x0000000b, virt = 0x00000000 fffb6f88 ]
9968400071584 (+   61752)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400075608 (+    4024)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c01610e7 ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db4 ]
9968400083528 (+    7920)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400087288 (+    3760)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c01610ee ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db0 ]
9968400097312 (+   10024)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400103064 (+    5752)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160f9c ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db4 ]
9968400116624 (+   13560)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400120424 (+    3800)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160fa1 ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db0 ]
9968400123856 (+    3432)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400128208 (+    4352)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160dab ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000009, virt = 0x00000000 fffb6d28 ]
9968400183848 (+   55640)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400188232 (+    4384)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160e4d ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db4 ]
9968400196160 (+    7928)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400199928 (+    3768)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160e54 ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db0 ]
9968400209864 (+    9936)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400214984 (+    5120)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160f9c ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db4 ]
9968400228232 (+   13248)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400232000 (+    3768)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160fa1 ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000003, virt = 0x00000000 c0009db0 ]
9968400235424 (+    3424)  VMENTRY       vcpu = 0x00000000  pid = 0x000011ea
9968400239816 (+    4392)  VMEXIT        vcpu = 0x00000000  pid = 0x000011ea [
exitcode = 0x00000000, rip = 0x00000000 c0160dab ]
0 (+       0)  PAGE_FAULT    vcpu = 0x00000000  pid = 0x000011ea [ errorcode =
0x00000009, virt = 0x00000000 fffb6d30 ]




Avi Kivity wrote:
> David S. Ahern wrote:
>> I have been looking at RHEL3 based guests lately, and to say the least
>> the
>> performance is horrible. Rather than write a long tome on what I've
>> done and
>> observed, I'd like to find out if anyone has some insights or known
>> problem
>> areas running 2.4 guests. The short of it is that % system time spikes
>> from time
>> to time (e.g., on exec of a new process such as running /bin/true).
>>
>> I do not see the problem running RHEL3 on ESX, and an equivalent VM
>> running
>> RHEL4 runs fine. That suggests that the 2.4 kernel is doing something
>> in a way
>> that is not handled efficiently by kvm.
>>
>> Can someone shed some light on it?
>>   
> 
> It's not something that I test regularly.  If you're running a 32-bit
> kernel, I'd suspect kmap(), or perhaps false positives from the fork
> detector.
> 
> kvmtrace will probably give enough info to tell exactly what's going on;
> 'kvmstat -1' while the badness is happening may also help.
> 

[kvm-devel] [patch 13/13] QEMU/KVM: get rid of global lock

[Marcelo T. <mto...@re...>]

Now that QEMU does all necessary locking itself, get rid of the global mutex lock.

Allows more than one thread inside QEMU simultaneously.

Index: kvm-userspace.io/qemu/qemu-kvm.c
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-kvm.c
+++ kvm-userspace.io/qemu/qemu-kvm.c
@@ -145,7 +145,6 @@ static int try_push_interrupts(void *opa
 static void post_kvm_run(void *opaque, int vcpu)
 {
 
-    pthread_mutex_lock(&qemu_mutex);
     kvm_arch_post_kvm_run(opaque, vcpu);
 }
 
@@ -157,7 +156,6 @@ static int pre_kvm_run(void *opaque, int
 
     if (env->interrupt_request & CPU_INTERRUPT_EXIT)
 	return 1;
-    pthread_mutex_unlock(&qemu_mutex);
     return 0;
 }
 
@@ -228,7 +226,6 @@ static int kvm_eat_signal(struct qemu_kv
     if (r == -1 && (errno == EAGAIN || errno == EINTR) && !timeout)
 	return 0;
     e = errno;
-    pthread_mutex_lock(&qemu_mutex);
     if (r == -1 && !(errno == EAGAIN || errno == EINTR)) {
 	printf("sigtimedwait: %s\n", strerror(e));
 	exit(1);
@@ -241,7 +238,6 @@ static int kvm_eat_signal(struct qemu_kv
 	vcpu_info[env->cpu_index].stopped = 1;
 	pthread_kill(io_thread, SIGUSR1);
     }
-    pthread_mutex_unlock(&qemu_mutex);
 
     return ret;
 }
@@ -264,9 +260,7 @@ static void kvm_eat_signals(CPUState *en
 
 static void kvm_main_loop_wait(CPUState *env, int timeout)
 {
-    pthread_mutex_unlock(&qemu_mutex);
     kvm_eat_signals(env, timeout);
-    pthread_mutex_lock(&qemu_mutex);
     vcpu_info[env->cpu_index].signalled = 0;
 }
 
@@ -339,7 +333,6 @@ static int kvm_main_loop_cpu(CPUState *e
     struct vcpu_info *info = &vcpu_info[env->cpu_index];
 
     setup_kernel_sigmask(env);
-    pthread_mutex_lock(&qemu_mutex);
 
     kvm_qemu_init_env(env);
     env->ready_for_interrupt_injection = 1;
@@ -367,7 +360,6 @@ static int kvm_main_loop_cpu(CPUState *e
 	    kvm_arch_load_regs(env);
 	}
     }
-    pthread_mutex_unlock(&qemu_mutex);
     return 0;
 }
 
@@ -456,12 +448,10 @@ void qemu_kvm_notify_work(void)
 int kvm_main_loop(void)
 {
     io_thread = pthread_self();
-    pthread_mutex_unlock(&qemu_mutex);
     while (1) {
         if (get_cpu_env())
             hw_error("io thread has valid env\n");
         kvm_eat_signal(&io_signal_table, NULL, 1000);
-        pthread_mutex_lock(&qemu_mutex);
         main_loop_wait(0);
         if (qemu_shutdown_requested())
             break;
@@ -471,10 +461,8 @@ int kvm_main_loop(void)
             pthread_kill(vcpu_info[0].thread, SIG_IPI);
             qemu_kvm_reset_requested = 1;
         }
-        pthread_mutex_unlock(&qemu_mutex);
     }
 
-    pthread_mutex_unlock(&qemu_mutex);
     return 0;
 }
 
@@ -614,7 +602,6 @@ int kvm_qemu_init()
     if (!kvm_context) {
       	return -1;
     }
-    pthread_mutex_lock(&qemu_mutex);
 
     return 0;
 }
@@ -833,12 +820,13 @@ void qemu_kvm_aio_wait(void)
 
     /* io thread */
     if (!env) {
-        pthread_mutex_unlock(&qemu_mutex);
         kvm_eat_signal(&io_signal_table, NULL, 1000);
-        pthread_mutex_lock(&qemu_mutex);
     /* vcpu thread */
-    } else
+    } else {
+        pthread_mutex_lock(&qemu_mutex);
         pthread_cond_wait(&qemu_aio_cond, &qemu_mutex);
+        pthread_mutex_unlock(&qemu_mutex);
+    }
 }
 
 void qemu_kvm_aio_wait_end(void)

-- 

[kvm-devel] [patch 12/13] QEMU: scsi-disk reentrancy fix

[Marcelo T. <mto...@re...>]

Unrelated to this series. Ignore it.

Index: kvm-userspace.io/qemu/hw/scsi-disk.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/scsi-disk.c
+++ kvm-userspace.io/qemu/hw/scsi-disk.c
@@ -196,12 +196,12 @@ static void scsi_read_data(SCSIDevice *d
         n = SCSI_DMA_BUF_SIZE / 512;
 
     r->buf_len = n * 512;
-    r->aiocb = bdrv_aio_read(s->bdrv, r->sector, r->dma_buf, n,
+    r->sector += n;
+    r->sector_count -= n;
+    r->aiocb = bdrv_aio_read(s->bdrv, r->sector - n, r->dma_buf, n,
                              scsi_read_complete, r);
     if (r->aiocb == NULL)
         scsi_command_complete(r, SENSE_HARDWARE_ERROR);
-    r->sector += n;
-    r->sector_count -= n;
 }
 
 static void scsi_write_complete(void * opaque, int ret)
@@ -248,12 +248,12 @@ static int scsi_write_data(SCSIDevice *d
         BADF("Data transfer already in progress\n");
     n = r->buf_len / 512;
     if (n) {
-        r->aiocb = bdrv_aio_write(s->bdrv, r->sector, r->dma_buf, n,
+        r->sector += n;
+        r->sector_count -= n;
+        r->aiocb = bdrv_aio_write(s->bdrv, r->sector - n, r->dma_buf, n,
                                   scsi_write_complete, r);
         if (r->aiocb == NULL)
             scsi_command_complete(r, SENSE_HARDWARE_ERROR);
-        r->sector += n;
-        r->sector_count -= n;
     } else {
         /* Invoke completion routine to fetch data from host.  */
         scsi_write_complete(r, 0);

-- 

[kvm-devel] [patch 11/13] QEMU: block device locking

[Marcelo T. <mto...@re...>]

Grab device locks when moving data through block devices in the host->guest 
direction.

Also protect the aio_list by a separate lock.

Index: kvm-userspace.io/qemu/block-qcow.c
===================================================================
--- kvm-userspace.io.orig/qemu/block-qcow.c
+++ kvm-userspace.io/qemu/block-qcow.c
@@ -543,7 +543,9 @@ static void qcow_aio_read_cb(void *opaqu
     acb->hd_aiocb = NULL;
     if (ret < 0) {
     fail:
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, ret);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
@@ -568,7 +570,9 @@ static void qcow_aio_read_cb(void *opaqu
 
     if (acb->nb_sectors == 0) {
         /* request completed */
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, 0);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
@@ -646,7 +650,9 @@ static void qcow_aio_write_cb(void *opaq
 
     if (ret < 0) {
     fail:
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, ret);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
@@ -657,7 +663,9 @@ static void qcow_aio_write_cb(void *opaq
 
     if (acb->nb_sectors == 0) {
         /* request completed */
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, 0);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
Index: kvm-userspace.io/qemu/block-raw-posix.c
===================================================================
--- kvm-userspace.io.orig/qemu/block-raw-posix.c
+++ kvm-userspace.io/qemu/block-raw-posix.c
@@ -260,11 +260,14 @@ static void aio_signal_handler(int signu
 #endif
 }
 
+qemu_mutex_t aio_list_lock;
+
 void qemu_aio_init(void)
 {
     struct sigaction act;
 
     aio_initialized = 1;
+    qemu_mutex_init(&aio_list_lock);
 
     sigfillset(&act.sa_mask);
     act.sa_flags = 0; /* do not restart syscalls to interrupt select() */
@@ -291,6 +294,7 @@ void qemu_aio_poll(void)
     int ret;
 
     for(;;) {
+        qemu_mutex_lock(&aio_list_lock);
         pacb = &first_aio;
         for(;;) {
             acb = *pacb;
@@ -300,7 +304,9 @@ void qemu_aio_poll(void)
             if (ret == ECANCELED) {
                 /* remove the request */
                 *pacb = acb->next;
+                qemu_mutex_unlock(&aio_list_lock);
                 qemu_aio_release(acb);
+                break;
             } else if (ret != EINPROGRESS) {
                 /* end of aio */
                 if (ret == 0) {
@@ -314,8 +320,11 @@ void qemu_aio_poll(void)
                 }
                 /* remove the request */
                 *pacb = acb->next;
+                qemu_mutex_unlock(&aio_list_lock);
                 /* call the callback */
+                qemu_aio_lock(acb->common.bs);
                 acb->common.cb(acb->common.opaque, ret);
+                qemu_aio_unlock(acb->common.bs);
                 qemu_aio_release(acb);
                 break;
             } else {
@@ -323,7 +332,8 @@ void qemu_aio_poll(void)
             }
         }
     }
- the_end: ;
+ the_end:
+    qemu_mutex_unlock(&aio_list_lock);
 }
 
 /* Wait for all IO requests to complete.  */
@@ -331,6 +341,7 @@ void qemu_aio_flush(void)
 {
     qemu_aio_wait_start();
     qemu_aio_poll();
+    /* FIXME: first_aio is protected by aio_list_lock. */
     while (first_aio) {
         qemu_aio_wait();
     }
@@ -410,8 +421,10 @@ static RawAIOCB *raw_aio_setup(BlockDriv
     else
         acb->aiocb.aio_nbytes = nb_sectors * 512;
     acb->aiocb.aio_offset = sector_num * 512;
+    qemu_mutex_lock(&aio_list_lock);
     acb->next = first_aio;
     first_aio = acb;
+    qemu_mutex_unlock(&aio_list_lock);
     return acb;
 }
 
@@ -451,7 +464,6 @@ static void raw_aio_cancel(BlockDriverAI
 {
     int ret;
     RawAIOCB *acb = (RawAIOCB *)blockacb;
-    RawAIOCB **pacb;
 
     ret = aio_cancel(acb->aiocb.aio_fildes, &acb->aiocb);
     if (ret == AIO_NOTCANCELED) {
@@ -459,7 +471,9 @@ static void raw_aio_cancel(BlockDriverAI
            it */
         while (aio_error(&acb->aiocb) == EINPROGRESS);
     }
+    /* qemu_aio_poll will remove it from the queue */
 
+#if 0
     /* remove the callback from the queue */
     pacb = &first_aio;
     for(;;) {
@@ -467,11 +481,14 @@ static void raw_aio_cancel(BlockDriverAI
             break;
         } else if (*pacb == acb) {
             *pacb = acb->next;
+            qemu_mutex_unlock(&aio_list_lock);
             qemu_aio_release(acb);
+            qemu_mutex_lock(&aio_list_lock);
             break;
         }
         pacb = &acb->next;
     }
+#endif
 }
 
 static void raw_close(BlockDriverState *bs)
Index: kvm-userspace.io/qemu/block.c
===================================================================
--- kvm-userspace.io.orig/qemu/block.c
+++ kvm-userspace.io/qemu/block.c
@@ -501,6 +501,21 @@ int bdrv_commit(BlockDriverState *bs)
     return 0;
 }
 
+void qemu_aio_lock(BlockDriverState *bs)
+{
+    if (!bs->qemu_dev) {
+        unsigned long *ptr = 0;
+        printf("bs->drv = NULL\n");
+        *ptr = 0;
+    }
+    qemu_mutex_lock(&bs->qemu_dev->lock);
+}
+
+void qemu_aio_unlock(BlockDriverState *bs)
+{
+    qemu_mutex_unlock(&bs->qemu_dev->lock);
+}
+
 /* return < 0 if error. See bdrv_write() for the return codes */
 int bdrv_read(BlockDriverState *bs, int64_t sector_num,
               uint8_t *buf, int nb_sectors)
@@ -1326,7 +1341,9 @@ static void bdrv_aio_cancel_em(BlockDriv
 static void bdrv_aio_bh_cb(void *opaque)
 {
     BlockDriverAIOCBSync *acb = opaque;
+    qemu_aio_lock(acb->common.bs);
     acb->common.cb(acb->common.opaque, acb->ret);
+    qemu_aio_unlock(acb->common.bs);
     qemu_aio_release(acb);
 }
 
@@ -1365,6 +1382,7 @@ static BlockDriverAIOCB *bdrv_aio_write_
 static void bdrv_aio_cancel_em(BlockDriverAIOCB *blockacb)
 {
     BlockDriverAIOCBSync *acb = (BlockDriverAIOCBSync *)blockacb;
+    /* FIXME: the bh list needs a lock */
     qemu_bh_cancel(acb->bh);
     qemu_aio_release(acb);
 }
@@ -1459,7 +1477,6 @@ void *qemu_aio_get(BlockDriverState *bs,
 
 void qemu_aio_release(void *p)
 {
-    BlockDriverAIOCB *acb = p;
     qemu_free(p);
 }
 
Index: kvm-userspace.io/qemu/block.h
===================================================================
--- kvm-userspace.io.orig/qemu/block.h
+++ kvm-userspace.io/qemu/block.h
@@ -96,6 +96,9 @@ void qemu_aio_wait_start(void);
 void qemu_aio_wait(void);
 void qemu_aio_wait_end(void);
 
+void qemu_aio_lock(BlockDriverState *bs);
+void qemu_aio_unlock(BlockDriverState *bs);
+
 int qemu_key_check(BlockDriverState *bs, const char *name);
 
 /* Ensure contents are flushed to disk.  */
Index: kvm-userspace.io/qemu/block_int.h
===================================================================
--- kvm-userspace.io.orig/qemu/block_int.h
+++ kvm-userspace.io/qemu/block_int.h
@@ -130,6 +130,7 @@ struct BlockDriverState {
     char device_name[32];
     /* PCI devfn of parent */
     int devfn;
+    QEMUDevice *qemu_dev;
     BlockDriverState *next;
 };
 
Index: kvm-userspace.io/qemu/block-qcow2.c
===================================================================
--- kvm-userspace.io.orig/qemu/block-qcow2.c
+++ kvm-userspace.io/qemu/block-qcow2.c
@@ -812,7 +812,9 @@ static void qcow_aio_read_cb(void *opaqu
     acb->hd_aiocb = NULL;
     if (ret < 0) {
     fail:
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, ret);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
@@ -837,7 +839,9 @@ static void qcow_aio_read_cb(void *opaqu
 
     if (acb->nb_sectors == 0) {
         /* request completed */
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, 0);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
@@ -933,7 +937,9 @@ static void qcow_aio_write_cb(void *opaq
 
     if (ret < 0) {
     fail:
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, ret);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
@@ -944,7 +950,9 @@ static void qcow_aio_write_cb(void *opaq
 
     if (acb->nb_sectors == 0) {
         /* request completed */
+        qemu_aio_lock(bs);
         acb->common.cb(acb->common.opaque, 0);
+        qemu_aio_unlock(bs);
         qemu_aio_release(acb);
         return;
     }
Index: kvm-userspace.io/qemu/hw/ide.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/ide.c
+++ kvm-userspace.io/qemu/hw/ide.c
@@ -2983,8 +2983,10 @@ void pci_piix3_ide_init(PCIBus *bus, Blo
     ide_init_ioport(&d->ide_if[2], 0x170, 0x376, &d->dev.qemu_dev);
 
     for (i = 0; i < 4; i++)
-        if (hd_table[i])
+        if (hd_table[i]) {
+            hd_table[i]->qemu_dev = &d->dev.qemu_dev;
             hd_table[i]->devfn = d->dev.devfn;
+        }
 
     register_savevm("ide", 0, 1, pci_ide_save, pci_ide_load, d);
 }
Index: kvm-userspace.io/qemu/hw/lsi53c895a.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/lsi53c895a.c
+++ kvm-userspace.io/qemu/hw/lsi53c895a.c
@@ -594,6 +594,7 @@ static void lsi_command_complete(void *o
 {
     LSIState *s = (LSIState *)opaque;
     int out;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     out = (s->sstat1 & PHASE_MASK) == PHASE_DO;
     if (reason == SCSI_REASON_DONE) {
@@ -735,6 +736,8 @@ static void lsi_do_msgout(LSIState *s)
     uint8_t msg;
     int len;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
+
     DPRINTF("MSG out len=%d\n", s->dbc);
     while (s->dbc) {
         msg = lsi_get_msgbyte(s);
@@ -1217,6 +1220,8 @@ static uint8_t lsi_reg_readb(LSIState *s
     case addr + 2: return (s->name >> 16) & 0xff; \
     case addr + 3: return (s->name >> 24) & 0xff;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
+
 #ifdef DEBUG_LSI_REG
     DPRINTF("Read reg %x\n", offset);
 #endif
@@ -1383,6 +1388,8 @@ static void lsi_reg_writeb(LSIState *s, 
     case addr + 2: s->name &= 0xff00ffff; s->name |= val << 16; break; \
     case addr + 3: s->name &= 0x00ffffff; s->name |= val << 24; break;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
+
 #ifdef DEBUG_LSI_REG
     DPRINTF("Write reg %x = %02x\n", offset, val);
 #endif
@@ -1592,6 +1599,7 @@ static void lsi_mmio_writeb(void *opaque
 {
     LSIState *s = (LSIState *)opaque;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
     lsi_reg_writeb(s, addr & 0xff, val);
 }
 
@@ -1599,6 +1607,7 @@ static void lsi_mmio_writew(void *opaque
 {
     LSIState *s = (LSIState *)opaque;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
     addr &= 0xff;
     lsi_reg_writeb(s, addr, val & 0xff);
     lsi_reg_writeb(s, addr + 1, (val >> 8) & 0xff);
@@ -1608,6 +1617,7 @@ static void lsi_mmio_writel(void *opaque
 {
     LSIState *s = (LSIState *)opaque;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
     addr &= 0xff;
     lsi_reg_writeb(s, addr, val & 0xff);
     lsi_reg_writeb(s, addr + 1, (val >> 8) & 0xff);
@@ -1619,6 +1629,7 @@ static uint32_t lsi_mmio_readb(void *opa
 {
     LSIState *s = (LSIState *)opaque;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
     return lsi_reg_readb(s, addr & 0xff);
 }
 
@@ -1627,6 +1638,7 @@ static uint32_t lsi_mmio_readw(void *opa
     LSIState *s = (LSIState *)opaque;
     uint32_t val;
 
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
     addr &= 0xff;
     val = lsi_reg_readb(s, addr);
     val |= lsi_reg_readb(s, addr + 1) << 8;
@@ -1662,6 +1674,7 @@ static void lsi_ram_writeb(void *opaque,
     LSIState *s = (LSIState *)opaque;
     uint32_t newval;
     int shift;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     addr &= 0x1fff;
     newval = s->script_ram[addr >> 2];
@@ -1675,6 +1688,7 @@ static void lsi_ram_writew(void *opaque,
 {
     LSIState *s = (LSIState *)opaque;
     uint32_t newval;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     addr &= 0x1fff;
     newval = s->script_ram[addr >> 2];
@@ -1690,6 +1704,7 @@ static void lsi_ram_writew(void *opaque,
 static void lsi_ram_writel(void *opaque, target_phys_addr_t addr, uint32_t val)
 {
     LSIState *s = (LSIState *)opaque;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     addr &= 0x1fff;
     s->script_ram[addr >> 2] = val;
@@ -1699,6 +1714,7 @@ static uint32_t lsi_ram_readb(void *opaq
 {
     LSIState *s = (LSIState *)opaque;
     uint32_t val;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     addr &= 0x1fff;
     val = s->script_ram[addr >> 2];
@@ -1710,6 +1726,7 @@ static uint32_t lsi_ram_readw(void *opaq
 {
     LSIState *s = (LSIState *)opaque;
     uint32_t val;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     addr &= 0x1fff;
     val = s->script_ram[addr >> 2];
@@ -1721,6 +1738,7 @@ static uint32_t lsi_ram_readw(void *opaq
 static uint32_t lsi_ram_readl(void *opaque, target_phys_addr_t addr)
 {
     LSIState *s = (LSIState *)opaque;
+    assert_is_locked(&s->pci_dev.qemu_dev.lock);
 
     addr &= 0x1fff;
     return le32_to_cpu(s->script_ram[addr >> 2]);
@@ -1850,6 +1868,7 @@ void lsi_scsi_attach(void *opaque, Block
     if (s->scsi_dev[id] == NULL)
         s->scsi_dev[id] = scsi_disk_init(bd, 1, lsi_command_complete, s);
     bd->devfn = s->pci_dev.devfn;
+    bd->qemu_dev = &s->pci_dev.qemu_dev;
 }
 
 int lsi_scsi_uninit(PCIDevice *d)

-- 

[kvm-devel] [patch 10/13] QEMU: get rid of aiocb cache

[Marcelo T. <mto...@re...>]

the malloc() implementation should do a decent job at this, simplifies
block device locking.

Index: kvm-userspace.io/qemu/block.c
===================================================================
--- kvm-userspace.io.orig/qemu/block.c
+++ kvm-userspace.io/qemu/block.c
@@ -1447,14 +1447,10 @@ void *qemu_aio_get(BlockDriverState *bs,
     BlockDriverAIOCB *acb;
 
     drv = bs->drv;
-    if (drv->free_aiocb) {
-        acb = drv->free_aiocb;
-        drv->free_aiocb = acb->next;
-    } else {
-        acb = qemu_mallocz(drv->aiocb_size);
-        if (!acb)
-            return NULL;
-    }
+    acb = qemu_mallocz(drv->aiocb_size);
+    if (!acb)
+        return NULL;
+
     acb->bs = bs;
     acb->cb = cb;
     acb->opaque = opaque;
@@ -1464,9 +1460,7 @@ void *qemu_aio_get(BlockDriverState *bs,
 void qemu_aio_release(void *p)
 {
     BlockDriverAIOCB *acb = p;
-    BlockDriver *drv = acb->bs->drv;
-    acb->next = drv->free_aiocb;
-    drv->free_aiocb = acb;
+    qemu_free(p);
 }
 
 /**************************************************************/
Index: kvm-userspace.io/qemu/block_int.h
===================================================================
--- kvm-userspace.io.orig/qemu/block_int.h
+++ kvm-userspace.io/qemu/block_int.h
@@ -85,7 +85,6 @@ struct BlockDriver {
     /* to control generic scsi devices */
     int (*bdrv_ioctl)(BlockDriverState *bs, unsigned long int req, void *buf);
 
-    BlockDriverAIOCB *free_aiocb;
     struct BlockDriver *next;
 };
 

-- 

[kvm-devel] [patch 09/13] QEMU: network device locking

[Marcelo T. <mto...@re...>]

Grab device locks when moving data through network devices in the host->guest
direction.

Index: kvm-userspace.io/qemu/hw/e1000.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/e1000.c
+++ kvm-userspace.io/qemu/hw/e1000.c
@@ -1008,6 +1008,7 @@ pci_e1000_init(PCIBus *bus, NICInfo *nd,
 
     d->vc = qemu_new_vlan_client(nd->vlan, e1000_receive,
                                  e1000_can_receive, d);
+    d->vc->qemu_dev = &d->dev.qemu_dev;
 
     snprintf(d->vc->info_str, sizeof(d->vc->info_str),
              "%s macaddr=%02x:%02x:%02x:%02x:%02x:%02x", info_str,
Index: kvm-userspace.io/qemu/net.h
===================================================================
--- kvm-userspace.io.orig/qemu/net.h
+++ kvm-userspace.io/qemu/net.h
@@ -13,6 +13,7 @@ struct VLANClientState {
     void *opaque;
     struct VLANClientState *next;
     struct VLANState *vlan;
+    QEMUDevice *qemu_dev;
     char info_str[256];
 };
 
Index: kvm-userspace.io/qemu/qemu-device.h
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-device.h
+++ kvm-userspace.io/qemu/qemu-device.h
@@ -88,6 +88,8 @@ static inline void qemu_mutex_init(qemu_
 #endif /* DEBUG_PTHREADS */
 #endif /* _POSIX_THREADS */
 
+#define NOLOCK -1
+
 struct QEMUDevice {
     qemu_mutex_t lock;
 };
Index: kvm-userspace.io/qemu/vl.c
===================================================================
--- kvm-userspace.io.orig/qemu/vl.c
+++ kvm-userspace.io/qemu/vl.c
@@ -3830,6 +3830,18 @@ VLANClientState *qemu_new_vlan_client(VL
     return vc;
 }
 
+void qemu_net_lock(VLANClientState *vc)
+{
+    if (vc->qemu_dev != NOLOCK)
+        qemu_mutex_lock(&vc->qemu_dev->lock);
+}
+
+void qemu_net_unlock(VLANClientState *vc)
+{
+    if (vc->qemu_dev != NOLOCK)
+        qemu_mutex_unlock(&vc->qemu_dev->lock);
+}
+
 int qemu_can_send_packet(VLANClientState *vc1)
 {
     VLANState *vlan = vc1->vlan;
@@ -3855,7 +3867,9 @@ void qemu_send_packet(VLANClientState *v
 #endif
     for(vc = vlan->first_client; vc != NULL; vc = vc->next) {
         if (vc != vc1) {
+            qemu_net_lock(vc);
             vc->fd_read(vc->opaque, buf, size);
+            qemu_net_unlock(vc);
         }
     }
 }
@@ -4131,6 +4145,7 @@ static TAPState *net_tap_fd_init(VLANSta
     s->no_poll = 0;
     enable_sigio_timer(fd);
     s->vc = qemu_new_vlan_client(vlan, tap_receive, NULL, s);
+    s->vc->qemu_dev = NOLOCK;
     qemu_set_fd_handler2(s->fd, tap_read_poll, tap_send, NULL, s);
     snprintf(s->vc->info_str, sizeof(s->vc->info_str), "tap: fd=%d", fd);
     return s;
Index: kvm-userspace.io/qemu/hw/virtio-net.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/virtio-net.c
+++ kvm-userspace.io/qemu/hw/virtio-net.c
@@ -64,7 +64,6 @@ typedef struct VirtIONet
     int can_receive;
     int tap_fd;
     struct VirtIONet *next;
-    int do_notify;
     QEMUTimer *tx_timer;
     int tx_timer_active;
 } VirtIONet;
@@ -113,6 +112,8 @@ static void virtio_net_receive(void *opa
     struct virtio_net_hdr *hdr;
     int offset, i;
 
+    assert_is_locked(&n->vdev.pci_dev.qemu_dev.lock);
+
     /* FIXME: the drivers really need to set their status better */
     if (n->rx_vq->vring.avail == NULL) {
 	n->can_receive = 0;
@@ -144,6 +145,18 @@ static void virtio_net_receive(void *opa
     virtio_notify(&n->vdev, n->rx_vq);
 }
 
+
+void virtio_net_lock(VirtIONet *vnet)
+{
+    qemu_mutex_lock(&vnet->vdev.pci_dev.qemu_dev.lock);
+}
+
+void virtio_net_unlock(VirtIONet *vnet)
+{
+    qemu_mutex_unlock(&vnet->vdev.pci_dev.qemu_dev.lock);
+}
+
+
 /* -net tap receive handler */
 void virtio_net_poll(void)
 {
@@ -160,31 +173,39 @@ void virtio_net_poll(void)
     tv.tv_sec = 0;
     tv.tv_usec = 0;
 
-    while (1) {
+    do {
+        did_notify = 0;
 
         // Prepare the set of device to select from
         for (vnet = VirtIONetHead; vnet; vnet = vnet->next) {
+            int ret;
 
             if (vnet->tap_fd == -1)
                 continue;
 
-            vnet->do_notify = 0;
+            virtio_net_lock(vnet);
             //first check if the driver is ok
-            if (!virtio_net_can_receive(vnet))
+            ret = virtio_net_can_receive(vnet);
+            if (!ret) {
+                virtio_net_unlock(vnet);
                 continue;
+            }
 
             /* FIXME: the drivers really need to set their status better */
             if (vnet->rx_vq->vring.avail == NULL) {
                 vnet->can_receive = 0;
+                virtio_net_unlock(vnet);
                 continue;
             }
 
+            virtio_net_unlock(vnet);
+
             FD_SET(vnet->tap_fd, &rfds);
             if (max_fd < vnet->tap_fd) max_fd = vnet->tap_fd;
         }
 
         if (select(max_fd + 1, &rfds, NULL, NULL, &tv) <= 0)
-            break;
+            return;
 
         // Now check who has data pending in the tap
         for (vnet = VirtIONetHead; vnet; vnet = vnet->next) {
@@ -192,14 +213,17 @@ void virtio_net_poll(void)
             if (!FD_ISSET(vnet->tap_fd, &rfds))
                 continue;
 
+            virtio_net_lock(vnet);
             if (virtqueue_pop(vnet->rx_vq, &elem) == 0) {
                 vnet->can_receive = 0;
+                virtio_net_unlock(vnet);
                 continue;
             }
 
             hdr = (void *)elem.in_sg[0].iov_base;
             hdr->flags = 0;
             hdr->gso_type = VIRTIO_NET_HDR_GSO_NONE;
+            virtio_net_unlock(vnet);
 again:
             len = readv(vnet->tap_fd, &elem.in_sg[1], elem.in_num - 1);
             if (len == -1) {
@@ -208,21 +232,13 @@ again:
                 else
                     fprintf(stderr, "reading network error %d", len);
             }
+            did_notify = 1;
+            virtio_net_lock(vnet);
             virtqueue_push(vnet->rx_vq, &elem, sizeof(*hdr) + len);
-            vnet->do_notify = 1;
+            virtio_notify(&vnet->vdev, vnet->rx_vq);
+            virtio_net_unlock(vnet);
         }
-
-        /* signal other side */
-        did_notify = 0;
-        for (vnet = VirtIONetHead; vnet; vnet = vnet->next)
-            if (vnet->do_notify) {
-                virtio_notify(&vnet->vdev, vnet->rx_vq);
-                did_notify++;
-            }
-        if (!did_notify)
-            break;
-     }
-
+    } while (did_notify);
 }
 
 /* TX */
@@ -231,6 +247,8 @@ static void virtio_net_flush_tx(VirtIONe
     VirtQueueElement elem;
     int count = 0;
 
+    assert_is_locked(&n->vdev.pci_dev.qemu_dev.lock);
+
     if (!(n->vdev.status & VIRTIO_CONFIG_S_DRIVER_OK))
         return;
 
@@ -280,8 +298,10 @@ static void virtio_net_tx_timer(void *op
     if (!(n->vdev.status & VIRTIO_CONFIG_S_DRIVER_OK))
         return;
 
+    virtio_net_lock(n);
     n->tx_vq->vring.used->flags &= ~VRING_USED_F_NO_NOTIFY;
     virtio_net_flush_tx(n, n->tx_vq);
+    virtio_net_unlock(n);
 }
 
 PCIDevice *virtio_net_init(PCIBus *bus, NICInfo *nd, int devfn)

-- 

[kvm-devel] [patch 08/13] QEMU: character device locking

[Marcelo T. <mto...@re...>]

Grab device locks when moving data through char devices in the host->guest direction.

Index: kvm-userspace.io/qemu/hw/serial.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/serial.c
+++ kvm-userspace.io/qemu/hw/serial.c
@@ -305,18 +305,21 @@ static void serial_receive_break(SerialS
 static int serial_can_receive1(void *opaque)
 {
     SerialState *s = opaque;
+    assert_is_locked(&s->qemu_dev.lock);
     return serial_can_receive(s);
 }
 
 static void serial_receive1(void *opaque, const uint8_t *buf, int size)
 {
     SerialState *s = opaque;
+    assert_is_locked(&s->qemu_dev.lock);
     serial_receive_byte(s, buf[0]);
 }
 
 static void serial_event(void *opaque, int event)
 {
     SerialState *s = opaque;
+    assert_is_locked(&s->qemu_dev.lock);
     if (event == CHR_EVENT_BREAK)
         serial_receive_break(s);
 }
@@ -400,6 +403,7 @@ SerialState *serial_init(int base, qemu_
     register_ioport_write(dev, base, 8, 1, serial_ioport_write, s);
     register_ioport_read(dev, base, 8, 1, serial_ioport_read, s);
     s->chr = chr;
+    chr->qemu_dev = dev;
     qemu_chr_add_handlers(chr, serial_can_receive1, serial_receive1,
                           serial_event, s);
     return s;
Index: kvm-userspace.io/qemu/qemu-char.h
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-char.h
+++ kvm-userspace.io/qemu/qemu-char.h
@@ -44,6 +44,7 @@ struct CharDriverState {
     void *opaque;
     int focus;
     QEMUBH *bh;
+    QEMUDevice *qemu_dev;
 };
 
 CharDriverState *qemu_chr_open(const char *filename);
Index: kvm-userspace.io/qemu/vl.c
===================================================================
--- kvm-userspace.io.orig/qemu/vl.c
+++ kvm-userspace.io/qemu/vl.c
@@ -1690,6 +1690,18 @@ int qemu_timedate_diff(struct tm *tm)
 /***********************************************************/
 /* character device */
 
+static void qemu_chr_lock(CharDriverState *s)
+{
+    if (!s->qemu_dev)
+        hw_error("%s: no qemu_dev\n", __func__);
+    qemu_mutex_lock(&s->qemu_dev->lock);
+}
+
+static void qemu_chr_unlock(CharDriverState *s)
+{
+   qemu_mutex_unlock(&s->qemu_dev->lock);
+}
+
 static void qemu_chr_event(CharDriverState *s, int event)
 {
     if (!s->chr_event)
@@ -1715,7 +1727,12 @@ void qemu_chr_reset(CharDriverState *s)
 
 int qemu_chr_write(CharDriverState *s, const uint8_t *buf, int len)
 {
-    return s->chr_write(s, buf, len);
+    int ret;
+
+    assert_is_dev_locked(s->qemu_dev);
+    ret = s->chr_write(s, buf, len);
+
+    return ret;
 }
 
 int qemu_chr_ioctl(CharDriverState *s, int cmd, void *arg)
@@ -1727,9 +1744,11 @@ int qemu_chr_ioctl(CharDriverState *s, i
 
 int qemu_chr_can_read(CharDriverState *s)
 {
+    int ret = 0;
     if (!s->chr_can_read)
-        return 0;
-    return s->chr_can_read(s->handler_opaque);
+        return ret;
+    ret = s->chr_can_read(s->handler_opaque);
+    return ret;
 }
 
 void qemu_chr_read(CharDriverState *s, uint8_t *buf, int len)
@@ -1755,8 +1774,10 @@ void qemu_chr_printf(CharDriverState *s,
 
 void qemu_chr_send_event(CharDriverState *s, int event)
 {
+    qemu_chr_lock(s);
     if (s->chr_send_event)
         s->chr_send_event(s, event);
+    qemu_chr_unlock(s);
 }
 
 void qemu_chr_add_handlers(CharDriverState *s,
@@ -1799,8 +1820,10 @@ typedef struct {
     IOCanRWHandler *chr_can_read[MAX_MUX];
     IOReadHandler *chr_read[MAX_MUX];
     IOEventHandler *chr_event[MAX_MUX];
+    QEMUDevice *devices[MAX_MUX];
     void *ext_opaque[MAX_MUX];
     CharDriverState *drv;
+    QEMUDevice qemu_dev;
     unsigned char buffer[MUX_BUFFER_SIZE];
     int prod;
     int cons;
@@ -1935,11 +1958,13 @@ static void mux_chr_accept_input(CharDri
     int m = chr->focus;
     MuxDriver *d = chr->opaque;
 
-    while (d->prod != d->cons &&
-           d->chr_can_read[m] &&
-           d->chr_can_read[m](d->ext_opaque[m])) {
-        d->chr_read[m](d->ext_opaque[m],
-                       &d->buffer[d->cons++ & MUX_BUFFER_MASK], 1);
+    while (d->prod != d->cons) {
+        qemu_mutex_lock(&d->devices[m]->lock);
+        if (d->chr_can_read[m] && d->chr_can_read[m](d->ext_opaque[m])) {
+            d->chr_read[m](d->ext_opaque[m],
+                           &d->buffer[d->cons++ & MUX_BUFFER_MASK], 1);
+        }
+        qemu_mutex_unlock(&d->devices[m]->lock);
     }
 }
 
@@ -1950,8 +1975,13 @@ static int mux_chr_can_read(void *opaque
 
     if ((d->prod - d->cons) < MUX_BUFFER_SIZE)
         return 1;
-    if (d->chr_can_read[chr->focus])
-        return d->chr_can_read[chr->focus](d->ext_opaque[chr->focus]);
+    if (d->chr_can_read[chr->focus]) {
+        int ret;
+        qemu_mutex_lock(&d->devices[chr->focus]->lock);
+        ret = d->chr_can_read[chr->focus](d->ext_opaque[chr->focus]);
+        qemu_mutex_unlock(&d->devices[chr->focus]->lock);
+        return ret;
+    }
     return 0;
 }
 
@@ -1966,11 +1996,13 @@ static void mux_chr_read(void *opaque, c
 
     for(i = 0; i < size; i++)
         if (mux_proc_byte(chr, d, buf[i])) {
-            if (d->prod == d->cons &&
-                d->chr_can_read[m] &&
-                d->chr_can_read[m](d->ext_opaque[m]))
-                d->chr_read[m](d->ext_opaque[m], &buf[i], 1);
-            else
+            if (d->prod == d->cons) {
+                qemu_mutex_lock(&d->devices[m]->lock);
+                if (d->chr_can_read[m] && d->chr_can_read[m](d->ext_opaque[m])) {
+                    d->chr_read[m](d->ext_opaque[m], &buf[i], 1);
+                }
+                qemu_mutex_unlock(&d->devices[m]->lock);
+            } else
                 d->buffer[d->prod++ & MUX_BUFFER_MASK] = buf[i];
         }
 }
@@ -1982,9 +2014,13 @@ static void mux_chr_event(void *opaque, 
     int i;
 
     /* Send the event to all registered listeners */
-    for (i = 0; i < d->mux_cnt; i++)
-        if (d->chr_event[i])
+    for (i = 0; i < d->mux_cnt; i++) {
+        if (d->chr_event[i]) {
+            qemu_mutex_lock(&d->devices[i]->lock);
             d->chr_event[i](d->ext_opaque[i], event);
+            qemu_mutex_unlock(&d->devices[i]->lock);
+        }
+    }
 }
 
 static void mux_chr_update_read_handler(CharDriverState *chr)
@@ -1999,6 +2035,7 @@ static void mux_chr_update_read_handler(
     d->chr_can_read[d->mux_cnt] = chr->chr_can_read;
     d->chr_read[d->mux_cnt] = chr->chr_read;
     d->chr_event[d->mux_cnt] = chr->chr_event;
+    d->devices[d->mux_cnt] = chr->qemu_dev;
     /* Fix up the real driver with mux routines */
     if (d->mux_cnt == 0) {
         qemu_chr_add_handlers(d->drv, mux_chr_can_read, mux_chr_read,
@@ -2021,8 +2058,10 @@ static CharDriverState *qemu_chr_open_mu
         free(chr);
         return NULL;
     }
+    qemu_register_device(&d->qemu_dev);
 
     chr->opaque = d;
+    chr->qemu_dev = &d->qemu_dev;
     d->drv = drv;
     chr->focus = -1;
     chr->chr_write = mux_chr_write;
Index: kvm-userspace.io/qemu/qemu-device.h
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-device.h
+++ kvm-userspace.io/qemu/qemu-device.h
@@ -10,6 +10,7 @@
 #define qemu_mutex_lock(mutex) pthread_mutex_lock(mutex)
 #define qemu_mutex_unlock(mutex) pthread_mutex_unlock(mutex)
 #define assert_is_locked(mutex) do { } while (0)
+#define assert_is_dev_locked(mutex) do { } while (0)
 static inline void qemu_mutex_init(qemu_mutex_t *mutex)
 {
     pthread_mutex_init(mutex, NULL);
@@ -42,6 +43,16 @@ static inline void assert_is_locked(qemu
     }
 }
 
+static inline void assert_is_dev_locked(void *dev)
+{
+    if (!dev) {
+        printf("assert failure, qemu_dev is null\n");
+        print_backtrace();
+        exit(0);
+    }
+    assert_is_locked((qemu_mutex_t *)dev);
+}
+
 static inline void qemu_mutex_lock(qemu_mutex_t *mutex)
 {
     if (!mutex) {

-- 

[kvm-devel] [patch 07/13] QEMU: grab device lock for ioport/iomem processing

[Marcelo T. <mto...@re...>]

Record which device owns ioports/memports, and use that to grab the appropriate
lock when entering ioport/iomem processing.

Make cpu_physical_memory_rw() unlocked (called from inside device code), and
locking optional from __cpu_physical_memory_rw(), to be called when a vcpu
enters mmio read/write.

The debugging checks should aid finding most problems.

Index: kvm-userspace.io/qemu/exec.c
===================================================================
--- kvm-userspace.io.orig/qemu/exec.c
+++ kvm-userspace.io/qemu/exec.c
@@ -170,6 +170,7 @@ PhysPageDesc **l1_phys_map;
 CPUWriteMemoryFunc *io_mem_write[IO_MEM_NB_ENTRIES][4];
 CPUReadMemoryFunc *io_mem_read[IO_MEM_NB_ENTRIES][4];
 void *io_mem_opaque[IO_MEM_NB_ENTRIES];
+QEMUDevice *io_mem_devices[IO_MEM_NB_ENTRIES];
 char io_mem_used[IO_MEM_NB_ENTRIES];
 #if defined(CONFIG_SOFTMMU)
 static int io_mem_watch;
@@ -2039,6 +2040,11 @@ static inline void tlb_set_dirty(CPUStat
 }
 #endif /* defined(CONFIG_USER_ONLY) */
 
+QEMUDevice *qemu_find_device_iomem(int io_index)
+{
+    return io_mem_devices[io_index >> IO_MEM_SHIFT];
+}
+
 static int subpage_register (subpage_t *mmio, uint32_t start, uint32_t end,
                              int memory);
 static void *subpage_init (target_phys_addr_t base, uint32_t *phys,
@@ -2502,11 +2508,12 @@ static void *subpage_init (target_phys_a
 {
     subpage_t *mmio;
     int subpage_memory;
+    QEMUDevice *dev = qemu_find_device_iomem(*phys);
 
     mmio = qemu_mallocz(sizeof(subpage_t));
     if (mmio != NULL) {
         mmio->base = base;
-        subpage_memory = cpu_register_io_memory(NULL, 0, subpage_read, subpage_write, mmio);
+        subpage_memory = cpu_register_io_memory(dev, 0, subpage_read, subpage_write, mmio);
 #if defined(DEBUG_SUBPAGE)
         printf("%s: %p base " TARGET_FMT_plx " len %08x %d\n", __func__,
                mmio, base, TARGET_PAGE_SIZE, subpage_memory);
@@ -2582,6 +2589,7 @@ int cpu_register_io_memory(QEMUDevice *d
         io_mem_write[io_index][i] = mem_write[i];
     }
     io_mem_opaque[io_index] = opaque;
+    io_mem_devices[io_index] = dev;
     return (io_index << IO_MEM_SHIFT) | subwidth;
 }
 
@@ -2651,8 +2659,42 @@ void cpu_physical_memory_rw(target_phys_
 }
 
 #else
-void cpu_physical_memory_rw(target_phys_addr_t addr, uint8_t *buf,
-                            int len, int is_write)
+
+#define WRITE 0x80
+
+void qemu_iomem_dev_lock(int io_index, void *unassigned_io_fn, int iotype,
+                         int has_lock)
+{
+    if (!io_mem_devices[io_index]) {
+        void *handler;
+
+        if (iotype & WRITE)
+            handler = io_mem_write[io_index][iotype & ~WRITE];
+        else
+            handler = io_mem_read[io_index][iotype];
+
+        if (handler != unassigned_io_fn)
+            hw_error("iomem %x has no registered QEMUDevice, but has valid "
+                     "handling fn\n", io_index);
+        return;
+    }
+    if (has_lock)
+        assert_is_locked(&io_mem_devices[io_index]->lock);
+    else
+        qemu_mutex_lock(&io_mem_devices[io_index]->lock);
+}
+
+void qemu_iomem_dev_unlock(int io_index, int has_lock)
+{
+    if (has_lock)
+        assert_is_locked(&io_mem_devices[io_index]->lock);
+    else if (io_mem_devices[io_index])
+        qemu_mutex_unlock(&io_mem_devices[io_index]->lock);
+}
+
+
+void __cpu_physical_memory_rw(target_phys_addr_t addr, uint8_t *buf,
+                            int len, int is_write, int has_lock)
 {
     int l, io_index;
     uint8_t *ptr;
@@ -2666,6 +2708,7 @@ void cpu_physical_memory_rw(target_phys_
         l = (page + TARGET_PAGE_SIZE) - addr;
         if (l > len)
             l = len;
+        /* FIXME: verify safety of concurrent phys_page_find */
         p = phys_page_find(page >> TARGET_PAGE_BITS);
         if (!p) {
             pd = IO_MEM_UNASSIGNED;
@@ -2681,19 +2724,26 @@ void cpu_physical_memory_rw(target_phys_
                 if (l >= 4 && ((addr & 3) == 0)) {
                     /* 32 bit write access */
                     val = ldl_p(buf);
+                    qemu_iomem_dev_lock(io_index, unassigned_mem_writeb,
+                                        2|WRITE, has_lock);
                     io_mem_write[io_index][2](io_mem_opaque[io_index], addr, val);
                     l = 4;
                 } else if (l >= 2 && ((addr & 1) == 0)) {
                     /* 16 bit write access */
                     val = lduw_p(buf);
+                    qemu_iomem_dev_lock(io_index, unassigned_mem_writeb,
+                                        1|WRITE, has_lock);
                     io_mem_write[io_index][1](io_mem_opaque[io_index], addr, val);
                     l = 2;
                 } else {
                     /* 8 bit write access */
                     val = ldub_p(buf);
+                    qemu_iomem_dev_lock(io_index, unassigned_mem_writeb,
+                                        0|WRITE, has_lock);
                     io_mem_write[io_index][0](io_mem_opaque[io_index], addr, val);
                     l = 1;
                 }
+                qemu_iomem_dev_unlock(io_index, has_lock);
             } else {
                 unsigned long addr1;
                 addr1 = (pd & TARGET_PAGE_MASK) + (addr & ~TARGET_PAGE_MASK);
@@ -2719,21 +2769,28 @@ void cpu_physical_memory_rw(target_phys_
                 /* I/O case */
                 io_index = (pd >> IO_MEM_SHIFT) & (IO_MEM_NB_ENTRIES - 1);
                 if (l >= 4 && ((addr & 3) == 0)) {
+                    qemu_iomem_dev_lock(io_index, unassigned_mem_readb, 2,
+                                        has_lock);
                     /* 32 bit read access */
                     val = io_mem_read[io_index][2](io_mem_opaque[io_index], addr);
                     stl_p(buf, val);
                     l = 4;
                 } else if (l >= 2 && ((addr & 1) == 0)) {
                     /* 16 bit read access */
+                    qemu_iomem_dev_lock(io_index, unassigned_mem_readb, 1,
+                                        has_lock);
                     val = io_mem_read[io_index][1](io_mem_opaque[io_index], addr);
                     stw_p(buf, val);
                     l = 2;
                 } else {
                     /* 8 bit read access */
+                    qemu_iomem_dev_lock(io_index, unassigned_mem_readb, 0,
+                                        has_lock);
                     val = io_mem_read[io_index][0](io_mem_opaque[io_index], addr);
                     stb_p(buf, val);
                     l = 1;
                 }
+                qemu_iomem_dev_unlock(io_index, has_lock);
             } else {
                 /* RAM case */
                 ptr = phys_ram_base + (pd & TARGET_PAGE_MASK) +
@@ -2747,6 +2804,13 @@ void cpu_physical_memory_rw(target_phys_
     }
 }
 
+void cpu_physical_memory_rw(target_phys_addr_t addr, uint8_t *buf,
+                            int len, int is_write)
+{
+    __cpu_physical_memory_rw(addr, buf, len, is_write, 1);
+    return;
+}
+
 /* used for ROM loading : can write in RAM and ROM */
 void cpu_physical_memory_write_rom(target_phys_addr_t addr,
                                    const uint8_t *buf, int len)
Index: kvm-userspace.io/qemu/vl.c
===================================================================
--- kvm-userspace.io.orig/qemu/vl.c
+++ kvm-userspace.io/qemu/vl.c
@@ -170,6 +170,7 @@ int inet_aton(const char *cp, struct in_
 const char *bios_dir = CONFIG_QEMU_SHAREDIR;
 const char *bios_name = NULL;
 void *ioport_opaque[MAX_IOPORTS];
+QEMUDevice *ioport_devices[MAX_IOPORTS];
 IOPortReadFunc *ioport_read_table[3][MAX_IOPORTS];
 IOPortWriteFunc *ioport_write_table[3][MAX_IOPORTS];
 /* Note: drives_table[MAX_DRIVES] is a dummy block driver if none available
@@ -366,6 +367,10 @@ int register_ioport_read(QEMUDevice *dev
         if (ioport_opaque[i] != NULL && ioport_opaque[i] != opaque)
             hw_error("register_ioport_read: invalid opaque");
         ioport_opaque[i] = opaque;
+
+        if (ioport_devices[i] != NULL && ioport_devices[i] != dev)
+            hw_error("register_ioport_read: invalid device");
+        ioport_devices[i] = dev;
     }
     return 0;
 }
@@ -391,6 +396,10 @@ int register_ioport_write(QEMUDevice *de
         if (ioport_opaque[i] != NULL && ioport_opaque[i] != opaque)
             hw_error("register_ioport_write: invalid opaque");
         ioport_opaque[i] = opaque;
+
+        if (ioport_devices[i] != NULL && ioport_devices[i] != dev)
+            hw_error("register_ioport_write: invalid device");
+        ioport_devices[i] = dev;
     }
     return 0;
 }
@@ -409,18 +418,47 @@ void isa_unassign_ioport(int start, int 
         ioport_write_table[2][i] = default_ioport_writel;
 
         ioport_opaque[i] = NULL;
+        ioport_devices[i] = NULL;
     }
 }
 
+#define WRITE 0x80
+
 /***********************************************************/
 
+void qemu_ioport_dev_lock(int addr, void *unassigned_io_fn, int iotype)
+{
+    if (!ioport_devices[addr]) {
+        void *handler;
+
+        if (iotype & WRITE)
+            handler = ioport_write_table[iotype & ~WRITE][addr];
+        else
+            handler = ioport_read_table[iotype][addr];
+
+        if (handler != unassigned_io_fn)
+            hw_error("ioport %x has no registered QEMUDevice, but has valid "
+                     "handling fn\n", addr);
+        return;
+    }
+    qemu_mutex_lock(&ioport_devices[addr]->lock);
+}
+
+void qemu_ioport_dev_unlock(int addr)
+{
+    if (ioport_devices[addr])
+        qemu_mutex_unlock(&ioport_devices[addr]->lock);
+}
+
 void cpu_outb(CPUState *env, int addr, int val)
 {
 #ifdef DEBUG_IOPORT
     if (loglevel & CPU_LOG_IOPORT)
         fprintf(logfile, "outb: %04x %02x\n", addr, val);
 #endif
+    qemu_ioport_dev_lock(addr, default_ioport_writeb, 0|WRITE);
     ioport_write_table[0][addr](ioport_opaque[addr], addr, val);
+    qemu_ioport_dev_unlock(addr);
 #ifdef USE_KQEMU
     if (env)
         env->last_io_time = cpu_get_time_fast();
@@ -433,7 +471,9 @@ void cpu_outw(CPUState *env, int addr, i
     if (loglevel & CPU_LOG_IOPORT)
         fprintf(logfile, "outw: %04x %04x\n", addr, val);
 #endif
+    qemu_ioport_dev_lock(addr, default_ioport_writew, 1|WRITE);
     ioport_write_table[1][addr](ioport_opaque[addr], addr, val);
+    qemu_ioport_dev_unlock(addr);
 #ifdef USE_KQEMU
     if (env)
         env->last_io_time = cpu_get_time_fast();
@@ -446,7 +486,9 @@ void cpu_outl(CPUState *env, int addr, i
     if (loglevel & CPU_LOG_IOPORT)
         fprintf(logfile, "outl: %04x %08x\n", addr, val);
 #endif
+    qemu_ioport_dev_lock(addr, default_ioport_writel, 2|WRITE);
     ioport_write_table[2][addr](ioport_opaque[addr], addr, val);
+    qemu_ioport_dev_unlock(addr);
 #ifdef USE_KQEMU
     if (env)
         env->last_io_time = cpu_get_time_fast();
@@ -456,7 +498,9 @@ void cpu_outl(CPUState *env, int addr, i
 int cpu_inb(CPUState *env, int addr)
 {
     int val;
+    qemu_ioport_dev_lock(addr, default_ioport_readb, 0);
     val = ioport_read_table[0][addr](ioport_opaque[addr], addr);
+    qemu_ioport_dev_unlock(addr);
 #ifdef DEBUG_IOPORT
     if (loglevel & CPU_LOG_IOPORT)
         fprintf(logfile, "inb : %04x %02x\n", addr, val);
@@ -471,7 +515,9 @@ int cpu_inb(CPUState *env, int addr)
 int cpu_inw(CPUState *env, int addr)
 {
     int val;
+    qemu_ioport_dev_lock(addr, default_ioport_readw, 1);
     val = ioport_read_table[1][addr](ioport_opaque[addr], addr);
+    qemu_ioport_dev_unlock(addr);
 #ifdef DEBUG_IOPORT
     if (loglevel & CPU_LOG_IOPORT)
         fprintf(logfile, "inw : %04x %04x\n", addr, val);
@@ -486,7 +532,9 @@ int cpu_inw(CPUState *env, int addr)
 int cpu_inl(CPUState *env, int addr)
 {
     int val;
+    qemu_ioport_dev_lock(addr, default_ioport_readl, 2);
     val = ioport_read_table[2][addr](ioport_opaque[addr], addr);
+    qemu_ioport_dev_unlock(addr);
 #ifdef DEBUG_IOPORT
     if (loglevel & CPU_LOG_IOPORT)
         fprintf(logfile, "inl : %04x %08x\n", addr, val);
Index: kvm-userspace.io/qemu/hw/pc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pc.c
+++ kvm-userspace.io/qemu/hw/pc.c
@@ -1046,7 +1046,6 @@ static void pc_init1(ram_addr_t ram_size
         }
     }
 
-    qemu_system_hot_add_init(cpu_model);
 #define USE_HYPERCALL
 #ifdef USE_HYPERCALL
     pci_hypercall_init(pci_bus);
@@ -1110,6 +1109,8 @@ static void pc_init1(ram_addr_t ram_size
         i440fx_init_memory_mappings(i440fx_state);
     }
 
+    qemu_system_hot_add_init(cpu_model);
+
     if (pci_enabled) {
 	int max_bus;
         int bus, unit;
Index: kvm-userspace.io/qemu/cpu-all.h
===================================================================
--- kvm-userspace.io.orig/qemu/cpu-all.h
+++ kvm-userspace.io/qemu/cpu-all.h
@@ -850,6 +850,10 @@ CPUReadMemoryFunc **cpu_get_io_memory_re
 
 void cpu_physical_memory_rw(target_phys_addr_t addr, uint8_t *buf,
                             int len, int is_write);
+/* the same as above, but specify whether device locks are held */
+void __cpu_physical_memory_rw_lock(target_phys_addr_t addr, uint8_t *buf,
+                            int len, int is_write, int has_lock);
+
 static inline void cpu_physical_memory_read(target_phys_addr_t addr,
                                             uint8_t *buf, int len)
 {
Index: kvm-userspace.io/qemu/qemu-kvm.c
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-kvm.c
+++ kvm-userspace.io/qemu/qemu-kvm.c
@@ -555,13 +555,13 @@ static int kvm_outl(void *opaque, uint16
 
 static int kvm_mmio_read(void *opaque, uint64_t addr, uint8_t *data, int len)
 {
-	cpu_physical_memory_rw(addr, data, len, 0);
+	__cpu_physical_memory_rw(addr, data, len, 0, 0);
 	return 0;
 }
 
 static int kvm_mmio_write(void *opaque, uint64_t addr, uint8_t *data, int len)
 {
-	cpu_physical_memory_rw(addr, data, len, 1);
+	__cpu_physical_memory_rw(addr, data, len, 1, 0);
 	return 0;
 }
 

-- 

[kvm-devel] [patch 06/13] QEMU: plug QEMUDevice pt2 / iomem awareness

[Marcelo T. <mto...@re...>]

Same as before, but make the iomem->device relationship visible.

Index: kvm-userspace.io/qemu/cpu-all.h
===================================================================
--- kvm-userspace.io.orig/qemu/cpu-all.h
+++ kvm-userspace.io/qemu/cpu-all.h
@@ -840,7 +840,7 @@ void cpu_register_physical_memory(target
 uint32_t cpu_get_physical_page_desc(target_phys_addr_t addr);
 ram_addr_t qemu_ram_alloc(unsigned long size);
 void qemu_ram_free(ram_addr_t addr);
-int cpu_register_io_memory(int io_index,
+int cpu_register_io_memory(QEMUDevice *dev, int io_index,
                            CPUReadMemoryFunc **mem_read,
                            CPUWriteMemoryFunc **mem_write,
                            void *opaque);
Index: kvm-userspace.io/qemu/exec.c
===================================================================
--- kvm-userspace.io.orig/qemu/exec.c
+++ kvm-userspace.io/qemu/exec.c
@@ -33,6 +33,7 @@
 #include <unistd.h>
 #include <inttypes.h>
 
+#include "qemu-device.h"
 #include "cpu.h"
 #include "exec-all.h"
 
@@ -2505,7 +2506,7 @@ static void *subpage_init (target_phys_a
     mmio = qemu_mallocz(sizeof(subpage_t));
     if (mmio != NULL) {
         mmio->base = base;
-        subpage_memory = cpu_register_io_memory(0, subpage_read, subpage_write, mmio);
+        subpage_memory = cpu_register_io_memory(NULL, 0, subpage_read, subpage_write, mmio);
 #if defined(DEBUG_SUBPAGE)
         printf("%s: %p base " TARGET_FMT_plx " len %08x %d\n", __func__,
                mmio, base, TARGET_PAGE_SIZE, subpage_memory);
@@ -2534,14 +2535,14 @@ static void io_mem_init(void)
 {
     int i;
 
-    cpu_register_io_memory(IO_MEM_ROM >> IO_MEM_SHIFT, error_mem_read, unassigned_mem_write, NULL);
-    cpu_register_io_memory(IO_MEM_UNASSIGNED >> IO_MEM_SHIFT, unassigned_mem_read, unassigned_mem_write, NULL);
-    cpu_register_io_memory(IO_MEM_NOTDIRTY >> IO_MEM_SHIFT, error_mem_read, notdirty_mem_write, NULL);
+    cpu_register_io_memory(NULL, IO_MEM_ROM >> IO_MEM_SHIFT, error_mem_read, unassigned_mem_write, NULL);
+    cpu_register_io_memory(NULL, IO_MEM_UNASSIGNED >> IO_MEM_SHIFT, unassigned_mem_read, unassigned_mem_write, NULL);
+    cpu_register_io_memory(NULL, IO_MEM_NOTDIRTY >> IO_MEM_SHIFT, error_mem_read, notdirty_mem_write, NULL);
     for (i=0; i<5; i++)
         io_mem_used[i] = 1;
 
 #if defined(CONFIG_SOFTMMU)
-    io_mem_watch = cpu_register_io_memory(-1, watch_mem_read,
+    io_mem_watch = cpu_register_io_memory(NULL, -1, watch_mem_read,
                                           watch_mem_write, NULL);
 #endif
     /* alloc dirty bits array */
@@ -2557,7 +2558,8 @@ static void io_mem_init(void)
    modified. If it is zero, a new io zone is allocated. The return
    value can be used with cpu_register_physical_memory(). (-1) is
    returned if error. */
-int cpu_register_io_memory(int io_index,
+int cpu_register_io_memory(QEMUDevice *dev,
+                           int io_index,
                            CPUReadMemoryFunc **mem_read,
                            CPUWriteMemoryFunc **mem_write,
                            void *opaque)
Index: kvm-userspace.io/qemu/hw/apic.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/apic.c
+++ kvm-userspace.io/qemu/hw/apic.c
@@ -70,6 +70,7 @@
 #define MAX_APIC_WORDS 8
 
 typedef struct APICState {
+    QEMUDevice qemu_dev;
     CPUState *cpu_env;
     uint32_t apicbase;
     uint8_t id;
@@ -93,6 +94,7 @@ typedef struct APICState {
 } APICState;
 
 struct IOAPICState {
+    QEMUDevice qemu_dev;
     uint8_t id;
     uint8_t ioregsel;
     uint64_t base_address;
@@ -958,6 +960,7 @@ int apic_init(CPUState *env)
     s = qemu_mallocz(sizeof(APICState));
     if (!s)
         return -1;
+    qemu_register_device(&s->qemu_dev);
     env->apic_state = s;
     apic_init_ipi(s);
     s->id = last_apic_id++;
@@ -977,7 +980,7 @@ int apic_init(CPUState *env)
     if (apic_io_memory == 0) {
         /* NOTE: the APIC is directly connected to the CPU - it is not
            on the global memory bus. */
-        apic_io_memory = cpu_register_io_memory(0, apic_mem_read,
+        apic_io_memory = cpu_register_io_memory(&s->qemu_dev, 0, apic_mem_read,
                                                 apic_mem_write, NULL);
         cpu_register_physical_memory(s->apicbase & ~0xfff, 0x1000,
                                      apic_io_memory);
@@ -1255,10 +1258,11 @@ IOAPICState *ioapic_init(void)
     s = qemu_mallocz(sizeof(IOAPICState));
     if (!s)
         return NULL;
+    qemu_register_device(&s->qemu_dev);
     ioapic_reset(s);
     s->id = last_apic_id++;
 
-    io_memory = cpu_register_io_memory(0, ioapic_mem_read,
+    io_memory = cpu_register_io_memory(&s->qemu_dev, 0, ioapic_mem_read,
                                        ioapic_mem_write, s);
     cpu_register_physical_memory(0xfec00000, 0x1000, io_memory);
 
Index: kvm-userspace.io/qemu/hw/cirrus_vga.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/cirrus_vga.c
+++ kvm-userspace.io/qemu/hw/cirrus_vga.c
@@ -3258,7 +3258,7 @@ static void cirrus_init_common(CirrusVGA
     register_ioport_read(dev, 0x3ba, 1, 1, vga_ioport_read, s);
     register_ioport_read(dev, 0x3da, 1, 1, vga_ioport_read, s);
 
-    vga_io_memory = cpu_register_io_memory(0, cirrus_vga_mem_read,
+    vga_io_memory = cpu_register_io_memory(dev, 0, cirrus_vga_mem_read,
                                            cirrus_vga_mem_write, s);
     cpu_register_physical_memory(isa_mem_base + 0x000a0000, 0x20000,
                                  vga_io_memory);
@@ -3300,18 +3300,18 @@ static void cirrus_init_common(CirrusVGA
 
     /* I/O handler for LFB */
     s->cirrus_linear_io_addr =
-	cpu_register_io_memory(0, cirrus_linear_read, cirrus_linear_write,
+	cpu_register_io_memory(dev, 0, cirrus_linear_read, cirrus_linear_write,
 			       s);
     s->cirrus_linear_write = cpu_get_io_memory_write(s->cirrus_linear_io_addr);
 
     /* I/O handler for LFB */
     s->cirrus_linear_bitblt_io_addr =
-	cpu_register_io_memory(0, cirrus_linear_bitblt_read, cirrus_linear_bitblt_write,
+	cpu_register_io_memory(dev, 0, cirrus_linear_bitblt_read, cirrus_linear_bitblt_write,
 			       s);
 
     /* I/O handler for memory-mapped I/O */
     s->cirrus_mmio_io_addr =
-	cpu_register_io_memory(0, cirrus_mmio_read, cirrus_mmio_write, s);
+	cpu_register_io_memory(dev, 0, cirrus_mmio_read, cirrus_mmio_write, s);
 
     /* XXX: s->vram_size must be a power of two */
     s->cirrus_addr_mask = s->real_vram_size - 1;
Index: kvm-userspace.io/qemu/hw/e1000.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/e1000.c
+++ kvm-userspace.io/qemu/hw/e1000.c
@@ -936,7 +936,8 @@ e1000_mmio_map(PCIDevice *pci_dev, int r
     DBGOUT(MMIO, "e1000_mmio_map addr=0x%08x 0x%08x\n", addr, size);
 
     d->mmio_base = addr;
-    cpu_register_physical_memory(addr, PNPMMIO_SIZE, d->mmio_index);
+    cpu_register_physical_memory(addr, PNPMMIO_SIZE,
+                                 d->mmio_index);
 }
 
 static int
@@ -976,7 +977,8 @@ pci_e1000_init(PCIBus *bus, NICInfo *nd,
 
     pci_conf[0x3d] = 1; // interrupt pin 0
 
-    d->mmio_index = cpu_register_io_memory(0, e1000_mmio_read,
+    d->mmio_index = cpu_register_io_memory(&d->dev.qemu_dev, 0,
+            e1000_mmio_read,
             e1000_mmio_write, d);
 
     pci_register_io_region((PCIDevice *)d, 0, PNPMMIO_SIZE,
Index: kvm-userspace.io/qemu/hw/eepro100.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/eepro100.c
+++ kvm-userspace.io/qemu/hw/eepro100.c
@@ -1767,7 +1767,8 @@ static PCIDevice *nic_init(PCIBus * bus,
 
     /* Handler for memory-mapped I/O */
     d->eepro100.mmio_index =
-        cpu_register_io_memory(0, pci_mmio_read, pci_mmio_write, s);
+        cpu_register_io_memory(&d->dev.qemu_dev, 0, pci_mmio_read,
+                               pci_mmio_write, s);
 
     pci_register_io_region(&d->dev, 0, PCI_MEM_SIZE,
                            PCI_ADDRESS_SPACE_MEM |
Index: kvm-userspace.io/qemu/hw/fdc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/fdc.c
+++ kvm-userspace.io/qemu/hw/fdc.c
@@ -755,8 +755,8 @@ fdctrl_t *fdctrl_init (qemu_irq irq, int
 
     fdctrl->sun4m = 0;
     if (mem_mapped) {
-        io_mem = cpu_register_io_memory(0, fdctrl_mem_read, fdctrl_mem_write,
-                                        fdctrl);
+        io_mem = cpu_register_io_memory(dev, 0, fdctrl_mem_read,
+                                        fdctrl_mem_write, fdctrl);
         cpu_register_physical_memory(io_base, 0x08, io_mem);
     } else {
         register_ioport_read(dev, (uint32_t)io_base + 0x01, 5, 1, &fdctrl_read,
@@ -777,10 +777,12 @@ fdctrl_t *sun4m_fdctrl_init (qemu_irq ir
 {
     fdctrl_t *fdctrl;
     int io_mem;
+    QEMUDevice *dev;
 
     fdctrl = fdctrl_init_common(irq, 0, io_base, fds);
+    dev = &fdctrl->qemu_dev;
     fdctrl->sun4m = 1;
-    io_mem = cpu_register_io_memory(0, fdctrl_mem_read_strict,
+    io_mem = cpu_register_io_memory(dev, 0, fdctrl_mem_read_strict,
                                     fdctrl_mem_write_strict,
                                     fdctrl);
     cpu_register_physical_memory(io_base, 0x08, io_mem);
Index: kvm-userspace.io/qemu/hw/isa_mmio.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/isa_mmio.c
+++ kvm-userspace.io/qemu/hw/isa_mmio.c
@@ -93,6 +93,11 @@ static CPUReadMemoryFunc *isa_mmio_read[
 
 static int isa_mmio_iomemtype = 0;
 
+
+/*
+ * FIXME to use QEMUDevice
+ */
+#if 0
 void isa_mmio_init(target_phys_addr_t base, target_phys_addr_t size)
 {
     if (!isa_mmio_iomemtype) {
@@ -101,3 +106,4 @@ void isa_mmio_init(target_phys_addr_t ba
     }
     cpu_register_physical_memory(base, size, isa_mmio_iomemtype);
 }
+#endif
Index: kvm-userspace.io/qemu/hw/lsi53c895a.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/lsi53c895a.c
+++ kvm-userspace.io/qemu/hw/lsi53c895a.c
@@ -1813,7 +1813,8 @@ static void lsi_ram_mapfunc(PCIDevice *p
 
     DPRINTF("Mapping ram at %08x\n", addr);
     s->script_ram_base = addr;
-    cpu_register_physical_memory(addr + 0, 0x2000, s->ram_io_addr);
+    cpu_register_physical_memory(addr + 0, 0x2000,
+                                 s->ram_io_addr);
 }
 
 static void lsi_mmio_mapfunc(PCIDevice *pci_dev, int region_num,
@@ -1822,7 +1823,8 @@ static void lsi_mmio_mapfunc(PCIDevice *
     LSIState *s = (LSIState *)pci_dev;
 
     DPRINTF("Mapping registers at %08x\n", addr);
-    cpu_register_physical_memory(addr + 0, 0x400, s->mmio_io_addr);
+    cpu_register_physical_memory(addr + 0, 0x400,
+                                 s->mmio_io_addr);
 }
 
 void lsi_scsi_attach(void *opaque, BlockDriverState *bd, int id)
@@ -1865,6 +1867,7 @@ int lsi_scsi_uninit(PCIDevice *d)
 void *lsi_scsi_init(PCIBus *bus, int devfn)
 {
     LSIState *s;
+    QEMUDevice *dev;
 
     s = (LSIState *)pci_register_device(bus, "LSI53C895A SCSI HBA",
                                         sizeof(*s), devfn, NULL, NULL);
@@ -1872,6 +1875,7 @@ void *lsi_scsi_init(PCIBus *bus, int dev
         fprintf(stderr, "lsi-scsi: Failed to register PCI device\n");
         return NULL;
     }
+    dev = &s->pci_dev.qemu_dev;
 
     s->pci_dev.config[0x00] = 0x00;
     s->pci_dev.config[0x01] = 0x10;
@@ -1880,9 +1884,9 @@ void *lsi_scsi_init(PCIBus *bus, int dev
     s->pci_dev.config[0x0b] = 0x01;
     s->pci_dev.config[0x3d] = 0x01; /* interrupt pin 1 */
 
-    s->mmio_io_addr = cpu_register_io_memory(0, lsi_mmio_readfn,
+    s->mmio_io_addr = cpu_register_io_memory(dev, 0, lsi_mmio_readfn,
                                              lsi_mmio_writefn, s);
-    s->ram_io_addr = cpu_register_io_memory(0, lsi_ram_readfn,
+    s->ram_io_addr = cpu_register_io_memory(dev, 0, lsi_ram_readfn,
                                             lsi_ram_writefn, s);
 
     pci_register_io_region((struct PCIDevice *)s, 0, 256,
Index: kvm-userspace.io/qemu/hw/mc146818rtc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/mc146818rtc.c
+++ kvm-userspace.io/qemu/hw/mc146818rtc.c
@@ -564,6 +564,7 @@ static CPUWriteMemoryFunc *rtc_mm_write[
     &cmos_mm_writel,
 };
 
+#ifdef TARGET_MIPS
 RTCState *rtc_mm_init(target_phys_addr_t base, int it_shift, qemu_irq irq)
 {
     RTCState *s;
@@ -598,3 +599,4 @@ RTCState *rtc_mm_init(target_phys_addr_t
     register_savevm("mc146818rtc", base, 1, rtc_save, rtc_load, s);
     return s;
 }
+#endif
Index: kvm-userspace.io/qemu/hw/parallel.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/parallel.c
+++ kvm-userspace.io/qemu/hw/parallel.c
@@ -523,6 +523,7 @@ static CPUWriteMemoryFunc *parallel_mm_w
     &parallel_mm_writel,
 };
 
+#ifdef TARGET_MIPS
 /* If fd is zero, it means that the parallel device uses the console */
 ParallelState *parallel_mm_init(target_phys_addr_t base, int it_shift, qemu_irq irq, CharDriverState *chr)
 {
@@ -540,3 +541,4 @@ ParallelState *parallel_mm_init(target_p
     cpu_register_physical_memory(base, 8 << it_shift, io_sw);
     return s;
 }
+#endif
Index: kvm-userspace.io/qemu/hw/pckbd.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pckbd.c
+++ kvm-userspace.io/qemu/hw/pckbd.c
@@ -432,6 +432,7 @@ static CPUWriteMemoryFunc *kbd_mm_write[
     &kbd_mm_writeb,
 };
 
+#ifdef TARGET_MIPS
 void i8042_mm_init(qemu_irq kbd_irq, qemu_irq mouse_irq,
                    target_phys_addr_t base, int it_shift)
 {
@@ -450,8 +451,7 @@ void i8042_mm_init(qemu_irq kbd_irq, qem
 
     s->kbd = ps2_kbd_init(kbd_update_kbd_irq, s);
     s->mouse = ps2_mouse_init(kbd_update_aux_irq, s);
-#ifdef TARGET_I386
     vmmouse_init(s->mouse);
-#endif
     qemu_register_reset(kbd_reset, s);
 }
+#endif
Index: kvm-userspace.io/qemu/hw/pcnet.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pcnet.c
+++ kvm-userspace.io/qemu/hw/pcnet.c
@@ -1944,7 +1944,8 @@ static void pcnet_mmio_map(PCIDevice *pc
     printf("pcnet_ioport_map addr=0x%08x 0x%08x\n", addr, size);
 #endif
 
-    cpu_register_physical_memory(addr, PCNET_PNPMMIO_SIZE, d->mmio_index);
+    cpu_register_physical_memory(addr, PCNET_PNPMMIO_SIZE,
+                                 d->mmio_index);
 }
 
 static void pci_physical_memory_write(void *dma_opaque, target_phys_addr_t addr,
@@ -1962,6 +1963,7 @@ static void pci_physical_memory_read(voi
 PCIDevice *pci_pcnet_init(PCIBus *bus, NICInfo *nd, int devfn)
 {
     PCNetState *d;
+    QEMUDevice *dev;
     uint8_t *pci_conf;
 
 #if 0
@@ -1972,6 +1974,8 @@ PCIDevice *pci_pcnet_init(PCIBus *bus, N
     d = (PCNetState *)pci_register_device(bus, "PCNet", sizeof(PCNetState),
                                           devfn, NULL, NULL);
 
+    dev = &d->dev.qemu_dev;
+
     pci_conf = d->dev.config;
 
     *(uint16_t *)&pci_conf[0x00] = cpu_to_le16(0x1022);
@@ -1993,7 +1997,7 @@ PCIDevice *pci_pcnet_init(PCIBus *bus, N
 
     /* Handler for memory-mapped I/O */
     d->mmio_index =
-      cpu_register_io_memory(0, pcnet_mmio_read, pcnet_mmio_write, d);
+      cpu_register_io_memory(dev, 0, pcnet_mmio_read, pcnet_mmio_write, d);
 
     pci_register_io_region((PCIDevice *)d, 0, PCNET_IOPORT_SIZE,
                            PCI_ADDRESS_SPACE_IO, pcnet_ioport_map);
Index: kvm-userspace.io/qemu/hw/piix_pci.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/piix_pci.c
+++ kvm-userspace.io/qemu/hw/piix_pci.c
@@ -62,6 +62,7 @@ static int pci_irq_levels[4];
 static void update_pam(PCIDevice *d, uint32_t start, uint32_t end, int r)
 {
     uint32_t addr;
+    QEMUDevice *dev = &d->qemu_dev;
 
     //    printf("ISA mapping %08x-0x%08x: %d\n", start, end, r);
     switch(r) {
@@ -90,6 +91,7 @@ static void i440fx_update_memory_mapping
 {
     int i, r;
     uint32_t smram, addr;
+    QEMUDevice *dev = &d->qemu_dev;
 
     update_pam(d, 0xf0000, 0x100000, (d->config[0x59] >> 4) & 3);
     for(i = 0; i < 12; i++) {
Index: kvm-userspace.io/qemu/hw/rtl8139.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/rtl8139.c
+++ kvm-userspace.io/qemu/hw/rtl8139.c
@@ -3316,7 +3316,8 @@ static void rtl8139_mmio_map(PCIDevice *
     PCIRTL8139State *d = (PCIRTL8139State *)pci_dev;
     RTL8139State *s = &d->rtl8139;
 
-    cpu_register_physical_memory(addr + 0, 0x100, s->rtl8139_mmio_io_addr);
+    cpu_register_physical_memory(addr + 0, 0x100,
+                                 s->rtl8139_mmio_io_addr);
 }
 
 static void rtl8139_ioport_map(PCIDevice *pci_dev, int region_num,
@@ -3407,11 +3408,13 @@ PCIDevice *pci_rtl8139_init(PCIBus *bus,
     RTL8139State *s;
     uint8_t *pci_conf;
     static int rtl8139_id;
+    QEMUDevice *dev;
 
     d = (PCIRTL8139State *)pci_register_device(bus,
                                               "RTL8139", sizeof(PCIRTL8139State),
                                               devfn,
                                               NULL, NULL);
+    dev = &d->dev.qemu_dev;
     pci_conf = d->dev.config;
     pci_conf[0x00] = 0xec; /* Realtek 8139 */
     pci_conf[0x01] = 0x10;
@@ -3429,7 +3432,7 @@ PCIDevice *pci_rtl8139_init(PCIBus *bus,
 
     /* I/O handler for memory-mapped I/O */
     s->rtl8139_mmio_io_addr =
-    cpu_register_io_memory(0, rtl8139_mmio_read, rtl8139_mmio_write, s);
+    cpu_register_io_memory(dev, 0, rtl8139_mmio_read, rtl8139_mmio_write, s);
 
     pci_register_io_region(&d->dev, 0, 0x100,
                            PCI_ADDRESS_SPACE_IO,  rtl8139_ioport_map);
Index: kvm-userspace.io/qemu/hw/serial.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/serial.c
+++ kvm-userspace.io/qemu/hw/serial.c
@@ -477,6 +477,7 @@ static CPUWriteMemoryFunc *serial_mm_wri
     &serial_mm_writel,
 };
 
+#ifndef TARGET_I386
 SerialState *serial_mm_init (target_phys_addr_t base, int it_shift,
                              qemu_irq irq, CharDriverState *chr,
                              int ioregister)
@@ -506,3 +507,4 @@ SerialState *serial_mm_init (target_phys
                           serial_event, s);
     return s;
 }
+#endif
Index: kvm-userspace.io/qemu/hw/usb-ohci.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/usb-ohci.c
+++ kvm-userspace.io/qemu/hw/usb-ohci.c
@@ -1592,8 +1592,8 @@ static CPUWriteMemoryFunc *ohci_writefn[
     ohci_mem_write
 };
 
-static void usb_ohci_init(OHCIState *ohci, int num_ports, int devfn,
-            qemu_irq irq, enum ohci_type type, const char *name)
+static void usb_ohci_init(QEMUDevice *dev, OHCIState *ohci, int num_ports,
+            int devfn, qemu_irq irq, enum ohci_type type, const char *name)
 {
     int i;
 
@@ -1613,7 +1613,7 @@ static void usb_ohci_init(OHCIState *ohc
                 usb_frame_time, usb_bit_time);
     }
 
-    ohci->mem = cpu_register_io_memory(0, ohci_readfn, ohci_writefn, ohci);
+    ohci->mem = cpu_register_io_memory(dev, 0, ohci_readfn, ohci_writefn, ohci);
     ohci->name = name;
 
     ohci->irq = irq;
@@ -1639,7 +1639,8 @@ static void ohci_mapfunc(PCIDevice *pci_
 {
     OHCIPCIState *ohci = (OHCIPCIState *)pci_dev;
     ohci->state.mem_base = addr;
-    cpu_register_physical_memory(addr, size, ohci->state.mem);
+    cpu_register_physical_memory(addr, size,
+                                 ohci->state.mem);
 }
 
 void usb_ohci_init_pci(struct PCIBus *bus, int num_ports, int devfn)
@@ -1664,13 +1665,16 @@ void usb_ohci_init_pci(struct PCIBus *bu
     ohci->pci_dev.config[0x0b] = 0xc;
     ohci->pci_dev.config[0x3d] = 0x01; /* interrupt pin 1 */
 
-    usb_ohci_init(&ohci->state, num_ports, devfn, ohci->pci_dev.irq[0],
-                  OHCI_TYPE_PCI, ohci->pci_dev.name);
+    usb_ohci_init(&ohci->pci_dev.qemu_dev, &ohci->state, num_ports, devfn,
+                  ohci->pci_dev.irq[0], OHCI_TYPE_PCI, ohci->pci_dev.name);
 
     pci_register_io_region((struct PCIDevice *)ohci, 0, 256,
                            PCI_ADDRESS_SPACE_MEM, ohci_mapfunc);
 }
 
+
+#if 0
+FIXME: use QEMUDevice
 void usb_ohci_init_pxa(target_phys_addr_t base, int num_ports, int devfn,
                        qemu_irq irq)
 {
@@ -1682,3 +1686,4 @@ void usb_ohci_init_pxa(target_phys_addr_
 
     cpu_register_physical_memory(ohci->mem_base, 0x1000, ohci->mem);
 }
+#endif
Index: kvm-userspace.io/qemu/hw/vga.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/vga.c
+++ kvm-userspace.io/qemu/hw/vga.c
@@ -2008,6 +2008,8 @@ static void vga_map(PCIDevice *pci_dev, 
 {
     PCIVGAState *d = (PCIVGAState *)pci_dev;
     VGAState *s = &d->vga_state;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
+
     if (region_num == PCI_ROM_SLOT) {
         cpu_register_physical_memory(addr, s->bios_size, s->bios_offset);
     } else {
@@ -2246,7 +2248,8 @@ void vga_init(VGAState *s, QEMUDevice *d
 #endif
 #endif /* CONFIG_BOCHS_VBE */
 
-    vga_io_memory = cpu_register_io_memory(0, vga_mem_read, vga_mem_write, s);
+    vga_io_memory = cpu_register_io_memory(dev, 0, vga_mem_read, vga_mem_write,
+                                           s);
     cpu_register_physical_memory(isa_mem_base + 0x000a0000, 0x20000,
                                  vga_io_memory);
 }
@@ -2313,11 +2316,12 @@ static void vga_mm_init(VGAState *s, tar
                         target_phys_addr_t ctrl_base, int it_shift)
 {
     int s_ioport_ctrl, vga_io_memory;
+    QEMUDevice *dev = &s->qemu_dev;
 
     s->base_ctrl = ctrl_base;
     s->it_shift = it_shift;
-    s_ioport_ctrl = cpu_register_io_memory(0, vga_mm_read_ctrl, vga_mm_write_ctrl, s);
-    vga_io_memory = cpu_register_io_memory(0, vga_mem_read, vga_mem_write, s);
+    s_ioport_ctrl = cpu_register_io_memory(dev, 0, vga_mm_read_ctrl, vga_mm_write_ctrl, s);
+    vga_io_memory = cpu_register_io_memory(dev, 0, vga_mem_read, vga_mem_write, s);
 
     register_savevm("vga", 0, 2, vga_save, vga_load, s);
 
@@ -2357,6 +2361,7 @@ int isa_vga_mm_init(DisplayState *ds, ui
                     int it_shift)
 {
     VGAState *s;
+    QEMUDevice *dev;
 
     s = qemu_mallocz(sizeof(VGAState));
     if (!s)
@@ -2365,6 +2370,7 @@ int isa_vga_mm_init(DisplayState *ds, ui
     vga_common_init(s, ds, vga_ram_base, vga_ram_offset, vga_ram_size);
     vga_mm_init(s, vram_base, ctrl_base, it_shift);
 
+    dev = &s->qemu_dev;
     graphic_console_init(s->ds, s->update, s->invalidate, s->screen_dump,
                          s->text_update, s);
 
Index: kvm-userspace.io/qemu/hw/vmware_vga.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/vmware_vga.c
+++ kvm-userspace.io/qemu/hw/vmware_vga.c
@@ -1181,11 +1181,12 @@ static void pci_vmsvga_map_mem(PCIDevice
 {
     struct pci_vmsvga_state_s *d = (struct pci_vmsvga_state_s *) pci_dev;
     struct vmsvga_state_s *s = &d->chip;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
     int iomemtype;
 
     s->vram_base = addr;
 #ifdef DIRECT_VRAM
-    iomemtype = cpu_register_io_memory(0, vmsvga_vram_read,
+    iomemtype = cpu_register_io_memory(dev, 0, vmsvga_vram_read,
                     vmsvga_vram_write, s);
 #else
     iomemtype = 0 | IO_MEM_RAM;
Index: kvm-userspace.io/qemu/kqemu.c
===================================================================
--- kvm-userspace.io.orig/qemu/kqemu.c
+++ kvm-userspace.io/qemu/kqemu.c
@@ -38,6 +38,7 @@
 #include <unistd.h>
 #include <inttypes.h>
 
+#include "qemu-common.h"
 #include "cpu.h"
 #include "exec-all.h"
 
Index: kvm-userspace.io/qemu/qemu-common.h
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-common.h
+++ kvm-userspace.io/qemu/qemu-common.h
@@ -29,6 +29,8 @@
 
 #include "qemu-device.h"
 
+#include "qemu-device.h"
+
 #ifdef _WIN32
 #define WIN32_LEAN_AND_MEAN
 #include <windows.h>
Index: kvm-userspace.io/qemu/hw/ide.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/ide.c
+++ kvm-userspace.io/qemu/hw/ide.c
@@ -3027,6 +3027,8 @@ void pci_piix4_ide_init(PCIBus *bus, Blo
     register_savevm("ide", 0, 1, pci_ide_save, pci_ide_load, d);
 }
 
+#ifdef TARGET_PPC
+
 /***********************************************************/
 /* MacIO based PowerPC IDE */
 
@@ -3151,6 +3153,7 @@ int pmac_ide_init (BlockDriverState **hd
                                              pmac_ide_write, &ide_if[0]);
     return pmac_ide_memory;
 }
+#endif
 
 /***********************************************************/
 /* CF-ATA Microdrive */

-- 

[kvm-devel] [patch 03/13] QEMU: make esp.c build conditional to SPARC target

[Marcelo T. <mto...@re...>]

Only used by qemu/hw/sun4m.c.

Index: kvm-userspace.io/qemu/Makefile.target
===================================================================
--- kvm-userspace.io.orig/qemu/Makefile.target
+++ kvm-userspace.io/qemu/Makefile.target
@@ -563,7 +563,7 @@ DEPLIBS += libfdt.a
 endif
 
 # SCSI layer
-OBJS+= lsi53c895a.o esp.o
+OBJS+= lsi53c895a.o
 
 # USB layer
 OBJS+= usb-ohci.o
@@ -642,6 +642,7 @@ OBJS+= ptimer.o
 OBJS+= pflash_cfi01.o
 endif
 ifeq ($(TARGET_BASE_ARCH), sparc)
+OBJS+= esp.o
 ifeq ($(TARGET_ARCH), sparc64)
 OBJS+= sun4u.o ide.o pckbd.o ps2.o vga.o apb_pci.o
 OBJS+= fdc.o mc146818rtc.o serial.o m48t59.o

-- 

[kvm-devel] [patch 05/13] QEMU: add a mutex to protect IRQ chip data structures

[Marcelo T. <mto...@re...>]

Subject says it all. 

With KVM in-kernel irqchip this is unused.

Index: kvm-userspace.io/qemu/hw/apic.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/apic.c
+++ kvm-userspace.io/qemu/hw/apic.c
@@ -478,6 +478,8 @@ int apic_get_interrupt(CPUState *env)
     APICState *s = env->apic_state;
     int intno;
 
+    assert_is_locked(&irq_mutex);
+
     /* if the APIC is installed or enabled, we let the 8259 handle the
        IRQs */
     if (!s)
@@ -597,6 +599,8 @@ static uint32_t apic_mem_readl(void *opa
         return 0;
     s = env->apic_state;
 
+    qemu_mutex_lock(&irq_mutex);
+
     index = (addr >> 4) & 0xff;
     switch(index) {
     case 0x02: /* id */
@@ -659,6 +663,7 @@ static uint32_t apic_mem_readl(void *opa
         val = 0;
         break;
     }
+    qemu_mutex_unlock(&irq_mutex);
 #ifdef DEBUG_APIC
     printf("APIC read: %08x = %08x\n", (uint32_t)addr, val);
 #endif
@@ -676,6 +681,8 @@ static void apic_mem_writel(void *opaque
         return;
     s = env->apic_state;
 
+    qemu_mutex_lock(&irq_mutex);
+
 #ifdef DEBUG_APIC
     printf("APIC write: %08x = %08x\n", (uint32_t)addr, val);
 #endif
@@ -748,6 +755,8 @@ static void apic_mem_writel(void *opaque
         s->esr |= ESR_ILLEGAL_ADDRESS;
         break;
     }
+
+    qemu_mutex_unlock(&irq_mutex);
 }
 
 #ifdef KVM_CAP_IRQCHIP
@@ -995,6 +1004,8 @@ static void ioapic_service(IOAPICState *
     uint8_t polarity;
     uint32_t deliver_bitmask[MAX_APIC_WORDS];
 
+    assert_is_locked(&irq_mutex);
+
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         mask = 1 << i;
         if (s->irr & mask) {
@@ -1087,9 +1098,12 @@ static void ioapic_mem_writel(void *opaq
     IOAPICState *s = opaque;
     int index;
 
+    qemu_mutex_lock(&irq_mutex);
+
     addr &= 0xff;
     if (addr == 0x00)  {
         s->ioregsel = val;
+        qemu_mutex_unlock(&irq_mutex);
         return;
     } else if (addr == 0x10) {
 #ifdef DEBUG_IOAPIC
@@ -1113,6 +1127,7 @@ static void ioapic_mem_writel(void *opaq
                         s->ioredtbl[index] |= val;
                     }
                     ioapic_service(s);
+                    qemu_mutex_unlock(&irq_mutex);
                 }
         }
     }
Index: kvm-userspace.io/qemu/hw/i8259.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/i8259.c
+++ kvm-userspace.io/qemu/hw/i8259.c
@@ -78,6 +78,7 @@ static uint64_t irq_count[16];
 static inline void pic_set_irq1(PicState *s, int irq, int level)
 {
     int mask;
+    assert_is_locked(&irq_mutex);
     mask = 1 << irq;
     if (s->elcr & mask) {
         /* level triggered */
@@ -118,6 +119,8 @@ static int pic_get_irq(PicState *s)
 {
     int mask, cur_priority, priority;
 
+    assert_is_locked(&irq_mutex);
+
     mask = s->irr & ~s->imr;
     priority = get_priority(s, mask);
     if (priority == 8)
@@ -189,6 +192,7 @@ static void i8259_set_irq(void *opaque, 
 	if (kvm_set_irq(irq, level))
 	    return;
 #endif
+    qemu_mutex_lock(&irq_mutex);
 #if defined(DEBUG_PIC) || defined(DEBUG_IRQ_COUNT)
     if (level != irq_level[irq]) {
 #if defined(DEBUG_PIC)
@@ -211,6 +215,7 @@ static void i8259_set_irq(void *opaque, 
     if (s->alt_irq_func)
         s->alt_irq_func(s->alt_irq_opaque, irq, level);
     pic_update_irq(s);
+    qemu_mutex_unlock(&irq_mutex);
 }
 
 /* acknowledge interrupt 'irq' */
@@ -309,6 +314,7 @@ static void pic_ioport_write(void *opaqu
     PicState *s = opaque;
     int priority, cmd, irq;
 
+    qemu_mutex_lock(&irq_mutex);
 #ifdef DEBUG_PIC
     printf("pic_write: addr=0x%02x val=0x%02x\n", addr, val);
 #endif
@@ -394,6 +400,7 @@ static void pic_ioport_write(void *opaqu
             break;
         }
     }
+    qemu_mutex_unlock(&irq_mutex);
 }
 
 static uint32_t pic_poll_read (PicState *s, uint32_t addr1)
@@ -424,6 +431,7 @@ static uint32_t pic_ioport_read(void *op
     unsigned int addr;
     int ret;
 
+    qemu_mutex_lock(&irq_mutex);
     addr = addr1;
     addr &= 1;
     if (s->poll) {
@@ -439,6 +447,7 @@ static uint32_t pic_ioport_read(void *op
             ret = s->imr;
         }
     }
+    qemu_mutex_unlock(&irq_mutex);
 #ifdef DEBUG_PIC
     printf("pic_read: addr=0x%02x val=0x%02x\n", addr1, ret);
 #endif
@@ -451,11 +460,13 @@ uint32_t pic_intack_read(PicState2 *s)
 {
     int ret;
 
+    qemu_mutex_lock(&irq_mutex);
     ret = pic_poll_read(&s->pics[0], 0x00);
     if (ret == 2)
         ret = pic_poll_read(&s->pics[1], 0x80) + 8;
     /* Prepare for ISR read */
     s->pics[0].read_reg_select = 1;
+    qemu_mutex_unlock(&irq_mutex);
 
     return ret;
 }
Index: kvm-userspace.io/qemu/hw/pc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pc.c
+++ kvm-userspace.io/qemu/hw/pc.c
@@ -53,6 +53,8 @@ static PITState *pit;
 static IOAPICState *ioapic;
 static PCIDevice *i440fx_state;
 
+qemu_mutex_t irq_mutex;
+
 static void ioport80_write(void *opaque, uint32_t addr, uint32_t data)
 {
 }
@@ -99,24 +101,30 @@ int cpu_get_pic_interrupt(CPUState *env)
 {
     int intno;
 
+    qemu_mutex_lock(&irq_mutex);
     intno = apic_get_interrupt(env);
     if (intno >= 0) {
         /* set irq request if a PIC irq is still pending */
         /* XXX: improve that */
         pic_update_irq(isa_pic);
-        return intno;
+        goto out;
     }
     /* read the irq from the PIC */
-    if (!apic_accept_pic_intr(env))
-        return -1;
+    if (!apic_accept_pic_intr(env)) {
+        intno = -1;
+        goto out;
+    }
 
     intno = pic_read_irq(isa_pic);
+out:
+    qemu_mutex_unlock(&irq_mutex);
     return intno;
 }
 
 static void pic_irq_request(void *opaque, int irq, int level)
 {
     CPUState *env = opaque;
+    assert_is_locked(&irq_mutex);
     if (level && apic_accept_pic_intr(env))
         cpu_interrupt(env, CPU_INTERRUPT_HARD);
 }
@@ -783,6 +791,8 @@ static void pc_init1(ram_addr_t ram_size
     BlockDriverState *hd[MAX_IDE_BUS * MAX_IDE_DEVS];
     BlockDriverState *fd[MAX_FD];
 
+    qemu_mutex_init(&irq_mutex);
+
     if (ram_size >= 0xe0000000 ) {
         above_4g_mem_size = ram_size - 0xe0000000;
         ram_size = 0xe0000000;
Index: kvm-userspace.io/qemu/hw/pc.h
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pc.h
+++ kvm-userspace.io/qemu/hw/pc.h
@@ -2,6 +2,8 @@
 #define HW_PC_H
 /* PC-style peripherals (also used by other machines).  */
 
+extern qemu_mutex_t irq_mutex;
+
 /* serial.c */
 
 SerialState *serial_init(int base, qemu_irq irq, CharDriverState *chr);

-- 

[kvm-devel] [patch 04/13] QEMU: plug QEMUDevice pt1 / ioport awareness

[Marcelo T. <mto...@re...>]

Plug QEMUDevice.

ISA drivers should add QEMUDevice to their structures and call qemu_register_device()
themselves, while the PCI layer does it automatically.

Pass QEMUDevice to register_ioport() making the ioport->device relationship
visible.

Index: kvm-userspace.io/qemu/hw/pci.h
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pci.h
+++ kvm-userspace.io/qemu/hw/pci.h
@@ -11,6 +11,28 @@
 /* PCI bus */
 extern target_phys_addr_t pci_mem_base;
 
+typedef void (*pci_set_irq_fn)(qemu_irq *pic, int irq_num, int level);
+typedef int (*pci_map_irq_fn)(PCIDevice *pci_dev, int irq_num);
+
+struct PCIBus {
+    QEMUDevice qemu_dev;
+    int bus_num;
+    int devfn_min;
+    pci_set_irq_fn set_irq;
+    pci_map_irq_fn map_irq;
+    uint32_t config_reg; /* XXX: suppress */
+    /* low level pic */
+    SetIRQFunc *low_set_irq;
+    qemu_irq *irq_opaque;
+    PCIDevice *devices[256];
+    PCIDevice *parent_dev;
+    PCIBus *next;
+    /* The bus IRQ state is the logical OR of the connected devices.
+       Keep a count of the number of devices with raised IRQs.  */
+    int nirq;
+    int irq_count[];
+};
+
 typedef void PCIConfigWriteFunc(PCIDevice *pci_dev,
                                 uint32_t address, uint32_t data, int len);
 typedef uint32_t PCIConfigReadFunc(PCIDevice *pci_dev,
@@ -47,6 +69,7 @@ typedef struct PCIIORegion {
 #define PCI_MAX_LAT		0x3f	/* 8 bits */
 
 struct PCIDevice {
+    QEMUDevice qemu_dev;
     /* PCI config space */
     uint8_t config[256];
 
@@ -88,8 +111,6 @@ void pci_default_write_config(PCIDevice 
 void pci_device_save(PCIDevice *s, QEMUFile *f);
 int pci_device_load(PCIDevice *s, QEMUFile *f);
 
-typedef void (*pci_set_irq_fn)(qemu_irq *pic, int irq_num, int level);
-typedef int (*pci_map_irq_fn)(PCIDevice *pci_dev, int irq_num);
 PCIBus *pci_register_bus(pci_set_irq_fn set_irq, pci_map_irq_fn map_irq,
                          qemu_irq *pic, int devfn_min, int nirq);
 
Index: kvm-userspace.io/qemu/hw/pci.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pci.c
+++ kvm-userspace.io/qemu/hw/pci.c
@@ -29,24 +29,6 @@
 
 //#define DEBUG_PCI
 
-struct PCIBus {
-    int bus_num;
-    int devfn_min;
-    pci_set_irq_fn set_irq;
-    pci_map_irq_fn map_irq;
-    uint32_t config_reg; /* XXX: suppress */
-    /* low level pic */
-    SetIRQFunc *low_set_irq;
-    qemu_irq *irq_opaque;
-    PCIDevice *devices[256];
-    PCIDevice *parent_dev;
-    PCIBus *next;
-    /* The bus IRQ state is the logical OR of the connected devices.
-       Keep a count of the number of devices with raised IRQs.  */
-    int nirq;
-    int irq_count[];
-};
-
 static void pci_update_mappings(PCIDevice *d);
 static void pci_set_irq(void *opaque, int irq_num, int level);
 
@@ -92,6 +74,7 @@ PCIBus *pci_register_bus(pci_set_irq_fn 
     static int nbus = 0;
 
     bus = qemu_mallocz(sizeof(PCIBus) + (nirq * sizeof(int)));
+    qemu_register_device(&bus->qemu_dev);
     bus->set_irq = set_irq;
     bus->map_irq = map_irq;
     bus->irq_opaque = pic;
@@ -168,6 +151,7 @@ PCIDevice *pci_register_device(PCIBus *b
     pci_dev = qemu_mallocz(instance_size);
     if (!pci_dev)
         return NULL;
+    qemu_register_device(&pci_dev->qemu_dev);
     pci_dev->bus = bus;
     pci_dev->devfn = devfn;
     pstrcpy(pci_dev->name, sizeof(pci_dev->name), name);
Index: kvm-userspace.io/qemu/hw/eepro100.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/eepro100.c
+++ kvm-userspace.io/qemu/hw/eepro100.c
@@ -1374,17 +1374,18 @@ static void pci_map(PCIDevice * pci_dev,
 {
     PCIEEPRO100State *d = (PCIEEPRO100State *) pci_dev;
     EEPRO100State *s = &d->eepro100;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
     logout("region %d, addr=0x%08x, size=0x%08x, type=%d\n",
            region_num, addr, size, type);
 
     assert(region_num == 1);
-    register_ioport_write(addr, size, 1, ioport_write1, s);
-    register_ioport_read(addr, size, 1, ioport_read1, s);
-    register_ioport_write(addr, size, 2, ioport_write2, s);
-    register_ioport_read(addr, size, 2, ioport_read2, s);
-    register_ioport_write(addr, size, 4, ioport_write4, s);
-    register_ioport_read(addr, size, 4, ioport_read4, s);
+    register_ioport_write(dev, addr, size, 1, ioport_write1, s);
+    register_ioport_read(dev, addr, size, 1, ioport_read1, s);
+    register_ioport_write(dev, addr, size, 2, ioport_write2, s);
+    register_ioport_read(dev, addr, size, 2, ioport_read2, s);
+    register_ioport_write(dev, addr, size, 4, ioport_write4, s);
+    register_ioport_read(dev, addr, size, 4, ioport_read4, s);
 
     s->region[region_num] = addr;
 }
Index: kvm-userspace.io/qemu/hw/isa.h
===================================================================
--- kvm-userspace.io.orig/qemu/hw/isa.h
+++ kvm-userspace.io/qemu/hw/isa.h
@@ -2,9 +2,9 @@
 
 extern target_phys_addr_t isa_mem_base;
 
-int register_ioport_read(int start, int length, int size,
+int register_ioport_read(QEMUDevice *dev, int start, int length, int size,
                          IOPortReadFunc *func, void *opaque);
-int register_ioport_write(int start, int length, int size,
+int register_ioport_write(QEMUDevice *dev, int start, int length, int size,
                           IOPortWriteFunc *func, void *opaque);
 void isa_unassign_ioport(int start, int length);
 
Index: kvm-userspace.io/qemu/hw/lsi53c895a.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/lsi53c895a.c
+++ kvm-userspace.io/qemu/hw/lsi53c895a.c
@@ -1794,15 +1794,16 @@ static void lsi_io_mapfunc(PCIDevice *pc
                            uint32_t addr, uint32_t size, int type)
 {
     LSIState *s = (LSIState *)pci_dev;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
     DPRINTF("Mapping IO at %08x\n", addr);
 
-    register_ioport_write(addr, 256, 1, lsi_io_writeb, s);
-    register_ioport_read(addr, 256, 1, lsi_io_readb, s);
-    register_ioport_write(addr, 256, 2, lsi_io_writew, s);
-    register_ioport_read(addr, 256, 2, lsi_io_readw, s);
-    register_ioport_write(addr, 256, 4, lsi_io_writel, s);
-    register_ioport_read(addr, 256, 4, lsi_io_readl, s);
+    register_ioport_write(dev, addr, 256, 1, lsi_io_writeb, s);
+    register_ioport_read(dev, addr, 256, 1, lsi_io_readb, s);
+    register_ioport_write(dev, addr, 256, 2, lsi_io_writew, s);
+    register_ioport_read(dev, addr, 256, 2, lsi_io_readw, s);
+    register_ioport_write(dev, addr, 256, 4, lsi_io_writel, s);
+    register_ioport_read(dev, addr, 256, 4, lsi_io_readl, s);
 }
 
 static void lsi_ram_mapfunc(PCIDevice *pci_dev, int region_num,
Index: kvm-userspace.io/qemu/hw/ne2000.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/ne2000.c
+++ kvm-userspace.io/qemu/hw/ne2000.c
@@ -122,6 +122,7 @@
 #define NE2000_MEM_SIZE     NE2000_PMEM_END
 
 typedef struct NE2000State {
+    QEMUDevice qemu_dev;
     uint8_t cmd;
     uint32_t start;
     uint32_t stop;
@@ -723,21 +724,25 @@ static int ne2000_load(QEMUFile* f,void*
 void isa_ne2000_init(int base, qemu_irq irq, NICInfo *nd)
 {
     NE2000State *s;
+    QEMUDevice *dev;
 
     s = qemu_mallocz(sizeof(NE2000State));
     if (!s)
         return;
 
-    register_ioport_write(base, 16, 1, ne2000_ioport_write, s);
-    register_ioport_read(base, 16, 1, ne2000_ioport_read, s);
+    dev = &s->qemu_dev;
+    qemu_register_device(dev);
 
-    register_ioport_write(base + 0x10, 1, 1, ne2000_asic_ioport_write, s);
-    register_ioport_read(base + 0x10, 1, 1, ne2000_asic_ioport_read, s);
-    register_ioport_write(base + 0x10, 2, 2, ne2000_asic_ioport_write, s);
-    register_ioport_read(base + 0x10, 2, 2, ne2000_asic_ioport_read, s);
+    register_ioport_write(dev, base, 16, 1, ne2000_ioport_write, s);
+    register_ioport_read(dev, base, 16, 1, ne2000_ioport_read, s);
 
-    register_ioport_write(base + 0x1f, 1, 1, ne2000_reset_ioport_write, s);
-    register_ioport_read(base + 0x1f, 1, 1, ne2000_reset_ioport_read, s);
+    register_ioport_write(dev, base + 0x10, 1, 1, ne2000_asic_ioport_write, s);
+    register_ioport_read(dev, base + 0x10, 1, 1, ne2000_asic_ioport_read, s);
+    register_ioport_write(dev, base + 0x10, 2, 2, ne2000_asic_ioport_write, s);
+    register_ioport_read(dev, base + 0x10, 2, 2, ne2000_asic_ioport_read, s);
+
+    register_ioport_write(dev, base + 0x1f, 1, 1, ne2000_reset_ioport_write, s);
+    register_ioport_read(dev, base + 0x1f, 1, 1, ne2000_reset_ioport_read, s);
     s->irq = irq;
     memcpy(s->macaddr, nd->macaddr, 6);
 
@@ -771,19 +776,20 @@ static void ne2000_map(PCIDevice *pci_de
 {
     PCINE2000State *d = (PCINE2000State *)pci_dev;
     NE2000State *s = &d->ne2000;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
-    register_ioport_write(addr, 16, 1, ne2000_ioport_write, s);
-    register_ioport_read(addr, 16, 1, ne2000_ioport_read, s);
+    register_ioport_write(dev, addr, 16, 1, ne2000_ioport_write, s);
+    register_ioport_read(dev, addr, 16, 1, ne2000_ioport_read, s);
 
-    register_ioport_write(addr + 0x10, 1, 1, ne2000_asic_ioport_write, s);
-    register_ioport_read(addr + 0x10, 1, 1, ne2000_asic_ioport_read, s);
-    register_ioport_write(addr + 0x10, 2, 2, ne2000_asic_ioport_write, s);
-    register_ioport_read(addr + 0x10, 2, 2, ne2000_asic_ioport_read, s);
-    register_ioport_write(addr + 0x10, 4, 4, ne2000_asic_ioport_writel, s);
-    register_ioport_read(addr + 0x10, 4, 4, ne2000_asic_ioport_readl, s);
+    register_ioport_write(dev, addr + 0x10, 1, 1, ne2000_asic_ioport_write, s);
+    register_ioport_read(dev, addr + 0x10, 1, 1, ne2000_asic_ioport_read, s);
+    register_ioport_write(dev, addr + 0x10, 2, 2, ne2000_asic_ioport_write, s);
+    register_ioport_read(dev, addr + 0x10, 2, 2, ne2000_asic_ioport_read, s);
+    register_ioport_write(dev, addr + 0x10, 4, 4, ne2000_asic_ioport_writel, s);
+    register_ioport_read(dev, addr + 0x10, 4, 4, ne2000_asic_ioport_readl, s);
 
-    register_ioport_write(addr + 0x1f, 1, 1, ne2000_reset_ioport_write, s);
-    register_ioport_read(addr + 0x1f, 1, 1, ne2000_reset_ioport_read, s);
+    register_ioport_write(dev, addr + 0x1f, 1, 1, ne2000_reset_ioport_write, s);
+    register_ioport_read(dev, addr + 0x1f, 1, 1, ne2000_reset_ioport_read, s);
 }
 
 PCIDevice *pci_ne2000_init(PCIBus *bus, NICInfo *nd, int devfn)
Index: kvm-userspace.io/qemu/vl.c
===================================================================
--- kvm-userspace.io.orig/qemu/vl.c
+++ kvm-userspace.io/qemu/vl.c
@@ -346,7 +346,7 @@ static void init_ioports(void)
 }
 
 /* size is the word size in byte */
-int register_ioport_read(int start, int length, int size,
+int register_ioport_read(QEMUDevice *dev, int start, int length, int size,
                          IOPortReadFunc *func, void *opaque)
 {
     int i, bsize;
@@ -371,7 +371,7 @@ int register_ioport_read(int start, int 
 }
 
 /* size is the word size in byte */
-int register_ioport_write(int start, int length, int size,
+int register_ioport_write(QEMUDevice *dev, int start, int length, int size,
                           IOPortWriteFunc *func, void *opaque)
 {
     int i, bsize;
Index: kvm-userspace.io/qemu/audio/audio.h
===================================================================
--- kvm-userspace.io.orig/qemu/audio/audio.h
+++ kvm-userspace.io/qemu/audio/audio.h
@@ -78,6 +78,7 @@ typedef struct CaptureVoiceOut CaptureVo
 typedef struct SWVoiceIn SWVoiceIn;
 
 typedef struct QEMUSoundCard {
+    QEMUDevice qemu_dev;
     AudioState *audio;
     char *name;
     LIST_ENTRY (QEMUSoundCard) entries;
Index: kvm-userspace.io/qemu/hw/acpi.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/acpi.c
+++ kvm-userspace.io/qemu/hw/acpi.c
@@ -80,6 +80,7 @@ typedef struct PIIX4PMState {
 #define SMBBLKDAT 0x07
 
 PIIX4PMState *pm_state;
+QEMUDevice *pm_dev;
 
 static void update_pmtmr(PIIX4PMState *s)
 {
@@ -425,10 +426,10 @@ static void pm_io_space_update(PIIX4PMSt
 #if defined(DEBUG)
         printf("PM: mapping to 0x%x\n", pm_io_base);
 #endif
-        register_ioport_write(pm_io_base, 64, 2, pm_ioport_writew, s);
-        register_ioport_read(pm_io_base, 64, 2, pm_ioport_readw, s);
-        register_ioport_write(pm_io_base, 64, 4, pm_ioport_writel, s);
-        register_ioport_read(pm_io_base, 64, 4, pm_ioport_readl, s);
+        register_ioport_write(pm_dev, pm_io_base, 64, 2, pm_ioport_writew, s);
+        register_ioport_read(pm_dev, pm_io_base, 64, 2, pm_ioport_readw, s);
+        register_ioport_write(pm_dev, pm_io_base, 64, 4, pm_ioport_writel, s);
+        register_ioport_read(pm_dev, pm_io_base, 64, 4, pm_ioport_readl, s);
     }
 }
 
@@ -490,6 +491,7 @@ i2c_bus *piix4_pm_init(PCIBus *bus, int 
                                          "PM", sizeof(PIIX4PMState),
                                          devfn, NULL, pm_write_config);
     pm_state = s;
+    pm_dev = &pm_state->dev.qemu_dev;
     pci_conf = s->dev.config;
     pci_conf[0x00] = 0x86;
     pci_conf[0x01] = 0x80;
@@ -506,10 +508,10 @@ i2c_bus *piix4_pm_init(PCIBus *bus, int 
 
     pci_conf[0x40] = 0x01; /* PM io base read only bit */
 
-    register_ioport_write(0xb2, 2, 1, pm_smi_writeb, s);
-    register_ioport_read(0xb2, 2, 1, pm_smi_readb, s);
+    register_ioport_write(pm_dev, 0xb2, 2, 1, pm_smi_writeb, s);
+    register_ioport_read(pm_dev, 0xb2, 2, 1, pm_smi_readb, s);
 
-    register_ioport_write(ACPI_DBG_IO_ADDR, 4, 4, acpi_dbg_writel, s);
+    register_ioport_write(pm_dev, ACPI_DBG_IO_ADDR, 4, 4, acpi_dbg_writel, s);
 
     /* XXX: which specification is used ? The i82731AB has different
        mappings */
@@ -521,8 +523,8 @@ i2c_bus *piix4_pm_init(PCIBus *bus, int 
     pci_conf[0x90] = smb_io_base | 1;
     pci_conf[0x91] = smb_io_base >> 8;
     pci_conf[0xd2] = 0x09;
-    register_ioport_write(smb_io_base, 64, 1, smb_ioport_writeb, s);
-    register_ioport_read(smb_io_base, 64, 1, smb_ioport_readb, s);
+    register_ioport_write(pm_dev, smb_io_base, 64, 1, smb_ioport_writeb, s);
+    register_ioport_read(pm_dev, smb_io_base, 64, 1, smb_ioport_readb, s);
 
     s->tmr_timer = qemu_new_timer(vm_clock, pm_tmr_timer, s);
 
@@ -689,17 +691,17 @@ static const char *model;
 
 void qemu_system_hot_add_init(const char *cpu_model)
 {
-    register_ioport_write(GPE_BASE, 4, 1, gpe_writeb, &gpe);
-    register_ioport_read(GPE_BASE, 4, 1,  gpe_readb, &gpe);
+    register_ioport_write(pm_dev, GPE_BASE, 4, 1, gpe_writeb, &gpe);
+    register_ioport_read(pm_dev, GPE_BASE, 4, 1,  gpe_readb, &gpe);
 
-    register_ioport_write(PROC_BASE, 4, 1, gpe_writeb, &gpe);
-    register_ioport_read(PROC_BASE, 4, 1,  gpe_readb, &gpe);
+    register_ioport_write(pm_dev, PROC_BASE, 4, 1, gpe_writeb, &gpe);
+    register_ioport_read(pm_dev, PROC_BASE, 4, 1,  gpe_readb, &gpe);
 
-    register_ioport_write(PCI_BASE, 8, 4, pcihotplug_write, &pci0_status);
-    register_ioport_read(PCI_BASE, 8, 4,  pcihotplug_read, &pci0_status);
+    register_ioport_write(pm_dev, PCI_BASE, 8, 4, pcihotplug_write, &pci0_status);
+    register_ioport_read(pm_dev, PCI_BASE, 8, 4,  pcihotplug_read, &pci0_status);
 
-    register_ioport_write(PCI_EJ_BASE, 4, 4, pciej_write, NULL);
-    register_ioport_read(PCI_EJ_BASE, 4, 4,  pciej_read, NULL);
+    register_ioport_write(pm_dev, PCI_EJ_BASE, 4, 4, pciej_write, NULL);
+    register_ioport_read(pm_dev, PCI_EJ_BASE, 4, 4,  pciej_read, NULL);
 
     model = cpu_model;
 }
Index: kvm-userspace.io/qemu/hw/cirrus_vga.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/cirrus_vga.c
+++ kvm-userspace.io/qemu/hw/cirrus_vga.c
@@ -235,6 +235,7 @@ typedef void (*cirrus_fill_t)(struct Cir
 typedef struct CirrusVGAState {
     VGA_STATE_COMMON
 
+    QEMUDevice qemu_dev;
     int cirrus_linear_io_addr;
     int cirrus_linear_bitblt_io_addr;
     int cirrus_mmio_io_addr;
@@ -3215,7 +3216,8 @@ static int cirrus_vga_load(QEMUFile *f, 
  *
  ***************************************/
 
-static void cirrus_init_common(CirrusVGAState * s, int device_id, int is_pci)
+static void cirrus_init_common(CirrusVGAState * s, int device_id, int is_pci,
+                               QEMUDevice *dev)
 {
     int vga_io_memory, i;
     static int inited;
@@ -3242,19 +3244,19 @@ static void cirrus_init_common(CirrusVGA
         rop_to_index[CIRRUS_ROP_NOTSRC_AND_NOTDST] = 15;
     }
 
-    register_ioport_write(0x3c0, 16, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3c0, 16, 1, vga_ioport_write, s);
 
-    register_ioport_write(0x3b4, 2, 1, vga_ioport_write, s);
-    register_ioport_write(0x3d4, 2, 1, vga_ioport_write, s);
-    register_ioport_write(0x3ba, 1, 1, vga_ioport_write, s);
-    register_ioport_write(0x3da, 1, 1, vga_ioport_write, s);
-
-    register_ioport_read(0x3c0, 16, 1, vga_ioport_read, s);
-
-    register_ioport_read(0x3b4, 2, 1, vga_ioport_read, s);
-    register_ioport_read(0x3d4, 2, 1, vga_ioport_read, s);
-    register_ioport_read(0x3ba, 1, 1, vga_ioport_read, s);
-    register_ioport_read(0x3da, 1, 1, vga_ioport_read, s);
+    register_ioport_write(dev, 0x3b4, 2, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3d4, 2, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3ba, 1, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3da, 1, 1, vga_ioport_write, s);
+
+    register_ioport_read(dev, 0x3c0, 16, 1, vga_ioport_read, s);
+
+    register_ioport_read(dev, 0x3b4, 2, 1, vga_ioport_read, s);
+    register_ioport_read(dev, 0x3d4, 2, 1, vga_ioport_read, s);
+    register_ioport_read(dev, 0x3ba, 1, 1, vga_ioport_read, s);
+    register_ioport_read(dev, 0x3da, 1, 1, vga_ioport_read, s);
 
     vga_io_memory = cpu_register_io_memory(0, cirrus_vga_mem_read,
                                            cirrus_vga_mem_write, s);
@@ -3334,12 +3336,16 @@ void isa_cirrus_vga_init(DisplayState *d
                          unsigned long vga_ram_offset, int vga_ram_size)
 {
     CirrusVGAState *s;
+    QEMUDevice *dev;
 
     s = qemu_mallocz(sizeof(CirrusVGAState));
 
+    dev = &s->qemu_dev;
+    qemu_register_device(dev);
+
     vga_common_init((VGAState *)s,
                     ds, vga_ram_base, vga_ram_offset, vga_ram_size);
-    cirrus_init_common(s, CIRRUS_ID_CLGD5430, 0);
+    cirrus_init_common(s, CIRRUS_ID_CLGD5430, 0, dev);
     /* XXX ISA-LFB support */
 }
 
@@ -3386,6 +3392,7 @@ void pci_cirrus_vga_init(PCIBus *bus, Di
     uint8_t *pci_conf;
     CirrusVGAState *s;
     int device_id;
+    QEMUDevice *dev;
 
     device_id = CIRRUS_ID_CLGD5446;
 
@@ -3403,11 +3410,13 @@ void pci_cirrus_vga_init(PCIBus *bus, Di
     pci_conf[0x0b] = PCI_CLASS_BASE_DISPLAY;
     pci_conf[0x0e] = PCI_CLASS_HEADERTYPE_00h;
 
+    dev = &d->dev.qemu_dev;
+
     /* setup VGA */
     s = &d->cirrus_vga;
     vga_common_init((VGAState *)s,
                     ds, vga_ram_base, vga_ram_offset, vga_ram_size);
-    cirrus_init_common(s, device_id, 1);
+    cirrus_init_common(s, device_id, 1, dev);
 
     graphic_console_init(s->ds, s->update, s->invalidate, s->screen_dump,
                          s->text_update, s);
Index: kvm-userspace.io/qemu/hw/dma.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/dma.c
+++ kvm-userspace.io/qemu/hw/dma.c
@@ -55,6 +55,7 @@ struct dma_regs {
 #define COUNT 1
 
 static struct dma_cont {
+    QEMUDevice qemu_dev;
     uint8_t status;
     uint8_t command;
     uint8_t mask;
@@ -448,30 +449,31 @@ static int dma_phony_handler (void *opaq
 static void dma_init2(struct dma_cont *d, int base, int dshift,
                       int page_base, int pageh_base)
 {
+    QEMUDevice *dev = &d->qemu_dev;
     const static int page_port_list[] = { 0x1, 0x2, 0x3, 0x7 };
     int i;
 
     d->dshift = dshift;
     for (i = 0; i < 8; i++) {
-        register_ioport_write (base + (i << dshift), 1, 1, write_chan, d);
-        register_ioport_read (base + (i << dshift), 1, 1, read_chan, d);
+        register_ioport_write (dev, base + (i << dshift), 1, 1, write_chan, d);
+        register_ioport_read (dev, base + (i << dshift), 1, 1, read_chan, d);
     }
     for (i = 0; i < LENOFA (page_port_list); i++) {
-        register_ioport_write (page_base + page_port_list[i], 1, 1,
+        register_ioport_write (dev, page_base + page_port_list[i], 1, 1,
                                write_page, d);
-        register_ioport_read (page_base + page_port_list[i], 1, 1,
+        register_ioport_read (dev, page_base + page_port_list[i], 1, 1,
                               read_page, d);
         if (pageh_base >= 0) {
-            register_ioport_write (pageh_base + page_port_list[i], 1, 1,
+            register_ioport_write (dev, pageh_base + page_port_list[i], 1, 1,
                                    write_pageh, d);
-            register_ioport_read (pageh_base + page_port_list[i], 1, 1,
+            register_ioport_read (dev, pageh_base + page_port_list[i], 1, 1,
                                   read_pageh, d);
         }
     }
     for (i = 0; i < 8; i++) {
-        register_ioport_write (base + ((i + 8) << dshift), 1, 1,
+        register_ioport_write (dev, base + ((i + 8) << dshift), 1, 1,
                                write_cont, d);
-        register_ioport_read (base + ((i + 8) << dshift), 1, 1,
+        register_ioport_read (dev, base + ((i + 8) << dshift), 1, 1,
                               read_cont, d);
     }
     qemu_register_reset(dma_reset, d);
Index: kvm-userspace.io/qemu/hw/es1370.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/es1370.c
+++ kvm-userspace.io/qemu/hw/es1370.c
@@ -917,18 +917,19 @@ static void es1370_map (PCIDevice *pci_d
 {
     PCIES1370State *d = (PCIES1370State *) pci_dev;
     ES1370State *s = &d->es1370;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
     (void) region_num;
     (void) size;
     (void) type;
 
-    register_ioport_write (addr, 0x40 * 4, 1, es1370_writeb, s);
-    register_ioport_write (addr, 0x40 * 2, 2, es1370_writew, s);
-    register_ioport_write (addr, 0x40, 4, es1370_writel, s);
-
-    register_ioport_read (addr, 0x40 * 4, 1, es1370_readb, s);
-    register_ioport_read (addr, 0x40 * 2, 2, es1370_readw, s);
-    register_ioport_read (addr, 0x40, 4, es1370_readl, s);
+    register_ioport_write (dev, addr, 0x40 * 4, 1, es1370_writeb, s);
+    register_ioport_write (dev, addr, 0x40 * 2, 2, es1370_writew, s);
+    register_ioport_write (dev, addr, 0x40, 4, es1370_writel, s);
+
+    register_ioport_read (dev, addr, 0x40 * 4, 1, es1370_readb, s);
+    register_ioport_read (dev, addr, 0x40 * 2, 2, es1370_readw, s);
+    register_ioport_read (dev, addr, 0x40, 4, es1370_readl, s);
 }
 
 static void es1370_save (QEMUFile *f, void *opaque)
Index: kvm-userspace.io/qemu/hw/extboot.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/extboot.c
+++ kvm-userspace.io/qemu/hw/extboot.c
@@ -116,6 +116,12 @@ static void extboot_write_cmd(void *opaq
     }
 }
 
+/*
+ * XXX: extboot should use the disk controller's QEMUDevice, but since this
+ * happens at boot only it should be fine wrt synchronization.
+ */
+QEMUDevice extboot_dev;
+
 void extboot_init(BlockDriverState *bs, int cmd)
 {
     int *pcmd;
@@ -125,8 +131,9 @@ void extboot_init(BlockDriverState *bs, 
 	fprintf(stderr, "Error allocating memory\n");
 	exit(1);
     }
+    qemu_register_device(&extboot_dev);
 
     *pcmd = cmd;
-    register_ioport_read(0x404, 1, 1, extboot_read, pcmd);
-    register_ioport_write(0x405, 1, 2, extboot_write_cmd, bs);
+    register_ioport_read(&extboot_dev, 0x404, 1, 1, extboot_read, pcmd);
+    register_ioport_write(&extboot_dev, 0x405, 1, 2, extboot_write_cmd, bs);
 }
Index: kvm-userspace.io/qemu/hw/fdc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/fdc.c
+++ kvm-userspace.io/qemu/hw/fdc.c
@@ -464,6 +464,7 @@ do { (state) = ((state) & ~FD_STATE_STAT
 #define FD_FORMAT_CMD(state) ((state) & FD_STATE_FORMAT)
 
 struct fdctrl_t {
+    QEMUDevice qemu_dev;
     fdctrl_t *fdctrl;
     /* Controller's identification */
     uint8_t version;
@@ -699,6 +700,7 @@ static fdctrl_t *fdctrl_init_common (qem
                                      BlockDriverState **fds)
 {
     fdctrl_t *fdctrl;
+    QEMUDevice *dev;
     int i;
 
     FLOPPY_DPRINTF("init controller\n");
@@ -712,6 +714,8 @@ static fdctrl_t *fdctrl_init_common (qem
     }
     fdctrl->result_timer = qemu_new_timer(vm_clock,
                                           fdctrl_result_timer, fdctrl);
+    dev = &fdctrl->qemu_dev;
+    qemu_register_device(dev);
 
     fdctrl->version = 0x90; /* Intel 82078 controller */
     fdctrl->irq = irq;
@@ -743,9 +747,11 @@ fdctrl_t *fdctrl_init (qemu_irq irq, int
                        BlockDriverState **fds)
 {
     fdctrl_t *fdctrl;
+    QEMUDevice *dev;
     int io_mem;
 
     fdctrl = fdctrl_init_common(irq, dma_chann, io_base, fds);
+    dev = &fdctrl->qemu_dev;
 
     fdctrl->sun4m = 0;
     if (mem_mapped) {
@@ -753,14 +759,14 @@ fdctrl_t *fdctrl_init (qemu_irq irq, int
                                         fdctrl);
         cpu_register_physical_memory(io_base, 0x08, io_mem);
     } else {
-        register_ioport_read((uint32_t)io_base + 0x01, 5, 1, &fdctrl_read,
+        register_ioport_read(dev, (uint32_t)io_base + 0x01, 5, 1, &fdctrl_read,
                              fdctrl);
-        register_ioport_read((uint32_t)io_base + 0x07, 1, 1, &fdctrl_read,
+        register_ioport_read(dev, (uint32_t)io_base + 0x07, 1, 1, &fdctrl_read,
                              fdctrl);
-        register_ioport_write((uint32_t)io_base + 0x01, 5, 1, &fdctrl_write,
-                              fdctrl);
-        register_ioport_write((uint32_t)io_base + 0x07, 1, 1, &fdctrl_write,
-                              fdctrl);
+        register_ioport_write(dev, (uint32_t)io_base + 0x01, 5, 1,
+                              &fdctrl_write, fdctrl);
+        register_ioport_write(dev, (uint32_t)io_base + 0x07, 1, 1,
+                              &fdctrl_write, fdctrl);
     }
 
     return fdctrl;
Index: kvm-userspace.io/qemu/hw/hypercall.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/hypercall.c
+++ kvm-userspace.io/qemu/hw/hypercall.c
@@ -189,9 +189,12 @@ static void hp_map(PCIDevice *pci_dev, i
 {
     PCIHypercallState *d = (PCIHypercallState *)pci_dev;
     HypercallState *s = &d->hp;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
-    register_ioport_write(addr, HYPERCALL_IOPORT_SIZE, 1, hp_ioport_write, s);
-    register_ioport_read(addr, HYPERCALL_IOPORT_SIZE, 1, hp_ioport_read, s);
+    register_ioport_write(dev, addr, HYPERCALL_IOPORT_SIZE, 1,
+                          hp_ioport_write, s);
+    register_ioport_read(dev, addr, HYPERCALL_IOPORT_SIZE, 1,
+                         hp_ioport_read, s);
 
 }
 
Index: kvm-userspace.io/qemu/hw/i8254.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/i8254.c
+++ kvm-userspace.io/qemu/hw/i8254.c
@@ -463,6 +463,9 @@ PITState *pit_init(int base, qemu_irq ir
 {
     PITState *pit = &pit_state;
     PITChannelState *s;
+    QEMUDevice *dev = &pit->qemu_dev;
+
+    qemu_register_device(dev);
 
     s = &pit->channels[0];
     /* the timer 0 is connected to an IRQ */
@@ -473,8 +476,8 @@ PITState *pit_init(int base, qemu_irq ir
 		    pit_save, pit_load, pit);
 
     qemu_register_reset(pit_reset, pit);
-    register_ioport_write(base, 4, 1, pit_ioport_write, pit);
-    register_ioport_read(base, 3, 1, pit_ioport_read, pit);
+    register_ioport_write(dev, base, 4, 1, pit_ioport_write, pit);
+    register_ioport_read(dev, base, 3, 1, pit_ioport_read, pit);
 
     pit_reset(pit);
 
Index: kvm-userspace.io/qemu/hw/i8254.h
===================================================================
--- kvm-userspace.io.orig/qemu/hw/i8254.h
+++ kvm-userspace.io/qemu/hw/i8254.h
@@ -54,6 +54,7 @@ typedef struct PITChannelState {
 } PITChannelState;
 
 struct PITState {
+    QEMUDevice qemu_dev;
     PITChannelState channels[3];
 };
 
Index: kvm-userspace.io/qemu/hw/i8259.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/i8259.c
+++ kvm-userspace.io/qemu/hw/i8259.c
@@ -35,6 +35,7 @@
 //#define DEBUG_IRQ_COUNT
 
 typedef struct PicState {
+    QEMUDevice qemu_dev;
     uint8_t last_irr; /* edge detection */
     uint8_t irr; /* interrupt request register */
     uint8_t imr; /* interrupt mask register */
@@ -594,11 +595,15 @@ static int pic_load(QEMUFile *f, void *o
 /* XXX: add generic master/slave system */
 static void pic_init1(int io_addr, int elcr_addr, PicState *s)
 {
-    register_ioport_write(io_addr, 2, 1, pic_ioport_write, s);
-    register_ioport_read(io_addr, 2, 1, pic_ioport_read, s);
+    QEMUDevice *dev = &s->qemu_dev;
+
+    qemu_register_device(dev);
+
+    register_ioport_write(dev, io_addr, 2, 1, pic_ioport_write, s);
+    register_ioport_read(dev, io_addr, 2, 1, pic_ioport_read, s);
     if (elcr_addr >= 0) {
-        register_ioport_write(elcr_addr, 1, 1, elcr_ioport_write, s);
-        register_ioport_read(elcr_addr, 1, 1, elcr_ioport_read, s);
+        register_ioport_write(dev, elcr_addr, 1, 1, elcr_ioport_write, s);
+        register_ioport_read(dev, elcr_addr, 1, 1, elcr_ioport_read, s);
     }
     register_savevm("i8259", io_addr, 1, pic_save, pic_load, s);
     qemu_register_reset(pic_reset, s);
Index: kvm-userspace.io/qemu/hw/ide.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/ide.c
+++ kvm-userspace.io/qemu/hw/ide.c
@@ -370,6 +370,7 @@ typedef void EndTransferFunc(struct IDES
 
 /* NOTE: IDEState represents in fact one drive */
 typedef struct IDEState {
+    QEMUDevice qemu_dev;
     /* ide config */
     int is_cdrom;
     int is_cf;
@@ -2499,20 +2500,21 @@ static void ide_init2(IDEState *ide_stat
     }
 }
 
-static void ide_init_ioport(IDEState *ide_state, int iobase, int iobase2)
+static void ide_init_ioport(IDEState *ide_state, int iobase, int iobase2,
+                            QEMUDevice *dev)
 {
-    register_ioport_write(iobase, 8, 1, ide_ioport_write, ide_state);
-    register_ioport_read(iobase, 8, 1, ide_ioport_read, ide_state);
+    register_ioport_write(dev, iobase, 8, 1, ide_ioport_write, ide_state);
+    register_ioport_read(dev, iobase, 8, 1, ide_ioport_read, ide_state);
     if (iobase2) {
-        register_ioport_read(iobase2, 1, 1, ide_status_read, ide_state);
-        register_ioport_write(iobase2, 1, 1, ide_cmd_write, ide_state);
+        register_ioport_read(dev, iobase2, 1, 1, ide_status_read, ide_state);
+        register_ioport_write(dev, iobase2, 1, 1, ide_cmd_write, ide_state);
     }
 
     /* data ports */
-    register_ioport_write(iobase, 2, 2, ide_data_writew, ide_state);
-    register_ioport_read(iobase, 2, 2, ide_data_readw, ide_state);
-    register_ioport_write(iobase, 4, 4, ide_data_writel, ide_state);
-    register_ioport_read(iobase, 4, 4, ide_data_readl, ide_state);
+    register_ioport_write(dev, iobase, 2, 2, ide_data_writew, ide_state);
+    register_ioport_read(dev, iobase, 2, 2, ide_data_readw, ide_state);
+    register_ioport_write(dev, iobase, 4, 4, ide_data_writel, ide_state);
+    register_ioport_read(dev, iobase, 4, 4, ide_data_readl, ide_state);
 }
 
 /* save per IDE drive data */
@@ -2578,13 +2580,17 @@ void isa_ide_init(int iobase, int iobase
                   BlockDriverState *hd0, BlockDriverState *hd1)
 {
     IDEState *ide_state;
+    QEMUDevice *dev;
 
     ide_state = qemu_mallocz(sizeof(IDEState) * 2);
     if (!ide_state)
         return;
 
+    dev = &ide_state->qemu_dev;
+    qemu_register_device(dev);
+
     ide_init2(ide_state, hd0, hd1, irq);
-    ide_init_ioport(ide_state, iobase, iobase2);
+    ide_init_ioport(ide_state, iobase, iobase2, dev);
 }
 
 /***********************************************************/
@@ -2597,21 +2603,24 @@ static void ide_map(PCIDevice *pci_dev, 
 {
     PCIIDEState *d = (PCIIDEState *)pci_dev;
     IDEState *ide_state;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
     if (region_num <= 3) {
         ide_state = &d->ide_if[(region_num >> 1) * 2];
         if (region_num & 1) {
-            register_ioport_read(addr + 2, 1, 1, ide_status_read, ide_state);
-            register_ioport_write(addr + 2, 1, 1, ide_cmd_write, ide_state);
+            register_ioport_read(dev, addr + 2, 1, 1, ide_status_read,
+                                 ide_state);
+            register_ioport_write(dev, addr + 2, 1, 1, ide_cmd_write,
+                                 ide_state);
         } else {
-            register_ioport_write(addr, 8, 1, ide_ioport_write, ide_state);
-            register_ioport_read(addr, 8, 1, ide_ioport_read, ide_state);
+            register_ioport_write(dev, addr, 8, 1, ide_ioport_write, ide_state);
+            register_ioport_read(dev, addr, 8, 1, ide_ioport_read, ide_state);
 
             /* data ports */
-            register_ioport_write(addr, 2, 2, ide_data_writew, ide_state);
-            register_ioport_read(addr, 2, 2, ide_data_readw, ide_state);
-            register_ioport_write(addr, 4, 4, ide_data_writel, ide_state);
-            register_ioport_read(addr, 4, 4, ide_data_readl, ide_state);
+            register_ioport_write(dev, addr, 2, 2, ide_data_writew, ide_state);
+            register_ioport_read(dev, addr, 2, 2, ide_data_readw, ide_state);
+            register_ioport_write(dev, addr, 4, 4, ide_data_writel, ide_state);
+            register_ioport_read(dev, addr, 4, 4, ide_data_readl, ide_state);
         }
     }
 }
@@ -2762,6 +2771,7 @@ static void bmdma_map(PCIDevice *pci_dev
                     uint32_t addr, uint32_t size, int type)
 {
     PCIIDEState *d = (PCIIDEState *)pci_dev;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
     int i;
 
     for(i = 0;i < 2; i++) {
@@ -2770,13 +2780,13 @@ static void bmdma_map(PCIDevice *pci_dev
         d->ide_if[2 * i + 1].bmdma = bm;
         bm->pci_dev = (PCIIDEState *)pci_dev;
 
-        register_ioport_write(addr, 1, 1, bmdma_cmd_writeb, bm);
+        register_ioport_write(dev, addr, 1, 1, bmdma_cmd_writeb, bm);
 
-        register_ioport_write(addr + 1, 3, 1, bmdma_writeb, bm);
-        register_ioport_read(addr, 4, 1, bmdma_readb, bm);
+        register_ioport_write(dev, addr + 1, 3, 1, bmdma_writeb, bm);
+        register_ioport_read(dev, addr, 4, 1, bmdma_readb, bm);
 
-        register_ioport_write(addr + 4, 4, 4, bmdma_addr_writel, bm);
-        register_ioport_read(addr + 4, 4, 4, bmdma_addr_readl, bm);
+        register_ioport_write(dev, addr + 4, 4, 4, bmdma_addr_writel, bm);
+        register_ioport_read(dev, addr + 4, 4, 4, bmdma_addr_readl, bm);
         addr += 8;
     }
 }
@@ -2969,8 +2979,8 @@ void pci_piix3_ide_init(PCIBus *bus, Blo
 
     ide_init2(&d->ide_if[0], hd_table[0], hd_table[1], pic[14]);
     ide_init2(&d->ide_if[2], hd_table[2], hd_table[3], pic[15]);
-    ide_init_ioport(&d->ide_if[0], 0x1f0, 0x3f6);
-    ide_init_ioport(&d->ide_if[2], 0x170, 0x376);
+    ide_init_ioport(&d->ide_if[0], 0x1f0, 0x3f6, &d->dev.qemu_dev);
+    ide_init_ioport(&d->ide_if[2], 0x170, 0x376, &d->dev.qemu_dev);
 
     for (i = 0; i < 4; i++)
         if (hd_table[i])
@@ -3011,8 +3021,8 @@ void pci_piix4_ide_init(PCIBus *bus, Blo
 
     ide_init2(&d->ide_if[0], hd_table[0], hd_table[1], pic[14]);
     ide_init2(&d->ide_if[2], hd_table[2], hd_table[3], pic[15]);
-    ide_init_ioport(&d->ide_if[0], 0x1f0, 0x3f6);
-    ide_init_ioport(&d->ide_if[2], 0x170, 0x376);
+    ide_init_ioport(&d->ide_if[0], 0x1f0, 0x3f6, &d->dev.qemu_dev);
+    ide_init_ioport(&d->ide_if[2], 0x170, 0x376, &d->dev.qemu_dev);
 
     register_savevm("ide", 0, 1, pci_ide_save, pci_ide_load, d);
 }
Index: kvm-userspace.io/qemu/hw/mc146818rtc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/mc146818rtc.c
+++ kvm-userspace.io/qemu/hw/mc146818rtc.c
@@ -55,6 +55,7 @@
 #define REG_B_UIE 0x10
 
 struct RTCState {
+    QEMUDevice qemu_dev;
     uint8_t cmos_data[128];
     uint8_t cmos_index;
     struct tm current_tm;
@@ -457,11 +458,15 @@ static int rtc_load(QEMUFile *f, void *o
 RTCState *rtc_init(int base, qemu_irq irq)
 {
     RTCState *s;
+    QEMUDevice *dev;
 
     s = qemu_mallocz(sizeof(RTCState));
     if (!s)
         return NULL;
 
+    dev = &s->qemu_dev;
+    qemu_register_device(dev);
+
     s->irq = irq;
     s->cmos_data[RTC_REG_A] = 0x26;
     s->cmos_data[RTC_REG_B] = 0x02;
@@ -480,8 +485,8 @@ RTCState *rtc_init(int base, qemu_irq ir
     s->next_second_time = qemu_get_clock(vm_clock) + (ticks_per_sec * 99) / 100;
     qemu_mod_timer(s->second_timer2, s->next_second_time);
 
-    register_ioport_write(base, 2, 1, cmos_ioport_write, s);
-    register_ioport_read(base, 2, 1, cmos_ioport_read, s);
+    register_ioport_write(dev, base, 2, 1, cmos_ioport_write, s);
+    register_ioport_read(dev, base, 2, 1, cmos_ioport_read, s);
 
     register_savevm("mc146818rtc", base, 1, rtc_save, rtc_load, s);
     return s;
Index: kvm-userspace.io/qemu/hw/parallel.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/parallel.c
+++ kvm-userspace.io/qemu/hw/parallel.c
@@ -64,6 +64,7 @@
 #define PARA_CTR_SIGNAL (PARA_CTR_SELECT|PARA_CTR_INIT|PARA_CTR_AUTOLF|PARA_CTR_STROBE)
 
 struct ParallelState {
+    QEMUDevice qemu_dev;
     uint8_t dataw;
     uint8_t datar;
     uint8_t status;
@@ -431,6 +432,7 @@ static void parallel_reset(ParallelState
 ParallelState *parallel_init(int base, qemu_irq irq, CharDriverState *chr)
 {
     ParallelState *s;
+    QEMUDevice *dev;
     uint8_t dummy;
 
     s = qemu_mallocz(sizeof(ParallelState));
@@ -438,24 +440,27 @@ ParallelState *parallel_init(int base, q
         return NULL;
     parallel_reset(s, irq, chr);
 
+    dev = &s->qemu_dev;
+    qemu_register_device(dev);
+
     if (qemu_chr_ioctl(chr, CHR_IOCTL_PP_READ_STATUS, &dummy) == 0) {
         s->hw_driver = 1;
         s->status = dummy;
     }
 
     if (s->hw_driver) {
-        register_ioport_write(base, 8, 1, parallel_ioport_write_hw, s);
-        register_ioport_read(base, 8, 1, parallel_ioport_read_hw, s);
-        register_ioport_write(base+4, 1, 2, parallel_ioport_eppdata_write_hw2, s);
-        register_ioport_read(base+4, 1, 2, parallel_ioport_eppdata_read_hw2, s);
-        register_ioport_write(base+4, 1, 4, parallel_ioport_eppdata_write_hw4, s);
-        register_ioport_read(base+4, 1, 4, parallel_ioport_eppdata_read_hw4, s);
-        register_ioport_write(base+0x400, 8, 1, parallel_ioport_ecp_write, s);
-        register_ioport_read(base+0x400, 8, 1, parallel_ioport_ecp_read, s);
+        register_ioport_write(dev, base, 8, 1, parallel_ioport_write_hw, s);
+        register_ioport_read(dev, base, 8, 1, parallel_ioport_read_hw, s);
+        register_ioport_write(dev, base+4, 1, 2, parallel_ioport_eppdata_write_hw2, s);
+        register_ioport_read(dev, base+4, 1, 2, parallel_ioport_eppdata_read_hw2, s);
+        register_ioport_write(dev, base+4, 1, 4, parallel_ioport_eppdata_write_hw4, s);
+        register_ioport_read(dev, base+4, 1, 4, parallel_ioport_eppdata_read_hw4, s);
+        register_ioport_write(dev, base+0x400, 8, 1, parallel_ioport_ecp_write, s);
+        register_ioport_read(dev, base+0x400, 8, 1, parallel_ioport_ecp_read, s);
     }
     else {
-        register_ioport_write(base, 8, 1, parallel_ioport_write_sw, s);
-        register_ioport_read(base, 8, 1, parallel_ioport_read_sw, s);
+        register_ioport_write(dev, base, 8, 1, parallel_ioport_write_sw, s);
+        register_ioport_read(dev, base, 8, 1, parallel_ioport_read_sw, s);
     }
     return s;
 }
Index: kvm-userspace.io/qemu/hw/pc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pc.c
+++ kvm-userspace.io/qemu/hw/pc.c
@@ -375,18 +375,20 @@ static void bochs_bios_write(void *opaqu
     }
 }
 
+QEMUDevice pc_basic_hw;
+
 static void bochs_bios_init(void)
 {
-    register_ioport_write(0x400, 1, 2, bochs_bios_write, NULL);
-    register_ioport_write(0x401, 1, 2, bochs_bios_write, NULL);
-    register_ioport_write(0x402, 1, 1, bochs_bios_write, NULL);
-    register_ioport_write(0x403, 1, 1, bochs_bios_write, NULL);
-    register_ioport_write(0x8900, 1, 1, bochs_bios_write, NULL);
-
-    register_ioport_write(0x501, 1, 2, bochs_bios_write, NULL);
-    register_ioport_write(0x502, 1, 2, bochs_bios_write, NULL);
-    register_ioport_write(0x500, 1, 1, bochs_bios_write, NULL);
-    register_ioport_write(0x503, 1, 1, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x400, 1, 2, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x401, 1, 2, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x402, 1, 1, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x403, 1, 1, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x8900, 1, 1, bochs_bios_write, NULL);
+
+    register_ioport_write(&pc_basic_hw, 0x501, 1, 2, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x502, 1, 2, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x500, 1, 1, bochs_bios_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x503, 1, 1, bochs_bios_write, NULL);
 }
 
 /* Generate an initial boot sector which sets state and jump to
@@ -929,6 +931,8 @@ static void pc_init1(ram_addr_t ram_size
         }
     }
 
+    qemu_register_device(&pc_basic_hw);
+
     bochs_bios_init();
 
     if (linux_boot)
@@ -946,9 +950,9 @@ static void pc_init1(ram_addr_t ram_size
     }
 
     /* init basic PC hardware */
-    register_ioport_write(0x80, 1, 1, ioport80_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0x80, 1, 1, ioport80_write, NULL);
 
-    register_ioport_write(0xf0, 1, 1, ioportF0_write, NULL);
+    register_ioport_write(&pc_basic_hw, 0xf0, 1, 1, ioportF0_write, NULL);
 
     if (cirrus_vga_enabled) {
         if (pci_enabled) {
@@ -977,8 +981,8 @@ static void pc_init1(ram_addr_t ram_size
 
     rtc_state = rtc_init(0x70, i8259[8]);
 
-    register_ioport_read(0x92, 1, 1, ioport92_read, NULL);
-    register_ioport_write(0x92, 1, 1, ioport92_write, NULL);
+    register_ioport_read(&pc_basic_hw, 0x92, 1, 1, ioport92_read, NULL);
+    register_ioport_write(&pc_basic_hw, 0x92, 1, 1, ioport92_write, NULL);
 
     if (pci_enabled) {
         ioapic = ioapic_init();
Index: kvm-userspace.io/qemu/hw/pckbd.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pckbd.c
+++ kvm-userspace.io/qemu/hw/pckbd.c
@@ -115,6 +115,7 @@
 #define KBD_PENDING_AUX         2
 
 typedef struct KBDState {
+    QEMUDevice qemu_dev;
     uint8_t write_cmd; /* if non zero, write data to port 60 is expected */
     uint8_t status;
     uint8_t mode;
@@ -368,16 +369,19 @@ static int kbd_load(QEMUFile* f, void* o
 void i8042_init(qemu_irq kbd_irq, qemu_irq mouse_irq, uint32_t io_base)
 {
     KBDState *s = &kbd_state;
+    QEMUDevice *dev = &s->qemu_dev;
 
     s->irq_kbd = kbd_irq;
     s->irq_mouse = mouse_irq;
 
+    qemu_register_device(dev);
+
     kbd_reset(s);
     register_savevm("pckbd", 0, 3, kbd_save, kbd_load, s);
-    register_ioport_read(io_base, 1, 1, kbd_read_data, s);
-    register_ioport_write(io_base, 1, 1, kbd_write_data, s);
-    register_ioport_read(io_base + 4, 1, 1, kbd_read_status, s);
-    register_ioport_write(io_base + 4, 1, 1, kbd_write_command, s);
+    register_ioport_read(dev, io_base, 1, 1, kbd_read_data, s);
+    register_ioport_write(dev, io_base, 1, 1, kbd_write_data, s);
+    register_ioport_read(dev, io_base + 4, 1, 1, kbd_read_status, s);
+    register_ioport_write(dev, io_base + 4, 1, 1, kbd_write_command, s);
 
     s->kbd = ps2_kbd_init(kbd_update_kbd_irq, s);
     s->mouse = ps2_mouse_init(kbd_update_aux_irq, s);
Index: kvm-userspace.io/qemu/hw/pcnet.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pcnet.c
+++ kvm-userspace.io/qemu/hw/pcnet.c
@@ -1726,18 +1726,19 @@ static void pcnet_ioport_map(PCIDevice *
                              uint32_t addr, uint32_t size, int type)
 {
     PCNetState *d = (PCNetState *)pci_dev;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
 #ifdef PCNET_DEBUG_IO
     printf("pcnet_ioport_map addr=0x%04x size=0x%04x\n", addr, size);
 #endif
 
-    register_ioport_write(addr, 16, 1, pcnet_aprom_writeb, d);
-    register_ioport_read(addr, 16, 1, pcnet_aprom_readb, d);
+    register_ioport_write(dev, addr, 16, 1, pcnet_aprom_writeb, d);
+    register_ioport_read(dev, addr, 16, 1, pcnet_aprom_readb, d);
 
-    register_ioport_write(addr + 0x10, 0x10, 2, pcnet_ioport_writew, d);
-    register_ioport_read(addr + 0x10, 0x10, 2, pcnet_ioport_readw, d);
-    register_ioport_write(addr + 0x10, 0x10, 4, pcnet_ioport_writel, d);
-    register_ioport_read(addr + 0x10, 0x10, 4, pcnet_ioport_readl, d);
+    register_ioport_write(dev, addr + 0x10, 0x10, 2, pcnet_ioport_writew, d);
+    register_ioport_read(dev, addr + 0x10, 0x10, 2, pcnet_ioport_readw, d);
+    register_ioport_write(dev, addr + 0x10, 0x10, 4, pcnet_ioport_writel, d);
+    register_ioport_read(dev, addr + 0x10, 0x10, 4, pcnet_ioport_readl, d);
 }
 
 static void pcnet_mmio_writeb(void *opaque, target_phys_addr_t addr, uint32_t val)
Index: kvm-userspace.io/qemu/hw/pcspk.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pcspk.c
+++ kvm-userspace.io/qemu/hw/pcspk.c
@@ -144,8 +144,11 @@ static void pcspk_ioport_write(void *opa
 void pcspk_init(PITState *pit)
 {
     PCSpkState *s = &pcspk_state;
+    QEMUDevice *dev = &pcspk_state.card.qemu_dev;
+
+    qemu_register_device(dev);
 
     s->pit = pit;
-    register_ioport_read(0x61, 1, 1, pcspk_ioport_read, s);
-    register_ioport_write(0x61, 1, 1, pcspk_ioport_write, s);
+    register_ioport_read(dev, 0x61, 1, 1, pcspk_ioport_read, s);
+    register_ioport_write(dev, 0x61, 1, 1, pcspk_ioport_write, s);
 }
Index: kvm-userspace.io/qemu/hw/piix_pci.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/piix_pci.c
+++ kvm-userspace.io/qemu/hw/piix_pci.c
@@ -174,20 +174,23 @@ PCIBus *i440fx_init(PCIDevice **pi440fx_
     PCIBus *b;
     PCIDevice *d;
     I440FXState *s;
+    QEMUDevice *dev;
 
     s = qemu_mallocz(sizeof(I440FXState));
     b = pci_register_bus(piix3_set_irq, pci_slot_get_pirq, pic, 0, 4);
     s->bus = b;
 
-    register_ioport_write(0xcf8, 4, 4, i440fx_addr_writel, s);
-    register_ioport_read(0xcf8, 4, 4, i440fx_addr_readl, s);
+    dev = &s->bus->qemu_dev;
 
-    register_ioport_write(0xcfc, 4, 1, pci_host_data_writeb, s);
-    register_ioport_write(0xcfc, 4, 2, pci_host_data_writew, s);
-    register_ioport_write(0xcfc, 4, 4, pci_host_data_writel, s);
-    register_ioport_read(0xcfc, 4, 1, pci_host_data_readb, s);
-    register_ioport_read(0xcfc, 4, 2, pci_host_data_readw, s);
-    register_ioport_read(0xcfc, 4, 4, pci_host_data_readl, s);
+    register_ioport_write(dev, 0xcf8, 4, 4, i440fx_addr_writel, s);
+    register_ioport_read(dev, 0xcf8, 4, 4, i440fx_addr_readl, s);
+
+    register_ioport_write(dev, 0xcfc, 4, 1, pci_host_data_writeb, s);
+    register_ioport_write(dev, 0xcfc, 4, 2, pci_host_data_writew, s);
+    register_ioport_write(dev, 0xcfc, 4, 4, pci_host_data_writel, s);
+    register_ioport_read(dev, 0xcfc, 4, 1, pci_host_data_readb, s);
+    register_ioport_read(dev, 0xcfc, 4, 2, pci_host_data_readw, s);
+    register_ioport_read(dev, 0xcfc, 4, 4, pci_host_data_readl, s);
 
     d = pci_register_device(b, "i440FX", sizeof(PCIDevice), 0,
                             NULL, i440fx_write_config);
Index: kvm-userspace.io/qemu/hw/rtl8139.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/rtl8139.c
+++ kvm-userspace.io/qemu/hw/rtl8139.c
@@ -3324,15 +3324,16 @@ static void rtl8139_ioport_map(PCIDevice
 {
     PCIRTL8139State *d = (PCIRTL8139State *)pci_dev;
     RTL8139State *s = &d->rtl8139;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
-    register_ioport_write(addr, 0x100, 1, rtl8139_ioport_writeb, s);
-    register_ioport_read( addr, 0x100, 1, rtl8139_ioport_readb,  s);
+    register_ioport_write(dev, addr, 0x100, 1, rtl8139_ioport_writeb, s);
+    register_ioport_read( dev, addr, 0x100, 1, rtl8139_ioport_readb,  s);
 
-    register_ioport_write(addr, 0x100, 2, rtl8139_ioport_writew, s);
-    register_ioport_read( addr, 0x100, 2, rtl8139_ioport_readw,  s);
+    register_ioport_write(dev, addr, 0x100, 2, rtl8139_ioport_writew, s);
+    register_ioport_read( dev, addr, 0x100, 2, rtl8139_ioport_readw,  s);
 
-    register_ioport_write(addr, 0x100, 4, rtl8139_ioport_writel, s);
-    register_ioport_read( addr, 0x100, 4, rtl8139_ioport_readl,  s);
+    register_ioport_write(dev, addr, 0x100, 4, rtl8139_ioport_writel, s);
+    register_ioport_read( dev, addr, 0x100, 4, rtl8139_ioport_readl,  s);
 }
 
 static CPUReadMemoryFunc *rtl8139_mmio_read[3] = {
Index: kvm-userspace.io/qemu/hw/sb16.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/sb16.c
+++ kvm-userspace.io/qemu/hw/sb16.c
@@ -1406,6 +1406,7 @@ int SB16_init (AudioState *audio, qemu_i
     int i;
     static const uint8_t dsp_write_ports[] = {0x6, 0xc};
     static const uint8_t dsp_read_ports[] = {0x6, 0xa, 0xc, 0xd, 0xe, 0xf};
+    QEMUDevice *dev;
 
     if (!audio) {
         dolog ("No audio state\n");
@@ -1419,6 +1420,9 @@ int SB16_init (AudioState *audio, qemu_i
         return -1;
     }
 
+    dev = &s->card.qemu_dev;
+    qemu_register_device(dev);
+
     s->cmd = -1;
     s->pic = pic;
     s->irq = conf.irq;
@@ -1441,17 +1445,19 @@ int SB16_init (AudioState *audio, qemu_i
     }
 
     for (i = 0; i < LENOFA (dsp_write_ports); i++) {
-        register_ioport_write (s->port + dsp_write_ports[i], 1, 1, dsp_write, s);
+        register_ioport_write (dev, s->port + dsp_write_ports[i], 1, 1,
+                               dsp_write, s);
     }
 
     for (i = 0; i < LENOFA (dsp_read_ports); i++) {
-        register_ioport_read (s->port + dsp_read_ports[i], 1, 1, dsp_read, s);
+        register_ioport_read (dev, s->port + dsp_read_ports[i], 1, 1,
+                              dsp_read, s);
     }
 
-    register_ioport_write (s->port + 0x4, 1, 1, mixer_write_indexb, s);
-    register_ioport_write (s->port + 0x4, 1, 2, mixer_write_indexw, s);
-    register_ioport_read (s->port + 0x5, 1, 1, mixer_read, s);
-    register_ioport_write (s->port + 0x5, 1, 1, mixer_write_datab, s);
+    register_ioport_write (dev, s->port + 0x4, 1, 1, mixer_write_indexb, s);
+    register_ioport_write (dev, s->port + 0x4, 1, 2, mixer_write_indexw, s);
+    register_ioport_read (dev, s->port + 0x5, 1, 1, mixer_read, s);
+    register_ioport_write (dev, s->port + 0x5, 1, 1, mixer_write_datab, s);
 
     DMA_register_channel (s->hdma, SB_read_DMA, s);
     DMA_register_channel (s->dma, SB_read_DMA, s);
Index: kvm-userspace.io/qemu/hw/serial.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/serial.c
+++ kvm-userspace.io/qemu/hw/serial.c
@@ -74,6 +74,7 @@
 #define UART_LSR_DR	0x01	/* Receiver data ready */
 
 struct SerialState {
+    QEMUDevice qemu_dev;
     uint16_t divider;
     uint8_t rbr; /* receive register */
     uint8_t ier;
@@ -381,19 +382,23 @@ static void serial_reset(void *opaque)
 SerialState *serial_init(int base, qemu_irq irq, CharDriverState *chr)
 {
     SerialState *s;
+    QEMUDevice *dev;
 
     s = qemu_mallocz(sizeof(SerialState));
     if (!s)
         return NULL;
     s->irq = irq;
 
+    dev = &s->qemu_dev;
+    qemu_register_device(dev);
+
     qemu_register_reset(serial_reset, s);
     serial_reset(s);
 
     register_savevm("serial", base, 2, serial_save, serial_load, s);
 
-    register_ioport_write(base, 8, 1, serial_ioport_write, s);
-    register_ioport_read(base, 8, 1, serial_ioport_read, s);
+    register_ioport_write(dev, base, 8, 1, serial_ioport_write, s);
+    register_ioport_read(dev, base, 8, 1, serial_ioport_read, s);
     s->chr = chr;
     qemu_chr_add_handlers(chr, serial_can_receive1, serial_receive1,
                           serial_event, s);
Index: kvm-userspace.io/qemu/hw/usb-uhci.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/usb-uhci.c
+++ kvm-userspace.io/qemu/hw/usb-uhci.c
@@ -873,13 +873,14 @@ static void uhci_map(PCIDevice *pci_dev,
                     uint32_t addr, uint32_t size, int type)
 {
     UHCIState *s = (UHCIState *)pci_dev;
+    QEMUDevice *dev = &pci_dev->qemu_dev;
 
-    register_ioport_write(addr, 32, 2, uhci_ioport_writew, s);
-    register_ioport_read(addr, 32, 2, uhci_ioport_readw, s);
-    register_ioport_write(addr, 32, 4, uhci_ioport_writel, s);
-    register_ioport_read(addr, 32, 4, uhci_ioport_readl, s);
-    register_ioport_write(addr, 32, 1, uhci_ioport_writeb, s);
-    register_ioport_read(addr, 32, 1, uhci_ioport_readb, s);
+    register_ioport_write(dev, addr, 32, 2, uhci_ioport_writew, s);
+    register_ioport_read(dev, addr, 32, 2, uhci_ioport_readw, s);
+    register_ioport_write(dev, addr, 32, 4, uhci_ioport_writel, s);
+    register_ioport_read(dev, addr, 32, 4, uhci_ioport_readl, s);
+    register_ioport_write(dev, addr, 32, 1, uhci_ioport_writeb, s);
+    register_ioport_read(dev, addr, 32, 1, uhci_ioport_readb, s);
 }
 
 void usb_uhci_piix3_init(PCIBus *bus, int devfn)
Index: kvm-userspace.io/qemu/hw/vga.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/vga.c
+++ kvm-userspace.io/qemu/hw/vga.c
@@ -2200,49 +2200,49 @@ void vga_common_init(VGAState *s, Displa
 }
 
 /* used by both ISA and PCI */
-void vga_init(VGAState *s)
+void vga_init(VGAState *s, QEMUDevice *dev)
 {
     int vga_io_memory;
 
     register_savevm("vga", 0, 2, vga_save, vga_load, s);
 
-    register_ioport_write(0x3c0, 16, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3c0, 16, 1, vga_ioport_write, s);
 
-    register_ioport_write(0x3b4, 2, 1, vga_ioport_write, s);
-    register_ioport_write(0x3d4, 2, 1, vga_ioport_write, s);
-    register_ioport_write(0x3ba, 1, 1, vga_ioport_write, s);
-    register_ioport_write(0x3da, 1, 1, vga_ioport_write, s);
-
-    register_ioport_read(0x3c0, 16, 1, vga_ioport_read, s);
-
-    register_ioport_read(0x3b4, 2, 1, vga_ioport_read, s);
-    register_ioport_read(0x3d4, 2, 1, vga_ioport_read, s);
-    register_ioport_read(0x3ba, 1, 1, vga_ioport_read, s);
-    register_ioport_read(0x3da, 1, 1, vga_ioport_read, s);
+    register_ioport_write(dev, 0x3b4, 2, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3d4, 2, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3ba, 1, 1, vga_ioport_write, s);
+    register_ioport_write(dev, 0x3da, 1, 1, vga_ioport_write, s);
+
+    register_ioport_read(dev, 0x3c0, 16, 1, vga_ioport_read, s);
+
+    register_ioport_read(dev, 0x3b4, 2, 1, vga_ioport_read, s);
+    register_ioport_read(dev, 0x3d4, 2, 1, vga_ioport_read, s);
+    register_ioport_read(dev, 0x3ba, 1, 1, vga_ioport_read, s);
+    register_ioport_read(dev, 0x3da, 1, 1, vga_ioport_read, s);
     s->bank_offset = 0;
 
 #ifdef CONFIG_BOCHS_VBE
     s->vbe_regs[VBE_DISPI_INDEX_ID] = VBE_DISPI_ID0;
     s->vbe_bank_mask = ((s->vram_size >> 16) - 1);
 #if defined (TARGET_I386)
-    register_ioport_read(0x1ce, 1, 2, vbe_ioport_read_index, s);
-    register_ioport_read(0x1cf, 1, 2, vbe_ioport_read_data, s);
+    register_ioport_read(dev, 0x1ce, 1, 2, vbe_ioport_read_index, s);
+    register_ioport_read(dev, 0x1cf, 1, 2, vbe_ioport_read_data, s);
 
-    register_ioport_write(0x1ce, 1, 2, vbe_ioport_write_index, s);
-    register_ioport_write(0x1cf, 1, 2, vbe_ioport_write_data, s);
+    register_ioport_write(dev, 0x1ce, 1, 2, vbe_ioport_write_index, s);
+    register_ioport_write(dev, 0x1cf, 1, 2, vbe_ioport_write_data, s);
 
     /* old Bochs IO ports */
-    register_ioport_read(0xff80, 1, 2, vbe_ioport_read_index, s);
-    register_ioport_read(0xff81, 1, 2, vbe_ioport_read_data, s);
+    register_ioport_read(dev, 0xff80, 1, 2, vbe_ioport_read_index, s);
+    register_ioport_read(dev, 0xff81, 1, 2, vbe_ioport_read_data, s);
 
-    register_ioport_write(0xff80, 1, 2, vbe_ioport_write_index, s);
-    register_ioport_write(0xff81, 1, 2, vbe_ioport_write_data, s);
+    register_ioport_write(dev, 0xff80, 1, 2, vbe_ioport_write_index, s);
+    register_ioport_write(dev, 0xff81, 1, 2, vbe_ioport_write_data, s);
 #else
-    register_ioport_read(0x1ce, 1, 2, vbe_ioport_read_index, s);
-    register_ioport_read(0x1d0, 1, 2, vbe_ioport_read_data, s);
+    register_ioport_read(dev, 0x1ce, 1, 2, vbe_ioport_read_index, s);
+    register_ioport_read(dev, 0x1d0, 1, 2, vbe_ioport_read_data, s);
 
-    register_ioport_write(0x1ce, 1, 2, vbe_ioport_write_index, s);
-    register_ioport_write(0x1d0, 1, 2, vbe_ioport_write_data, s);
+    register_ioport_write(dev, 0x1ce, 1, 2, vbe_ioport_write_index, s);
+    register_ioport_write(dev, 0x1d0, 1, 2, vbe_ioport_write_data, s);
 #endif
 #endif /* CONFIG_BOCHS_VBE */
 
@@ -2330,13 +2330,15 @@ int isa_vga_init(DisplayState *ds, uint8
                  unsigned long vga_ram_offset, int vga_ram_size)
 {
     VGAState *s;
+    QEMUDevice *dev = &s->qemu_dev;
 
     s = qemu_mallocz(sizeof(VGAState));
     if (!s)
         return -1;
+    qemu_register_device(dev);
 
     vga_common_init(s, ds, vga_ram_base, vga_ram_offset, vga_ram_size);
-    vga_init(s);
+    vga_init(s, dev);
 
     graphic_console_init(s->ds, s->update, s->invalidate, s->screen_dump,
                          s->text_update, s);
@@ -2380,6 +2382,7 @@ int pci_vga_init(PCIBus *bus, DisplaySta
 {
     PCIVGAState *d;
     VGAState *s;
+    QEMUDevice *dev;
     uint8_t *pci_conf;
 
     d = (PCIVGAState *)pci_register_device(bus, "VGA",
@@ -2388,9 +2391,10 @@ int pci_vga_init(PCIBus *bus, DisplaySta
     if (!d)
         return -1;
     s = &d->vga_state;
+    dev = &d->dev.qemu_dev;
 
     vga_common_init(s, ds, vga_ram_base, vga_ram_offset, vga_ram_size);
-    vga_init(s);
+    vga_init(s, dev);
 
     graphic_console_init(s->ds, s->update, s->invalidate, s->screen_dump,
                          s->text_update, s);
Index: kvm-userspace.io/qemu/hw/vga_int.h
===================================================================
--- kvm-userspace.io.orig/qemu/hw/vga_int.h
+++ kvm-userspace.io/qemu/hw/vga_int.h
@@ -157,6 +157,7 @@
 
 typedef struct VGAState {
     VGA_STATE_COMMON
+    QEMUDevice qemu_dev;
 } VGAState;
 
 static inline int c6_to_8(int v)
@@ -169,7 +170,7 @@ static inline int c6_to_8(int v)
 
 void vga_common_init(VGAState *s, DisplayState *ds, uint8_t *vga_ram_base,
                      unsigned long vga_ram_offset, int vga_ram_size);
-void vga_init(VGAState *s);
+void vga_init(VGAState *s, QEMUDevice *dev);
 uint32_t vga_mem_readb(void *opaque, target_phys_addr_t addr);
 void vga_mem_writeb(void *opaque, target_phys_addr_t addr, uint32_t val);
 void vga_invalidate_scanlines(VGAState *s, int y1, int y2);
Index: kvm-userspace.io/qemu/hw/virtio.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/virtio.c
+++ kvm-userspace.io/qemu/hw/virtio.c
@@ -337,26 +337,27 @@ static void virtio_map(PCIDevice *pci_de
 		       uint32_t addr, uint32_t size, int type)
 {
     VirtIODevice *vdev = to_virtio_device(pci_dev);
+    QEMUDevice *dev = &pci_dev->qemu_dev;
     int i;
 
     vdev->addr = addr;
     for (i = 0; i < 3; i++) {
-	register_ioport_write(addr, 20, 1 << i, virtio_ioport_write, vdev);
-	register_ioport_read(addr, 20, 1 << i, virtio_ioport_read, vdev);
+	register_ioport_write(dev, addr, 20, 1 << i, virtio_ioport_write, vdev);
+	register_ioport_read(dev, addr, 20, 1 << i, virtio_ioport_read, vdev);
     }
 
     if (vdev->config_len) {
-	register_ioport_write(addr + 20, vdev->config_len, 1,
+	register_ioport_write(dev, addr + 20, vdev->config_len, 1,
 			      virtio_config_writeb, vdev);
-	register_ioport_write(addr + 20, vdev->config_len, 2,
+	register_ioport_write(dev, addr + 20, vdev->config_len, 2,
 			      virtio_config_writew, vdev);
-	register_ioport_write(addr + 20, vdev->config_len, 4,
+	register_ioport_write(dev, addr + 20, vdev->config_len, 4,
 			      virtio_config_writel, vdev);
-	register_ioport_read(addr + 20...
 
[truncated message content]