kvm-devel — kernel virtual machine development

Re: [kvm-devel] [PATCH] gfxboot VMX workaround v2

[Guillaume T. <gui...@ex...>]

On Fri, 18 Apr 2008 08:23:07 -0500
Anthony Liguori <an...@co...> wrote:

 
> This doesn't seem right.  You should have been able to break out of the 
> emulator long before encountering an out instruction.  The next 
> instruction you encounter should be a mov instruction.  Are you sure 
> you're updating eip correctly?

I think that eip is updated correctly but you're right, I think that
the condition to stop emulation is not well implemented. I emulate a
lot of mov instructions and I remain blocked in the emulation loop
until I reach the "out" instruction. The loop is the following:

  [...]
  cs_rpl = vmcs_read16(GUEST_CS_SELECTOR) & SELECTOR_RPL_MASK;
  ss_rpl = vmcs_read16(GUEST_SS_SELECTOR) & SELECTOR_RPL_MASK;

  while (cs_rpl != ss_rpl) {
      if (emulate_instruction(vcpu, NULL, 0,0, 0) == EMULATE_FAIL) {
          printk(KERN_INFO "%s: emulation of 0x%x failed\n",
                           __FUNCTION__,
                           vcpu->arch.emulate_ctxt.decode.b);
          return -1;
       }
       cs_rpl = vmcs_read16(GUEST_CS_SELECTOR) & SELECTOR_RPL_MASK;
       ss_rpl = vmcs_read16(GUEST_SS_SELECTOR) & SELECTOR_RPL_MASK;
  }
  printk(KERN_INFO "%s: VMX friendly state recovered\n", __FUNCTION__);
  // I never reach this point

Maybe CS and SS selector are not well updated. I will add trace to see
their values before and after the emulation.


Regards,
Guillaume

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[Nguyen A. Q. <aq...@gm...>]

On 4/18/08, Anthony Liguori <ali...@us...> wrote:
> Nguyen Anh Quynh wrote:
>
> > On Thu, Apr 17, 2008 at 3:00 PM, H. Peter Anvin <hp...@zy...> wrote:
> >
> >
> > >
> > > > +       .globl linux_boot
> > > > +linux_boot:
> > > > +       cli
> > > > +       cld
> > > > +       mov $0x9000, %ax
> > > > +       mov %ax, %ds
> > > > +       mov %ax, %es
> > > > +       mov %ax, %fs
> > > > +       mov %ax, %gs
> > > > +       mov %ax, %ss
> > > > +       mov $0x8ffe, %sp
> > > > +       ljmp $0x9000 + 0x20, $0
> > > >
> > > >
> > > >
> > >  The hard use of segment 9000 is really highly unfortunate for bzImage,
> > > since it restricts its heap more than necessary.  I suggest following
> the
> > > patterns used by the (new) Qemu loader.
> > >
> > >
> >
> > Actually, this code is left from the original code of Anthony, and it
> > seems he took it from qemu 0.8 version.
> >
> > Anthony, may you explain why you want to hijact the linux boot process
> > here? If I understand correctly, we can just let the original int19
> > execute, and if linux boot is desired, it would work in normal way. So
> > why you want to do this?
> >
> >
>
>  The thinking is to eliminate the need to hijack the boot sector when using
> the -kernel option.

I see, but does that offer any advantage over the current approach?

Thanks,
Q

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[H. P. A. <hp...@zy...>]

Anthony Liguori wrote:
> 
> The thinking is to eliminate the need to hijack the boot sector when 
> using the -kernel option.  However, the linux boot stuff in extboot has 
> been broken since hpa rewrote the boot code.  It can be removed for now 
> and I'll eventually revisit it.
> 

It probably makes more sense to have a different boot ROM for that.  I 
thought this was extboot, but apparently not.  I probably can throw 
something together.

	-hpa

Re: [kvm-devel] [PATCH 3/6] KVM: MMU: Add EPT support

[Anthony L. <an...@co...>]

Yang, Sheng wrote:
> @@ -1048,17 +1071,18 @@ static void mmu_set_spte(struct kvm_vcpu *vcpu, u64 
> *shadow_pte,
>  	 * whether the guest actually used the pte (in order to detect
>  	 * demand paging).
>  	 */
> -	spte = PT_PRESENT_MASK | PT_DIRTY_MASK;
> +	spte = shadow_base_present_pte | shadow_dirty_mask;
>  	if (!speculative)
>  		pte_access |= PT_ACCESSED_MASK;
>  	if (!dirty)
>  		pte_access &= ~ACC_WRITE_MASK;
> -	if (!(pte_access & ACC_EXEC_MASK))
> -		spte |= PT64_NX_MASK;
> -
> -	spte |= PT_PRESENT_MASK;
> +	if (pte_access & ACC_EXEC_MASK) {
> +		if (shadow_x_mask)
> +			spte |= shadow_x_mask;
> +	} else if (shadow_nx_mask)
> +		spte |= shadow_nx_mask;
>   

This looks like it may be a bug.  The old behavior sets NX if 
(pte_access & ACC_EXEC_MASK).  The new behavior unconditionally sets NX 
and never sets PRESENT.  Also, the if (shadow_x_mask) checks are 
unnecessary.  spte |= 0 is a nop.

>  	if (pte_access & ACC_USER_MASK)
> -		spte |= PT_USER_MASK;
> +		spte |= shadow_user_mask;
>  	if (largepage)
>  		spte |= PT_PAGE_SIZE_MASK;
>   

Regards,

Anthony Liguori

Re: [kvm-devel] [PATCH] gfxboot VMX workaround v2

[Anthony L. <an...@co...>]

Guillaume Thouvenin wrote:
> On Tue, 15 Apr 2008 16:06:43 +0300
> Avi Kivity <av...@qu...> wrote:
>
>   
>>> ...
>>> handle_vmentry_failure: invalid guest state
>>> handle_vmentry_failure: start emulation
>>> handle_vmentry_failure: emulation failed
>>>   
>>>       
>> What instruction failed, exactly?
>>
>>     
>
> I added the code do dump the instruction and it seems that it's the
> emulation of 0xe6 (== out imm8, al) that failed. I made modifications
> to emulate it (see below) and now I have another problem in kvm
> userspace with the following message (and the emulation doesn't work):
>
> enterprise:~ $ kvm_run: Operation not permitted
> enterprise:~ $ kvm_run returned -1
>  
>   
>> You need to load rip as well.
>>     
>
> Ooops, yes. So jump far emulation is now like:
>
> +       case 0xea: /* jmp far */ {
> +               struct kvm_segment kvm_seg;
> +               long int eip;
> +               int ret;
> +
> +               kvm_x86_ops->get_segment(ctxt->vcpu, &kvm_seg, VCPU_SREG_CS); 
> +
> +               ret = load_segment_descriptor(ctxt->vcpu, kvm_seg.selector, 9, VCPU_SREG_CS);
> +               if (ret < 0){
> +                       printk(KERN_INFO "%s: Failed to load CS descriptor\n", __FUNCTION__);
> +                       goto cannot_emulate;
> +               }
> +
> +               switch (c->op_bytes) {
> +               case 2:
> +                       eip = insn_fetch(s16, 2, c->eip);
> +                       break;
> +               case 4:
> +                       eip = insn_fetch(s32, 4, c->eip);
> +                       break;
> +               default:
> +                       DPRINTF("jmp far: Invalid op_bytes\n");
> +                       goto cannot_emulate;
> +               }
> +               printk(KERN_INFO "eip == 0x%lx\n", eip);
> +               c->eip = eip;
> +               break;
> +       }
>
> It seems that the jump to cs:eip works and now I have the following error:
>
> [18535.446917] handle_vmentry_failure: invalid guest state
> [18535.449519] handle_vmentry_failure: start emulation
> [18535.457519] eip == 0x6e18
> [18535.467685] handle_vmentry_failure: emulation of 0xe6 failed
>
> For the emulation of 0xe6 I used the following one that I found in
> nitin's tree:
>   

This doesn't seem right.  You should have been able to break out of the 
emulator long before encountering an out instruction.  The next 
instruction you encounter should be a mov instruction.  Are you sure 
you're updating eip correctly?

Regards,

Anthony Liguori

> +       case 0xe6: /* out imm8, al */
> +       case 0xe7: /* out imm8, ax/eax */ {
> +               struct kvm_io_device *pio_dev;
> +               
> +               pio_dev = vcpu_find_pio_dev(ctxt->vcpu, c->src.val);
> +               kvm_iodevice_write(pio_dev, c->src.val,
> +                               (c->d & ByteOp) ? 1 : c->op_bytes,
> +                               &c->regs[VCPU_REGS_RAX]);
> +               }
> +               break;
>
> I will look closer where is the problem and as you suggested, I will
> display the instruction to be emulated and the register state before
> and after, and compare with the expected state.
>
>
> Thanks for your help,
> Regards,
> Guillaume
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> kvm-devel mailing list
> kvm...@li...
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>   

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[Anthony L. <ali...@us...>]

H. Peter Anvin wrote:
> Nguyen Anh Quynh wrote:
>>
>> Actually, this code is left from the original code of Anthony, and it
>> seems he took it from qemu 0.8 version.
>>
>> Anthony, may you explain why you want to hijact the linux boot process
>> here? If I understand correctly, we can just let the original int19
>> execute, and if linux boot is desired, it would work in normal way. So
>> why you want to do this?
>>
>
> I'm having exactly the *opposite* question... why does extboot have 
> code to hook int 13h?

extboot is primarily intended to allow scsi boot or boot from a pv 
disk.  It hooks int13h to fake out disk access.

Regards,

Anthony Liguori

>     -hpa

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[Anthony L. <ali...@us...>]

Nguyen Anh Quynh wrote:
> On Thu, Apr 17, 2008 at 3:00 PM, H. Peter Anvin <hp...@zy...> wrote:
>   
>>> +       .globl linux_boot
>>> +linux_boot:
>>> +       cli
>>> +       cld
>>> +       mov $0x9000, %ax
>>> +       mov %ax, %ds
>>> +       mov %ax, %es
>>> +       mov %ax, %fs
>>> +       mov %ax, %gs
>>> +       mov %ax, %ss
>>> +       mov $0x8ffe, %sp
>>> +       ljmp $0x9000 + 0x20, $0
>>>
>>>       
>>  The hard use of segment 9000 is really highly unfortunate for bzImage,
>> since it restricts its heap more than necessary.  I suggest following the
>> patterns used by the (new) Qemu loader.
>>     
>
> Actually, this code is left from the original code of Anthony, and it
> seems he took it from qemu 0.8 version.
>
> Anthony, may you explain why you want to hijact the linux boot process
> here? If I understand correctly, we can just let the original int19
> execute, and if linux boot is desired, it would work in normal way. So
> why you want to do this?
>   

The thinking is to eliminate the need to hijack the boot sector when 
using the -kernel option.  However, the linux boot stuff in extboot has 
been broken since hpa rewrote the boot code.  It can be removed for now 
and I'll eventually revisit it.

Regards,

Anthony Liguori

> Thanks,
> Q
>   

Re: [kvm-devel] [PATCH] gfxboot VMX workaround v2

[Guillaume T. <gui...@ex...>]

On Fri, 18 Apr 2008 14:18:16 +0200
Guillaume Thouvenin <gui...@ex...> wrote:

> I added the code do dump the instruction and it seems that it's the
> emulation of 0xe6 (== out imm8, al) that failed. I made modifications
> to emulate it (see below) and now I have another problem in kvm
> userspace with the following message (and the emulation doesn't work):
> 
> enterprise:~ $ kvm_run: Operation not permitted
> enterprise:~ $ kvm_run returned -1

Ok for this one it seems to be a wrong value in the opcode_table[]. Now
it generates an oops. I'm investigating... 

Regards,
Guillaume

---

Apr 18 14:48:53 enterprise kernel: [22321.010006] handle_vmentry_failure: invalid guest state
Apr 18 14:48:53 enterprise kernel: [22321.011953] handle_vmentry_failure: start emulation
Apr 18 14:48:53 enterprise kernel: [22321.015875] c->op_bytes == 2
Apr 18 14:48:53 enterprise kernel: [22321.019862] eip == 0x6e18

Message from syslogd@enterprise at Fri Apr 18 14:48:54 2008 ...
enterprise kernel: [22321.027850] Oops: 0000 [2] SMP

Message from syslogd@enterprise at Fri Apr 18 14:48:54 2008 ...
enterprise kernel: [22321.027850] Code: 75 58 48 8b 7d 00 e8 64 4f ff ff f6 85 98 00 00 00 01 ba 01 00 00 00 75 04 0f b6 55 4c 48 8b 75 58 48 8d 8d a0 00 00 00 48 89 c7 <ff> 50 08 e9 f1 07 00 00 8a 45 4c 3c 02 74 0a 3c 04 0f 85 73 13

Message from syslogd@enterprise at Fri Apr 18 14:48:54 2008 ...
enterprise kernel: [22321.027850] CR2: 0000000000000008
Apr 18 14:48:54 enterprise kernel: [22321.027850] PGD 36f1a8067 PUD 327c17067 PMD 0
Apr 18 14:48:54 enterprise kernel: [22321.027850] CPU 1
Apr 18 14:48:54 enterprise kernel: [22321.027850] Modules linked in: kvm_intel kvm aic94xx libsas scsi_transport_sas [last unloaded: kvm]
Apr 18 14:48:54 enterprise kernel: [22321.027850] Pid: 7814, comm: qemu-system-x86 Tainted: G      D  2.6.25 #207
Apr 18 14:48:54 enterprise kernel: [22321.027850] RIP: 0010:[<ffffffff88043933>]  [<ffffffff88043933>] :kvm:x86_emulate_insn+0x2d97/0x414c
Apr 18 14:48:54 enterprise kernel: [22321.027850] RSP: 0018:ffff81033005fb68  EFLAGS: 00010202
Apr 18 14:48:54 enterprise kernel: [22321.027850] RAX: 0000000000000000 RBX: ffff810344cf9440 RCX: ffff810344cf9498
Apr 18 14:48:54 enterprise kernel: [22321.027850] RDX: 0000000000000001 RSI: 000000000000007a RDI: 0000000000000000
Apr 18 14:48:54 enterprise kernel: [22321.027850] RBP: ffff810344cf93f8 R08: 0000000000000000 R09: 0000000000000000
Apr 18 14:48:54 enterprise kernel: [22321.027850] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Apr 18 14:48:54 enterprise kernel: [22321.027850] R13: ffffffff88051e50 R14: ffff810344cf9498 R15: 0000000000007ad6
Apr 18 14:48:54 enterprise kernel: [22321.027850] FS:  000000004108b950(0000) GS:ffff810397c250c0(0000) knlGS:0000000000000000
Apr 18 14:48:54 enterprise kernel: [22321.027850] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
Apr 18 14:48:54 enterprise kernel: [22321.027850] CR2: 0000000000000008 CR3: 00000003301b2000 CR4: 00000000000026e0
Apr 18 14:48:54 enterprise kernel: [22321.027850] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 18 14:48:54 enterprise kernel: [22321.027850] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Apr 18 14:48:54 enterprise kernel: [22321.027850] Process qemu-system-x86 (pid: 7814, threadinfo ffff81033005e000, task ffff810396023080)
Apr 18 14:48:54 enterprise kernel: [22321.027850] Stack:  ffff81033005fb04 0000000000000088 ffff810344cf9438 ffff810344cf9440
Apr 18 14:48:54 enterprise kernel: [22321.027850]  0000000000040040 0000000000055e1c 0000000000055e1c ffff810344cf9498
Apr 18 14:48:54 enterprise kernel: [22321.027850]  0000000000800009 ffffffff8805087a 0000000000000000 ffff810344cf80c0
Apr 18 14:48:54 enterprise kernel: [22321.027850] Call Trace:
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff88038d91>] ? :kvm:emulate_instruction+0x1e5/0x2b9
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff88057cd1>] ? :kvm_intel:kvm_handle_exit+0xea/0x1e8
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff88057a96>] ? :kvm_intel:vmx_intr_assist+0x68/0x1b9
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff80563398>] ? __down_read+0x12/0xa1
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff8803b940>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x4ae/0x631
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff80291ec9>] ? touch_atime+0xae/0xed
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff8803672e>] ? :kvm:kvm_vcpu_ioctl+0xf3/0x3a1
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff802802c0>] ? do_sync_read+0xd1/0x118
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff880363b1>] ? :kvm:kvm_vm_ioctl+0x1ab/0x1c3
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff8028ae49>] ? vfs_ioctl+0x21/0x6b
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff8028b0e6>] ? do_vfs_ioctl+0x253/0x264
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff80280aa0>] ? vfs_read+0x11e/0x132
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff8028b133>] ? sys_ioctl+0x3c/0x5d
Apr 18 14:48:54 enterprise kernel: [22321.027850]  [<ffffffff8020b08a>] ? system_call_after_swapgs+0x8a/0x8f
Apr 18 14:48:54 enterprise kernel: [22321.027850]
Apr 18 14:48:54 enterprise kernel: [22321.027850]
Apr 18 14:48:54 enterprise kernel: [22321.027850]  RSP <ffff81033005fb68>
Apr 18 14:48:54 enterprise kernel: [22321.028054] ---[ end trace 153cd0388d4ba429 ]---
q

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[H. P. A. <hp...@zy...>]

Nguyen Anh Quynh wrote:
> 
> Actually, this code is left from the original code of Anthony, and it
> seems he took it from qemu 0.8 version.
> 
> Anthony, may you explain why you want to hijact the linux boot process
> here? If I understand correctly, we can just let the original int19
> execute, and if linux boot is desired, it would work in normal way. So
> why you want to do this?
> 

I'm having exactly the *opposite* question... why does extboot have code 
to hook int 13h?

	-hpa

Re: [kvm-devel] [Qemu-devel] Re: [PATCH 1/3] Refactor AIO interface to allow other AIO implementations

[Jamie L. <ja...@sh...>]

Daniel P. Berrange wrote:
> > Those cases aren't always discoverable.  Linux-aio just falls back to 
> > using synchronous IO.  It's pretty terrible.  We need a new AIO 
> > interface for Linux (and yes, we're working on this).  Once we have 
> > something better, we'll change that to be the default and things will 
> > Just Work for most users.
> 
> If QEMU can't discover cases where it won't work, what criteria should
> the end user use to decide between the impls, or for that matter, what
> criteria should a management api/app like libvirt use ? If the only decision
> logic is  'try it & benchmark your VM' then its not a particularly useful
> option.

Good use of Linux-AIO requires that you basically "know" which cases
it handles well, and which ones it doesn't.  Falling back to
synchronous I/O with no indication (except speed) is a pretty
atrocious API imho.  But that's what the Linux folks decided to do.

I suspect what you have to do is:

    1. Try opening the file with O_DIRECT.
    2. Use fstat to check the filesystem type and block device type.
    3. If it's on a whitelist of filesystem types,
    4. and a whitelist of block device types,
    5. and the kernel version is later than an fs+bd-dependent value,
    6. then select an alignment size (kernel version dependent)
       and use Linux-AIO with it.

Otherwise don't use Linux-AIO.  You may then decide to use Glibc's
POSIX-AIO (which uses threads), or use threads for I/O yourself.

In future, the above recipe will be more complicated, in that you have
to use the same decision tree to decide between:

    - Synchronous IO.
    - Your own thread based IO.
    - Glibc POSIX-AIO using threads.
    - Linux-AIO.
    - Virtio thing or whatever is based around vringfd.
    - Syslets if they gain traction and perform well.

> I've basically got a choice of making libvirt always ad '-aio linux'
> or never add it at all. My inclination is to the latter since it is
> compatible with existing QEMU which has no -aio option. Presumably
> '-aio linux' is intended to provide some performance benefit so it'd
> be nice to use it. If we can't express some criteria under which it
> should be turned on, I can't enable it; where as if you can express
> some criteria, then QEMU should apply them automatically.

I'm of the view that '-aio auto' would be a really good option - and
when it's proven itself, it should be the default.  It could work on
all QEMU hosts: it would pick synchronous IO when there is nothing else.

The criteria for selecting a good AIO strategy on Linux are quite
complex, and might be worth hard coding.  In that case, putting that
into QEMU itself would be much better than every program which
launches QEMU having it's own implementation of the criteria.

> Pushing this choice of AIO impls to the app or user invoking QEMU just
> does not seem like a win here.

I think having the choice is very good, because whatever the hard
coded selection criteria, there will be times when it's wrong (ideally
in conservative ways - it should always be functional, just suboptimal).

So I do support this patch to add the switch.

But _forcing_ the user to decide is not good, since the criteria are
rather obscure and change with things like filesystem.  At least, a
set of command line options to QEMU ought to work when you copy a VM
to another machine!

So I think '-aio auto', which invokes the selection criteria of the
day and is guaranteed to work (conservatively picking a slower method
if it cannot be sure a faster one will work) would be the most useful
option of all.

-- Jamie

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[H. P. A. <hp...@zy...>]

Nguyen Anh Quynh wrote:
> 
> Actually, this code is left from the original code of Anthony, and it
> seems he took it from qemu 0.8 version.
> 
> Anthony, may you explain why you want to hijact the linux boot process
> here? If I understand correctly, we can just let the original int19
> execute, and if linux boot is desired, it would work in normal way. So
> why you want to do this?
> 

I'm having exactly the *opposite* question... why does extboot have code 
to hook int 13h?

	-hpa

Re: [kvm-devel] kvm-65/66 bug with Solaris 10 U4 ?

[Chris L. <cla...@re...>]

Ian Kirk wrote:
> 
> I can't compile them against 2.6.24.4-64.fc8PAE as the module magic name
> mismatches, and I don't know how to change kernel-devel to know it's PAE.

You just need to install kernel-PAE-devel package, and then build against that.

Chris Lalancette

Re: [kvm-devel] [PATCH] gfxboot VMX workaround v2

[Guillaume T. <gui...@ex...>]

On Tue, 15 Apr 2008 16:06:43 +0300
Avi Kivity <av...@qu...> wrote:

> > ...
> > handle_vmentry_failure: invalid guest state
> > handle_vmentry_failure: start emulation
> > handle_vmentry_failure: emulation failed
> >   
> 
> What instruction failed, exactly?
> 

I added the code do dump the instruction and it seems that it's the
emulation of 0xe6 (== out imm8, al) that failed. I made modifications
to emulate it (see below) and now I have another problem in kvm
userspace with the following message (and the emulation doesn't work):

enterprise:~ $ kvm_run: Operation not permitted
enterprise:~ $ kvm_run returned -1
 
> You need to load rip as well.

Ooops, yes. So jump far emulation is now like:

+       case 0xea: /* jmp far */ {
+               struct kvm_segment kvm_seg;
+               long int eip;
+               int ret;
+
+               kvm_x86_ops->get_segment(ctxt->vcpu, &kvm_seg, VCPU_SREG_CS); 
+
+               ret = load_segment_descriptor(ctxt->vcpu, kvm_seg.selector, 9, VCPU_SREG_CS);
+               if (ret < 0){
+                       printk(KERN_INFO "%s: Failed to load CS descriptor\n", __FUNCTION__);
+                       goto cannot_emulate;
+               }
+
+               switch (c->op_bytes) {
+               case 2:
+                       eip = insn_fetch(s16, 2, c->eip);
+                       break;
+               case 4:
+                       eip = insn_fetch(s32, 4, c->eip);
+                       break;
+               default:
+                       DPRINTF("jmp far: Invalid op_bytes\n");
+                       goto cannot_emulate;
+               }
+               printk(KERN_INFO "eip == 0x%lx\n", eip);
+               c->eip = eip;
+               break;
+       }

It seems that the jump to cs:eip works and now I have the following error:

[18535.446917] handle_vmentry_failure: invalid guest state
[18535.449519] handle_vmentry_failure: start emulation
[18535.457519] eip == 0x6e18
[18535.467685] handle_vmentry_failure: emulation of 0xe6 failed

For the emulation of 0xe6 I used the following one that I found in
nitin's tree:

+       case 0xe6: /* out imm8, al */
+       case 0xe7: /* out imm8, ax/eax */ {
+               struct kvm_io_device *pio_dev;
+               
+               pio_dev = vcpu_find_pio_dev(ctxt->vcpu, c->src.val);
+               kvm_iodevice_write(pio_dev, c->src.val,
+                               (c->d & ByteOp) ? 1 : c->op_bytes,
+                               &c->regs[VCPU_REGS_RAX]);
+               }
+               break;

I will look closer where is the problem and as you suggested, I will
display the instruction to be emulated and the register state before
and after, and compare with the expected state.


Thanks for your help,
Regards,
Guillaume

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[Nguyen A. Q. <aq...@gm...>]

On Thu, Apr 17, 2008 at 4:36 PM, Carlo Marcelo Arenas Belon
<ca...@sa...> wrote:
> On Thu, Apr 17, 2008 at 10:30:27AM +0900, Nguyen Anh Quynh wrote:
>
>  +++ b/extboot/farvar.h
>  @@ -0,0 +1,113 @@
>  +// Code to access multiple segments within gcc.
>  +//
>  +// Copyright (C) 2008  Kevin O'Connor <ke...@ko...>
>  +//
>  +// This file may be distributed under the terms of the GNU GPLv3 license.
>
>  IANAL but wouldn't this make extboot GPLv3 only? how that will interact
>  with the GPLv2 extboot qemu?

I am not sure if that is fine, but it might be better to have the same
license for every code. I will contact Kevin when the next version is
ready (I am still fixing something)

Thanks,
Q

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[Nguyen A. Q. <aq...@gm...>]

On Thu, Apr 17, 2008 at 3:00 PM, H. Peter Anvin <hp...@zy...> wrote:
> > +       .globl linux_boot
> > +linux_boot:
> > +       cli
> > +       cld
> > +       mov $0x9000, %ax
> > +       mov %ax, %ds
> > +       mov %ax, %es
> > +       mov %ax, %fs
> > +       mov %ax, %gs
> > +       mov %ax, %ss
> > +       mov $0x8ffe, %sp
> > +       ljmp $0x9000 + 0x20, $0
> >
>
>  The hard use of segment 9000 is really highly unfortunate for bzImage,
> since it restricts its heap more than necessary.  I suggest following the
> patterns used by the (new) Qemu loader.

Actually, this code is left from the original code of Anthony, and it
seems he took it from qemu 0.8 version.

Anthony, may you explain why you want to hijact the linux boot process
here? If I understand correctly, we can just let the original int19
execute, and if linux boot is desired, it would work in normal way. So
why you want to do this?

Thanks,
Q

Re: [kvm-devel] Extboot Option ROM rewritten in C - v3

[Nguyen A. Q. <aq...@gm...>]

On Thu, Apr 17, 2008 at 2:58 PM, H. Peter Anvin <hp...@zy...> wrote:
> Nguyen Anh Quynh wrote:
>
> > This patch replaces the current assembly code of Extboot option rom
> > with new C code. Patch is against kvm-66.
> >
> > This version returns an error code in case int 13 handler cannot
> > handle a requested function.
> >
> > Signed-off-by: Nguyen Anh Quynh <aq...@gm...>
> >
> >
>
>  +       /* -fomit-frame-pointer might clobber %ebp */
>  +       pushl %ebp
>  +       call setup
>  +       popl  %ebp
>
>
>  No, it might not.  %ebx, %ebp, %esi, and %edi are guaranteed preserved;
> %eax, %ecx and %edx are clobbered.
>
>  It's also prudent to call cld before jumping to C code.

OK, I will fix these in the next version.

Thanks,
Q

Re: [kvm-devel] [PATCH 1/1] QEMU/KVM: Support for PCI Passthrough

[Samuel M. <sam...@gm...>]

On Fri, Apr 18, 2008 at 2:39 PM, Amit Shah <ami...@qu...> wrote:
> * On Monday 14 Apr 2008 06:01:07 Samuel Masham wrote:
>  >
>  > Please keep the userspace support alive.
>  >
>  > I am particularly interested in using the pci-passthough to qemu
>  > running non x86 system emulation
>  > (at the moment mips)
>  >
>  > My hope is that the pci - passthough could help with developing
>  > drivers and testing across architectures...
>
>  OK; keeping support around won't be too much of a hassle, though the current
>  support for pci-passthrough and the irqhook module are developed with x86 in
>  mind (and only tested on x86).
>
>  Since it's not tested on any other architecture, I've marked it TARGET_I386
>  and TARGET_X86_64 for now. Feel free to extend it to other architectures.

Thanks,

I will let you know if I get anything useful out of it.

Samuel

[kvm-devel] Чем НЕЛЬЗЯ пренебрегать?

[Корпоративный з. <al...@hr...>]

  Приглашатся!

  Руководители и сотрудники служб безопасности, юрисконсульты,
  руководители предприятий, финансовые и коммерческие директора

  принять участие в мероприятии:

                     ...Э к о н о м и ч е с к а я
                      безопасность  предприятия...

  12 - 16 мая 2008г. в Санкт-Петербурге

  Информационный отдел: (812)... 983... 37... 96

  Основные блоки программы:...

  Основы экономической безопасности предприятия.
  Общие положения теории экономической безопасности.
  Основные направления обеспечения безопасности предприятия.
  Внешние и внутренние угрозы.

  Определение эффективности работы службы экономической безопасности.
  Функции, задачи и направления деятельности СБ.
  Планирование работы.

  Система и методы анализа и управления экономическими рисками.
  Корпоративные захваты: инструменты обнаружения и противодействия
  корпоративным захватам. Правовые и экономические аспекты процесса
  недружественного поглощения. Сценарии проведения.

  Место деловой разведки в обеспечении экономической безопасности
  бизнеса. Понятие деловой разведки. Роль бизнесразведки в принятии
  управленческого решения. Бизнесразведка и безопасность бизнеса.

  Внезапная проверка. Процедура проведения проверок, виды проверок
  и основания проведения.

  Интегрированная система охраны объектов.

  Основы информационной безопасности предприятия.

  Правовые основы защиты конфиденциальной информации.

  Мероприятия по защите конфиденциальной информации.

  Меры по обеспечению информационной безопасности предприятия,
  связанные с кадровой работой.

  Технические средства промышленного шпионажа и средства
  их обнаружения.

  Защита компьютерной информации.

  Практическая демонстрация возможностей средств контроля и
  управления доступом, средств противодействия промышленному шпионажу.

  По запросу высылается полная программа (812)... 983... 37... 96

[kvm-devel] [PATCH 1/6] KVM: VMX: EPT Feature Detection

[Yang, S. <she...@in...>]

From 9e723871299268e844c9e72f3903ba5f4eb71751 Mon Sep 17 00:00:00 2001
From: Sheng Yang <she...@in...>
Date: Fri, 18 Apr 2008 17:02:59 +0800
Subject: [PATCH 1/5] KVM: VMX: EPT Feature Detection


Signed-off-by: Sheng Yang <she...@in...>
---
 arch/x86/kvm/vmx.c |   63 +++++++++++++++++++++++++++++++++++++++++++++++----
 arch/x86/kvm/vmx.h |   25 ++++++++++++++++++++
 2 files changed, 83 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 8e5d664..d93250d 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -42,6 +42,9 @@ module_param(enable_vpid, bool, 0);
 static int flexpriority_enabled = 1;
 module_param(flexpriority_enabled, bool, 0);

+static int enable_ept;
+module_param(enable_ept, bool, 0);
+
 struct vmcs {
 	u32 revision_id;
 	u32 abort;
@@ -107,6 +110,11 @@ static struct vmcs_config {
 	u32 vmentry_ctrl;
 } vmcs_config;

+struct vmx_capability {
+	u32 ept;
+	u32 vpid;
+} vmx_capability;
+
 #define VMX_SEGMENT_FIELD(seg)					\
 	[VCPU_SREG_##seg] = {                                   \
 		.selector = GUEST_##seg##_SELECTOR,		\
@@ -214,6 +222,32 @@ static inline bool 
cpu_has_vmx_virtualize_apic_accesses(void)
 		    SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES);
 }

+static inline int cpu_has_vmx_invept_individual_addr(void)
+{
+	return (!!(vmx_capability.ept & VMX_EPT_EXTENT_INDIVIDUAL_BIT));
+}
+
+static inline int cpu_has_vmx_invept_context(void)
+{
+	return (!!(vmx_capability.ept & VMX_EPT_EXTENT_CONTEXT_BIT));
+}
+
+static inline int cpu_has_vmx_invept_global(void)
+{
+	return (!!(vmx_capability.ept & VMX_EPT_EXTENT_GLOBAL_BIT));
+}
+
+static inline int cpu_has_vmx_ept(void)
+{
+	return (vmcs_config.cpu_based_2nd_exec_ctrl &
+		SECONDARY_EXEC_ENABLE_EPT);
+}
+
+static inline int vm_need_ept(void)
+{
+	return (cpu_has_vmx_ept() && enable_ept);
+}
+
 static inline int vm_need_virtualize_apic_accesses(struct kvm *kvm)
 {
 	return ((cpu_has_vmx_virtualize_apic_accesses()) &&
@@ -985,7 +1019,7 @@ static __init int adjust_vmx_controls(u32 ctl_min, u32 
ctl_opt,
 static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf)
 {
 	u32 vmx_msr_low, vmx_msr_high;
-	u32 min, opt;
+	u32 min, opt, min2, opt2;
 	u32 _pin_based_exec_control = 0;
 	u32 _cpu_based_exec_control = 0;
 	u32 _cpu_based_2nd_exec_control = 0;
@@ -1003,6 +1037,8 @@ static __init int setup_vmcs_config(struct vmcs_config 
*vmcs_conf)
 	      CPU_BASED_CR8_LOAD_EXITING |
 	      CPU_BASED_CR8_STORE_EXITING |
 #endif
+	      CPU_BASED_CR3_LOAD_EXITING |
+	      CPU_BASED_CR3_STORE_EXITING |
 	      CPU_BASED_USE_IO_BITMAPS |
 	      CPU_BASED_MOV_DR_EXITING |
 	      CPU_BASED_USE_TSC_OFFSETING;
@@ -1018,11 +1054,13 @@ static __init int setup_vmcs_config(struct vmcs_config 
*vmcs_conf)
 					   ~CPU_BASED_CR8_STORE_EXITING;
 #endif
 	if (_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS) {
-		min = 0;
-		opt = SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
+		min2 = 0;
+		opt2 = SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
 			SECONDARY_EXEC_WBINVD_EXITING |
-			SECONDARY_EXEC_ENABLE_VPID;
-		if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_PROCBASED_CTLS2,
+			SECONDARY_EXEC_ENABLE_VPID |
+			SECONDARY_EXEC_ENABLE_EPT;
+		if (adjust_vmx_controls(min2, opt2,
+					MSR_IA32_VMX_PROCBASED_CTLS2,
 					&_cpu_based_2nd_exec_control) < 0)
 			return -EIO;
 	}
@@ -1031,6 +1069,16 @@ static __init int setup_vmcs_config(struct vmcs_config 
*vmcs_conf)
 				SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES))
 		_cpu_based_exec_control &= ~CPU_BASED_TPR_SHADOW;
 #endif
+	if (_cpu_based_2nd_exec_control & SECONDARY_EXEC_ENABLE_EPT) {
+		/* CR3 accesses don't need to cause VM Exits when EPT enabled */
+		min &= ~(CPU_BASED_CR3_LOAD_EXITING |
+			 CPU_BASED_CR3_STORE_EXITING);
+		if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_PROCBASED_CTLS,
+					&_cpu_based_exec_control) < 0)
+			return -EIO;
+		rdmsr(MSR_IA32_VMX_EPT_VPID_CAP,
+		      vmx_capability.ept, vmx_capability.vpid);
+	}

 	min = 0;
 #ifdef CONFIG_X86_64
@@ -1638,6 +1686,9 @@ static int vmx_vcpu_setup(struct vcpu_vmx *vmx)
 				CPU_BASED_CR8_LOAD_EXITING;
 #endif
 	}
+	if (!vm_need_ept())
+		exec_control |= CPU_BASED_CR3_STORE_EXITING |
+				CPU_BASED_CR3_LOAD_EXITING;
 	vmcs_write32(CPU_BASED_VM_EXEC_CONTROL, exec_control);

 	if (cpu_has_secondary_exec_ctrls()) {
@@ -1647,6 +1698,8 @@ static int vmx_vcpu_setup(struct vcpu_vmx *vmx)
 				~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES;
 		if (vmx->vpid == 0)
 			exec_control &= ~SECONDARY_EXEC_ENABLE_VPID;
+		if (!vm_need_ept())
+			exec_control &= ~SECONDARY_EXEC_ENABLE_EPT;
 		vmcs_write32(SECONDARY_VM_EXEC_CONTROL, exec_control);
 	}

diff --git a/arch/x86/kvm/vmx.h b/arch/x86/kvm/vmx.h
index 5dff460..5894f26 100644
--- a/arch/x86/kvm/vmx.h
+++ b/arch/x86/kvm/vmx.h
@@ -35,6 +35,8 @@
 #define CPU_BASED_MWAIT_EXITING                 0x00000400
 #define CPU_BASED_RDPMC_EXITING                 0x00000800
 #define CPU_BASED_RDTSC_EXITING                 0x00001000
+#define CPU_BASED_CR3_LOAD_EXITING		0x00008000
+#define CPU_BASED_CR3_STORE_EXITING		0x00010000
 #define CPU_BASED_CR8_LOAD_EXITING              0x00080000
 #define CPU_BASED_CR8_STORE_EXITING             0x00100000
 #define CPU_BASED_TPR_SHADOW                    0x00200000
@@ -49,6 +51,7 @@
  * Definitions of Secondary Processor-Based VM-Execution Controls.
  */
 #define SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES 0x00000001
+#define SECONDARY_EXEC_ENABLE_EPT               0x00000002
 #define SECONDARY_EXEC_ENABLE_VPID              0x00000020
 #define SECONDARY_EXEC_WBINVD_EXITING		0x00000040

@@ -100,10 +103,22 @@ enum vmcs_field {
 	VIRTUAL_APIC_PAGE_ADDR_HIGH     = 0x00002013,
 	APIC_ACCESS_ADDR		= 0x00002014,
 	APIC_ACCESS_ADDR_HIGH		= 0x00002015,
+	EPT_POINTER                     = 0x0000201a,
+	EPT_POINTER_HIGH                = 0x0000201b,
+	GUEST_PHYSICAL_ADDRESS          = 0x00002400,
+	GUEST_PHYSICAL_ADDRESS_HIGH     = 0x00002401,
 	VMCS_LINK_POINTER               = 0x00002800,
 	VMCS_LINK_POINTER_HIGH          = 0x00002801,
 	GUEST_IA32_DEBUGCTL             = 0x00002802,
 	GUEST_IA32_DEBUGCTL_HIGH        = 0x00002803,
+	GUEST_PDPTR0                    = 0x0000280a,
+	GUEST_PDPTR0_HIGH               = 0x0000280b,
+	GUEST_PDPTR1                    = 0x0000280c,
+	GUEST_PDPTR1_HIGH               = 0x0000280d,
+	GUEST_PDPTR2                    = 0x0000280e,
+	GUEST_PDPTR2_HIGH               = 0x0000280f,
+	GUEST_PDPTR3                    = 0x00002810,
+	GUEST_PDPTR3_HIGH               = 0x00002811,
 	PIN_BASED_VM_EXEC_CONTROL       = 0x00004000,
 	CPU_BASED_VM_EXEC_CONTROL       = 0x00004002,
 	EXCEPTION_BITMAP                = 0x00004004,
@@ -226,6 +241,8 @@ enum vmcs_field {
 #define EXIT_REASON_MWAIT_INSTRUCTION   36
 #define EXIT_REASON_TPR_BELOW_THRESHOLD 43
 #define EXIT_REASON_APIC_ACCESS         44
+#define EXIT_REASON_EPT_VIOLATION       48
+#define EXIT_REASON_EPT_MISCONFIG       49
 #define EXIT_REASON_WBINVD		54

 /*
@@ -316,6 +333,7 @@ enum vmcs_field {
 #define MSR_IA32_VMX_CR4_FIXED1                 0x489
 #define MSR_IA32_VMX_VMCS_ENUM                  0x48a
 #define MSR_IA32_VMX_PROCBASED_CTLS2            0x48b
+#define MSR_IA32_VMX_EPT_VPID_CAP               0x48c

 #define MSR_IA32_FEATURE_CONTROL                0x3a
 #define MSR_IA32_FEATURE_CONTROL_LOCKED         0x1
@@ -327,4 +345,11 @@ enum vmcs_field {
 #define VMX_VPID_EXTENT_SINGLE_CONTEXT		1
 #define VMX_VPID_EXTENT_ALL_CONTEXT		2

+#define VMX_EPT_EXTENT_INDIVIDUAL_ADDR		0
+#define VMX_EPT_EXTENT_CONTEXT			1
+#define VMX_EPT_EXTENT_GLOBAL			2
+#define VMX_EPT_EXTENT_INDIVIDUAL_BIT		(1ul << 24)
+#define VMX_EPT_EXTENT_CONTEXT_BIT		(1ul << 25)
+#define VMX_EPT_EXTENT_GLOBAL_BIT		(1ul << 26)
+
 #endif
--
1.5.4.5

[kvm-devel] [PATCH 6/6] kvm: qemu: Enable EPT support for real mode

[Yang, S. <she...@in...>]

From 73c33765f3d879001818cd0719038c78a0c65561 Mon Sep 17 00:00:00 2001
From: Sheng Yang <she...@in...>
Date: Fri, 18 Apr 2008 17:15:39 +0800
Subject: [PATCH] kvm: qemu: Enable EPT support for real mode

This patch build a identity page table on the last page of VGA bios, and use 
it as the guest page table in nonpaging mode for EPT.

Signed-off-by: Sheng Yang <she...@in...>
---
 qemu/hw/pc.c |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/qemu/hw/pc.c b/qemu/hw/pc.c
index ae87ab9..dcb98c6 100644
--- a/qemu/hw/pc.c
+++ b/qemu/hw/pc.c
@@ -780,6 +780,9 @@ static void pc_init1(ram_addr_t ram_size, int 
vga_ram_size,
     int index;
     BlockDriverState *hd[MAX_IDE_BUS * MAX_IDE_DEVS];
     BlockDriverState *fd[MAX_FD];
+#ifdef USE_KVM
+    uint32_t *table_items;
+#endif

     if (ram_size >= 0xe0000000 ) {
         above_4g_mem_size = ram_size - 0xe0000000;
@@ -857,6 +860,17 @@ static void pc_init1(ram_addr_t ram_size, int 
vga_ram_size,
         exit(1);
     }

+#ifdef USE_KVM
+    if (kvm_allowed) {
+	/* set up identity map for EPT at the last page of VGA BIOS region.
+	 * 0xe7 = _PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED |
+	 *	  _PAGE_DIRTY   | _PAGE_PSE */
+	table_items = (void *)(phys_ram_base + vga_bios_offset + 0xf000);
+	for (i = 0; i < 1024; i++)
+	    table_items[i] = (i << 22) + 0xe7;
+    }
+#endif
+
     /* above 4giga memory allocation */
     if (above_4g_mem_size > 0) {
         ram_addr = qemu_ram_alloc(above_4g_mem_size);
--
1.5.4.5

[kvm-devel] [PATCH 5/6] KVM: VMX: Enable EPT feature for KVM

[Yang, S. <she...@in...>]

From 43eb727046349aac3df52317dbbfd3b4b33c084d Mon Sep 17 00:00:00 2001
From: Sheng Yang <she...@in...>
Date: Fri, 18 Apr 2008 17:07:31 +0800
Subject: [PATCH 5/5] KVM: VMX: Enable EPT feature for KVM


Signed-off-by: Sheng Yang <she...@in...>
---
 arch/x86/kvm/mmu.c         |   11 ++-
 arch/x86/kvm/vmx.c         |  227 
++++++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/vmx.h         |   11 ++
 include/asm-x86/kvm_host.h |    1 +
 4 files changed, 241 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 1828837..c7b7335 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -1187,8 +1187,15 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t v, 
int write,
 				return -ENOMEM;
 			}

-			table[index] = __pa(new_table->spt) | PT_PRESENT_MASK
-				| PT_WRITABLE_MASK | shadow_user_mask;
+			if (shadow_user_mask)
+				table[index] = __pa(new_table->spt)
+					| PT_PRESENT_MASK | PT_WRITABLE_MASK
+					| shadow_user_mask;
+			else
+				table[index] = __pa(new_table->spt)
+					| PT_PRESENT_MASK | PT_WRITABLE_MASK
+					| shadow_x_mask;
+			table[index] = __pa(new_table->spt) | 0x7;
 		}
 		table_addr = table[index] & PT64_BASE_ADDR_MASK;
 	}
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index d93250d..2a85930 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -42,7 +42,7 @@ module_param(enable_vpid, bool, 0);
 static int flexpriority_enabled = 1;
 module_param(flexpriority_enabled, bool, 0);

-static int enable_ept;
+static int enable_ept = 1;
 module_param(enable_ept, bool, 0);

 struct vmcs {
@@ -284,6 +284,18 @@ static inline void __invvpid(int ext, u16 vpid, gva_t 
gva)
 		  : : "a"(&operand), "c"(ext) : "cc", "memory");
 }

+static inline void __invept(int ext, u64 eptp, gpa_t gpa)
+{
+	struct {
+		u64 eptp, gpa;
+	} operand = {eptp, gpa};
+
+	asm volatile (ASM_VMX_INVEPT
+			/* CF==1 or ZF==1 --> rc = -1 */
+			"; ja 1f ; ud2 ; 1:\n"
+			: : "a" (&operand), "c" (ext) : "cc", "memory");
+}
+
 static struct kvm_msr_entry *find_msr_entry(struct vcpu_vmx *vmx, u32 msr)
 {
 	int i;
@@ -335,6 +347,33 @@ static inline void vpid_sync_vcpu_all(struct vcpu_vmx 
*vmx)
 	__invvpid(VMX_VPID_EXTENT_SINGLE_CONTEXT, vmx->vpid, 0);
 }

+static inline void ept_sync_global(void)
+{
+	if (cpu_has_vmx_invept_global())
+		__invept(VMX_EPT_EXTENT_GLOBAL, 0, 0);
+}
+
+static inline void ept_sync_context(u64 eptp)
+{
+	if (vm_need_ept()) {
+		if (cpu_has_vmx_invept_context())
+			__invept(VMX_EPT_EXTENT_CONTEXT, eptp, 0);
+		else
+			ept_sync_global();
+	}
+}
+
+static inline void ept_sync_individual_addr(u64 eptp, gpa_t gpa)
+{
+	if (vm_need_ept()) {
+		if (cpu_has_vmx_invept_individual_addr())
+			__invept(VMX_EPT_EXTENT_INDIVIDUAL_ADDR,
+					eptp, gpa);
+		else
+			ept_sync_context(eptp);
+	}
+}
+
 static unsigned long vmcs_readl(unsigned long field)
 {
 	unsigned long value;
@@ -422,6 +461,8 @@ static void update_exception_bitmap(struct kvm_vcpu *vcpu)
 		eb |= 1u << 1;
 	if (vcpu->arch.rmode.active)
 		eb = ~0;
+	if (vm_need_ept())
+		eb &= ~(1u << PF_VECTOR); /* bypass_guest_pf = 0 */
 	vmcs_write32(EXCEPTION_BITMAP, eb);
 }

@@ -1352,8 +1393,64 @@ static void vmx_decache_cr4_guest_bits(struct kvm_vcpu 
*vcpu)
 	vcpu->arch.cr4 |= vmcs_readl(GUEST_CR4) & ~KVM_GUEST_CR4_MASK;
 }

+static void ept_load_pdptrs(struct kvm_vcpu *vcpu)
+{
+	if (is_paging(vcpu) && is_pae(vcpu) && !is_long_mode(vcpu)) {
+		if (!load_pdptrs(vcpu, vcpu->arch.cr3)) {
+			printk(KERN_ERR "EPT: Fail to load pdptrs!\n");
+			return;
+		}
+		vmcs_write64(GUEST_PDPTR0, vcpu->arch.pdptrs[0]);
+		vmcs_write64(GUEST_PDPTR1, vcpu->arch.pdptrs[1]);
+		vmcs_write64(GUEST_PDPTR2, vcpu->arch.pdptrs[2]);
+		vmcs_write64(GUEST_PDPTR3, vcpu->arch.pdptrs[3]);
+	}
+}
+
+static void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4);
+
+static void ept_update_paging_mode_cr0(unsigned long *hw_cr0,
+					unsigned long cr0,
+					struct kvm_vcpu *vcpu)
+{
+	if (!(cr0 & X86_CR0_PG)) {
+		/* From paging/starting to nonpaging */
+		vmcs_write32(CPU_BASED_VM_EXEC_CONTROL,
+			     vmcs_config.cpu_based_exec_ctrl |
+			     (CPU_BASED_CR3_LOAD_EXITING |
+			      CPU_BASED_CR3_STORE_EXITING));
+		vcpu->arch.cr0 = cr0;
+		vmx_set_cr4(vcpu, vcpu->arch.cr4);
+		*hw_cr0 |= X86_CR0_PE | X86_CR0_PG;
+		*hw_cr0 &= ~X86_CR0_WP;
+	} else if (!is_paging(vcpu)) {
+		/* From nonpaging to paging */
+		vmcs_write32(CPU_BASED_VM_EXEC_CONTROL,
+			     vmcs_config.cpu_based_exec_ctrl &
+			     ~(CPU_BASED_CR3_LOAD_EXITING |
+			       CPU_BASED_CR3_STORE_EXITING));
+		vcpu->arch.cr0 = cr0;
+		vmx_set_cr4(vcpu, vcpu->arch.cr4);
+		if (!(vcpu->arch.cr0 & X86_CR0_WP))
+			*hw_cr0 &= ~X86_CR0_WP;
+	}
+}
+
+static void ept_update_paging_mode_cr4(unsigned long *hw_cr4,
+					struct kvm_vcpu *vcpu)
+{
+	if (!is_paging(vcpu)) {
+		*hw_cr4 &= ~X86_CR4_PAE;
+		*hw_cr4 |= X86_CR4_PSE;
+	} else if (!(vcpu->arch.cr4 & X86_CR4_PAE))
+		*hw_cr4 &= ~X86_CR4_PAE;
+}
+
 static void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
 {
+	unsigned long hw_cr0 = (cr0 & ~KVM_GUEST_CR0_MASK) |
+				KVM_VM_CR0_ALWAYS_ON;
+
 	vmx_fpu_deactivate(vcpu);

 	if (vcpu->arch.rmode.active && (cr0 & X86_CR0_PE))
@@ -1371,28 +1468,60 @@ static void vmx_set_cr0(struct kvm_vcpu *vcpu, 
unsigned long cr0)
 	}
 #endif

+	if (vm_need_ept())
+		ept_update_paging_mode_cr0(&hw_cr0, cr0, vcpu);
+
 	vmcs_writel(CR0_READ_SHADOW, cr0);
-	vmcs_writel(GUEST_CR0,
-		    (cr0 & ~KVM_GUEST_CR0_MASK) | KVM_VM_CR0_ALWAYS_ON);
+	vmcs_writel(GUEST_CR0, hw_cr0);
 	vcpu->arch.cr0 = cr0;

 	if (!(cr0 & X86_CR0_TS) || !(cr0 & X86_CR0_PE))
 		vmx_fpu_activate(vcpu);
 }

+static u64 construct_eptp(unsigned long root_hpa)
+{
+	u64 eptp;
+
+	/* TODO write the value reading from MSR */
+	eptp = VMX_EPT_DEFAULT_MT |
+		VMX_EPT_DEFAULT_GAW << VMX_EPT_GAW_EPTP_SHIFT;
+	eptp |= (root_hpa & PAGE_MASK);
+
+	return eptp;
+}
+
 static void vmx_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
 {
+	unsigned long guest_cr3;
+	u64 eptp;
+
+	guest_cr3 = cr3;
+	if (vm_need_ept()) {
+		eptp = construct_eptp(cr3);
+		vmcs_write64(EPT_POINTER, eptp);
+		ept_sync_context(eptp);
+		ept_load_pdptrs(vcpu);
+		guest_cr3 = is_paging(vcpu) ? vcpu->arch.cr3 :
+			VMX_EPT_IDENTITY_PAGETABLE_ADDR;
+	}
+
 	vmx_flush_tlb(vcpu);
-	vmcs_writel(GUEST_CR3, cr3);
+	vmcs_writel(GUEST_CR3, guest_cr3);
 	if (vcpu->arch.cr0 & X86_CR0_PE)
 		vmx_fpu_deactivate(vcpu);
 }

 static void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
 {
+	unsigned long hw_cr4 = cr4 | (vcpu->arch.rmode.active ?
+		    KVM_RMODE_VM_CR4_ALWAYS_ON : KVM_PMODE_VM_CR4_ALWAYS_ON);
+
+	if (vm_need_ept())
+		ept_update_paging_mode_cr4(&hw_cr4, vcpu);
+
 	vmcs_writel(CR4_READ_SHADOW, cr4);
-	vmcs_writel(GUEST_CR4, cr4 | (vcpu->arch.rmode.active ?
-		    KVM_RMODE_VM_CR4_ALWAYS_ON : KVM_PMODE_VM_CR4_ALWAYS_ON));
+	vmcs_writel(GUEST_CR4, hw_cr4);
 	vcpu->arch.cr4 = cr4;
 }

@@ -2047,6 +2176,9 @@ static int handle_exception(struct kvm_vcpu *vcpu, 
struct kvm_run *kvm_run)
 	if (intr_info & INTR_INFO_DELIVER_CODE_MASK)
 		error_code = vmcs_read32(VM_EXIT_INTR_ERROR_CODE);
 	if (is_page_fault(intr_info)) {
+		/* EPT won't cause page fault directly */
+		if (vm_need_ept())
+			BUG();
 		cr2 = vmcs_readl(EXIT_QUALIFICATION);
 		KVMTRACE_3D(PAGE_FAULT, vcpu, error_code, (u32)cr2,
 			    (u32)((u64)cr2 >> 32), handler);
@@ -2376,6 +2508,64 @@ static int handle_task_switch(struct kvm_vcpu *vcpu, 
struct kvm_run *kvm_run)
 	return kvm_task_switch(vcpu, tss_selector, reason);
 }

+static int handle_ept_violation(struct kvm_vcpu *vcpu, struct kvm_run 
*kvm_run)
+{
+	u64 exit_qualification;
+	enum emulation_result er;
+	gpa_t gpa;
+	unsigned long hva;
+	int gla_validity;
+	int r;
+
+	exit_qualification = vmcs_read64(EXIT_QUALIFICATION);
+
+	if (exit_qualification & (1 << 6)) {
+		printk(KERN_ERR "EPT: GPA exceeds GAW!\n");
+		return -ENOTSUPP;
+	}
+
+	gla_validity = (exit_qualification >> 7) & 0x3;
+	if (gla_validity != 0x3 && gla_validity != 0x1 && gla_validity != 0) {
+		printk(KERN_ERR "EPT: Handling EPT violation failed!\n");
+		printk(KERN_ERR "EPT: GPA: 0x%lx, GVA: 0x%lx\n",
+			(long unsigned int)vmcs_read64(GUEST_PHYSICAL_ADDRESS),
+			(long unsigned int)vmcs_read64(GUEST_LINEAR_ADDRESS));
+		printk(KERN_ERR "EPT: Exit qualification is 0x%lx\n",
+			(long unsigned int)exit_qualification);
+		kvm_run->exit_reason = KVM_EXIT_UNKNOWN;
+		kvm_run->hw.hardware_exit_reason = 0;
+		return -ENOTSUPP;
+	}
+
+	gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
+	hva = gfn_to_hva(vcpu->kvm, gpa >> PAGE_SHIFT);
+	if (!kvm_is_error_hva(hva)) {
+		r = kvm_mmu_page_fault(vcpu, gpa & PAGE_MASK, 0);
+		if (r < 0) {
+			printk(KERN_ERR "EPT: Not enough memory!\n");
+			return -ENOMEM;
+		}
+		return 1;
+	} else {
+		/* must be MMIO */
+		er = emulate_instruction(vcpu, kvm_run, 0, 0, 0);
+
+		if (er == EMULATE_FAIL) {
+			printk(KERN_ERR
+			 "EPT: Fail to handle EPT violation vmexit!er is %d\n",
+			 er);
+			printk(KERN_ERR "EPT: GPA: 0x%lx, GVA: 0x%lx\n",
+			 (long unsigned int)vmcs_read64(GUEST_PHYSICAL_ADDRESS),
+			 (long unsigned int)vmcs_read64(GUEST_LINEAR_ADDRESS));
+			printk(KERN_ERR "EPT: Exit qualification is 0x%lx\n",
+				(long unsigned int)exit_qualification);
+			return -ENOTSUPP;
+		} else if (er == EMULATE_DO_MMIO)
+			return 0;
+	}
+	return 1;
+}
+
 /*
  * The exit handlers return 1 if the exit was handled fully and guest 
execution
  * may resume.  Otherwise they set the kvm_run parameter to indicate what 
needs
@@ -2399,6 +2589,7 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu 
*vcpu,
 	[EXIT_REASON_APIC_ACCESS]             = handle_apic_access,
 	[EXIT_REASON_WBINVD]                  = handle_wbinvd,
 	[EXIT_REASON_TASK_SWITCH]             = handle_task_switch,
+	[EXIT_REASON_EPT_VIOLATION]	      = handle_ept_violation,
 };

 static const int kvm_vmx_max_exit_handlers =
@@ -2425,7 +2616,8 @@ static int kvm_handle_exit(struct kvm_run *kvm_run, 
struct kvm_vcpu *vcpu)
 	}

 	if ((vectoring_info & VECTORING_INFO_VALID_MASK) &&
-				exit_reason != EXIT_REASON_EXCEPTION_NMI)
+			(exit_reason != EXIT_REASON_EXCEPTION_NMI &&
+			exit_reason != EXIT_REASON_EPT_VIOLATION))
 		printk(KERN_WARNING "%s: unexpected, valid vectoring info and "
 		       "exit reason is 0x%x\n", __func__, exit_reason);
 	if (exit_reason < kvm_vmx_max_exit_handlers
@@ -2672,6 +2864,13 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu, struct 
kvm_run *kvm_run)
 #endif
 	      );

+	/* Access CR3 don't cause VMExit in paging mode, so we need
+	 * to sync with guest real CR3. */
+	if (vm_need_ept() && is_paging(vcpu)) {
+		vcpu->arch.cr3 = vmcs_readl(GUEST_CR3);
+		ept_load_pdptrs(vcpu);
+	}
+
 	vmx->idt_vectoring_info = vmcs_read32(IDT_VECTORING_INFO_FIELD);
 	if (vmx->rmode.irq.pending)
 		fixup_rmode_irq(vmx);
@@ -2727,6 +2926,15 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm 
*kvm, unsigned int id)
 		return ERR_PTR(-ENOMEM);

 	allocate_vpid(vmx);
+	if (id == 0 && vm_need_ept()) {
+		kvm_mmu_set_base_ptes(VMX_EPT_READABLE_MASK |
+			VMX_EPT_WRITABLE_MASK |
+			VMX_EPT_DEFAULT_MT << VMX_EPT_MT_EPTE_SHIFT);
+		kvm_mmu_set_mask_ptes(0ull, VMX_EPT_FAKE_ACCESSED_MASK,
+				VMX_EPT_FAKE_DIRTY_MASK, 0ull,
+				VMX_EPT_EXECUTABLE_MASK);
+		kvm_enable_tdp();
+	}

 	err = kvm_vcpu_init(&vmx->vcpu, kvm, id);
 	if (err)
@@ -2896,9 +3104,14 @@ static int __init vmx_init(void)
 	vmx_disable_intercept_for_msr(vmx_msr_bitmap, MSR_IA32_SYSENTER_ESP);
 	vmx_disable_intercept_for_msr(vmx_msr_bitmap, MSR_IA32_SYSENTER_EIP);

+	if (cpu_has_vmx_ept())
+		bypass_guest_pf = 0;
+
 	if (bypass_guest_pf)
 		kvm_mmu_set_nonpresent_ptes(~0xffeull, 0ull);

+	ept_sync_global();
+
 	return 0;

 out2:
diff --git a/arch/x86/kvm/vmx.h b/arch/x86/kvm/vmx.h
index 5894f26..5c18d68 100644
--- a/arch/x86/kvm/vmx.h
+++ b/arch/x86/kvm/vmx.h
@@ -351,5 +351,16 @@ enum vmcs_field {
 #define VMX_EPT_EXTENT_INDIVIDUAL_BIT		(1ul << 24)
 #define VMX_EPT_EXTENT_CONTEXT_BIT		(1ul << 25)
 #define VMX_EPT_EXTENT_GLOBAL_BIT		(1ul << 26)
+#define VMX_EPT_IDENTITY_PAGETABLE_ADDR		0xcf000ul
+#define VMX_EPT_MAX_GAW				0x4
+#define VMX_EPT_MT_EPTE_SHIFT			3
+#define VMX_EPT_GAW_EPTP_SHIFT			3
+#define VMX_EPT_DEFAULT_GAW			0x3ull
+#define VMX_EPT_DEFAULT_MT			0x6ull
+#define VMX_EPT_READABLE_MASK			0x1ull
+#define VMX_EPT_WRITABLE_MASK			0x2ull
+#define VMX_EPT_EXECUTABLE_MASK			0x4ull
+#define VMX_EPT_FAKE_ACCESSED_MASK		(1ul << 62)
+#define VMX_EPT_FAKE_DIRTY_MASK			(1ul << 63)

 #endif
diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h
index 9f62773..639657f 100644
--- a/include/asm-x86/kvm_host.h
+++ b/include/asm-x86/kvm_host.h
@@ -646,6 +646,7 @@ static inline void kvm_inject_gp(struct kvm_vcpu *vcpu, 
u32 error_code)
 #define ASM_VMX_VMWRITE_RSP_RDX   ".byte 0x0f, 0x79, 0xd4"
 #define ASM_VMX_VMXOFF            ".byte 0x0f, 0x01, 0xc4"
 #define ASM_VMX_VMXON_RAX         ".byte 0xf3, 0x0f, 0xc7, 0x30"
+#define ASM_VMX_INVEPT		  ".byte 0x66, 0x0f, 0x38, 0x80, 0x08"
 #define ASM_VMX_INVVPID		  ".byte 0x66, 0x0f, 0x38, 0x81, 0x08"

 #define MSR_IA32_TIME_STAMP_COUNTER		0x010
--
1.5.4.5

[kvm-devel] [PATCH 4/6] KVM: Export necessary function for EPT

[Yang, S. <she...@in...>]

From 5d4a79e5edfc09b54bd83a3a289cbb82058e3daa Mon Sep 17 00:00:00 2001
From: Sheng Yang <she...@in...>
Date: Fri, 18 Apr 2008 17:05:20 +0800
Subject: [PATCH 4/5] KVM: Export necessary function for EPT

The function gfn_to_gva is necessary for handling EPT violation.

Signed-off-by: Sheng Yang <she...@in...>
---
 virt/kvm/kvm_main.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 0998455..d028e07 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -522,6 +522,7 @@ unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn)
 		return bad_hva();
 	return (slot->userspace_addr + (gfn - slot->base_gfn) * PAGE_SIZE);
 }
+EXPORT_SYMBOL_GPL(gfn_to_hva);

 /*
  * Requires current->mm->mmap_sem to be held
--
1.5.4.5

[kvm-devel] [PATCH 2/6] KVM: MMU: Move some defination

[Yang, S. <she...@in...>]

From a5ee291f056256f8a892393410bc5923ff575a3b Mon Sep 17 00:00:00 2001
From: Sheng Yang <she...@in...>
Date: Fri, 18 Apr 2008 17:03:53 +0800
Subject: [PATCH 2/5] KVM: MMU: Move some defination for building common 
entries

Move some defination to mmu.h in order to building common table entries.

Signed-off-by: Sheng Yang <she...@in...>
---
 arch/x86/kvm/mmu.c |   25 -------------------------
 arch/x86/kvm/mmu.h |   24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 078a7f1..108886d 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -84,31 +84,6 @@ static int dbg = 1;
 #define PT32_PT_BITS 10
 #define PT32_ENT_PER_PAGE (1 << PT32_PT_BITS)

-#define PT_WRITABLE_SHIFT 1
-
-#define PT_PRESENT_MASK (1ULL << 0)
-#define PT_WRITABLE_MASK (1ULL << PT_WRITABLE_SHIFT)
-#define PT_USER_MASK (1ULL << 2)
-#define PT_PWT_MASK (1ULL << 3)
-#define PT_PCD_MASK (1ULL << 4)
-#define PT_ACCESSED_MASK (1ULL << 5)
-#define PT_DIRTY_MASK (1ULL << 6)
-#define PT_PAGE_SIZE_MASK (1ULL << 7)
-#define PT_PAT_MASK (1ULL << 7)
-#define PT_GLOBAL_MASK (1ULL << 8)
-#define PT64_NX_SHIFT 63
-#define PT64_NX_MASK (1ULL << PT64_NX_SHIFT)
-
-#define PT_PAT_SHIFT 7
-#define PT_DIR_PAT_SHIFT 12
-#define PT_DIR_PAT_MASK (1ULL << PT_DIR_PAT_SHIFT)
-
-#define PT32_DIR_PSE36_SIZE 4
-#define PT32_DIR_PSE36_SHIFT 13
-#define PT32_DIR_PSE36_MASK \
-	(((1ULL << PT32_DIR_PSE36_SIZE) - 1) << PT32_DIR_PSE36_SHIFT)
-
-
 #define PT_FIRST_AVAIL_BITS_SHIFT 9
 #define PT64_SECOND_AVAIL_BITS_SHIFT 52

diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index e64e9f5..271c011 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -9,6 +9,30 @@
 #define TDP_ROOT_LEVEL PT32E_ROOT_LEVEL
 #endif

+#define PT_WRITABLE_SHIFT 1
+
+#define PT_PRESENT_MASK (1ULL << 0)
+#define PT_WRITABLE_MASK (1ULL << PT_WRITABLE_SHIFT)
+#define PT_USER_MASK (1ULL << 2)
+#define PT_PWT_MASK (1ULL << 3)
+#define PT_PCD_MASK (1ULL << 4)
+#define PT_ACCESSED_MASK (1ULL << 5)
+#define PT_DIRTY_MASK (1ULL << 6)
+#define PT_PAGE_SIZE_MASK (1ULL << 7)
+#define PT_PAT_MASK (1ULL << 7)
+#define PT_GLOBAL_MASK (1ULL << 8)
+#define PT64_NX_SHIFT 63
+#define PT64_NX_MASK (1ULL << PT64_NX_SHIFT)
+
+#define PT_PAT_SHIFT 7
+#define PT_DIR_PAT_SHIFT 12
+#define PT_DIR_PAT_MASK (1ULL << PT_DIR_PAT_SHIFT)
+
+#define PT32_DIR_PSE36_SIZE 4
+#define PT32_DIR_PSE36_SHIFT 13
+#define PT32_DIR_PSE36_MASK \
+	(((1ULL << PT32_DIR_PSE36_SIZE) - 1) << PT32_DIR_PSE36_SHIFT)
+
 static inline void kvm_mmu_free_some_pages(struct kvm_vcpu *vcpu)
 {
 	if (unlikely(vcpu->kvm->arch.n_free_mmu_pages < KVM_MIN_FREE_MMU_PAGES))
--
1.5.4.5

[kvm-devel] [PATCH 3/6] KVM: MMU: Add EPT support

[Yang, S. <she...@in...>]

From cb851671421832d37c7d90976b603b59a5c75c79 Mon Sep 17 00:00:00 2001
From: Sheng Yang <she...@in...>
Date: Fri, 18 Apr 2008 17:05:06 +0800
Subject: [PATCH 3/5] KVM: MMU: Add EPT support

Enable kvm_set_spte() to generate EPT entries.

Signed-off-by: Sheng Yang <she...@in...>
---
 arch/x86/kvm/mmu.c         |   44 
++++++++++++++++++++++++++++++++++----------
 arch/x86/kvm/x86.c         |    3 +++
 include/asm-x86/kvm_host.h |    3 +++
 3 files changed, 40 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 108886d..1828837 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -161,6 +161,12 @@ static struct kmem_cache *mmu_page_header_cache;

 static u64 __read_mostly shadow_trap_nonpresent_pte;
 static u64 __read_mostly shadow_notrap_nonpresent_pte;
+static u64 __read_mostly shadow_base_present_pte;
+static u64 __read_mostly shadow_nx_mask;
+static u64 __read_mostly shadow_x_mask;	/* mutual exclusive with nx_mask */
+static u64 __read_mostly shadow_user_mask;
+static u64 __read_mostly shadow_accessed_mask;
+static u64 __read_mostly shadow_dirty_mask;

 void kvm_mmu_set_nonpresent_ptes(u64 trap_pte, u64 notrap_pte)
 {
@@ -169,6 +175,23 @@ void kvm_mmu_set_nonpresent_ptes(u64 trap_pte, u64 
notrap_pte)
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_nonpresent_ptes);

+void kvm_mmu_set_base_ptes(u64 base_pte)
+{
+	shadow_base_present_pte = base_pte;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_set_base_ptes);
+
+void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
+		u64 dirty_mask, u64 nx_mask, u64 x_mask)
+{
+	shadow_user_mask = user_mask;
+	shadow_accessed_mask = accessed_mask;
+	shadow_dirty_mask = dirty_mask;
+	shadow_nx_mask = nx_mask;
+	shadow_x_mask = x_mask;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
+
 static int is_write_protection(struct kvm_vcpu *vcpu)
 {
 	return vcpu->arch.cr0 & X86_CR0_WP;
@@ -207,7 +230,7 @@ static int is_writeble_pte(unsigned long pte)

 static int is_dirty_pte(unsigned long pte)
 {
-	return pte & PT_DIRTY_MASK;
+	return pte & shadow_dirty_mask;
 }

 static int is_rmap_pte(u64 pte)
@@ -522,7 +545,7 @@ static void rmap_remove(struct kvm *kvm, u64 *spte)
 		return;
 	sp = page_header(__pa(spte));
 	pfn = spte_to_pfn(*spte);
-	if (*spte & PT_ACCESSED_MASK)
+	if (*spte & shadow_accessed_mask)
 		kvm_set_pfn_accessed(pfn);
 	if (is_writeble_pte(*spte))
 		kvm_release_pfn_dirty(pfn);
@@ -1048,17 +1071,18 @@ static void mmu_set_spte(struct kvm_vcpu *vcpu, u64 
*shadow_pte,
 	 * whether the guest actually used the pte (in order to detect
 	 * demand paging).
 	 */
-	spte = PT_PRESENT_MASK | PT_DIRTY_MASK;
+	spte = shadow_base_present_pte | shadow_dirty_mask;
 	if (!speculative)
 		pte_access |= PT_ACCESSED_MASK;
 	if (!dirty)
 		pte_access &= ~ACC_WRITE_MASK;
-	if (!(pte_access & ACC_EXEC_MASK))
-		spte |= PT64_NX_MASK;
-
-	spte |= PT_PRESENT_MASK;
+	if (pte_access & ACC_EXEC_MASK) {
+		if (shadow_x_mask)
+			spte |= shadow_x_mask;
+	} else if (shadow_nx_mask)
+		spte |= shadow_nx_mask;
 	if (pte_access & ACC_USER_MASK)
-		spte |= PT_USER_MASK;
+		spte |= shadow_user_mask;
 	if (largepage)
 		spte |= PT_PAGE_SIZE_MASK;

@@ -1164,7 +1188,7 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t v, 
int write,
 			}

 			table[index] = __pa(new_table->spt) | PT_PRESENT_MASK
-				| PT_WRITABLE_MASK | PT_USER_MASK;
+				| PT_WRITABLE_MASK | shadow_user_mask;
 		}
 		table_addr = table[index] & PT64_BASE_ADDR_MASK;
 	}
@@ -1608,7 +1632,7 @@ static bool last_updated_pte_accessed(struct kvm_vcpu 
*vcpu)
 {
 	u64 *spte = vcpu->arch.last_pte_updated;

-	return !!(spte && (*spte & PT_ACCESSED_MASK));
+	return !!(spte && (*spte & shadow_accessed_mask));
 }

 static void mmu_guess_page_from_pte_write(struct kvm_vcpu *vcpu, gpa_t gpa,
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0ce5563..0735efb 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2417,6 +2417,9 @@ int kvm_arch_init(void *opaque)

 	kvm_x86_ops = ops;
 	kvm_mmu_set_nonpresent_ptes(0ull, 0ull);
+	kvm_mmu_set_base_ptes(PT_PRESENT_MASK);
+	kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
+			PT_DIRTY_MASK, PT64_NX_MASK, 0);
 	return 0;

 out:
diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h
index 31aa7d6..9f62773 100644
--- a/include/asm-x86/kvm_host.h
+++ b/include/asm-x86/kvm_host.h
@@ -432,6 +432,9 @@ void kvm_mmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_mmu_create(struct kvm_vcpu *vcpu);
 int kvm_mmu_setup(struct kvm_vcpu *vcpu);
 void kvm_mmu_set_nonpresent_ptes(u64 trap_pte, u64 notrap_pte);
+void kvm_mmu_set_base_ptes(u64 base_pte);
+void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
+		u64 dirty_mask, u64 nx_mask, u64 x_mask);

 int kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
 void kvm_mmu_slot_remove_write_access(struct kvm *kvm, int slot);
--
1.5.4.5

[kvm-devel] [PATCH 0/6] Enable EPT on KVM v3

[Yang, S. <she...@in...>]

Hi

This patchset enabled EPT on KVM.

The most obvious improvement is the separate construction of EPT table has 
been discarded completely. Now EPT reused ordinary MMU for building the EPT 
table. The code size is greatly reduced and this also solved the display 
problem. But I think it also have impact of scalability...

But currently, S/R and live migration still got problem. I am working on it 
now.

--
Thanks
Yang, Sheng