Re: [tboot-devel] VT-d PMRs

[Cihula, J. <jos...@in...>]

> From: Jonathan M. McCune [mailto:jon...@cm...]
> Sent: Tuesday, October 14, 2008 2:51 PM
> 
> Hi Joe, list,
> 
> I'm able to read the registers now, but I'm still not sure what I'm
> doing wrong.  Details...
> 
> With the latest xen-unstable.hg and tboot.hg, I added some code to
> xen/drivers/passthrough/vtd/utils.c:dump_iommu_info() to also dump the
> Protected Memory Registers (enable, low base/limit, high base/limit).
> I
> inserted a call so that these registers get printed before more
> sophisticated remapping is enabled.  I then printed them again
manually
> after Xen has fully booted (using 3xCtrl+A, V).
> 
> The 0xfed93000 from my previous email was 4K aligned, but it turns out
> that the only DRHD with non-zero PMR registers on my system is
> 0xfed91000 (low base of 16MB, size 16MB), and only during boot time
> (this makes sense, since disable_pmr() is called for each DRHD):
> 
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 80000001
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 1000000
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 1000000
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 200000
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> 
> 
> When I invoke SENTER with my code (on a non-Xen, vanilla Linux
system),
> I configure the low PMR region to be 2 MB in size, starting at 12 MB
of
> physical memory (kmalloc()'d).
> 
> TBOOT: os_sinit_data->vtd_pmr_lo_base: 0xc00000
> TBOOT: os_sinit_data->vtd_pmr_lo_size: 0x200000
> TBOOT: os_sinit_data->vtd_pmr_hi_base: 0x0
> TBOOT: os_sinit_data->vtd_pmr_hi_size: 0x0
> 
> It uses these same memory regions every time, and reliably works once
> but then fails (triggering a reboot) with:
> 
> TBOOT: LT.ERRORCODE=c00038f1
> TBOOT: AC module error : acm_type=1, progress=0f, error=e
> 
>            1110      Current values of VT-d PMR registers do not match
>                      requested values in SinitMleData
> 
> 
> If I try to read the PMR registers between sessions (i.e., while the
> system is _not_ in any kind of secure mode; SEXIT _did_ execute
> following SENTER), I get all 0s.

Are you sure that this code that reads them is correct?  I don't see why
they should be all 0s if you have not explicitly cleared them yourself.
The SEXIT instruction does not do anything to the PMRs.

> I have been assuming that "do not match" means "were not different
when
> they were last used."  Is it the case that I must re-enable Protected
> Memory before invoking SENTER again?  Something else?

Can you try disabling the PMRs between calls to SENTER and see if that
fixes it?

Joe

> 
> Thanks,
> -Jon
> 
> 
> 
> *** big debug output follows ***
> 
> (XEN) JON: dump_iommu_info()
> (XEN)
> (XEN) iommu 0: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed92000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c0000020230270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 0
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 0
> (XEN) print_iommu_regs: DMAR_GSTS_REG = 40000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7bf000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 800000000000000
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 4090
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN)
> (XEN) iommu 1: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed91000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c0000020230270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 10
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 0
> (XEN) print_iommu_regs: DMAR_GSTS_REG = 40000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7be000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 800000000000000
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 4098
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 80000001
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 1000000
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 1000000
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 200000
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN)
> (XEN) iommu 2: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed90000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c9008020a30270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 0
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 0
> (XEN) print_iommu_regs: DMAR_GSTS_REG = 40000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7bd000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 0
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 40a0
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN)
> (XEN) iommu 3: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed93000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c9008020230270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 0
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 0
> (XEN) print_iommu_regs: DMAR_GSTS_REG = 40000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7bc000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 0
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 40a8
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN) I/O virtualisation enabled
> (XEN) I/O virtualisation for PV guests disabled
> (XEN) JON: disable_pmr invoked
> (XEN) JON: disable_pmr invoked
> (XEN) JON: disable_pmr invoked
> (XEN) JON: disable_pmr invoked
> (XEN) *** LOADING DOMAIN 0 ***
> 
> 
> 
> (XEN) *** Serial input -> Xen (type 'CTRL-a' three times to switch
> input
> to DOM0)
> (XEN)
> (XEN) iommu 0: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed92000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c0000020230270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 0
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 80000000
> (XEN) print_iommu_regs: DMAR_GSTS_REG = c0000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7bf000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 4800000000000000
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 4090
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN)
> (XEN) iommu 1: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed91000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c0000020230270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 10
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 80000000
> (XEN) print_iommu_regs: DMAR_GSTS_REG = c0000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7be000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 2800000000000000
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 4098
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 1000000
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 1000000
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 200000
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN)
> (XEN) iommu 2: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed90000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c9008020a30270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 0
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 80000000
> (XEN) print_iommu_regs: DMAR_GSTS_REG = c0000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7bd000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 4800000000000001
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 40a0
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> (XEN)
> (XEN) iommu 3: nr_pt_levels = 3.
> (XEN)   Queued Invalidation: not supported.
> (XEN)   Interrupt Remapping: not supported.
> (XEN) ---- print_iommu_regs ----
> (XEN) print_iommu_regs: drhd->address = fed93000
> (XEN) print_iommu_regs: DMAR_VER_REG = 10
> (XEN) print_iommu_regs: DMAR_CAP_REG = c9008020230270
> (XEN) print_iommu_regs: n_fault_reg = 1
> (XEN) print_iommu_regs: fault_recording_offset_l = 200
> (XEN) print_iommu_regs: fault_recording_offset_h = 208
> (XEN) print_iommu_regs: fault_recording_reg_l = 0
> (XEN) print_iommu_regs: fault_recording_reg_h = 0
> (XEN) print_iommu_regs: DMAR_ECAP_REG = 1000
> (XEN) print_iommu_regs: DMAR_GCMD_REG = 80000000
> (XEN) print_iommu_regs: DMAR_GSTS_REG = c0000000
> (XEN) print_iommu_regs: DMAR_RTADDR_REG = 7c7bc000
> (XEN) print_iommu_regs: DMAR_CCMD_REG = 5000000000000001
> (XEN) print_iommu_regs: DMAR_FSTS_REG = 0
> (XEN) print_iommu_regs: DMAR_FECTL_REG = 0
> (XEN) print_iommu_regs: DMAR_FEDATA_REG = 40a8
> (XEN) print_iommu_regs: DMAR_FEADDR_REG = fee00000
> (XEN) print_iommu_regs: DMAR_FEUADDR_REG = 0
> (XEN) print_iommu_regs: DMAR_PMEN_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PLMLIMIT_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMBASE_REG = 0
> (XEN) print_iommu_regs: DMAR_PHMLIMIT_REG = 0
> 
> 
> 
> Cihula, Joseph wrote:
> >> From: Jonathan M. McCune [mailto:jon...@cm...]
> >> Sent: Friday, October 10, 2008 3:11 PM
> >>
> >> Hi Joe, list,
> >>
> >> I started digging through this.
> >>
> >> In the VT-d spec, the very beginning of chapter 7 says that each
DMA
> >> remapping hardware unit is placed at a system-specific 4K-aligned
> >> memory mapped address, and that the BIOS will report where these
> are.
> >>
> >> I realized I don't know what these addresses are, and Ch. 5 told me
> to
> >> look at the DMAR table.
> >>
> >> I wrote some code to decode the DMAR table, and the subsequent DRHD
> /
> >> RMRR entries it contains, and the subsequent Device Scope
Structures
> >> that they contain.
> >>
> >> When run on my system following execution of SENTER, the big long
> >> printout at the end of this email ensues.  I made an educated guess
> >> that the DRHD structure with flags = 1 (see Table 5-3 in the VT-d
> > spec)
> >> might be the DMA remapping hardware unit??? of choice.  I tried to
> > print its
> >> values at the relevant offsets for the PMRs (see Table 7-2), but I
> get
> >> all 0s:
> >>
> >> TBOOT: ACPI VT-d DRHD structure @ 7c6a3e78:
> >> TBOOT:  Type: 0
> >> TBOOT:  Length: 0x10 (16)
> >> TBOOT:  Flags: 1
> >> TBOOT:  Register Base Address: 0xfed93000
> >> TBOOT:          DMAR DRHD Registers @ fed93000:
> >> TBOOT:                  PMR Enable: 0x0
> >> TBOOT:                  PMR Low Base: 0x0
> >> TBOOT:                  PMR Low Limit: 0x0
> >> TBOOT:                  PMR High Base: 0x0
> >> TBOOT:                  PMR High Limit: 0x0
> >>
> >>
> >> Am I on the right track here?  How do I know which of these "DMA
> >> remapping hardware units" is the right one?
> >
> > These entries are not 4k aligned, so they aren't the right places.
> The
> > easiest thing to do is to look at the Xen code that parses the
tables
> > and reads the registers to see how to do it.
> >
> >> Thanks!
> >> -Jon
> >>
> >>
> >>
> >>
> >>
> >> TBOOT: acpi_dmar = 7c6a3df0
> >> TBOOT: VT-d DMAR @ 7c6a3df0 (len 360 bytes):
> >> 44 4d 41 52 68 01 00 00 01 c7 31 30 31 36 30 37
> >> 4f 45 4d 44 4d 41 52 20 01 00 00 00 4d 53 46 54
> >> 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00
> >> 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00
> >> 01 08 00 00 00 00 1b 00 00 00 18 00 00 00 00 00
> >> 00 10 d9 fe 00 00 00 00 01 08 00 00 00 00 02 00
> >> 00 00 28 00 00 00 00 00 00 20 d9 fe 00 00 00 00
> >> 01 08 00 00 00 00 03 00 01 08 00 00 00 00 03 02
> >> 01 08 00 00 00 00 03 03 00 00 10 00 01 00 00 00
> >> 00 30 d9 fe 00 00 00 00 01 00 58 00 00 00 00 00
> >> 00 d0 0e 00 00 00 00 00 ff ff 0e 00 00 00 00 00
> >> 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01
> >> 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07
> >> 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01
> >> 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07
> >> 01 00 58 00 00 00 00 00 00 00 2f 7d 00 00 00 00
> >> ff ff 2f 7d 00 00 00 00 01 08 00 00 00 00 1d 00
> >> 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02
> >> 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00
> >> 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02
> >> 01 08 00 00 00 00 1a 07 01 00 20 00 00 00 00 00
> >> 00 00 60 7d 00 00 00 00 ff ff ff 7d 00 00 00 00
> >> 01 08 00 00 00 00 02 00
> >> TBOOT: ACPI VT-d DMAR table @ 7c6a3df0 (len 48 bytes):
> >> TBOOT:  Signature: DMAR
> >> TBOOT:  Length: 0x168 (360)
> >> TBOOT:  Revision: 1
> >> TBOOT:  Checksum: c7
> >> TBOOT:  OEMID: 101607
> >> TBOOT:  OEM Table ID: OEMDMAR
> >> TBOOT:  OEM Revision: 1
> >> TBOOT:  Creator ID: MSFT
> >> TBOOT:  Creator Revision: 0x97
> >> TBOOT:  Host Address Width: 0x23 (35)
> >> TBOOT: ACPI VT-d DRHD structure @ 7c6a3e20:
> >> TBOOT:  Type: 0
> >> TBOOT:  Length: 0x18 (24)
> >> TBOOT:  Flags: 0
> >> TBOOT:  Register Base Address: 0xfed90000
> >> TBOOT:  Device Scope entry @ 7c6a3e30:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1b 00
> >>
> >> TBOOT: ACPI VT-d DRHD structure @ 7c6a3e38:
> >> TBOOT:  Type: 0
> >> TBOOT:  Length: 0x18 (24)
> >> TBOOT:  Flags: 0
> >> TBOOT:  Register Base Address: 0xfed91000
> >> TBOOT:  Device Scope entry @ 7c6a3e48:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 02 00
> >>
> >> TBOOT: ACPI VT-d DRHD structure @ 7c6a3e50:
> >> TBOOT:  Type: 0
> >> TBOOT:  Length: 0x28 (40)
> >> TBOOT:  Flags: 0
> >> TBOOT:  Register Base Address: 0xfed92000
> >> TBOOT:  Device Scope entry @ 7c6a3e60:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 03 00
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3e68:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 03 02
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3e70:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 03 03
> >>
> >> TBOOT: ACPI VT-d DRHD structure @ 7c6a3e78:
> >> TBOOT:  Type: 0
> >> TBOOT:  Length: 0x10 (16)
> >> TBOOT:  Flags: 1
> >> TBOOT:  Register Base Address: 0xfed93000
> >> TBOOT:          DMAR DRHD Registers @ fed93000:
> >> TBOOT:                  PMR Enable: 0x0
> >> TBOOT:                  PMR Low Base: 0x0
> >> TBOOT:                  PMR Low Limit: 0x0
> >> TBOOT:                  PMR High Base: 0x0
> >> TBOOT:                  PMR High Limit: 0x0
> >> TBOOT: ACPI VT-d RMRR structure @ 7c6a3e88:
> >> TBOOT:  Type: 0x1 (1)
> >> TBOOT:  Length: 0x58 (88)
> >> TBOOT:  Flag: 0x0
> >> TBOOT:  RMR Base Address:  ed000
> >> TBOOT:  RMR Limit Address: effff
> >> TBOOT:  Device Scope entry @ 7c6a3ea0:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 00
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3ea8:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 01
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3eb0:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 02
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3eb8:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 07
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3ec0:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 00
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3ec8:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 01
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3ed0:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 02
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3ed8:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 07
> >>
> >> TBOOT: ACPI VT-d RMRR structure @ 7c6a3ee0:
> >> TBOOT:  Type: 0x1 (1)
> >> TBOOT:  Length: 0x58 (88)
> >> TBOOT:  Flag: 0x0
> >> TBOOT:  RMR Base Address:  7d2f0000
> >> TBOOT:  RMR Limit Address: 7d2fffff
> >> TBOOT:  Device Scope entry @ 7c6a3ef8:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 00
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f00:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 01
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f08:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 02
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f10:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1d 07
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f18:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 00
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f20:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 01
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f28:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 02
> >>
> >> TBOOT:  Device Scope entry @ 7c6a3f30:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 1a 07
> >>
> >> TBOOT: ACPI VT-d RMRR structure @ 7c6a3f38:
> >> TBOOT:  Type: 0x1 (1)
> >> TBOOT:  Length: 0x20 (32)
> >> TBOOT:  Flag: 0x0
> >> TBOOT:  RMR Base Address:  7d600000
> >> TBOOT:  RMR Limit Address: 7dffffff
> >> TBOOT:  Device Scope entry @ 7c6a3f50:
> >> TBOOT:          Type: 0x1
> >> TBOOT:          Length: 0x8
> >> TBOOT:          Segment Number: 0x0
> >> TBOOT:          Starting Bus Number: 0x0
> >> TBOOT:          Raw PCI Path (Length - 4): 00 00 02 00
> >>
> >> TBOOT: VT-d DMAR table OK
> >>
> >>
> >> Cihula, Joseph wrote:
> >>>> From: Jonathan M. McCune [mailto:jon...@cm...]
> >>>> Sent: Thursday, October 09, 2008 10:27 AM
> >>>>
> >>>> Hi Joe,
> >>>>
> >>>> Cihula, Joseph wrote:
> >>>>> It is really:
> >>>>> 	Current values of VT-d PMR registers do not match
requested
> >>>>> values in SinitMleData
> >>>>>
> >>>>> which means that some code has already programmed the PMRs but
> not
> >>> to
> >>>>> the same values that the MLE is requesting.  Because the PMRs
> >> cannot
> >>>> be
> >>>>> changed reliably once they are set/enabled, this is not an
> allowed
> >>>>> condition.
> >>>> I am successfully invoking SENTER / SEXIT and then returning
> > control
> >>> to
> >>>> the legacy OS once per boot cycle.  When I try to execute SENTER
a
> >>>> second time, the system reboots and LT.ERRORCODE is populated
with
> >> the
> >>>> above error.  However, I cannot figure out what is wrong with my
> > PMR
> >>>> values.
> >>>>
> >>>> Is there a way to read the current values, so that I can see how
> >> they
> >>>> are set following the first SENTER? Looking through the MLE
manual
> >> and
> >>>> the Sw Dev Manual Vol 2b has left me without much insight.
> >>> VT-d registers are described in the VT-d spec at:
> >>>
> >
>
http://download.intel.com/technology/computing/vptech/Intel(r)_VT_for_D
> >> i
> >>> rect_IO.pdf
> >>>
> >>>> My MLE resides at physical address 0x00c00000 and consumes less
> > than
> >>>> 0x200000 (2MB) bytes.
> >>>>
> >>>> TBOOT:   vtd_pmr_lo_base: 0xc00000
> >>>> TBOOT:   vtd_pmr_lo_size: 0x200000
> >>>> TBOOT:   vtd_pmr_hi_base: 0x0
> >>>> TBOOT:   vtd_pmr_hi_size: 0x0
> >>>>
> >>>> I'm fairly confident that the os_sinit_data_t.vtd_pmr_* values
are
> >>>> being
> >>>> set identically prior to both invocations of SENTER (when I print
> >> them
> >>>> out, the above is what I see).
> >>> If you could use the VT-d spec to read and output the PMRs before
> >>> launch, that should determine if they are somehow getting changed
> >>> between invocations.
> >>>
> >>>> Thanks,
> >>>> -Jon
> >