kvm-ppc-devel

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis B. <ho...@us...>]

On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
> So gdb on power relies only on those few hw-breakpoints? With x86 you
> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
> (currently based on hw-BPs, but the same would be true for soft-BPs
> inserted by the gdbstub).

GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. It 
does not rely on the hardware breakpoints.

It gets a little more complicated when you involve a GDB stub. IIRC, GDB will 
ask the stub to set the breakpoint, and if the stub doesn't support it, GDB 
will fall back to overwriting the instructions in memory. Does the Qemu GDB 
stub advertise breakpoint support?

If not, the only support needed in KVM would be to send all debug interrupts 
to qemu, and allow qemu to send them back down for in-guest breakpoints.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis B. <ho...@us...>]

On Wednesday 14 May 2008 14:10:06 Jan Kiszka wrote:
> Hollis Blanchard wrote:
> > On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
> >> So gdb on power relies only on those few hw-breakpoints? With x86 you
> >> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
> >> (currently based on hw-BPs, but the same would be true for soft-BPs
> >> inserted by the gdbstub).
> > 
> > GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. 
It 
> > does not rely on the hardware breakpoints.
> > 
> > It gets a little more complicated when you involve a GDB stub. IIRC, GDB 
will 
> > ask the stub to set the breakpoint, and if the stub doesn't support it, 
GDB 
> > will fall back to overwriting the instructions in memory. Does the Qemu 
GDB 
> > stub advertise breakpoint support?
> 
> Yes, QEMU reacts on both Z0 (soft-BP) and Z1 (hard-BP). That's something
> even gdbserver does not do! It just handles watchpoints (Z2..4).
> 
> > 
> > If not, the only support needed in KVM would be to send all debug 
interrupts 
> > to qemu, and allow qemu to send them back down for in-guest breakpoints.
> > 
> 
> Simply returning "unsupported" on Z0 is not yet enough for x86, KVM's
> kernel side should not yet inform QEMU about soft-BP exceptions. But in
> theory, this should be easily fixable (or is already the case for other
> archs). And it would safe us from keeping track of N software
> breakpoints, where N could even become larger than 32, the current
> hardcoded limit for plain QEMU. :)
> 
> Meanwhile I realized that the proposed KVM_DEBUG_GUEST API is
> insufficient: We need a return channel for the debug register state
> (specifically to figure out details about hit watchpoints). I'm now
> favoring KVM_SET_DEBUG and KVM_GET_DEBUG as two new IOCTLs, enabling us
> to write _and_ read-back the suggested data structure.

How about simply extending kvm_exit.debug to contain the virtual address of 
the breakpoint hit? In Qemu, when exit_reason == KVM_EXIT_DEBUG, it would 
just need to see if that address is for a breakpoint Qemu set or not. If so, 
it's happy. If not, (commence handwaving) tell KVM to forward the debug 
interrupt to the guest. This way, the list of breakpoints is maintained in 
userspace (in the qemu gdb stub), which is nice because it could be 
arbitrarily large.

Also, this is not specific to hardware debug registers: soft and hard 
breakpoint interrupts would follow the same path. There's still a question of 
whether the GDB stub should set the breakpoint itself (Z0/Z1) or force GDB to 
modify memory, but either way the KVM code is simple.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Jan K. <jan...@we...>]

Hollis Blanchard wrote:
> On Wednesday 14 May 2008 14:10:06 Jan Kiszka wrote:
>> Hollis Blanchard wrote:
>>> On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
>>>> So gdb on power relies only on those few hw-breakpoints? With x86 you
>>>> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
>>>> (currently based on hw-BPs, but the same would be true for soft-BPs
>>>> inserted by the gdbstub).
>>> GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. 
> It 
>>> does not rely on the hardware breakpoints.
>>>
>>> It gets a little more complicated when you involve a GDB stub. IIRC, GDB 
> will 
>>> ask the stub to set the breakpoint, and if the stub doesn't support it, 
> GDB 
>>> will fall back to overwriting the instructions in memory. Does the Qemu 
> GDB 
>>> stub advertise breakpoint support?
>> Yes, QEMU reacts on both Z0 (soft-BP) and Z1 (hard-BP). That's something
>> even gdbserver does not do! It just handles watchpoints (Z2..4).
>>
>>> If not, the only support needed in KVM would be to send all debug 
> interrupts 
>>> to qemu, and allow qemu to send them back down for in-guest breakpoints.
>>>
>> Simply returning "unsupported" on Z0 is not yet enough for x86, KVM's
>> kernel side should not yet inform QEMU about soft-BP exceptions. But in
>> theory, this should be easily fixable (or is already the case for other
>> archs). And it would safe us from keeping track of N software
>> breakpoints, where N could even become larger than 32, the current
>> hardcoded limit for plain QEMU. :)
>>
>> Meanwhile I realized that the proposed KVM_DEBUG_GUEST API is
>> insufficient: We need a return channel for the debug register state
>> (specifically to figure out details about hit watchpoints). I'm now
>> favoring KVM_SET_DEBUG and KVM_GET_DEBUG as two new IOCTLs, enabling us
>> to write _and_ read-back the suggested data structure.
> 
> How about simply extending kvm_exit.debug to contain the virtual address of 
> the breakpoint hit?

Ah, there is an interface for such stuff already! And it can even take
quite some payload...

> In Qemu, when exit_reason == KVM_EXIT_DEBUG, it would 
> just need to see if that address is for a breakpoint Qemu set or not. If so, 
> it's happy. If not, (commence handwaving) tell KVM to forward the debug 
> interrupt to the guest. This way, the list of breakpoints is maintained in 
> userspace (in the qemu gdb stub), which is nice because it could be 
> arbitrarily large.

Yes, but I would rather pass the debug registers (more general: some
arch dependent state set) back in this slot. Those contain everything
the gdbstub needs to know to catch relevant hardware-BP/watchpoint
events (and report them to the gdb frontend).

> 
> Also, this is not specific to hardware debug registers: soft and hard 
> breakpoint interrupts would follow the same path. There's still a question of 
> whether the GDB stub should set the breakpoint itself (Z0/Z1) or force GDB to 
> modify memory, but either way the KVM code is simple.

Only rejecting Z0 will enable us to avoid any soft-BP tracking in
qemu-kvm, and that is definitely my plan. Z1 may become an option to add
later on and would be answered as "unsupported" for now.

Jan

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis B. <ho...@us...>]

On Wednesday 14 May 2008 14:49:02 Jan Kiszka wrote:
> > In Qemu, when exit_reason == KVM_EXIT_DEBUG, it would 
> > just need to see if that address is for a breakpoint Qemu set or not. If 
so, 
> > it's happy. If not, (commence handwaving) tell KVM to forward the debug 
> > interrupt to the guest. This way, the list of breakpoints is maintained in 
> > userspace (in the qemu gdb stub), which is nice because it could be 
> > arbitrarily large.
> 
> Yes, but I would rather pass the debug registers (more general: some
> arch dependent state set) back in this slot. Those contain everything
> the gdbstub needs to know to catch relevant hardware-BP/watchpoint
> events (and report them to the gdb frontend).

But what would the stub *do* with the contents of the debug registers? The 
only reason they were set is on behalf of the stub in the first place. In 
fact, in the case of soft breakpoints, KVM doesn't even know where all the 
set breakpoints are. The only thing KVM needs to report is the address of the 
breakpoint that was just hit.

Sorry if this gets formatted badly:

gdb					qemu stub				KVM
break *0xf00
sends Z0 packet 0xf00
					0xf00 -> BP list
					ioctl(KVM_DEBUG, 0xf00)
continue
					ioctl(KVM_RUN)
											running...
											breakpoint hit
											exit_reason = KVM_EXIT_DEBUG
											kvm_run.debug.address = current PC value
					search BP list for address
bp hit			<---	present        not present --->	send debug interrupt to guest

Notes:
- KVM_DEBUG in this case will set a hardware breakpoint. The alternative is to 
write an int3 into guest memory.
- The stub doesn't care how the hardware registers were configured. All it 
needs to know is a) that a breakpoint was hit, and b) at what address.

Does this make sense?

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis B. <ho...@us...>]

On Wednesday 14 May 2008 16:06:00 Hollis Blanchard wrote:
> In 
> fact, in the case of soft breakpoints, KVM doesn't even know where all the 
> set breakpoints are.

Side note: I'm retract this sentence: I wrote it before I sketched out the 
pseudocode, and forgot to remove it. :)

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis B. <ho...@us...>]

On Wednesday 14 May 2008 16:11:39 Hollis Blanchard wrote:
> On Wednesday 14 May 2008 16:06:00 Hollis Blanchard wrote:
> > In 
> > fact, in the case of soft breakpoints, KVM doesn't even know where all the 
> > set breakpoints are.
> 
> Side note: I'm retract this sentence: I wrote it before I sketched out the 
> pseudocode, and forgot to remove it. :)

Er no, let me try that again:

In my proposed design, the stub needs to know where all breakpoints are, HW or 
SW. (That means it must implement Z0/Z1.)

However, KVM itself doesn't need to know any of that: all breakpoints are 
referred to the stub for handling, and the stub will notify KVM if further 
action is needed in-kernel.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [kvm-ppc-devel] [kvm-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Jan K. <jan...@we...>]

Hollis Blanchard wrote:
> On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
>> So gdb on power relies only on those few hw-breakpoints? With x86 you
>> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
>> (currently based on hw-BPs, but the same would be true for soft-BPs
>> inserted by the gdbstub).
> 
> GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. It 
> does not rely on the hardware breakpoints.
> 
> It gets a little more complicated when you involve a GDB stub. IIRC, GDB will 
> ask the stub to set the breakpoint, and if the stub doesn't support it, GDB 
> will fall back to overwriting the instructions in memory. Does the Qemu GDB 
> stub advertise breakpoint support?

Yes, QEMU reacts on both Z0 (soft-BP) and Z1 (hard-BP). That's something
even gdbserver does not do! It just handles watchpoints (Z2..4).

> 
> If not, the only support needed in KVM would be to send all debug interrupts 
> to qemu, and allow qemu to send them back down for in-guest breakpoints.
> 

Simply returning "unsupported" on Z0 is not yet enough for x86, KVM's
kernel side should not yet inform QEMU about soft-BP exceptions. But in
theory, this should be easily fixable (or is already the case for other
archs). And it would safe us from keeping track of N software
breakpoints, where N could even become larger than 32, the current
hardcoded limit for plain QEMU. :)

Meanwhile I realized that the proposed KVM_DEBUG_GUEST API is
insufficient: We need a return channel for the debug register state
(specifically to figure out details about hit watchpoints). I'm now
favoring KVM_SET_DEBUG and KVM_GET_DEBUG as two new IOCTLs, enabling us
to write _and_ read-back the suggested data structure.

Jan