dle-develop — Dependable Linux Effort mailing list for developers

[Dle-develop] [PATCH tracing/kprobes 0/7] tracing/kprobes: kprobe-based event tracer update and perf support

[Masami H. <mhi...@re...>]

Hi Frederic,

This series fixes bugs and upgrades kprobe-based event tracer
as a dynamic event tracer on ftrace/perf tools. This also enhances
tracer output format to show each argument name and event name on
each entry.

With this series, users can add trace events dynamically on ftrace
and use those events with perf tools as below.

(Step.1) Define new events under new group

$ echo p:mygroup/myprobe do_sys_open dfd=a0 filename=a1 flags=a2 mode=a3 \
 > /debug/tracing/kprobes_events
$ echo r:mygroup/myretprobe do_sys_open rv >> /debug/tracing/kprobes_events
$ cat /debug/tracing/kprobes_events
p:myprobe do_sys_open+0 dfd=a0 filename=a1 flags=a2 mode=a3
r:myretprobe do_sys_open+0 rv=rv

(You can see that each argument has its name.)


(Step.2) Perf shows new events

$ perf list
...
   mygroup:myretprobe                         [Tracepoint event]
   mygroup:myprobe                            [Tracepoint event]
...


(Step.3) Record events with perf

$ perf record -f -e mygroup:myprobe:record -F 1 -a ls
...
[ perf record: Captured and wrote 0.081 MB perf.data (~3544 samples) ]


(Step.4) Perf trace shows the result

$ perf trace
version = 0.5
            perf-1405  [000]     0.000000: myprobe: (c04b0a5c) dfd=ffffff9c filename=810d3f7 flags=98800 mode=1
            perf-1405  [000]     0.000000: myprobe: (c04b0a5c) dfd=ffffff9c filename=bff7650c flags=8000 mode=1b6
            perf-1405  [000]     0.000000: myprobe: (c04b0a5c) dfd=ffffff9c filename=bff7650c flags=98800 mode=bff7450c
            perf-1405  [000]     0.000000: myprobe: (c04b0a5c) dfd=ffffff9c filename=bff7650c flags=8000 mode=1b6


(Step.5) You can also use return probes.

$ perf record -f -e mygroup:myretprobe:record -F 1 -a ls
...
[ perf record: Captured and wrote 0.081 MB perf.data (~3544 samples) ]
$ perf trace
version = 0.5
            perf-1408  [000]     0.000000: myretprobe: (c04b0a5c <- c04b0b7d) rv=b
            perf-1408  [000]     0.000000: myretprobe: (c04b0a5c <- c04b0b7d) rv=c
            perf-1408  [000]     0.000000: myretprobe: (c04b0a5c <- c04b0b7d) rv=d
            perf-1408  [000]     0.000000: myretprobe: (c04b0a5c <- c04b0b7d) rv=c


TODO:
- Implement perf kprobe command to help defining new probes.


Thank you,

---

Masami Hiramatsu (7):
      tracing/kprobes: Support custom subsystem for each kprobe event
      tracing/kprobes: Show event name in trace output
      tracing/kprobes: Add argument name support
      tracing/kprobes: Add event profiling support
      tracing/kprobes: Cleanup kprobe tracer code.
      tracing/kprobes: Fix probe offset to be unsigned
      x86/ptrace: Fix regs_get_argument_nth() to add correct offset


 Documentation/trace/kprobetrace.txt |   65 +++---
 arch/x86/kernel/ptrace.c            |    2 
 kernel/trace/trace_kprobe.c         |  369 +++++++++++++++++++++++------------
 3 files changed, 274 insertions(+), 162 deletions(-)

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH tracing/kprobes 1/7] x86/ptrace: Fix regs_get_argument_nth() to add correct offset

[Masami H. <mhi...@re...>]

Fix regs_get_argument_nth() to add correct offset bytes. Because
offset_of() returns offset in byte, the offset should be added
to char * instead of unsigned long *.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Andi Kleen <ak...@li...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Frank Ch. Eigler <fc...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Ingo Molnar <mi...@el...>
Cc: Jason Baron <jb...@re...>
Cc: K.Prasad <pr...@li...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Peter Zijlstra <pe...@in...>
Cc: Srikar Dronamraju <sr...@li...>
Cc: Steven Rostedt <ro...@go...>
Cc: Tom Zanussi <tza...@gm...>
---

 arch/x86/kernel/ptrace.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/ptrace.c b/arch/x86/kernel/ptrace.c
index a33a17d..caffb68 100644
--- a/arch/x86/kernel/ptrace.c
+++ b/arch/x86/kernel/ptrace.c
@@ -150,7 +150,7 @@ static const int arg_offs_table[] = {
 unsigned long regs_get_argument_nth(struct pt_regs *regs, unsigned int n)
 {
 	if (n < ARRAY_SIZE(arg_offs_table))
-		return *((unsigned long *)regs + arg_offs_table[n]);
+		return *(unsigned long *)((char *)regs + arg_offs_table[n]);
 	else {
 		/*
 		 * The typical case: arg n is on the stack.


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH tracing/kprobes] x86: Add MMX support for instruction decoder

[Frederic W. <fwe...@gm...>]

On Tue, Sep 08, 2009 at 12:32:46PM -0400, Masami Hiramatsu wrote:
> Add MMX/SSE instructions to x86 opcode maps, since some of those
> instructions are used in the kernel.
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Jim Keniston <jke...@us...>
> Cc: H. Peter Anvin <hp...@zy...>
> Cc: Sam Ravnborg <sa...@ra...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ingo Molnar <mi...@el...>
> ---


Hehe :) 

	"Succeed: decoded and checked 1574884 instructions"

I'm taking the three patches, thanks!

[Dle-develop] [RFC PATCH tracing/kprobes] kprobes: Call vmalloc_sync_all() for avoiding in-kernel paging on kprobes

[Masami H. <mhi...@re...>]

Call vmalloc_sync_all() before registering new kprobes and using new
instruction buffers to sync page table entries of the pages on where
kprobes data structure and instruction buffers are.
With this patch, kprobes can avoid in-kernel on-demand paging on these
kprobes, and it allows kprobes to put probes on page-fault path.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Jim Keniston <jke...@us...>
Cc: Ingo Molnar <mi...@el...>
Cc: Frederic Weisbecker <fwe...@gm...>
---

 kernel/kprobes.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 00d01b0..7cc8266 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -44,6 +44,7 @@
 #include <linux/debugfs.h>
 #include <linux/kdebug.h>
 #include <linux/memory.h>
+#include <linux/vmalloc.h>
 
 #include <asm-generic/sections.h>
 #include <asm/cacheflush.h>
@@ -191,6 +192,7 @@ static kprobe_opcode_t __kprobes *__get_insn_slot(void)
 		kfree(kip);
 		return NULL;
 	}
+	vmalloc_sync_all();	/* Prohibit on-demand kernel paging on kprobe */
 	INIT_LIST_HEAD(&kip->list);
 	list_add(&kip->list, &kprobe_insn_pages);
 	memset(kip->slot_used, SLOT_CLEAN, INSNS_PER_PAGE);
@@ -724,6 +726,7 @@ int __kprobes register_kprobe(struct kprobe *p)
 	}
 	preempt_enable();
 
+	vmalloc_sync_all();	/* Prohibit on-demand kernel paging on kprobe */
 	p->nmissed = 0;
 	INIT_LIST_HEAD(&p->list);
 	mutex_lock(&kprobe_mutex);


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH tracing/kprobes] kprobes/x86-32: Move irq-exit functions to kprobes section

[Masami H. <mhi...@re...>]

Move irq-exit functions to .kprobes.text section for protecting against
kprobes.

When I ran kprobe stress test on x86-32, I found below symbols causes
unrecoverable recursive probing:
ret_from_exception
ret_from_intr
check_userspace
restore_all
restore_all_notrace
restore_nocheck
irq_return

And also, I found some interrupt/exception entry points will cause
problems.

This patch moves those symbols (including its container functions)
to .kprobes.text section for protecting from kprobes probing.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Jim Keniston <jke...@us...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/kernel/entry_32.S |   24 ++++++++++++++++++++++++
 kernel/kprobes.c           |    2 ++
 2 files changed, 26 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
index c097e7d..beb30da 100644
--- a/arch/x86/kernel/entry_32.S
+++ b/arch/x86/kernel/entry_32.S
@@ -334,6 +334,10 @@ ENTRY(ret_from_fork)
 END(ret_from_fork)
 
 /*
+ * Interrupt exit functions should be protected against kprobes
+ */
+	.pushsection .kprobes.text, "ax"
+/*
  * Return to user mode is not as complex as all this looks,
  * but we want the default path for a system call return to
  * go as quickly as possible which is why some of this is
@@ -383,6 +387,10 @@ need_resched:
 END(resume_kernel)
 #endif
 	CFI_ENDPROC
+/*
+ * End of kprobes section
+ */
+	.popsection
 
 /* SYSENTER_RETURN points to after the "sysenter" instruction in
    the vsyscall page.  See vsyscall-sysentry.S, which defines the symbol.  */
@@ -513,6 +521,10 @@ sysexit_audit:
 	PTGS_TO_GS_EX
 ENDPROC(ia32_sysenter_target)
 
+/*
+ * syscall stub including irq exit should be protected against kprobes
+ */
+	.pushsection .kprobes.text, "ax"
 	# system call handler stub
 ENTRY(system_call)
 	RING0_INT_FRAME			# can't unwind into user space anyway
@@ -705,6 +717,10 @@ syscall_badsys:
 	jmp resume_userspace
 END(syscall_badsys)
 	CFI_ENDPROC
+/*
+ * End of kprobes section
+ */
+	.popsection
 
 /*
  * System calls that need a pt_regs pointer.
@@ -814,6 +830,10 @@ common_interrupt:
 ENDPROC(common_interrupt)
 	CFI_ENDPROC
 
+/*
+ *  Irq entries should be protected against kprobes
+ */
+	.pushsection .kprobes.text, "ax"
 #define BUILD_INTERRUPT3(name, nr, fn)	\
 ENTRY(name)				\
 	RING0_INT_FRAME;		\
@@ -980,6 +1000,10 @@ ENTRY(spurious_interrupt_bug)
 	jmp error_code
 	CFI_ENDPROC
 END(spurious_interrupt_bug)
+/*
+ * End of kprobes section
+ */
+	.popsection
 
 ENTRY(kernel_thread_helper)
 	pushl $0		# fake return address for unwinder
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 3267d90..00d01b0 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -91,6 +91,8 @@ static spinlock_t *kretprobe_table_lock_ptr(unsigned long hash)
 static struct kprobe_blackpoint kprobe_blacklist[] = {
 	{"preempt_schedule",},
 	{"native_get_debugreg",},
+	{"irq_entries_start",},
+	{"common_interrupt",},
 	{NULL}    /* Terminator */
 };
 


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH tracing/kprobes] x86: Add MMX support for instruction decoder

[Masami H. <mhi...@re...>]

Add MMX/SSE instructions to x86 opcode maps, since some of those
instructions are used in the kernel.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Sam Ravnborg <sa...@ra...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/lib/x86-opcode-map.txt |  307 +++++++++++++++++++++++++--------------
 1 files changed, 200 insertions(+), 107 deletions(-)

diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt
index 083dd59..59e20d5 100644
--- a/arch/x86/lib/x86-opcode-map.txt
+++ b/arch/x86/lib/x86-opcode-map.txt
@@ -310,14 +310,14 @@ Referrer: 2-byte escape
 0e:
 0f:
 # 0x0f 0x10-0x1f
-10:
-11:
-12:
-13:
-14:
-15:
-16:
-17:
+10: movups Vps,Wps | movss Vss,Wss (F3) | movupd Vpd,Wpd (66) | movsd Vsd,Wsd (F2)
+11: movups Wps,Vps | movss Wss,Vss (F3) | movupd Wpd,Vpd (66) | movsd Wsd,Vsd (F2)
+12: movlps Vq,Mq | movlpd Vq,Mq (66) | movhlps Vq,Uq | movddup Vq,Wq (F2) | movsldup Vq,Wq (F3)
+13: mpvlps Mq,Vq | movlpd Mq,Vq (66)
+14: unpcklps Vps,Wq | unpcklpd Vpd,Wq (66)
+15: unpckhps Vps,Wq | unpckhpd Vpd,Wq (66)
+16: movhps Vq,Mq | movhpd Vq,Mq (66) | movlsps Vq,Uq | movshdup Vq,Wq (F3)
+17: movhps Mq,Vq | movhpd Mq,Vq (66)
 18: Grp16 (1A)
 19:
 1a:
@@ -337,12 +337,12 @@ Referrer: 2-byte escape
 27:
 28: movaps Vps,Wps | movapd Vpd,Wpd (66)
 29: movaps Wps,Vps | movapd Wpd,Vpd (66)
-2a:
-2b:
-2c:
-2d:
-2e:
-2f:
+2a: cvtpi2ps Vps,Qpi | cvtsi2ss Vss,Ed/q (F3) | cvtpi2pd Vpd,Qpi (66) | cvtsi2sd Vsd,Ed/q (F2)
+2b: movntps Mps,Vps | movntpd Mpd,Vpd (66)
+2c: cvttps2pi Ppi,Wps | cvttss2si  Gd/q,Wss (F3) | cvttpd2pi Ppi,Wpd (66) | cvttsd2si Gd/q,Wsd (F2)
+2d: cvtps2pi Ppi,Wps | cvtss2si Gd/q,Wss (F3) | cvtpd2pi Qpi,Wpd (66) | cvtsd2si Gd/q,Wsd (F2)
+2e: ucomiss Vss,Wss | ucomisd  Vsd,Wsd (66)
+2f: comiss Vss,Wss | comisd  Vsd,Wsd (66)
 # 0x0f 0x30-0x3f
 30: WRMSR
 31: RDTSC
@@ -378,56 +378,56 @@ Referrer: 2-byte escape
 4e: CMOVLE/NG Gv,Ev
 4f: CMOVNLE/G Gv,Ev
 # 0x0f 0x50-0x5f
-50:
-51:
-52:
-53:
-54:
-55:
-56:
-57:
-58:
-59:
-5a:
-5b:
-5c:
-5d:
-5e:
-5f:
+50: movmskps Gd/q,Ups | movmskpd Gd/q,Upd (66)
+51: sqrtps Vps,Wps | sqrtss Vss,Wss (F3) | sqrtpd Vpd,Wpd (66) | sqrtsd Vsd,Wsd (F2)
+52: rsqrtps Vps,Wps | rsqrtss Vss,Wss (F3)
+53: rcpps Vps,Wps | rcpss Vss,Wss (F3)
+54: andps Vps,Wps | andpd Vpd,Wpd (66)
+55: andnps Vps,Wps | andnpd Vpd,Wpd (66)
+56: orps Vps,Wps | orpd Vpd,Wpd (66)
+57: xorps Vps,Wps | xorpd Vpd,Wpd (66)
+58: addps Vps,Wps | addss Vss,Wss (F3) | addpd Vpd,Wpd (66) | addsd Vsd,Wsd (F2)
+59: mulps Vps,Wps | mulss Vss,Wss (F3) | mulpd Vpd,Wpd (66) | mulsd Vsd,Wsd (F2)
+5a: cvtps2pd Vpd,Wps | cvtss2sd Vsd,Wss (F3) | cvtpd2ps Vps,Wpd (66) | cvtsd2ss Vsd,Wsd (F2)
+5b: cvtdq2ps Vps,Wdq | cvtps2dq Vdq,Wps (66) | cvttps2dq Vdq,Wps (F3)
+5c: subps Vps,Wps | subss Vss,Wss (F3) | subpd Vpd,Wpd (66) | subsd Vsd,Wsd (F2)
+5d: minps Vps,Wps | minss Vss,Wss (F3) | minpd Vpd,Wpd (66) | minsd Vsd,Wsd (F2)
+5e: divps Vps,Wps | divss Vss,Wss (F3) | divpd Vpd,Wpd (66) | divsd Vsd,Wsd (F2)
+5f: maxps Vps,Wps | maxss Vss,Wss (F3) | maxpd Vpd,Wpd (66) | maxsd Vsd,Wsd (F2)
 # 0x0f 0x60-0x6f
-60:
-61:
-62:
-63:
-64:
-65:
-66:
-67:
-68:
-69:
-6a:
-6b:
-6c:
-6d:
-6e:
-6f:
+60: punpcklbw Pq,Qd | punpcklbw Vdq,Wdq (66)
+61: punpcklwd Pq,Qd | punpcklwd Vdq,Wdq (66)
+62: punpckldq Pq,Qd | punpckldq Vdq,Wdq (66)
+63: packsswb Pq,Qq | packsswb Vdq,Wdq (66)
+64: pcmpgtb Pq,Qq | pcmpgtb Vdq,Wdq (66)
+65: pcmpgtw Pq,Qq | pcmpgtw(66) Vdq,Wdq
+66: pcmpgtd Pq,Qq | pcmpgtd Vdq,Wdq (66)
+67: packuswb Pq,Qq | packuswb(66) Vdq,Wdq
+68: punpckhbw Pq,Qd | punpckhbw Vdq,Wdq (66)
+69: punpckhwd Pq,Qd | punpckhwd Vdq,Wdq (66)
+6a: punpckhdq Pq,Qd | punpckhdq Vdq,Wdq (66)
+6b: packssdw Pq,Qd | packssdw Vdq,Wdq (66)
+6c: punpcklqdq Vdq,Wdq (66)
+6d: punpckhqdq Vdq,Wdq (66)
+6e: movd/q/ Pd,Ed/q | movd/q Vdq,Ed/q (66)
+6f: movq Pq,Qq | movdqa Vdq,Wdq (66) | movdqu Vdq,Wdq (F3)
 # 0x0f 0x70-0x7f
-70:
+70: pshufw Pq,Qq,Ib | pshufd Vdq,Wdq,Ib (66) | pshufhw Vdq,Wdq,Ib (F3) | pshuflw VdqWdq,Ib (F2)
 71: Grp12 (1A)
 72: Grp13 (1A)
 73: Grp14 (1A)
-74:
-75:
-76:
-77:
+74: pcmpeqb Pq,Qq | pcmpeqb Vdq,Wdq (66)
+75: pcmpeqw Pq,Qq | pcmpeqw Vdq,Wdq (66)
+76: pcmpeqd Pq,Qq | pcmpeqd Vdq,Wdq (66)
+77: emms
 78: VMREAD Ed/q,Gd/q
 79: VMWRITE Gd/q,Ed/q
 7a:
 7b:
-7c:
-7d:
-7e:
-7f:
+7c: haddps(F2) Vps,Wps | haddpd(66) Vpd,Wpd
+7d: hsubps(F2) Vps,Wps | hsubpd(66) Vpd,Wpd
+7e: movd/q Ed/q,Pd | movd/q Ed/q,Vdq (66) | movq Vq,Wq (F3)
+7f: movq Qq,Pq | movdqa Wdq,Vdq (66) | movdqu Wdq,Vdq (F3)
 # 0x0f 0x80-0x8f
 80: JO Jz (f64)
 81: JNO Jz (f64)
@@ -499,11 +499,11 @@ bf: MOVSX Gv,Ew
 # 0x0f 0xc0-0xcf
 c0: XADD Eb,Gb
 c1: XADD Ev,Gv
-c2:
+c2: cmpps Vps,Wps,Ib | cmpss Vss,Wss,Ib (F3) | cmppd Vpd,Wpd,Ib (66) | cmpsd Vsd,Wsd,Ib (F2)
 c3: movnti Md/q,Gd/q
-c4:
-c5:
-c6:
+c4: pinsrw Pq,Rd/q/Mw,Ib | pinsrw Vdq,Rd/q/Mw,Ib (66)
+c5: pextrw Gd,Nq,Ib | pextrw Gd,Udq,Ib (66)
+c6: shufps Vps,Wps,Ib | shufpd Vpd,Wpd,Ib (66)
 c7: Grp9 (1A)
 c8: BSWAP RAX/EAX/R8/R8D
 c9: BSWAP RCX/ECX/R9/R9D
@@ -514,60 +514,131 @@ cd: BSWAP RBP/EBP/R13/R13D
 ce: BSWAP RSI/ESI/R14/R14D
 cf: BSWAP RDI/EDI/R15/R15D
 # 0x0f 0xd0-0xdf
-d0:
-d1:
-d2:
-d3:
-d4:
-d5:
-d6:
-d7:
-d8:
-d9:
-da:
-db:
-dc:
-dd:
-de:
-df:
+d0: addsubps Vps,Wps (F2) | addsubpd Vpd,Wpd (66)
+d1: psrlw Pq,Qq | psrlw Vdq,Wdq (66)
+d2: psrld Pq,Qq | psrld Vdq,Wdq (66)
+d3: psrlq Pq,Qq | psrlq Vdq,Wdq (66)
+d4: paddq Pq,Qq | paddq Vdq,Wdq (66)
+d5: pmullw Pq,Qq | pmullw Vdq,Wdq (66)
+d6: movq Wq,Vq (66) | movq2dq Vdq,Nq (F3) | movdq2q Pq,Uq (F2)
+d7: pmovmskb Gd,Nq | pmovmskb Gd,Udq (66)
+d8: psubusb Pq,Qq | psubusb Vdq,Wdq (66)
+d9: psubusw Pq,Qq | psubusw Vdq,Wdq (66)
+da: pminub Pq,Qq | pminub Vdq,Wdq (66)
+db: pand Pq,Qq | pand Vdq,Wdq (66)
+dc: paddusb Pq,Qq | paddusb Vdq,Wdq (66)
+dd: paddusw Pq,Qq | paddusw Vdq,Wdq (66)
+de: pmaxub Pq,Qq | pmaxub Vdq,Wdq (66)
+df: pandn Pq,Qq | pandn Vdq,Wdq (66)
 # 0x0f 0xe0-0xef
-e0:
-e1:
-e2:
-e3:
-e4:
-e5:
-e6:
-e7:
-e8:
-e9:
-ea:
-eb:
-ec:
-ed:
-ee:
-ef:
+e0: pavgb Pq,Qq | pavgb Vdq,Wdq (66)
+e1: psraw Pq,Qq | psraw Vdq,Wdq (66)
+e2: psrad Pq,Qq | psrad Vdq,Wdq (66)
+e3: pavgw Pq,Qq | pavgw Vdq,Wdq (66)
+e4: pmulhuw Pq,Qq | pmulhuw Vdq,Wdq (66)
+e5: pmulhw Pq,Qq | pmulhw Vdq,Wdq (66)
+e6: cvtpd2dq Vdq,Wpd (F2) | cvttpd2dq Vdq,Wpd (66) | cvtdq2pd Vpd,Wdq (F3)
+e7: movntq Mq,Pq | movntdq Mdq,Vdq (66)
+e8: psubsb Pq,Qq | psubsb Vdq,Wdq (66)
+e9: psubsw Pq,Qq | psubsw Vdq,Wdq (66)
+ea: pminsw Pq,Qq | pminsw Vdq,Wdq (66)
+eb: por Pq,Qq | por Vdq,Wdq (66)
+ec: paddsb Pq,Qq | paddsb Vdq,Wdq (66)
+ed: paddsw Pq,Qq | paddsw Vdq,Wdq (66)
+ee: pmaxsw Pq,Qq | pmaxsw Vdq,Wdq (66)
+ef: pxor Pq,Qq | pxor Vdq,Wdq (66)
 # 0x0f 0xf0-0xff
-f0:
-f1:
-f2:
-f3:
-f4:
-f5:
-f6:
-f7:
-f8:
-f9:
-fa:
-fb:
-fc:
-fd:
-fe:
+f0: lddqu Vdq,Mdq (F2)
+f1: psllw Pq,Qq | psllw Vdq,Wdq (66)
+f2: pslld Pq,Qq | pslld Vdq,Wdq (66)
+f3: psllq Pq,Qq | psllq Vdq,Wdq (66)
+f4: pmuludq Pq,Qq | pmuludq Vdq,Wdq (66)
+f5: pmaddwd Pq,Qq | pmaddwd Vdq,Wdq (66)
+f6: psadbw Pq,Qq | psadbw Vdq,Wdq (66)
+f7: maskmovq Pq,Nq | maskmovdqu Vdq,Udq (66)
+f8: psubb Pq,Qq | psubb Vdq,Wdq (66)
+f9: psubw Pq,Qq | psubw Vdq,Wdq (66)
+fa: psubd Pq,Qq | psubd Vdq,Wdq (66)
+fb: psubq Pq,Qq | psubq Vdq,Wdq (66)
+fc: paddb Pq,Qq | paddb Vdq,Wdq (66)
+fd: paddw Pq,Qq | paddw Vdq,Wdq (66)
+fe: paddd Pq,Qq | paddd Vdq,Wdq (66)
 ff:
 EndTable
 
 Table: 3-byte opcode 1
 Referrer: 3-byte escape 1
+# 0x0f 0x38 0x00-0x0f
+00: pshufb Pq,Qq | pshufb Vdq,Wdq (66)
+01: phaddw Pq,Qq | phaddw Vdq,Wdq (66)
+02: phaddd Pq,Qq | phaddd Vdq,Wdq (66)
+03: phaddsw Pq,Qq | phaddsw Vdq,Wdq (66)
+04: pmaddubsw Pq,Qq | pmaddubsw (66)Vdq,Wdq
+05: phsubw Pq,Qq | phsubw Vdq,Wdq (66)
+06: phsubd Pq,Qq | phsubd Vdq,Wdq (66)
+07: phsubsw Pq,Qq | phsubsw Vdq,Wdq (66)
+08: psignb Pq,Qq | psignb Vdq,Wdq (66)
+09: psignw Pq,Qq | psignw Vdq,Wdq (66)
+0a: psignd Pq,Qq | psignd Vdq,Wdq (66)
+0b: pmulhrsw Pq,Qq | pmulhrsw Vdq,Wdq (66)
+0c:
+0d:
+0e:
+0f:
+# 0x0f 0x38 0x10-0x1f
+10: pblendvb Vdq,Wdq (66)
+11:
+12:
+13:
+14: blendvps Vdq,Wdq (66)
+15: blendvpd Vdq,Wdq (66)
+16:
+17: ptest Vdq,Wdq (66)
+18:
+19:
+1a:
+1b:
+1c: pabsb Pq,Qq | pabsb Vdq,Wdq (66)
+1d: pabsw Pq,Qq | pabsw Vdq,Wdq (66)
+1e: pabsd Pq,Qq | pabsd Vdq,Wdq (66)
+1f:
+# 0x0f 0x38 0x20-0x2f
+20: pmovsxbw Vdq,Udq/Mq (66)
+21: pmovsxbd Vdq,Udq/Md (66)
+22: pmovsxbq Vdq,Udq/Mw (66)
+23: pmovsxwd Vdq,Udq/Mq (66)
+24: pmovsxwq Vdq,Udq/Md (66)
+25: pmovsxdq Vdq,Udq/Mq (66)
+26:
+27:
+28: pmuldq Vdq,Wdq (66)
+29: pcmpeqq Vdq,Wdq (66)
+2a: movntdqa Vdq,Mdq (66)
+2b: packusdw Vdq,Wdq (66)
+2c:
+2d:
+2e:
+2f:
+# 0x0f 0x38 0x30-0x3f
+30: pmovzxbw Vdq,Udq/Mq (66)
+31: pmovzxbd Vdq,Udq/Md (66)
+32: pmovzxbq Vdq,Udq/Mw (66)
+33: pmovzxwd Vdq,Udq/Mq (66)
+34: pmovzxwq Vdq,Udq/Md (66)
+35: pmovzxdq Vdq,Udq/Mq (66)
+36:
+37: pcmpgtq Vdq,Wdq (66)
+38: pminsb Vdq,Wdq (66)
+39: pminsd Vdq,Wdq (66)
+3a: pminuw Vdq,Wdq (66)
+3b: pminud Vdq,Wdq (66)
+3c: pmaxsb Vdq,Wdq (66)
+3d: pmaxsd Vdq,Wdq (66)
+3e: pmaxuw Vdq,Wdq (66)
+3f: pmaxud Vdq,Wdq (66)
+# 0x0f 0x38 0x4f-0xff
+40: pmulld Vdq,Wdq (66)
+41: phminposuw Vdq,Wdq (66)
 80: INVEPT Gd/q,Mdq (66)
 81: INVPID Gd/q,Mdq (66)
 f0: MOVBE Gv,Mv | CRC32 Gd,Eb (F2)
@@ -576,7 +647,29 @@ EndTable
 
 Table: 3-byte opcode 2
 Referrer: 3-byte escape 2
-# all opcode is for SSE
+# 0x0f 0x3a 0x00-0xff
+08: roundps Vdq,Wdq,Ib (66)
+09: roundpd Vdq,Wdq,Ib (66)
+0a: roundss Vss,Wss,Ib (66)
+0b: roundsd Vsd,Wsd,Ib (66)
+0c: blendps Vdq,Wdq,Ib (66)
+0d: blendpd Vdq,Wdq,Ib (66)
+0e: pblendw Vdq,Wdq,Ib (66)
+0f: palignr Pq,Qq,Ib | palignr Vdq,Wdq,Ib (66)
+14: pextrb Rd/Mb,Vdq,Ib (66)
+15: pextrw Rd/Mw,Vdq,Ib (66)
+16: pextrd/pextrq Ed/q,Vdq,Ib (66)
+17: extractps Ed,Vdq,Ib (66)
+20: pinsrb Vdq,Rd/q/Mb,Ib (66)
+21: insertps Vdq,Udq/Md,Ib (66)
+22: pinsrd/pinsrq Vdq,Ed/q,Ib (66)
+40: dpps Vdq,Wdq,Ib (66)
+41: dppd Vdq,Wdq,Ib (66)
+42: mpsadbw Vdq,Wdq,Ib (66)
+60: pcmpestrm Vdq,Wdq,Ib (66)
+61: pcmpestri Vdq,Wdq,Ib (66)
+62: pcmpistrm Vdq,Wdq,Ib (66)
+63: pcmpistri Vdq,Wdq,Ib (66)
 EndTable
 
 GrpTable: Grp1


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> On Wed, Sep 02, 2009 at 08:58:05AM -0400, Masami Hiramatsu wrote:
>> Frederic Weisbecker wrote:
>>> BTW, after your patchset:
>>>
>>> TEST    posttest
>>>    Building modules, stage 2.
>>> Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
>>> Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
>>> make[1]: *** [posttest] Erreur 2
>>> make: *** [bzImage] Erreur 2
>>> make: *** Attente des tâches non terminées....
>>>    MODPOST 4 modules
>>>
>>>
>>> I'm cross compiling from 64 to 32 using:
>>>
>>> make ARCH=x86
>>>
>>> I've attached my config.
>>
>> Aah, you might miss an important patch :D
>> http://lkml.org/lkml/2009/8/28/367
>>
>> Thank you,
>>
>
>
> Well, even with this patch I have this problem occuring with
> make ARCH=x86
>
> Error: c125d65d:	0f 6f 02             	movq   (%edx),%mm0
> Error: objdump says 3 bytes, but insn_get_length() says 2 (attr:0)
> make[1]: *** [posttest] Erreur 2
> make: *** [bzImage] Erreur 2
>
> Hmm, may be that's because it doesn't support mmx instructions.
> The whole patchset is on
>
> git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
> 	tracing/kprobes
>
> Config and make V=1 log attached.

Aah, Indeed. It's time to add MMX/SSE support on x86 insn decoder.

Thank you for reporting!

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Frederic W. <fwe...@gm...>]

On Wed, Sep 02, 2009 at 08:58:05AM -0400, Masami Hiramatsu wrote:
> Frederic Weisbecker wrote:
> > BTW, after your patchset:
> > 
> > TEST    posttest
> >   Building modules, stage 2.
> > Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
> > Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
> > make[1]: *** [posttest] Erreur 2
> > make: *** [bzImage] Erreur 2
> > make: *** Attente des tâches non terminées....
> >   MODPOST 4 modules
> > 
> > 
> > I'm cross compiling from 64 to 32 using:
> > 
> > make ARCH=x86
> > 
> > I've attached my config.
> 
> Aah, you might miss an important patch :D
> http://lkml.org/lkml/2009/8/28/367
> 
> Thank you,
> 


Well, even with this patch I have this problem occuring with
make ARCH=x86

Error: c125d65d:	0f 6f 02             	movq   (%edx),%mm0
Error: objdump says 3 bytes, but insn_get_length() says 2 (attr:0)
make[1]: *** [posttest] Erreur 2
make: *** [bzImage] Erreur 2

Hmm, may be that's because it doesn't support mmx instructions.
The whole patchset is on

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
	tracing/kprobes

Config and make V=1 log attached.

[Dle-develop] Eir degrees and follow practice, and solemnly advised travelling s

[Zuerlein A. <aqu...@ma...>]

Ar'd;" and she turned pale and shook like a leaf; but the spirit was
willing, and she persisted she was ready to go. However it turned out
that there was a labourer's cottage about a quarter of a mile off, and
she was finally dispatched there for assistance. Few people know the
ready humanity which exists among the lower orders: the man must have
run all the way to Oxford, for he returned in little more than half an
hour, before the surgeon could dress and mount his horse. However,
Chesterton was evidently still living; and when the surgeon did arrive
he gave some hopes of his recovery. The weight of the blow had been in
some degree broken by the gun which poor Harry had raised in his hand,
and this only could have saved the skull from fracture. Of course we had
soon plenty of volunteers who were ready to be useful in any way; and
when at last the police had made their appearance, and removed both the
living and the dead, and Chesterton had been laid in Brown's room, and
the surgeon, having applied the usual remedies, had composedly accepted
Mrs Nutt's offer to make up a bed for him, and betaken himself thereto,
as if such events were to him matters of everyday occurrence--I suppose
they were--it struck me, for the first time, that there wa

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Frederic W. <fwe...@gm...>]

On Wed, Sep 02, 2009 at 08:58:05AM -0400, Masami Hiramatsu wrote:
> Frederic Weisbecker wrote:
> > BTW, after your patchset:
> > 
> > TEST    posttest
> >   Building modules, stage 2.
> > Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
> > Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
> > make[1]: *** [posttest] Erreur 2
> > make: *** [bzImage] Erreur 2
> > make: *** Attente des tâches non terminées....
> >   MODPOST 4 modules
> > 
> > 
> > I'm cross compiling from 64 to 32 using:
> > 
> > make ARCH=x86
> > 
> > I've attached my config.
> 
> Aah, you might miss an important patch :D
> http://lkml.org/lkml/2009/8/28/367
> 
> Thank you,


I thought I applied it.
I'm really an idiot :-(

Sorry for the noise.


 
> -- 
> Masami Hiramatsu
> 
> Software Engineer
> Hitachi Computer Products (America), Inc.
> Software Solutions Division
> 
> e-mail: mhi...@re...
> 

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> BTW, after your patchset:
> 
> TEST    posttest
>   Building modules, stage 2.
> Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
> Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
> make[1]: *** [posttest] Erreur 2
> make: *** [bzImage] Erreur 2
> make: *** Attente des tâches non terminées....
>   MODPOST 4 modules
> 
> 
> I'm cross compiling from 64 to 32 using:
> 
> make ARCH=x86
> 
> I've attached my config.

Aah, you might miss an important patch :D
http://lkml.org/lkml/2009/8/28/367

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Masami Hiramatsu wrote:
> Frederic Weisbecker wrote:
>> BTW, after your patchset:
>>
>> TEST    posttest
>>    Building modules, stage 2.
>> Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
>> Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
>> make[1]: *** [posttest] Erreur 2
>> make: *** [bzImage] Erreur 2
>> make: *** Attente des tâches non terminées....
>>    MODPOST 4 modules
>>
>>
>> I'm cross compiling from 64 to 32 using:
>>
>> make ARCH=x86
>>
>> I've attached my config.
>
> Hmm, if you can, could you tell me the output of kernel build with V=1?
> I'd like to know whether posttest passed $(CONFIG_64BIT) or not.
>
> Thank you,
>
>>
>> Thanks.
>

Here is my result of 'make ARCH=x86' with your config-32 on x86-64.
---
   TEST    posttest
Succeed: decoded and checked 1735629 instructions
---

And also, I've tested test_get_len manually.

$ echo -e 'c0101038:\t2e a1 f8 b0 9e c0\t   mov    %cs:0xc09eb0f8,%eax' | arch/x86/tools/test_get_len
Succeed: decoded and checked 1 instructions

$ echo -e 'c0101038:\t2e a1 f8 b0 9e c0\t   mov    %cs:0xc09eb0f8,%eax' | arch/x86/tools/test_get_len y
Error: c0101038:        2e a1 f8 b0 9e c0          mov    %cs:0xc09eb0f8,%eax
Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)

So, I assume that CONFIG_64BIT=y in your test environment. (I'm not sure why...)

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [TOOL] c2kpe: C expression to kprobe event format converter

[Frederic W. <fwe...@gm...>]

On Mon, Aug 31, 2009 at 12:14:34AM -0400, Masami Hiramatsu wrote:
> Frederic Weisbecker wrote:
> > On Thu, Aug 13, 2009 at 04:59:19PM -0400, Masami Hiramatsu wrote:
> >> This program converts probe point in C expression to kprobe event
> >> format for kprobe-based event tracer. This helps to define kprobes
> >> events by C source line number or function name, and local variable
> >> name. Currently, this supports only x86(32/64) kernels.
> >>
> >>
> >> Compile
> >> --------
> >> Before compilation, please install libelf and libdwarf development
> >> packages.
> >> (e.g. elfutils-libelf-devel and libdwarf-devel on Fedora)
> > 
> > 
> > This may probably need a specific libdwarf version?
> > 
> > c2kpe.c: In function ‘die_get_entrypc’:
> > c2kpe.c:422: erreur: ‘Dwarf_Ranges’ undeclared (first use in this function)
> > c2kpe.c:422: erreur: (Each undeclared identifier is reported only once
> > c2kpe.c:422: erreur: for each function it appears in.)
> > c2kpe.c:422: erreur: ‘ranges’ undeclared (first use in this function)
> > c2kpe.c:447: attention : implicit declaration of function ‘dwarf_get_ranges’
> > c2kpe.c:451: attention : implicit declaration of function ‘dwarf_ranges_dealloc’
> 
> Aah, sure, it should be compiled with libdwarf newer than 20090324.
> You can find it in http://reality.sgiweb.org/davea/dwarf.html


Ah ok.

 
> BTW, libdwarf and libdw (which is the yet another implementation of
> dwarf library) are still under development, e.g. libdwarf doesn't
> support gcc-4.4.1(very new) and only the latest libdw(0.142) can
> support it. So, perhaps I might better port it on libdw, even that is
> less documented...:(



May be let's continue with libdwarf for now and we'll see if support
for libdw is required later.



> >> TODO
> >> ----
> >>  - Fix bugs.
> >>  - Support multiple probepoints from stdin.
> >>  - Better kmodule support.
> >>  - Use elfutils-libdw?
> >>  - Merge into trace-cmd or perf-tools?
> > 
> > 
> > Yeah definetly, that would be a veeery interesting thing to have.
> > I've played with kprobe ftrace to debug something this evening.
> > 
> > It's very cool to be able to put dynamic tracepoints in desired places.
> > 
> > But...
> > I firstly needed to put random trace_printk() in some places to
> > observe some variables values. And then I thought about the kprobes
> > tracer and realized I could do that without the need of rebuilding
> > my kernel. Then I've played with it and indeed it works well and
> > it's useful, but at the cost of reading objdump based assembly
> > code to find the places where I could find my variables values.
> > And after two or three probes in such conditions, I've become
> > tired of that, then I wanted to try this tool.
> > 
> > 
> > While I cannot yet because of this build error, I can imagine
> > the power of such facility from perf.
> > 
> > We could have a perf probe that creates a kprobe event in debugfs
> > (default enable = 0) and which then rely on perf record for the actual
> > recording.
> > 
> > Then we could analyse it through perf trace.
> > Let's imagine a simple example:
> > 
> > int foo(int arg1, int arg2)
> > {
> > 	int var1;
> > 
> > 	var1 = arg1;
> > 	var1 *= arg2;
> > 	var1 -= arg1;
> > 
> > ------> insert a probe here (file bar.c : line 60)
> > 
> > 	var1 ^= ...
> > 
> > 	return var1;
> > }
> > 
> > ./perf kprobe --file bar.c:60 --action "arg1=%d","arg2=%d","var1=%d" -- ls -R /
> 
> I recommend it should be separated from record, like below:
> 
> # set new event
> ./perf kprobe --add kprobe:event1 --file bar.c:60 --action "arg1=%d","arg2=%d","var1=%d"
> # record new event
> ./perf record -e kprobe:event1 -a -R -- ls -R /


That indeed solves the command line overkill, but that also
breaks a bit the workflow :)

Well, I guess we can start simple in the beginning and follow the above
mockup which is indeed better decoupled. 
And if something more intuitive comes in mind later, then we can still
change it.


> This will allow us to focus on one thing -- convert C to kprobe-tracer.
> And also, it can be listed as like as tracepoint events.


Yeah.


> > ./perf trace
> > 	arg1=1 arg2=1 var1=0
> > 	arg1=2 arg2=2 var1=2
> > 	etc..
> > 
> > You may want to sort by field:
> > 
> > ./perf trace -s arg1 --order desc
> >         arg1=1
> >           |
> >           ------- arg2=1 var=1
> >           |
> >           ------- arg2=2 var=1
> > 
> >         arg1=2
> >           |
> >           ------- arg2=1 var=0
> >           |
> >           ------- [...]
> > 
> > ./perf trace -s arg1,arg2 --order asc
> >         arg1=1
> >           |
> >           ------- arg2=1
> >                     |
> >                     --------- var1=0
> >                     |
> >                     --------- var1=....
> >                   arg2=...
> >                     |
> > 
> > Ok the latter is a bad example because var1 will always have only one
> > value for a given arg1 and arg2. But I guess you see the point.
> > 
> > You won't have to care about the perf trace part, it's already
> > implemented and I'll soon handle the sorting part.
> > 
> > All we need is the perf kprobes that translate a C level
> > probing expression to a /debug/tracing/kprobe_events compliant
> > thing. And then just call perf record with the new created
> > event as an argument.
> 
> Indeed, that's what I imagine.



Cool, thanks!




> Thank you,
> 
> -- 
> Masami Hiramatsu
> 
> Software Engineer
> Hitachi Computer Products (America), Inc.
> Software Solutions Division
> 
> e-mail: mhi...@re...
> 

Re: [Dle-develop] [TOOL] c2kpe: C expression to kprobe event format converter

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> On Thu, Aug 13, 2009 at 04:59:19PM -0400, Masami Hiramatsu wrote:
>> This program converts probe point in C expression to kprobe event
>> format for kprobe-based event tracer. This helps to define kprobes
>> events by C source line number or function name, and local variable
>> name. Currently, this supports only x86(32/64) kernels.
>>
>>
>> Compile
>> --------
>> Before compilation, please install libelf and libdwarf development
>> packages.
>> (e.g. elfutils-libelf-devel and libdwarf-devel on Fedora)
> 
> 
> This may probably need a specific libdwarf version?
> 
> c2kpe.c: In function ‘die_get_entrypc’:
> c2kpe.c:422: erreur: ‘Dwarf_Ranges’ undeclared (first use in this function)
> c2kpe.c:422: erreur: (Each undeclared identifier is reported only once
> c2kpe.c:422: erreur: for each function it appears in.)
> c2kpe.c:422: erreur: ‘ranges’ undeclared (first use in this function)
> c2kpe.c:447: attention : implicit declaration of function ‘dwarf_get_ranges’
> c2kpe.c:451: attention : implicit declaration of function ‘dwarf_ranges_dealloc’

Aah, sure, it should be compiled with libdwarf newer than 20090324.
You can find it in http://reality.sgiweb.org/davea/dwarf.html

BTW, libdwarf and libdw (which is the yet another implementation of
dwarf library) are still under development, e.g. libdwarf doesn't
support gcc-4.4.1(very new) and only the latest libdw(0.142) can
support it. So, perhaps I might better port it on libdw, even that is
less documented...:(

>> TODO
>> ----
>>  - Fix bugs.
>>  - Support multiple probepoints from stdin.
>>  - Better kmodule support.
>>  - Use elfutils-libdw?
>>  - Merge into trace-cmd or perf-tools?
> 
> 
> Yeah definetly, that would be a veeery interesting thing to have.
> I've played with kprobe ftrace to debug something this evening.
> 
> It's very cool to be able to put dynamic tracepoints in desired places.
> 
> But...
> I firstly needed to put random trace_printk() in some places to
> observe some variables values. And then I thought about the kprobes
> tracer and realized I could do that without the need of rebuilding
> my kernel. Then I've played with it and indeed it works well and
> it's useful, but at the cost of reading objdump based assembly
> code to find the places where I could find my variables values.
> And after two or three probes in such conditions, I've become
> tired of that, then I wanted to try this tool.
> 
> 
> While I cannot yet because of this build error, I can imagine
> the power of such facility from perf.
> 
> We could have a perf probe that creates a kprobe event in debugfs
> (default enable = 0) and which then rely on perf record for the actual
> recording.
> 
> Then we could analyse it through perf trace.
> Let's imagine a simple example:
> 
> int foo(int arg1, int arg2)
> {
> 	int var1;
> 
> 	var1 = arg1;
> 	var1 *= arg2;
> 	var1 -= arg1;
> 
> ------> insert a probe here (file bar.c : line 60)
> 
> 	var1 ^= ...
> 
> 	return var1;
> }
> 
> ./perf kprobe --file bar.c:60 --action "arg1=%d","arg2=%d","var1=%d" -- ls -R /

I recommend it should be separated from record, like below:

# set new event
./perf kprobe --add kprobe:event1 --file bar.c:60 --action "arg1=%d","arg2=%d","var1=%d"
# record new event
./perf record -e kprobe:event1 -a -R -- ls -R /

This will allow us to focus on one thing -- convert C to kprobe-tracer.
And also, it can be listed as like as tracepoint events.

> ./perf trace
> 	arg1=1 arg2=1 var1=0
> 	arg1=2 arg2=2 var1=2
> 	etc..
> 
> You may want to sort by field:
> 
> ./perf trace -s arg1 --order desc
>         arg1=1
>           |
>           ------- arg2=1 var=1
>           |
>           ------- arg2=2 var=1
> 
>         arg1=2
>           |
>           ------- arg2=1 var=0
>           |
>           ------- [...]
> 
> ./perf trace -s arg1,arg2 --order asc
>         arg1=1
>           |
>           ------- arg2=1
>                     |
>                     --------- var1=0
>                     |
>                     --------- var1=....
>                   arg2=...
>                     |
> 
> Ok the latter is a bad example because var1 will always have only one
> value for a given arg1 and arg2. But I guess you see the point.
> 
> You won't have to care about the perf trace part, it's already
> implemented and I'll soon handle the sorting part.
> 
> All we need is the perf kprobes that translate a C level
> probing expression to a /debug/tracing/kprobe_events compliant
> thing. And then just call perf record with the new created
> event as an argument.

Indeed, that's what I imagine.

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> BTW, after your patchset:
> 
> TEST    posttest
>   Building modules, stage 2.
> Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
> Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
> make[1]: *** [posttest] Erreur 2
> make: *** [bzImage] Erreur 2
> make: *** Attente des tâches non terminées....
>   MODPOST 4 modules
> 
> 
> I'm cross compiling from 64 to 32 using:
> 
> make ARCH=x86
> 
> I've attached my config.

Hmm, if you can, could you tell me the output of kernel build with V=1?
I'd like to know whether posttest passed $(CONFIG_64BIT) or not.

Thank you,

> 
> Thanks.

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [TOOL] c2kpe: C expression to kprobe event format converter

[Frederic W. <fwe...@gm...>]

On Thu, Aug 13, 2009 at 04:59:19PM -0400, Masami Hiramatsu wrote:
> This program converts probe point in C expression to kprobe event
> format for kprobe-based event tracer. This helps to define kprobes
> events by C source line number or function name, and local variable
> name. Currently, this supports only x86(32/64) kernels.
>
>
> Compile
> --------
> Before compilation, please install libelf and libdwarf development
> packages.
> (e.g. elfutils-libelf-devel and libdwarf-devel on Fedora)


This may probably need a specific libdwarf version?

c2kpe.c: In function ‘die_get_entrypc’:
c2kpe.c:422: erreur: ‘Dwarf_Ranges’ undeclared (first use in this function)
c2kpe.c:422: erreur: (Each undeclared identifier is reported only once
c2kpe.c:422: erreur: for each function it appears in.)
c2kpe.c:422: erreur: ‘ranges’ undeclared (first use in this function)
c2kpe.c:447: attention : implicit declaration of function ‘dwarf_get_ranges’
c2kpe.c:451: attention : implicit declaration of function ‘dwarf_ranges_dealloc’


> TODO
> ----
>  - Fix bugs.
>  - Support multiple probepoints from stdin.
>  - Better kmodule support.
>  - Use elfutils-libdw?
>  - Merge into trace-cmd or perf-tools?


Yeah definetly, that would be a veeery interesting thing to have.
I've played with kprobe ftrace to debug something this evening.

It's very cool to be able to put dynamic tracepoints in desired places.

But...
I firstly needed to put random trace_printk() in some places to
observe some variables values. And then I thought about the kprobes
tracer and realized I could do that without the need of rebuilding
my kernel. Then I've played with it and indeed it works well and
it's useful, but at the cost of reading objdump based assembly
code to find the places where I could find my variables values.
And after two or three probes in such conditions, I've become
tired of that, then I wanted to try this tool.


While I cannot yet because of this build error, I can imagine
the power of such facility from perf.

We could have a perf probe that creates a kprobe event in debugfs
(default enable = 0) and which then rely on perf record for the actual
recording.

Then we could analyse it through perf trace.
Let's imagine a simple example:

int foo(int arg1, int arg2)
{
	int var1;

	var1 = arg1;
	var1 *= arg2;
	var1 -= arg1;

------> insert a probe here (file bar.c : line 60)

	var1 ^= ...

	return var1;
}

./perf kprobe --file bar.c:60 --action "arg1=%d","arg2=%d","var1=%d" -- ls -R /
./perf trace
	arg1=1 arg2=1 var1=0
	arg1=2 arg2=2 var1=2
	etc..

You may want to sort by field:

./perf trace -s arg1 --order desc
        arg1=1
          |
          ------- arg2=1 var=1
          |
          ------- arg2=2 var=1

        arg1=2
          |
          ------- arg2=1 var=0
          |
          ------- [...]

./perf trace -s arg1,arg2 --order asc
        arg1=1
          |
          ------- arg2=1
                    |
                    --------- var1=0
                    |
                    --------- var1=....
                  arg2=...
                    |

Ok the latter is a bad example because var1 will always have only one
value for a given arg1 and arg2. But I guess you see the point.

You won't have to care about the perf trace part, it's already
implemented and I'll soon handle the sorting part.

All we need is the perf kprobes that translate a C level
probing expression to a /debug/tracing/kprobe_events compliant
thing. And then just call perf record with the new created
event as an argument.

Frederic.

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Frederic W. <fwe...@gm...>]

On Sat, Aug 29, 2009 at 10:49:47PM -0400, Masami Hiramatsu wrote:
> Frederic Weisbecker wrote:
> > On Thu, Aug 27, 2009 at 01:23:11PM -0400, Masami Hiramatsu wrote:
> >> Add __kprobes to the functions which handles in-kernel fixable page faults.
> >> Since kprobes can cause those in-kernel page faults by accessing kprobe data
> >> structures, probing those fault functions will cause fault-int3-loop
> >> (do_page_fault has already been marked as __kprobes).
> > 
> > 
> > BTW, is it possible to assert a struct kprobe (and its pointer members)
> > haven't been allocated through vmalloc() ?
> > 
> 
> How about calling vmalloc_sync() always before registering kprobes? :-)
> 
> Thank you,


Sounds good :)

BTW, after your patchset:

TEST    posttest
  Building modules, stage 2.
Error: c0101038:	2e a1 f8 b0 9e c0    	mov    %cs:0xc09eb0f8,%eax
Error: objdump says 6 bytes, but insn_get_length() says 10 (attr:40000)
make[1]: *** [posttest] Erreur 2
make: *** [bzImage] Erreur 2
make: *** Attente des tâches non terminées....
  MODPOST 4 modules


I'm cross compiling from 64 to 32 using:

make ARCH=x86

I've attached my config.

Thanks.

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> On Thu, Aug 27, 2009 at 01:23:11PM -0400, Masami Hiramatsu wrote:
>> Add __kprobes to the functions which handles in-kernel fixable page faults.
>> Since kprobes can cause those in-kernel page faults by accessing kprobe data
>> structures, probing those fault functions will cause fault-int3-loop
>> (do_page_fault has already been marked as __kprobes).
> 
> 
> BTW, is it possible to assert a struct kprobe (and its pointer members)
> haven't been allocated through vmalloc() ?
> 

How about calling vmalloc_sync() always before registering kprobes? :-)

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> On Thu, Aug 27, 2009 at 01:23:11PM -0400, Masami Hiramatsu wrote:
>> Add __kprobes to the functions which handles in-kernel fixable page faults.
>> Since kprobes can cause those in-kernel page faults by accessing kprobe data
>> structures, probing those fault functions will cause fault-int3-loop
>> (do_page_fault has already been marked as __kprobes).
>>
>> Signed-off-by: Masami Hiramatsu <mhi...@re...>
>> Cc: Frederic Weisbecker <fwe...@gm...>
>> Cc: Ananth N Mavinakayanahalli <an...@in...>
>> Cc: Ingo Molnar <mi...@el...>
>> ---
>>
>>  arch/x86/mm/fault.c |   11 ++++++-----
>>  1 files changed, 6 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
>> index bfae139..c322e59 100644
>> --- a/arch/x86/mm/fault.c
>> +++ b/arch/x86/mm/fault.c
>> @@ -38,7 +38,8 @@ enum x86_pf_error_code {
>>   * Returns 0 if mmiotrace is disabled, or if the fault is not
>>   * handled by mmiotrace:
>>   */
>> -static inline int kmmio_fault(struct pt_regs *regs, unsigned long addr)
>> +static inline int __kprobes
>> +kmmio_fault(struct pt_regs *regs, unsigned long addr)
>>  {
>>  	if (unlikely(is_kmmio_active()))
>>  		if (kmmio_handler(regs, addr) == 1)
> 
> 
> I guess the problem also resides in a lot of subfunctions such as kmmio_handler
> and such...

Hmm, sure.
kmmio functions should be marked as __kprobes( or __kmmio or __debug? :)).

Thank you,

>> @@ -46,7 +47,7 @@ static inline int kmmio_fault(struct pt_regs *regs, unsigned long addr)
>>  	return 0;
>>  }
>>  
>> -static inline int notify_page_fault(struct pt_regs *regs)
>> +static inline int __kprobes notify_page_fault(struct pt_regs *regs)
>>  {
>>  	int ret = 0;
>>  
>> @@ -239,7 +240,7 @@ void vmalloc_sync_all(void)
>>   *
>>   *   Handle a fault on the vmalloc or module mapping area
>>   */
>> -static noinline int vmalloc_fault(unsigned long address)
>> +static noinline __kprobes int vmalloc_fault(unsigned long address)
>>  {
>>  	unsigned long pgd_paddr;
>>  	pmd_t *pmd_k;
>> @@ -361,7 +362,7 @@ void vmalloc_sync_all(void)
>>   *
>>   * This assumes no large pages in there.
>>   */
>> -static noinline int vmalloc_fault(unsigned long address)
>> +static noinline __kprobes int vmalloc_fault(unsigned long address)
>>  {
>>  	pgd_t *pgd, *pgd_ref;
>>  	pud_t *pud, *pud_ref;
>> @@ -858,7 +859,7 @@ static int spurious_fault_check(unsigned long error_code, pte_t *pte)
>>   * There are no security implications to leaving a stale TLB when
>>   * increasing the permissions on a page.
>>   */
>> -static noinline int
>> +static noinline __kprobes int
>>  spurious_fault(unsigned long error_code, unsigned long address)
>>  {
>>  	pgd_t *pgd;
>>
>>
>> -- 
>> Masami Hiramatsu
>>
>> Software Engineer
>> Hitachi Computer Products (America), Inc.
>> Software Solutions Division
>>
>> e-mail: mhi...@re...
> 

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 1/2] x86: Allow x86-32 instruction decoder selftest on x86-64

[Frederic W. <fwe...@gm...>]

On Fri, Aug 28, 2009 at 06:13:19PM -0400, Masami Hiramatsu wrote:
> Pass $(CONFIG_64BIT) to the x86 insn decoder selftest for decoding 32bit code
> on x86-64, which will happen when building kernel with ARCH=i386 on x86-64.
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Jim Keniston <jke...@us...>
> Cc: Ingo Molnar <mi...@el...>



Also applied those two, thanks!

Re: [Dle-develop] [PATCH -tip tracing/kprobes 1/6] kprobes/x86: Call BUG() when reentering probe into KPROBES_HIT_SS

[Frederic W. <fwe...@gm...>]

On Thu, Aug 27, 2009 at 01:22:58PM -0400, Masami Hiramatsu wrote:
> Call BUG() when a probe have been hit on the way of kprobe processing path,
> because that kind of probes are currently unrecoverable (recovering it will
> cause an infinit loop and stack overflow).
> 
> The original code seems to assume that it will be caused by an int3 which
> another subsystem inserted on out-of-line singlestep buffer if the hitting
> probe is same as current probe. However, in that case, int3-hitting-address
> is on the out-of-line buffer and should be different from first (current)
> int3 address.
> Thus, I decided to remove the code.
> 
> I also removes arch_disarm_kprobe() because it will involve other stuffs
> in text_poke().
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>



Applied the six patches to

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
	tracing/kprobes

Thanks!

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Frederic W. <fwe...@gm...>]

On Thu, Aug 27, 2009 at 01:23:11PM -0400, Masami Hiramatsu wrote:
> Add __kprobes to the functions which handles in-kernel fixable page faults.
> Since kprobes can cause those in-kernel page faults by accessing kprobe data
> structures, probing those fault functions will cause fault-int3-loop
> (do_page_fault has already been marked as __kprobes).


BTW, is it possible to assert a struct kprobe (and its pointer members)
haven't been allocated through vmalloc() ?

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Frederic W. <fwe...@gm...>]

On Thu, Aug 27, 2009 at 01:23:11PM -0400, Masami Hiramatsu wrote:
> Add __kprobes to the functions which handles in-kernel fixable page faults.
> Since kprobes can cause those in-kernel page faults by accessing kprobe data
> structures, probing those fault functions will cause fault-int3-loop
> (do_page_fault has already been marked as __kprobes).
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>
> ---
> 
>  arch/x86/mm/fault.c |   11 ++++++-----
>  1 files changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index bfae139..c322e59 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -38,7 +38,8 @@ enum x86_pf_error_code {
>   * Returns 0 if mmiotrace is disabled, or if the fault is not
>   * handled by mmiotrace:
>   */
> -static inline int kmmio_fault(struct pt_regs *regs, unsigned long addr)
> +static inline int __kprobes
> +kmmio_fault(struct pt_regs *regs, unsigned long addr)
>  {
>  	if (unlikely(is_kmmio_active()))
>  		if (kmmio_handler(regs, addr) == 1)


I guess the problem also resides in a lot of subfunctions such as kmmio_handler
and such...



> @@ -46,7 +47,7 @@ static inline int kmmio_fault(struct pt_regs *regs, unsigned long addr)
>  	return 0;
>  }
>  
> -static inline int notify_page_fault(struct pt_regs *regs)
> +static inline int __kprobes notify_page_fault(struct pt_regs *regs)
>  {
>  	int ret = 0;
>  
> @@ -239,7 +240,7 @@ void vmalloc_sync_all(void)
>   *
>   *   Handle a fault on the vmalloc or module mapping area
>   */
> -static noinline int vmalloc_fault(unsigned long address)
> +static noinline __kprobes int vmalloc_fault(unsigned long address)
>  {
>  	unsigned long pgd_paddr;
>  	pmd_t *pmd_k;
> @@ -361,7 +362,7 @@ void vmalloc_sync_all(void)
>   *
>   * This assumes no large pages in there.
>   */
> -static noinline int vmalloc_fault(unsigned long address)
> +static noinline __kprobes int vmalloc_fault(unsigned long address)
>  {
>  	pgd_t *pgd, *pgd_ref;
>  	pud_t *pud, *pud_ref;
> @@ -858,7 +859,7 @@ static int spurious_fault_check(unsigned long error_code, pte_t *pte)
>   * There are no security implications to leaving a stale TLB when
>   * increasing the permissions on a page.
>   */
> -static noinline int
> +static noinline __kprobes int
>  spurious_fault(unsigned long error_code, unsigned long address)
>  {
>  	pgd_t *pgd;
> 
> 
> -- 
> Masami Hiramatsu
> 
> Software Engineer
> Hitachi Computer Products (America), Inc.
> Software Solutions Division
> 
> e-mail: mhi...@re...

[Dle-develop] l and prop

[Buttray M. <sch...@a-...>]

Ked to him, and got him to be reconciled to his enemy. It all happened
when I was a child, and I never just knew the rights of it. But I know
that father was very glad when Mr. Hassel sold his farm joining ours,
and bought another at the foot of Spring Mountain, where he has lived
ever since. It troubled me very much that our folks felt so set against
the family; for Ned was the best-looking young man in our place, and had
such a dashing sort of a way with him, that he took my fancy
considerable, and I must confess I was rather blind to his faults. I
went to sleep that night with my head full of the picnic, and dreamed
that I rode up the mountain on Ned's Lightning, and just as I got to the
steepest part, the horse gave a jump, and tumbled me over i

[Dle-develop] [PATCH -tip tracing/kprobes 2/2] x86: Remove unused config macros from instruction decoder selftest

[Masami H. <mhi...@re...>]

Remove dummy definitions of CONFIG_X86_64 and CONFIG_X86_32 because those
macros are not used in the instruction decoder anymore.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Jim Keniston <jke...@us...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/tools/test_get_len.c |    5 -----
 1 files changed, 0 insertions(+), 5 deletions(-)

diff --git a/arch/x86/tools/test_get_len.c b/arch/x86/tools/test_get_len.c
index a3273f4..376d338 100644
--- a/arch/x86/tools/test_get_len.c
+++ b/arch/x86/tools/test_get_len.c
@@ -21,11 +21,6 @@
 #include <string.h>
 #include <assert.h>
 
-#ifdef __x86_64__
-#define CONFIG_X86_64
-#else
-#define CONFIG_X86_32
-#endif
 #define unlikely(cond) (cond)
 
 #include <asm/insn.h>


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...