dle-develop — Dependable Linux Effort mailing list for developers

[Dle-develop] [PATCH -tip tracing/kprobes 1/2] x86: Allow x86-32 instruction decoder selftest on x86-64

[Masami H. <mhi...@re...>]

Pass $(CONFIG_64BIT) to the x86 insn decoder selftest for decoding 32bit code
on x86-64, which will happen when building kernel with ARCH=i386 on x86-64.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Jim Keniston <jke...@us...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/tools/Makefile       |    2 +-
 arch/x86/tools/test_get_len.c |   14 +++++++-------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/x86/tools/Makefile b/arch/x86/tools/Makefile
index 95e9cc4..1bd006c 100644
--- a/arch/x86/tools/Makefile
+++ b/arch/x86/tools/Makefile
@@ -1,6 +1,6 @@
 PHONY += posttest
 quiet_cmd_posttest = TEST    $@
-      cmd_posttest = $(OBJDUMP) -d -j .text $(objtree)/vmlinux | awk -f $(srctree)/arch/x86/tools/distill.awk | $(obj)/test_get_len
+      cmd_posttest = $(OBJDUMP) -d -j .text $(objtree)/vmlinux | awk -f $(srctree)/arch/x86/tools/distill.awk | $(obj)/test_get_len $(CONFIG_64BIT)
 
 posttest: $(obj)/test_get_len vmlinux
 	$(call cmd,posttest)
diff --git a/arch/x86/tools/test_get_len.c b/arch/x86/tools/test_get_len.c
index 1e81adb..a3273f4 100644
--- a/arch/x86/tools/test_get_len.c
+++ b/arch/x86/tools/test_get_len.c
@@ -45,7 +45,7 @@ const char *prog;
 static void usage(void)
 {
 	fprintf(stderr, "Usage: objdump -d a.out | awk -f distill.awk |"
-		" ./test_get_len\n");
+		" %s [y|n](64bit flag)\n", prog);
 	exit(1);
 }
 
@@ -63,11 +63,15 @@ int main(int argc, char **argv)
 	unsigned char insn_buf[16];
 	struct insn insn;
 	int insns = 0;
+	int x86_64 = 0;
 
 	prog = argv[0];
-	if (argc > 1)
+	if (argc > 2)
 		usage();
 
+	if (argc == 2 && argv[1][0] == 'y')
+		x86_64 = 1;
+
 	while (fgets(line, BUFSIZE, stdin)) {
 		char copy[BUFSIZE], *s, *tab1, *tab2;
 		int nb = 0;
@@ -93,11 +97,7 @@ int main(int argc, char **argv)
 				break;
 		}
 		/* Decode an instruction */
-#ifdef __x86_64__
-		insn_init(&insn, insn_buf, 1);
-#else
-		insn_init(&insn, insn_buf, 0);
-#endif
+		insn_init(&insn, insn_buf, x86_64);
 		insn_get_length(&insn);
 		if (insn.length != nb) {
 			fprintf(stderr, "Error: %s", line);


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 4/6] kprobes: Fix to add __kprobes to notify_die

[Ananth N M. <an...@in...>]

On Thu, Aug 27, 2009 at 01:23:18PM -0400, Masami Hiramatsu wrote:
> Add __kprobes to notify_die() because do_int3() calls notify_die() instead of
> atomic_notify_call_chain() which is already marked as __kprobes.
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>

Acked-by: Ananth N Mavinakayanahalli <an...@in...>

Re: [Dle-develop] [PATCH -tip tracing/kprobes 2/6] kprobes/x86-64: Allow to reenter probe on post_handler

[Ananth N M. <an...@in...>]

On Thu, Aug 27, 2009 at 01:23:04PM -0400, Masami Hiramatsu wrote:
> Allow to reenter probe on the post_handler of another probe on x86-64,
> because x86-64 already allows reentering int3.
> In that case, reentered probe just increases kp.nmissed and returns.
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>
> ---

Acked-by: Ananth N Mavinakayanahalli <an...@in...>

Re: [Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Ananth N M. <an...@in...>]

On Thu, Aug 27, 2009 at 01:23:11PM -0400, Masami Hiramatsu wrote:
> Add __kprobes to the functions which handles in-kernel fixable page faults.
> Since kprobes can cause those in-kernel page faults by accessing kprobe data
> structures, probing those fault functions will cause fault-int3-loop
> (do_page_fault has already been marked as __kprobes).
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>
> ---

Acked-by: Ananth N Mavinakayanahalli <an...@in...>

Re: [Dle-develop] [PATCH -tip tracing/kprobes 1/6] kprobes/x86: Call BUG() when reentering probe into KPROBES_HIT_SS

[Ananth N M. <an...@in...>]

On Thu, Aug 27, 2009 at 01:22:58PM -0400, Masami Hiramatsu wrote:
> Call BUG() when a probe have been hit on the way of kprobe processing path,
> because that kind of probes are currently unrecoverable (recovering it will
> cause an infinit loop and stack overflow).
> 
> The original code seems to assume that it will be caused by an int3 which
> another subsystem inserted on out-of-line singlestep buffer if the hitting
> probe is same as current probe. However, in that case, int3-hitting-address
> is on the out-of-line buffer and should be different from first (current)
> int3 address.
> Thus, I decided to remove the code.
> 
> I also removes arch_disarm_kprobe() because it will involve other stuffs
> in text_poke().
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>

Acked-by: Ananth N Mavinakayanahalli <an...@in...>

Re: [Dle-develop] [PATCH -tip tracing/kprobes 6/6] kprobes: Prohibit to probe native_get_debugreg

[Ananth N M. <an...@in...>]

On Thu, Aug 27, 2009 at 01:23:32PM -0400, Masami Hiramatsu wrote:
> Since do_debug() calls get_debugreg(), native_get_debugreg() will be called
> from singlestepping. This can cause an int3 infinit loop.
> 
> Signed-off-by: Masami Hiramatsu <mhi...@re...>
> Cc: Frederic Weisbecker <fwe...@gm...>
> Cc: Ananth N Mavinakayanahalli <an...@in...>
> Cc: Ingo Molnar <mi...@el...>
> ---

Acked-by: Ananth N Mavinakayanahalli <an...@in...>

[Dle-develop] [PATCH -tip tracing/kprobes 5/6] kprobes/x86-64: Fix to move common_interrupt to .kprobes.text

[Masami H. <mhi...@re...>]

Since nmi, debug and int3 returns to irq_return in the common_interrupt,
probing this function will cause int3-loop, so it should be marked
as __kprobes.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/kernel/entry_64.S |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
index c251be7..36e2ef5 100644
--- a/arch/x86/kernel/entry_64.S
+++ b/arch/x86/kernel/entry_64.S
@@ -809,6 +809,10 @@ END(interrupt)
 	call \func
 	.endm
 
+/*
+ * Interrupt entry/exit should be protected against kprobes
+ */
+	.pushsection .kprobes.text, "ax"
 	/*
 	 * The interrupt stubs push (~vector+0x80) onto the stack and
 	 * then jump to common_interrupt.
@@ -947,6 +951,10 @@ ENTRY(retint_kernel)
 
 	CFI_ENDPROC
 END(common_interrupt)
+/*
+ * End of kprobes section
+ */
+       .popsection
 
 /*
  * APIC interrupts.


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip tracing/kprobes 6/6] kprobes: Prohibit to probe native_get_debugreg

[Masami H. <mhi...@re...>]

Since do_debug() calls get_debugreg(), native_get_debugreg() will be called
from singlestepping. This can cause an int3 infinit loop.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Ingo Molnar <mi...@el...>
---

 kernel/kprobes.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index f72e96c..3267d90 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -90,6 +90,7 @@ static spinlock_t *kretprobe_table_lock_ptr(unsigned long hash)
  */
 static struct kprobe_blackpoint kprobe_blacklist[] = {
 	{"preempt_schedule",},
+	{"native_get_debugreg",},
 	{NULL}    /* Terminator */
 };
 


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip tracing/kprobes 4/6] kprobes: Fix to add __kprobes to notify_die

[Masami H. <mhi...@re...>]

Add __kprobes to notify_die() because do_int3() calls notify_die() instead of
atomic_notify_call_chain() which is already marked as __kprobes.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Ingo Molnar <mi...@el...>
---

 kernel/notifier.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/kernel/notifier.c b/kernel/notifier.c
index 61d5aa5..acd24e7 100644
--- a/kernel/notifier.c
+++ b/kernel/notifier.c
@@ -558,7 +558,7 @@ EXPORT_SYMBOL(unregister_reboot_notifier);
 
 static ATOMIC_NOTIFIER_HEAD(die_chain);
 
-int notrace notify_die(enum die_val val, const char *str,
+int notrace __kprobes notify_die(enum die_val val, const char *str,
 	       struct pt_regs *regs, long err, int trap, int sig)
 {
 	struct die_args args = {


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip tracing/kprobes 1/6] kprobes/x86: Call BUG() when reentering probe into KPROBES_HIT_SS

[Masami H. <mhi...@re...>]

Call BUG() when a probe have been hit on the way of kprobe processing path,
because that kind of probes are currently unrecoverable (recovering it will
cause an infinit loop and stack overflow).

The original code seems to assume that it will be caused by an int3 which
another subsystem inserted on out-of-line singlestep buffer if the hitting
probe is same as current probe. However, in that case, int3-hitting-address
is on the out-of-line buffer and should be different from first (current)
int3 address.
Thus, I decided to remove the code.

I also removes arch_disarm_kprobe() because it will involve other stuffs
in text_poke().

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/kernel/kprobes.c |   26 ++++++++++----------------
 1 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kernel/kprobes.c b/arch/x86/kernel/kprobes.c
index ecee3d2..e0fb615 100644
--- a/arch/x86/kernel/kprobes.c
+++ b/arch/x86/kernel/kprobes.c
@@ -482,22 +482,16 @@ static int __kprobes reenter_kprobe(struct kprobe *p, struct pt_regs *regs,
 		kcb->kprobe_status = KPROBE_REENTER;
 		break;
 	case KPROBE_HIT_SS:
-		if (p == kprobe_running()) {
-			regs->flags &= ~X86_EFLAGS_TF;
-			regs->flags |= kcb->kprobe_saved_flags;
-			return 0;
-		} else {
-			/* A probe has been hit in the codepath leading up
-			 * to, or just after, single-stepping of a probed
-			 * instruction. This entire codepath should strictly
-			 * reside in .kprobes.text section.
-			 * Raise a BUG or we'll continue in an endless
-			 * reentering loop and eventually a stack overflow.
-			 */
-			arch_disarm_kprobe(p);
-			dump_kprobe(p);
-			BUG();
-		}
+		/* A probe has been hit in the codepath leading up to, or just
+		 * after, single-stepping of a probed instruction. This entire
+		 * codepath should strictly reside in .kprobes.text section.
+		 * Raise a BUG or we'll continue in an endless reentering loop
+		 * and eventually a stack overflow.
+		 */
+		printk(KERN_WARNING "Unrecoverable kprobe detected at %p.\n",
+		       p->addr);
+		dump_kprobe(p);
+		BUG();
 	default:
 		/* impossible cases */
 		WARN_ON(1);


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip tracing/kprobes 3/6] kprobes/x86: Fix to add __kprobes to in-kernel fault handing functions

[Masami H. <mhi...@re...>]

Add __kprobes to the functions which handles in-kernel fixable page faults.
Since kprobes can cause those in-kernel page faults by accessing kprobe data
structures, probing those fault functions will cause fault-int3-loop
(do_page_fault has already been marked as __kprobes).

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/mm/fault.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index bfae139..c322e59 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -38,7 +38,8 @@ enum x86_pf_error_code {
  * Returns 0 if mmiotrace is disabled, or if the fault is not
  * handled by mmiotrace:
  */
-static inline int kmmio_fault(struct pt_regs *regs, unsigned long addr)
+static inline int __kprobes
+kmmio_fault(struct pt_regs *regs, unsigned long addr)
 {
 	if (unlikely(is_kmmio_active()))
 		if (kmmio_handler(regs, addr) == 1)
@@ -46,7 +47,7 @@ static inline int kmmio_fault(struct pt_regs *regs, unsigned long addr)
 	return 0;
 }
 
-static inline int notify_page_fault(struct pt_regs *regs)
+static inline int __kprobes notify_page_fault(struct pt_regs *regs)
 {
 	int ret = 0;
 
@@ -239,7 +240,7 @@ void vmalloc_sync_all(void)
  *
  *   Handle a fault on the vmalloc or module mapping area
  */
-static noinline int vmalloc_fault(unsigned long address)
+static noinline __kprobes int vmalloc_fault(unsigned long address)
 {
 	unsigned long pgd_paddr;
 	pmd_t *pmd_k;
@@ -361,7 +362,7 @@ void vmalloc_sync_all(void)
  *
  * This assumes no large pages in there.
  */
-static noinline int vmalloc_fault(unsigned long address)
+static noinline __kprobes int vmalloc_fault(unsigned long address)
 {
 	pgd_t *pgd, *pgd_ref;
 	pud_t *pud, *pud_ref;
@@ -858,7 +859,7 @@ static int spurious_fault_check(unsigned long error_code, pte_t *pte)
  * There are no security implications to leaving a stale TLB when
  * increasing the permissions on a page.
  */
-static noinline int
+static noinline __kprobes int
 spurious_fault(unsigned long error_code, unsigned long address)
 {
 	pgd_t *pgd;


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip tracing/kprobes 2/6] kprobes/x86-64: Allow to reenter probe on post_handler

[Masami H. <mhi...@re...>]

Allow to reenter probe on the post_handler of another probe on x86-64,
because x86-64 already allows reentering int3.
In that case, reentered probe just increases kp.nmissed and returns.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/kernel/kprobes.c |   11 -----------
 1 files changed, 0 insertions(+), 11 deletions(-)

diff --git a/arch/x86/kernel/kprobes.c b/arch/x86/kernel/kprobes.c
index e0fb615..c5f1f11 100644
--- a/arch/x86/kernel/kprobes.c
+++ b/arch/x86/kernel/kprobes.c
@@ -463,17 +463,6 @@ static int __kprobes reenter_kprobe(struct kprobe *p, struct pt_regs *regs,
 {
 	switch (kcb->kprobe_status) {
 	case KPROBE_HIT_SSDONE:
-#ifdef CONFIG_X86_64
-		/* TODO: Provide re-entrancy from post_kprobes_handler() and
-		 * avoid exception stack corruption while single-stepping on
-		 * the instruction of the new probe.
-		 */
-		arch_disarm_kprobe(p);
-		regs->ip = (unsigned long)p->addr;
-		reset_current_kprobe();
-		preempt_enable_no_resched();
-		break;
-#endif
 	case KPROBE_HIT_ACTIVE:
 		save_previous_kprobe(kcb);
 		set_current_kprobe(p, regs, kcb);


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] Eam had crippled eleven other boys from an

[Gaden <ef...@tb...>]

 on in life, when I have found it desirable to emphasize my authority in
my home by means of a stout stick, that emblem of power has had no
glamor about it to weaken its force as an argument.... Then as to the
number of wives that a man should be permitted to have, I am in distinct
disagreement with the majority of my neighbors, who maintain that it is
entirely a matter of individual choice as to whether a man should have
five, ten or a thousand. I should not advocate the limitation to an
arbitrary number, but I believe that the question of one's actual needs
should rule. If a man's possessions enable him to maintain a large
establishment requiring the services of a cook, a laundress, two
waitresses and four upstairs girls, eight wives would be sufficient; but
on the other hand, for a young man beginning his career who needs only a
general house-worker, one is enough. Individual cases should regulate
the law as applied to the individual, and those who claim that they may
marry any number of women, whether they need them or not, entirely
regardless of whether or not they can keep them occupied, should be told
that no man is entitled to more of the good things of this life than he
can avail himself of in his daily procedure. Any other course than this
will sooner or later result in a great scarcity of nuptial raw material,
and it is not impossible to conceive of a day when all the women in the
land will become the property of a select, privileged few. A monopoly of
this sort would enable a few men to control posterity and build up a
Trust in the Matrimonial Industry that would engender not only a great
deal of bitter feeling between the masses and the classes, but enforce a
system of compulsory bachelorhood which ... Nevertheless, if woman wants
to vote let her do so. In spite o

Re: [Dle-develop] [PATCH tracing/kprobes 1/4] x86: Fix x86 instruction decoder selftest to check only .text

[Frederic W. <fwe...@gm...>]

On Fri, Aug 21, 2009 at 03:43:07PM -0400, Masami Hiramatsu wrote:
> Fix x86 instruction decoder selftest to check only .text because other
> sections (e.g. .notes) will have random bytes which don't need to be checked.


Applied these 4 patches in

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git \
	tracing/kprobes

Thanks!

[Dle-develop] [tip:perfcounters/tracing] perf trace: Add OPT_END to option array of perf-trace

[tip-bot f. M. H. <mhi...@re...>]

Commit-ID:  1909629fb1ec9800cf2cb0091870d6a1c1ca665f
Gitweb:     http://git.kernel.org/tip/1909629fb1ec9800cf2cb0091870d6a1c1ca665f
Author:     Masami Hiramatsu <mhi...@re...>
AuthorDate: Fri, 21 Aug 2009 14:56:03 -0400
Committer:  Ingo Molnar <mi...@el...>
CommitDate: Fri, 21 Aug 2009 21:42:43 +0200

perf trace: Add OPT_END to option array of perf-trace

Add OPT_END to option array of perf-trace for fixing a SEGV bug when
showing perf-trace help message.

Without this patch;
 ./perf trace -h

 usage: perf trace [<options>] <command>

    -D, --dump-raw-trace  dump raw trace in ASCII
    -v, --verbose         be more verbose (show symbol address, etc)
    -f, Segmentation fault

With this patch:
 ./perf trace -h

 usage: perf trace [<options>] <command>

    -D, --dump-raw-trace  dump raw trace in ASCII
    -v, --verbose         be more verbose (show symbol address, etc)

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: systemtap <sys...@so...>
Cc: DLE <dle...@li...>
Cc: Masami Hiramatsu <mhi...@re...>
Cc: Steven Rostedt <ro...@go...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Peter Zijlstra <pe...@in...>
Cc: Arnaldo Carvalho de Melo <ac...@re...>
Cc: Mike Galbraith <ef...@gm...>
Cc: Xiao Guangrong <xia...@cn...>
Cc: Zhaolei <zh...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Tom Zanussi <tza...@gm...>
LKML-Reference: <200...@lo...>
Signed-off-by: Ingo Molnar <mi...@el...>


---
 tools/perf/builtin-trace.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 88eef71..dd3c2e7 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -257,6 +257,7 @@ static const struct option options[] = {
 		    "dump raw trace in ASCII"),
 	OPT_BOOLEAN('v', "verbose", &verbose,
 		    "be more verbose (show symbol address, etc)"),
+	OPT_END()
 };
 
 int cmd_trace(int argc, const char **argv, const char *prefix __used)

Re: [Dle-develop] [BUGFIX PATCH -tip] perf tools: Add OPT_END to option array of perf-trace

[Ingo M. <mi...@el...>]

* Masami Hiramatsu <mhi...@re...> wrote:

> Add OPT_END to option array of perf-trace for fixing a SEGV bug 
> when showing perf-trace help message.

applied, thanks!

	Ingo

[Dle-develop] [PATCH tracing/kprobes 4/4] tracing/kprobes: Change trace_arg to probe_arg

[Masami H. <mhi...@re...>]

Change trace_arg_string() and parse_trace_arg() to probe_arg_string()
and parse_probe_arg(), since those are kprobe-tracer local functions.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Avi Kivity <av...@re...>
Cc: Andi Kleen <ak...@li...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Frank Ch. Eigler <fc...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
Cc: Jason Baron <jb...@re...>
Cc: K.Prasad <pr...@li...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Przemysław Pawełczyk <prz...@pa...>
Cc: Roland McGrath <ro...@re...>
Cc: Sam Ravnborg <sa...@ra...>
Cc: Srikar Dronamraju <sr...@li...>
Cc: Steven Rostedt <ro...@go...>
Cc: Tom Zanussi <tza...@gm...>
Cc: Vegard Nossum <veg...@gm...>
---

 kernel/trace/trace_kprobe.c |   18 +++++++++---------
 1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 22e91c0..783d2db 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -220,7 +220,7 @@ static __kprobes void *probe_address(struct trace_probe *tp)
 	return (probe_is_return(tp)) ? tp->rp.kp.addr : tp->kp.addr;
 }
 
-static int trace_arg_string(char *buf, size_t n, struct fetch_func *ff)
+static int probe_arg_string(char *buf, size_t n, struct fetch_func *ff)
 {
 	int ret = -EINVAL;
 
@@ -250,7 +250,7 @@ static int trace_arg_string(char *buf, size_t n, struct fetch_func *ff)
 		if (ret >= n)
 			goto end;
 		l += ret;
-		ret = trace_arg_string(buf + l, n - l, &id->orig);
+		ret = probe_arg_string(buf + l, n - l, &id->orig);
 		if (ret < 0)
 			goto end;
 		l += ret;
@@ -408,7 +408,7 @@ static int split_symbol_offset(char *symbol, long *offset)
 #define PARAM_MAX_ARGS 16
 #define PARAM_MAX_STACK (THREAD_SIZE / sizeof(unsigned long))
 
-static int parse_trace_arg(char *arg, struct fetch_func *ff, int is_return)
+static int parse_probe_arg(char *arg, struct fetch_func *ff, int is_return)
 {
 	int ret = 0;
 	unsigned long param;
@@ -499,7 +499,7 @@ static int parse_trace_arg(char *arg, struct fetch_func *ff, int is_return)
 			if (!id)
 				return -ENOMEM;
 			id->offset = offset;
-			ret = parse_trace_arg(arg, &id->orig, is_return);
+			ret = parse_probe_arg(arg, &id->orig, is_return);
 			if (ret)
 				kfree(id);
 			else {
@@ -617,7 +617,7 @@ static int create_trace_probe(int argc, char **argv)
 			ret = -ENOSPC;
 			goto error;
 		}
-		ret = parse_trace_arg(argv[i], &tp->args[i], is_return);
+		ret = parse_probe_arg(argv[i], &tp->args[i], is_return);
 		if (ret)
 			goto error;
 	}
@@ -680,7 +680,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
 		seq_printf(m, " 0x%p", probe_address(tp));
 
 	for (i = 0; i < tp->nr_args; i++) {
-		ret = trace_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
+		ret = probe_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
 		if (ret < 0) {
 			pr_warning("Argument%d decoding error(%d).\n", i, ret);
 			return ret;
@@ -996,7 +996,7 @@ static int kprobe_event_define_fields(struct ftrace_event_call *event_call)
 		sprintf(buf, "arg%d", i);
 		DEFINE_FIELD(unsigned long, args[i], buf, 0);
 		/* Set argument string as an alias field */
-		ret = trace_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
+		ret = probe_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
 		if (ret < 0)
 			return ret;
 		DEFINE_FIELD(unsigned long, args[i], buf, 0);
@@ -1023,7 +1023,7 @@ static int kretprobe_event_define_fields(struct ftrace_event_call *event_call)
 		sprintf(buf, "arg%d", i);
 		DEFINE_FIELD(unsigned long, args[i], buf, 0);
 		/* Set argument string as an alias field */
-		ret = trace_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
+		ret = probe_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
 		if (ret < 0)
 			return ret;
 		DEFINE_FIELD(unsigned long, args[i], buf, 0);
@@ -1040,7 +1040,7 @@ static int __probe_event_show_format(struct trace_seq *s,
 
 	/* Show aliases */
 	for (i = 0; i < tp->nr_args; i++) {
-		ret = trace_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
+		ret = probe_arg_string(buf, MAX_ARGSTR_LEN, &tp->args[i]);
 		if (ret < 0)
 			return ret;
 		if (!trace_seq_printf(s, "\talias: %s;\toriginal: arg%d;\n",


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH tracing/kprobes 3/4] tracing/kprobes: Fix format typo in trace_kprobes

[Masami H. <mhi...@re...>]

Fix a format typo in kprobe-tracer.

Currently, it shows 'tsize' in format;

$ cat /debug/tracing/events/kprobes/event/format 
...
        field: unsigned long ip;        offset:16;tsize:8;
        field: int nargs;       offset:24;tsize:4;
...

This should be '\tsize';

$ cat /debug/tracing/events/kprobes/event/format 
...
        field: unsigned long ip;        offset:16;	size:8;
        field: int nargs;       offset:24;	size:4;
...

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Avi Kivity <av...@re...>
Cc: Andi Kleen <ak...@li...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Frank Ch. Eigler <fc...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
Cc: Jason Baron <jb...@re...>
Cc: K.Prasad <pr...@li...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Przemysław Pawełczyk <prz...@pa...>
Cc: Roland McGrath <ro...@re...>
Cc: Sam Ravnborg <sa...@ra...>
Cc: Srikar Dronamraju <sr...@li...>
Cc: Steven Rostedt <ro...@go...>
Cc: Tom Zanussi <tza...@gm...>
Cc: Vegard Nossum <veg...@gm...>
---

 kernel/trace/trace_kprobe.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 7cd726e..22e91c0 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -1069,7 +1069,7 @@ static int __probe_event_show_format(struct trace_seq *s,
 #define SHOW_FIELD(type, item, name)					\
 	do {								\
 		ret = trace_seq_printf(s, "\tfield: " #type " %s;\t"	\
-				"offset:%u;tsize:%u;\n", name,		\
+				"offset:%u;\tsize:%u;\n", name,		\
 				(unsigned int)offsetof(typeof(field), item),\
 				(unsigned int)sizeof(type));		\
 		if (!ret)						\


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH tracing/kprobes 2/4] x86: Check awk features before generating inat-tables.c

[Masami H. <mhi...@re...>]

Check some awk features which old mawk doesn't support.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Avi Kivity <av...@re...>
Cc: Andi Kleen <ak...@li...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Frank Ch. Eigler <fc...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
Cc: Jason Baron <jb...@re...>
Cc: K.Prasad <pr...@li...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Przemysław Pawełczyk <prz...@pa...>
Cc: Roland McGrath <ro...@re...>
Cc: Sam Ravnborg <sa...@ra...>
Cc: Srikar Dronamraju <sr...@li...>
Cc: Steven Rostedt <ro...@go...>
Cc: Tom Zanussi <tza...@gm...>
Cc: Vegard Nossum <veg...@gm...>
---

 arch/x86/tools/gen-insn-attr-x86.awk |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/arch/x86/tools/gen-insn-attr-x86.awk b/arch/x86/tools/gen-insn-attr-x86.awk
index 93b62c9..19ba096 100644
--- a/arch/x86/tools/gen-insn-attr-x86.awk
+++ b/arch/x86/tools/gen-insn-attr-x86.awk
@@ -4,7 +4,25 @@
 #
 # Usage: awk -f gen-insn-attr-x86.awk x86-opcode-map.txt > inat-tables.c
 
+# Awk implementation sanity check
+function check_awk_implement() {
+	if (!match("abc", "[[:lower:]]+"))
+		return "Your awk doesn't support charactor-class."
+	if (sprintf("%x", 0) != "0")
+		return "Your awk has a printf-format problem."
+	return ""
+}
+
 BEGIN {
+	# Implementation error checking
+	awkchecked = check_awk_implement()
+	if (awkchecked != "") {
+		print "Error: " awkchecked > "/dev/stderr"
+		print "Please try to use gawk." > "/dev/stderr"
+		exit 1
+	}
+
+	# Setup generating tables
 	print "/* x86 opcode map generated from x86-opcode-map.txt */"
 	print "/* Do not change this code. */"
 	ggid = 1
@@ -293,6 +311,8 @@ function convert_operands(opnd,       i,imm,mod)
 }
 
 END {
+	if (awkchecked != "")
+		exit 1
 	# print escape opcode map's array
 	print "/* Escape opcode map array */"
 	print "const insn_attr_t const *inat_escape_tables[INAT_ESC_MAX + 1]" \


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH tracing/kprobes 1/4] x86: Fix x86 instruction decoder selftest to check only .text

[Masami H. <mhi...@re...>]

Fix x86 instruction decoder selftest to check only .text because other
sections (e.g. .notes) will have random bytes which don't need to be checked.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Avi Kivity <av...@re...>
Cc: Andi Kleen <ak...@li...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Frank Ch. Eigler <fc...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
Cc: Jason Baron <jb...@re...>
Cc: K.Prasad <pr...@li...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Przemysław Pawełczyk <prz...@pa...>
Cc: Roland McGrath <ro...@re...>
Cc: Sam Ravnborg <sa...@ra...>
Cc: Srikar Dronamraju <sr...@li...>
Cc: Steven Rostedt <ro...@go...>
Cc: Tom Zanussi <tza...@gm...>
Cc: Vegard Nossum <veg...@gm...>
---

 arch/x86/tools/Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/tools/Makefile b/arch/x86/tools/Makefile
index 3dd626b..95e9cc4 100644
--- a/arch/x86/tools/Makefile
+++ b/arch/x86/tools/Makefile
@@ -1,6 +1,6 @@
 PHONY += posttest
 quiet_cmd_posttest = TEST    $@
-      cmd_posttest = $(OBJDUMP) -d $(objtree)/vmlinux | awk -f $(srctree)/arch/x86/tools/distill.awk | $(obj)/test_get_len
+      cmd_posttest = $(OBJDUMP) -d -j .text $(objtree)/vmlinux | awk -f $(srctree)/arch/x86/tools/distill.awk | $(obj)/test_get_len
 
 posttest: $(obj)/test_get_len vmlinux
 	$(call cmd,posttest)


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [BUGFIX PATCH -tip] perf tools: Add OPT_END to option array of perf-trace

[Masami H. <mhi...@re...>]

Add OPT_END to option array of perf-trace for fixing a SEGV bug when
showing perf-trace help message.

Without this patch;
 ./perf trace -h

 usage: perf trace [<options>] <command>

    -D, --dump-raw-trace  dump raw trace in ASCII
    -v, --verbose         be more verbose (show symbol address, etc)
    -f, Segmentation fault

With this patch:
 ./perf trace -h

 usage: perf trace [<options>] <command>

    -D, --dump-raw-trace  dump raw trace in ASCII
    -v, --verbose         be more verbose (show symbol address, etc)

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Steven Rostedt <ro...@go...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Peter Zijlstra <pe...@in...>
Cc: Arnaldo Carvalho de Melo <ac...@re...>
Cc: Mike Galbraith <ef...@gm...>
Cc: Xiao Guangrong <xia...@cn...>
Cc: Zhaolei <zh...@cn...>
Cc: Li Zefan <li...@cn...>
Cc: Lai Jiangshan <la...@cn...>
Cc: Tom Zanussi <tza...@gm...>
---

 tools/perf/builtin-trace.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 88eef71..dd3c2e7 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -257,6 +257,7 @@ static const struct option options[] = {
 		    "dump raw trace in ASCII"),
 	OPT_BOOLEAN('v', "verbose", &verbose,
 		    "be more verbose (show symbol address, etc)"),
+	OPT_END()
 };
 
 int cmd_trace(int argc, const char **argv, const char *prefix __used)


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [TOOL] kprobestest : Kprobe stress test tool

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
>> Most of them can be fixed just by adding __kprobes.
>> Some of them which are already in the another section, kprobes
>> should check the symbols are in the section.
>
>
> You mean the blacklist?
>
> I also fear that putting bad kprobed functions into the kprobe
> section or into the blacklist may hide some kprobe internal bugs.
>
> Doing so is indeed mandatory for functions that trigger tracing
> recursion of things like that, but what if kprobe has an internal
> bug that only triggers while probe a certain class of function.
>
> Ie: it would be nice to identify the reason of the crash for
> each culprit in these lists.
 >
 > That may even help to find the others in advance.

Indeed, actually I've found some bugs while making jump-optimization
patches by using this stress test.
But some of them are obviously what we just forget to add __kprobes,
since those will be called from kprobes int3 handling functions.

And also, many lock-related code has been changed. I think
kprobes should use raw_*_lock, or prohibit to probe lock monitoring
functions like lockdep, because it will cause recursive call.

>
> Also kprobes seems to be a very fragile feature (that's what
> this selftest unearthes at least for me).
> And it really needs a recursion detection that stops every kprobing
> while reaching a given threshold of recursion. Something
> that would dump the stack and the falling kprobe structure.

Hmm, kprobes already has recursion detection(kp->nmiss), so
maybe, we can check it.

>
> That would avoid such hard lockups and also help to identify
> the dangerous symbols to probe.
>
>
>
>>> The problem is that I don't have any serial line in this
>>> box then I can't catch any crash log.
>>> My K7 testbox also died in my arms this afternoon.
>>>
>>> But I still have two other testboxes (one P2 and one P3),
>>> hopefully I could reproduce the problem in these boxes
>>> in which I can connect a serial line.
>>
>> Thank you for helping me to find it!
>>
>>> I've pushed your patches in the following git tree:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/fgrederic/random-tracing.git \
>>> 	tracing/kprobes
>>>
>>> So you can send patches on top of this one.
>>
>> Great! I've found another trivial bugs, so I'll fix those on it.
>
> Cool :)
>
> Btw, here is the result of your stress test in a PIII (attaching the log
> and the config).

Thanks, I'll check that.

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip v14 01/12] x86: instruction decoder API

[Frederic W. <fwe...@gm...>]

On Thu, Aug 20, 2009 at 03:01:25PM -0400, Masami Hiramatsu wrote:
> Frederic Weisbecker wrote:
>> On Thu, Aug 20, 2009 at 12:16:05PM -0400, Masami Hiramatsu wrote:
>>> Frederic Weisbecker wrote:
>>>> On Thu, Aug 20, 2009 at 11:03:40AM -0400, Masami Hiramatsu wrote:
>>>>> Frederic Weisbecker wrote:
>>>>>> On Thu, Aug 20, 2009 at 01:42:31AM +0200, Frederic Weisbecker wrote:
>>>>>>> On Thu, Aug 13, 2009 at 04:34:13PM -0400, Masami Hiramatsu wrote:
>>>>>>>> Add x86 instruction decoder to arch-specific libraries. This decoder
>>>>>>>> can decode x86 instructions used in kernel into prefix, opcode, modrm,
>>>>>>>> sib, displacement and immediates. This can also show the length of
>>>>>>>> instructions.
>>>>>>>>
>>>>>>>> This version introduces instruction attributes for decoding instructions.
>>>>>>>> The instruction attribute tables are generated from the opcode map file
>>>>>>>> (x86-opcode-map.txt) by the generator script(gen-insn-attr-x86.awk).
>>>>>>>>
>>>>>>>> Currently, the opcode maps are based on opcode maps in Intel(R) 64 and
>>>>>>>> IA-32 Architectures Software Developers Manual Vol.2: Appendix.A,
>>>>>>>> and consist of below two types of opcode tables.
>>>>>>>>
>>>>>>>> 1-byte/2-bytes/3-bytes opcodes, which has 256 elements, are
>>>>>>>> written as below;
>>>>>>>>
>>>>>>>>     Table: table-name
>>>>>>>>     Referrer: escaped-name
>>>>>>>>     opcode: mnemonic|GrpXXX [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...]
>>>>>>>>      (or)
>>>>>>>>     opcode: escape # escaped-name
>>>>>>>>     EndTable
>>>>>>>>
>>>>>>>> Group opcodes, which has 8 elements, are written as below;
>>>>>>>>
>>>>>>>>     GrpTable: GrpXXX
>>>>>>>>     reg:  mnemonic [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...]
>>>>>>>>     EndTable
>>>>>>>>
>>>>>>>> These opcode maps include a few SSE and FP opcodes (for setup), because
>>>>>>>> those opcodes are used in the kernel.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I'm getting the following build error on an old K7 box:
>>>>>>>
>>>>>>> arch/x86/lib/inat.c: In function ‘inat_get_opcode_attribute’:
>>>>>>> arch/x86/lib/inat.c:29: erreur: ‘inat_primary_table’ undeclared (first use in this function)
>>>>>>> arch/x86/lib/inat.c:29: erreur: (Each undeclared identifier is reported only once
>>>>>>> arch/x86/lib/inat.c:29: erreur: for each function it appears in.)
>>>>>>>
>>>>>>>
>>>>>>> I've attached my config. I haven't such problem on a dual x86-64 box.
>>>>>>
>>>>>>
>>>>>> Actually I have the same problem in x86-64
>>>>>> The content of my arch/x86/lib/inat-tables.c:
>>>>>>
>>>>>> /* x86 opcode map generated from x86-opcode-map.txt */
>>>>>> /* Do not change this code. */
>>>>>> /* Table: one byte opcode */
>>>>>> /* Escape opcode map array */
>>>>>> const insn_attr_t const *inat_escape_tables[INAT_ESC_MAX + 1][INAT_LPREFIX_MAX + 1] = {
>>>>>> };
>>>>>>
>>>>>> /* Group opcode map array */
>>>>>> const insn_attr_t const *inat_group_tables[INAT_GRP_MAX + 1][INAT_LPREFIX_MAX + 1] = {
>>>>>> };
>>>>>>
>>>>>>
>>>>>> I guess there is a problem with the generation of this file.
>>>>>
>>>>> Aah, you may use mawk on Ubuntu 9.04, right?
>>>>> If so, unfortunately, mawk is still under development.
>>>>>
>>>>> http://invisible-island.net/mawk/CHANGES
>>>>
>>>>
>>>>
>>>> Aargh...
>>>>
>>>>
>>>>>> 20090727
>>>>>> 	add check/fix to prevent gsub from recurring to modify on a substring
>>>>>> 	of the current line when the regular expression is anchored to the
>>>>>> 	beginning of the line; fixes gawk's anchgsub testcase.
>>>>>>
>>>>>> 	add check for implicit concatenation mistaken for exponent; fixes
>>>>>> 	gawk's hex testcase.
>>>>>>
>>>>>> 	add character-classes to built-in regular expressions.
>>>>>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>> Look, this means we can't use char-class expressions like
>>>>> [:lower:] until this version...
>>>>>
>>>>> And I've found another bug in mawk-1.3.3-20090728(the latest one).
>>>>> it almost works, but;
>>>>>
>>>>> $ mawk 'BEGIN {printf("0x%x\n", 0)}'
>>>>> 0x1
>>>>
>>>>
>>>> Ouch, indeed.
>>>>
>>>>
>>>>
>>>>> $ gawk 'BEGIN {printf("0x%x\n", 0)}'
>>>>> 0x0
>>>>>
>>>>> This bug skips an array element index 0x0 in inat-tables.c :(
>>>>>
>>>>> So, I recommend you to install gawk instead mawk until that
>>>>> supports all posix-awk features, since I don't think it is
>>>>> good idea to avoid all those bugs which depends on
>>>>> implementation (not specification).
>>>>>
>>>>>
>>>>> Thank you,
>>>>
>>>>
>>>>
>>>> Yeah, indeed. May be add a warning (or build error) in case the user uses
>>>> mawk?
>>>
>>> Hmm, it is possible that mawk will fix those bugs and catch up soon,
>>> so, I think checking mawk is not a good idea.
>>> (and since there will be other awk implementations, it's not fair.)
>>>
>>> I think what all I can do now is reporting bugs to
>>> mawk and ubuntu people.:-)
>>
>>
>>
>> Yeah, but without your tip I couldn't be able to find the origin
>> before some time.
>> And the kernel couldn't build anyway.
>>
>> At least we should do something with this version of mawk.
>
> Hm, indeed.
> Maybe, we can run additional sanity check script before using
> awk, like this;
>
> ---
> res=`echo a | $AWK '/[[:lower:]]+/{print "OK"}'`
> [ "$res" != "OK" ] && exit 1
>
> res=`$AWK 'BEGIN {printf("%x", 0)}'`
> [ "$res" != "0" ] && exit 1
>
> exit 0
> ---
>
> Thanks,
>


Yeah, that looks good.

Thanks.

Re: [Dle-develop] [TOOL] kprobestest : Kprobe stress test tool

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> On Thu, Aug 13, 2009 at 04:57:20PM -0400, Masami Hiramatsu wrote:
>> This script tests kprobes to probe on all symbols in the kernel and finds
>> symbols which must be blacklisted.
>>
>>
>> Usage
>> -----
>>    kprobestest [-s SYMLIST] [-b BLACKLIST] [-w WHITELIST]
>>       Run stress test. If SYMLIST file is specified, use it as
>>       an initial symbol list (This is useful for verifying white list
>>       after diagnosing all symbols).
>>
>>    kprobestest cleanup
>>       Cleanup all lists
>>
>>
>> How to Work
>> -----------
>> This tool list up all symbols in the kernel via /proc/kallsyms, and sorts
>> it into groups (each of them including 64 symbols in default). And then,
>> it tests each group by using kprobe-tracer. If a kernel crash occurred,
>> that group is moved into 'failed' dir. If the group passed the test, this
>> script moves it into 'passed' dir and saves kprobe_profile into
>> 'passed/profiles/'.
>> After testing all groups, all 'failed' groups are merged and sorted into
>> smaller groups (divided by 4, in default). And those are tested again.
>> This loop will be repeated until all group has just 1 symbol.
>>
>> Finally, the script sorts all 'passed' symbols into 'tested', 'untested',
>> and 'missed' based on profiles.
>>
>>
>> Note
>> ----
>>   - This script just gives us some clues to the blacklisted functions.
>>     In some cases, a combination of probe points will cause a problem, but
>>     each of them doesn't cause the problem alone.
>>
>> Thank you,
>>
>
>
> This script makes my x86-64 dual core easily and hardly locking-up
> on the 1st batch of symbols to test.
> I have one sym list in the failed and unset directories:
>
> int_very_careful
> int_signal
> int_restore_rest
> stub_clone
> stub_fork
> stub_vfork
> stub_sigaltstack
> stub_iopl
> ptregscall_common
> stub_execve
> stub_rt_sigreturn
> irq_entries_start
> common_interrupt
> ret_from_intr
> exit_intr
> retint_with_reschedule
> retint_check
> retint_swapgs
> retint_restore_args
> restore_args
> irq_return
> retint_careful
> retint_signal
> retint_kernel
> irq_move_cleanup_interrupt
> reboot_interrupt
> apic_timer_interrupt
> generic_interrupt
> invalidate_interrupt0
> invalidate_interrupt1
> invalidate_interrupt2
> invalidate_interrupt3
> invalidate_interrupt4
> invalidate_interrupt5
> invalidate_interrupt6
> invalidate_interrupt7
> threshold_interrupt
> thermal_interrupt
> mce_self_interrupt
> call_function_single_interrupt
> call_function_interrupt
> reschedule_interrupt
> error_interrupt
> spurious_interrupt
> perf_pending_interrupt
> divide_error
> overflow
> bounds
> invalid_op
> device_not_available
> double_fault
> coprocessor_segment_overrun
> invalid_TSS
> segment_not_present
> spurious_interrupt_bug
> coprocessor_error
> alignment_check
> simd_coprocessor_error
> native_load_gs_index
> gs_change
> kernel_thread
> child_rip
> kernel_execve
> call_softirq
>
>
> I don't have a crash log because I was running with X.
> But it also happened with other batch of symbols.

Thank you for reporting, here, I also have a result
tested on KVM@x86-64.

native_read_tscp
native_read_msr_safe
native_read_msr_amd_safe
native_write_msr_safe
vmalloc_fault
spurious_fault
search_exception_tables
notify_die
trace_hardirqs_off_caller
ident_complete
lock_acquire
lock_release
bad_address
secondary_startup_64
stack_start
bad_address
restore_args
irq_return
restore
trace_hardirqs_off_thunk
init_level4_pgt
level3_ident_pgt
level3_kernel_pgt
level2_fixmap_pgt
_text
startup_64
level1_fixmap_pgt
level2_ident_pgt
level2_kernel_pgt
level2_spare_pgt
native_get_debugreg
native_set_debugreg
native_set_iopl_mask
native_load_sp0
debug_show_all_locks
debug_check_no_locks_held
valid_state
mark_lock
mark_held_locks
lockdep_trace_alloc
trace_softirqs_on
trace_hardirqs_on_caller
__down_write
__down_read
trace_hardirqs_on_thunk
lockdep_sys_exit_thunk

Most of them can be fixed just by adding __kprobes.
Some of them which are already in the another section, kprobes
should check the symbols are in the section.

> The problem is that I don't have any serial line in this
> box then I can't catch any crash log.
> My K7 testbox also died in my arms this afternoon.
>
> But I still have two other testboxes (one P2 and one P3),
> hopefully I could reproduce the problem in these boxes
> in which I can connect a serial line.

Thank you for helping me to find it!

> I've pushed your patches in the following git tree:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/fgrederic/random-tracing.git \
> 	tracing/kprobes
>
> So you can send patches on top of this one.

Great! I've found another trivial bugs, so I'll fix those on it.

Thank you,


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip v14 01/12] x86: instruction decoder API

[Masami H. <mhi...@re...>]

Frederic Weisbecker wrote:
> On Thu, Aug 20, 2009 at 12:16:05PM -0400, Masami Hiramatsu wrote:
>> Frederic Weisbecker wrote:
>>> On Thu, Aug 20, 2009 at 11:03:40AM -0400, Masami Hiramatsu wrote:
>>>> Frederic Weisbecker wrote:
>>>>> On Thu, Aug 20, 2009 at 01:42:31AM +0200, Frederic Weisbecker wrote:
>>>>>> On Thu, Aug 13, 2009 at 04:34:13PM -0400, Masami Hiramatsu wrote:
>>>>>>> Add x86 instruction decoder to arch-specific libraries. This decoder
>>>>>>> can decode x86 instructions used in kernel into prefix, opcode, modrm,
>>>>>>> sib, displacement and immediates. This can also show the length of
>>>>>>> instructions.
>>>>>>>
>>>>>>> This version introduces instruction attributes for decoding instructions.
>>>>>>> The instruction attribute tables are generated from the opcode map file
>>>>>>> (x86-opcode-map.txt) by the generator script(gen-insn-attr-x86.awk).
>>>>>>>
>>>>>>> Currently, the opcode maps are based on opcode maps in Intel(R) 64 and
>>>>>>> IA-32 Architectures Software Developers Manual Vol.2: Appendix.A,
>>>>>>> and consist of below two types of opcode tables.
>>>>>>>
>>>>>>> 1-byte/2-bytes/3-bytes opcodes, which has 256 elements, are
>>>>>>> written as below;
>>>>>>>
>>>>>>>     Table: table-name
>>>>>>>     Referrer: escaped-name
>>>>>>>     opcode: mnemonic|GrpXXX [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...]
>>>>>>>      (or)
>>>>>>>     opcode: escape # escaped-name
>>>>>>>     EndTable
>>>>>>>
>>>>>>> Group opcodes, which has 8 elements, are written as below;
>>>>>>>
>>>>>>>     GrpTable: GrpXXX
>>>>>>>     reg:  mnemonic [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...]
>>>>>>>     EndTable
>>>>>>>
>>>>>>> These opcode maps include a few SSE and FP opcodes (for setup), because
>>>>>>> those opcodes are used in the kernel.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> I'm getting the following build error on an old K7 box:
>>>>>>
>>>>>> arch/x86/lib/inat.c: In function ‘inat_get_opcode_attribute’:
>>>>>> arch/x86/lib/inat.c:29: erreur: ‘inat_primary_table’ undeclared (first use in this function)
>>>>>> arch/x86/lib/inat.c:29: erreur: (Each undeclared identifier is reported only once
>>>>>> arch/x86/lib/inat.c:29: erreur: for each function it appears in.)
>>>>>>
>>>>>>
>>>>>> I've attached my config. I haven't such problem on a dual x86-64 box.
>>>>>
>>>>>
>>>>> Actually I have the same problem in x86-64
>>>>> The content of my arch/x86/lib/inat-tables.c:
>>>>>
>>>>> /* x86 opcode map generated from x86-opcode-map.txt */
>>>>> /* Do not change this code. */
>>>>> /* Table: one byte opcode */
>>>>> /* Escape opcode map array */
>>>>> const insn_attr_t const *inat_escape_tables[INAT_ESC_MAX + 1][INAT_LPREFIX_MAX + 1] = {
>>>>> };
>>>>>
>>>>> /* Group opcode map array */
>>>>> const insn_attr_t const *inat_group_tables[INAT_GRP_MAX + 1][INAT_LPREFIX_MAX + 1] = {
>>>>> };
>>>>>
>>>>>
>>>>> I guess there is a problem with the generation of this file.
>>>>
>>>> Aah, you may use mawk on Ubuntu 9.04, right?
>>>> If so, unfortunately, mawk is still under development.
>>>>
>>>> http://invisible-island.net/mawk/CHANGES
>>>
>>>
>>>
>>> Aargh...
>>>
>>>
>>>>> 20090727
>>>>> 	add check/fix to prevent gsub from recurring to modify on a substring
>>>>> 	of the current line when the regular expression is anchored to the
>>>>> 	beginning of the line; fixes gawk's anchgsub testcase.
>>>>>
>>>>> 	add check for implicit concatenation mistaken for exponent; fixes
>>>>> 	gawk's hex testcase.
>>>>>
>>>>> 	add character-classes to built-in regular expressions.
>>>>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>> Look, this means we can't use char-class expressions like
>>>> [:lower:] until this version...
>>>>
>>>> And I've found another bug in mawk-1.3.3-20090728(the latest one).
>>>> it almost works, but;
>>>>
>>>> $ mawk 'BEGIN {printf("0x%x\n", 0)}'
>>>> 0x1
>>>
>>>
>>> Ouch, indeed.
>>>
>>>
>>>
>>>> $ gawk 'BEGIN {printf("0x%x\n", 0)}'
>>>> 0x0
>>>>
>>>> This bug skips an array element index 0x0 in inat-tables.c :(
>>>>
>>>> So, I recommend you to install gawk instead mawk until that
>>>> supports all posix-awk features, since I don't think it is
>>>> good idea to avoid all those bugs which depends on
>>>> implementation (not specification).
>>>>
>>>>
>>>> Thank you,
>>>
>>>
>>>
>>> Yeah, indeed. May be add a warning (or build error) in case the user uses
>>> mawk?
>>
>> Hmm, it is possible that mawk will fix those bugs and catch up soon,
>> so, I think checking mawk is not a good idea.
>> (and since there will be other awk implementations, it's not fair.)
>>
>> I think what all I can do now is reporting bugs to
>> mawk and ubuntu people.:-)
>
>
>
> Yeah, but without your tip I couldn't be able to find the origin
> before some time.
> And the kernel couldn't build anyway.
>
> At least we should do something with this version of mawk.

Hm, indeed.
Maybe, we can run additional sanity check script before using
awk, like this;

---
res=`echo a | $AWK '/[[:lower:]]+/{print "OK"}'`
[ "$res" != "OK" ] && exit 1

res=`$AWK 'BEGIN {printf("%x", 0)}'`
[ "$res" != "0" ] && exit 1

exit 0
---

Thanks,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...