dle-develop — Dependable Linux Effort mailing list for developers

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Frederic W. <fwe...@gm...>]

On Mon, Oct 19, 2009 at 02:18:24PM -0200, Arnaldo Carvalho de Melo wrote:
> Em Mon, Oct 19, 2009 at 01:00:57PM +0200, Frederic Weisbecker escreveu:
> > On Mon, Oct 19, 2009 at 09:51:03AM +0200, Ingo Molnar wrote:
> > > > So, what would you think about using -D (def) and -U (undef) ?
> > > The simpest case should be no extra character at all:
> > >   perf probe schedule
> > Yeah, I really prefer that too.
> 
> /me too
>  
> > If you want more precision, it also means you have you code editor opened
> > and want to set a precise point. Since you also have the absolute
> > line directly displayed by your editor, you don't want to calculate the relative
> > line but rather the absolute one.
> 
> Perhaps this should come with vim/emacs key bindings so that all this
> gets hidden by just opening the source code and pressing
> control+SOMEKEY?
> 
> - Arnaldo


Yep, would be a nice idea.

[Dle-develop] [PATCH -tip perf/probes 2/2] x86: Add AES opcodes to opcode map

[Masami H. <mhi...@re...>]

Add Intel AES opcodes to x86 opcode map. These opcodes are
used in arch/x86/crypt/aesni-intel_asm.S.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/lib/x86-opcode-map.txt |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt
index 894497f..701c467 100644
--- a/arch/x86/lib/x86-opcode-map.txt
+++ b/arch/x86/lib/x86-opcode-map.txt
@@ -567,7 +567,7 @@ fe: paddd Pq,Qq | paddd Vdq,Wdq (66)
 ff:
 EndTable
 
-Table: 3-byte opcode 1
+Table: 3-byte opcode 1 (0x0f 0x38)
 Referrer: 3-byte escape 1
 # 0x0f 0x38 0x00-0x0f
 00: pshufb Pq,Qq | pshufb Vdq,Wdq (66)
@@ -642,11 +642,16 @@ Referrer: 3-byte escape 1
 41: phminposuw Vdq,Wdq (66)
 80: INVEPT Gd/q,Mdq (66)
 81: INVPID Gd/q,Mdq (66)
+db: aesimc Vdq,Wdq (66)
+dc: aesenc Vdq,Wdq (66)
+dd: aesenclast Vdq,Wdq (66)
+de: aesdec Vdq,Wdq (66)
+df: aesdeclast Vdq,Wdq (66)
 f0: MOVBE Gv,Mv | CRC32 Gd,Eb (F2)
 f1: MOVBE Mv,Gv | CRC32 Gd,Ev (F2)
 EndTable
 
-Table: 3-byte opcode 2
+Table: 3-byte opcode 2 (0x0f 0x3a)
 Referrer: 3-byte escape 2
 # 0x0f 0x3a 0x00-0xff
 08: roundps Vdq,Wdq,Ib (66)
@@ -671,6 +676,7 @@ Referrer: 3-byte escape 2
 61: pcmpestri Vdq,Wdq,Ib (66)
 62: pcmpistrm Vdq,Wdq,Ib (66)
 63: pcmpistri Vdq,Wdq,Ib (66)
+df: aeskeygenassist Vdq,Wdq,Ib (66)
 EndTable
 
 GrpTable: Grp1


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip perf/probes 1/2] x86: Fix group attribute decoding bug

[Masami H. <mhi...@re...>]

Fix a typo in inat_get_group_attribute() which should refer
inat_group_tables, not inat_escape_tables.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Jim Keniston <jke...@us...>
Cc: H. Peter Anvin <hp...@zy...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
---

 arch/x86/lib/inat.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/lib/inat.c b/arch/x86/lib/inat.c
index 054656a..3fb5998 100644
--- a/arch/x86/lib/inat.c
+++ b/arch/x86/lib/inat.c
@@ -68,7 +68,7 @@ insn_attr_t inat_get_group_attribute(insn_byte_t modrm, insn_byte_t last_pfx,
 	if (!table)
 		return inat_group_common_attribute(grp_attr);
 	if (inat_has_variant(table[X86_MODRM_REG(modrm)]) && m) {
-		table = inat_escape_tables[n][m];
+		table = inat_group_tables[n][m];
 		if (!table)
 			return inat_group_common_attribute(grp_attr);
 	}


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip perf/probes 0/2] x86 insn decoder bugfixes

[Masami H. <mhi...@re...>]

Hi Ingo,

Here are x86 insn decoder bugfixes.
I checked it with your config and allyesconfig too.

Thank you,

---

Masami Hiramatsu (2):
      x86: Add AES opcodes to opcode map
      x86: Fix group attribute decoding bug


 arch/x86/lib/inat.c             |    2 +-
 arch/x86/lib/x86-opcode-map.txt |   10 ++++++++--
 2 files changed, 9 insertions(+), 3 deletions(-)

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division
e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Masami H. <mhi...@re...>]

Ingo Molnar wrote:
>>> The point is to prefer intuitive, non-mechanic, fundamentally human
>>> expressions of information above mechanic ones (absolute line numbers,
>>> addresses, ways of probing, etc.) - and to have a rich variety of them.
>>>
>>> String based pattern matching and intuitive syntax that reuses existing
>>> paradigms of arithmetics and pattern-matching is good - limited syntax
>>> and extra, arbitrary syntactic hoops to jump through is bad.
>>>
>>> If we provide all that, people will start using this stuff - and i'd
>>> only like to merge this upstream once it's clear that people like me
>>> will (be able to) use this facility for ad-hoc probe insertion.
>>>
>>> In other words: this facility has to 'live within' our source code and
>>> has to be able to interact with it on a very broad basis - for it to be
>>> maximally useful for everyday development.
>>
>> Hmm, so you mean perf-probe should work with source-code? Without 
>> source code (but with debuginfo), maybe we can't use string matching, 
>> is that OK?
> 
> Well most forms of debuginfo embedd the full source code in the 
> debuginfo, right? If it's not there (or we dont know where it is) then 
> we cannot use it, obviously.

Um, actually debuginfo doesn't have the full source code, but has
the source file path. So, only if there are source files,
we can use string-based matching. Even if there are no source files,
that means users can't change their kernel:-). So we don't care
about kernel-version dependency.

> But we obviously want the whole 'perf probe' workflow to primarily 
> operate on source code - we are humans.

Sure :-)

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Masami H. <mhi...@re...>]

Ingo Molnar wrote:
> 
> * Masami Hiramatsu <mhi...@re...> wrote:
> 
>> How about the following syntax?
>> <adding>
>> perf probe schedule
>> perf probe --add schedule
>>
>> <deleting>
>> perf probe --del schedule
>> perf probe --del all	/* delete all probepoints */
>>
>> So, this doesn't symmetric, but provides simple way to add a probe.
> 
> agreed. For deletion, eventually this:
> 
>   perf probe --del "*"
> 
> should also work - as should other regexp (or glob) matches on a range 
> of existing probes.

Ah, that's a nice idea :-), it allows user to delete a group of
probes, like --del "schedule*".

> plus there should be a perf probe call to output all current probes as 
> perf probe commands - so that it can be saved (and restored).

Sure. 'perf probe --save file' & 'perf probe --restore file' will be
fine for me.

Thank you,
-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Masami Hiramatsu <mhi...@re...> wrote:

> How about the following syntax?
> <adding>
> perf probe schedule
> perf probe --add schedule
> 
> <deleting>
> perf probe --del schedule
> perf probe --del all	/* delete all probepoints */
> 
> So, this doesn't symmetric, but provides simple way to add a probe.

agreed. For deletion, eventually this:

  perf probe --del "*"

should also work - as should other regexp (or glob) matches on a range 
of existing probes.

plus there should be a perf probe call to output all current probes as 
perf probe commands - so that it can be saved (and restored).

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Masami Hiramatsu <mhi...@re...> wrote:

> Masami Hiramatsu wrote:
> > Ingo Molnar wrote:
> >> For example you might want to probe the point within schedule that calls
> >> switch_mm() - this could be done via:
> >>
> >>    perf probe schedule@switch_mm
> >>
> >> Or the point where 'next' gets assigned? Sure, you dont need to even
> >> open the editor, if you know the rough outline of the function you can
> >> probe it via:
> >>
> >>    perf probe schedule@'next ='
> >>
> >> Note that i was able to specify both probes without having opened an
> >> editor - just based on the general knowledge of the scheduler.
> > 
> > It may be useful for return probe too :-)
> > 
> >  perf probe schedule@return
> 
> Hmm, IMHO,
> 
> >>    perf probe schedule@switch_mm
> 
> might be confused as 'probe schedule() called from switch_mm()'.
> 
> BTW, there might be several local/inline functions which have
> same name.
> I think we'd better provide a syntax for solving this issue.
> And current syntax uses @ for this purpose as below.
> 
>  perf probe localfunc@file
> 
> Maybe, we still can use % for special matching,
> 
>  perf probe schedule%switch_mm
> 
> These can be combined with each other, as below.
> 
>  perf probe schedule@kernel/sched.c%switch_mm
> 
> Or, supporting lazy string pattern matching
>  (reusing glob matching in ftrace?)
> 
>  perf probe schedule:'switch_mm(*);'
> 
> Just my thought.

I'm not attached to any particular form of syntax here (other than it 
should be simple and obvious) - we can try and see how it works out.

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Masami Hiramatsu <mhi...@re...> wrote:

> Ah, I've gotten what you said, and many lines will be optimized out it 
> would be better to show line numbers which can be probed, as below.
> 
> 000	asmlinkage void __sched schedule(void)
> 	{
> 		struct task_struct *prev, *next;
> 		unsigned long *switch_count;
> 		struct rq *rq;
> 		int cpu;
> 
> 007	need_resched:
> 008		preempt_disable();
> 009		cpu = smp_processor_id();
> 010		rq = cpu_rq(cpu);
> 011		rcu_sched_qs(cpu);
> 012		prev = rq->curr;
> 013		switch_count = &prev->nivcsw;
> 
> 015		release_kernel_lock(prev);

Agreed, that's a very good idea!

( You could also make use of color_fprintf() to color-code some of the 
  numbers - for example lines with existing probes might be marked red. )

> > That way the following two are equivalent:
> >
> >   perf probe schedule
> >   perf probe schedule+0
> >
> > The advantage of relative line numbers is that they are much more
> > version invariant than absolute line numbers. Relative line numbers into
> > schedule() will only change if the function itself changes.
> >
> > This means that expressions like 'schedule+16' will have a lot longer
> > life-time than absolute line number driven probes. You can quote it in
> > an email and chances are that it will still be valid even a few kernel
> > releases down the road.
> 
> Hmm, I imagines 'schedule() + 16 byte offset' from 'schedule+16', 
> because many in-kernel(and oops) messages means that. :-) So I'd like 
> to use 'schedule:16'.

kernel symbol addresses are in the form of schedule+0x16/0x510.

I like the plus sign because it also allows the negative positions: 
relative position from the _end_ of the function. So for example 
'schedule-1' would probe the (last) return line of the function. 
[admittedly this is not as useful as the + operation though.]

But schedule:16 [and schedule:+16] would be fine too i suspect.

> >>> We also want to have functionality that helps people find probe
> >>> spots within a function:
> >>>
> >>>   perf probe --list-lines schedule
> >>>
> >>> Would list the line numbers and source code of the schedule()
> >>> function. (similar to how GDB 'list' works) That way someone can
> >>> have an ad-hoc session of deciding what place to probe, and the line
> >>> numbers make for an easy ID of the statement to probe.
> >>
> >> Agreed!
> >
> > Furthermore - to answer another question you raised above - the
> > following syntax:
> >
> >    perf probe schedule:'switch_count = &prev->nivcsw'
> >
> > is basically a 'fuzzy string match' based probe to within a function.
> >
> > For example you might want to probe the point within schedule that calls
> > switch_mm() - this could be done via:
> >
> >    perf probe schedule@switch_mm
> >
> > Or the point where 'next' gets assigned? Sure, you dont need to even
> > open the editor, if you know the rough outline of the function you can
> > probe it via:
> >
> >    perf probe schedule@'next ='
> >
> > Note that i was able to specify both probes without having opened an
> > editor - just based on the general knowledge of the scheduler.
> 
> It may be useful for return probe too :-)
> 
>  perf probe schedule@return
> 
> > The point is to prefer intuitive, non-mechanic, fundamentally human
> > expressions of information above mechanic ones (absolute line numbers,
> > addresses, ways of probing, etc.) - and to have a rich variety of them.
> >
> > String based pattern matching and intuitive syntax that reuses existing
> > paradigms of arithmetics and pattern-matching is good - limited syntax
> > and extra, arbitrary syntactic hoops to jump through is bad.
> >
> > If we provide all that, people will start using this stuff - and i'd
> > only like to merge this upstream once it's clear that people like me
> > will (be able to) use this facility for ad-hoc probe insertion.
> >
> > In other words: this facility has to 'live within' our source code and
> > has to be able to interact with it on a very broad basis - for it to be
> > maximally useful for everyday development.
> 
> Hmm, so you mean perf-probe should work with source-code? Without 
> source code (but with debuginfo), maybe we can't use string matching, 
> is that OK?

Well most forms of debuginfo embedd the full source code in the 
debuginfo, right? If it's not there (or we dont know where it is) then 
we cannot use it, obviously.

But we obviously want the whole 'perf probe' workflow to primarily 
operate on source code - we are humans.

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Frederic Weisbecker <fwe...@gm...> wrote:

[...]
>
> I think absolute and relative line modes are not colliding/contending 
> at all but actually fit two different needs.

Definitely so.

> - absolute is nice when you are lonely doing kernel debugging.
>   (can be expanded at will once you imagine user probes)
>   You are stuck in your code editor, trying to figure out the
>   origin of your problem and then you think it would be nice
>   to set a probe in branch 1 and in branch 2 inside func_foo().
>   Then you already have absolute lines and relying in
>   perf probe --list func_foo() to resolve an absolute line into
>   a relative one is a very undesired middle step.

Of course - absolute numbers definitely rule for everything that works 
on a whole-file basis. (I'd argue that if you do that from an editor 
then you want a short macro that just sets a probe there - much like a 
breakpoint. Such an editor macro would want to use absolute numbers.))

> - relative is nice in some other cases. When you already have
>   the function target in mind, you even don't need to check your
>   editor, just a quick check to this command and get the relative
>   line. But also when you want to transmit a probe reference
>   in a mailing list because of its better lifetime.

also useful for command line workflows: 'perf probe --list' output - i 
think we users to generate func_symbol+rel_position kind of probes.

Plus a relative position is more intuitive as well. If you see 
'schedule+10' versus 'schedule+102', you'll know it immediate that the 
first one is early in the function while the second one is near the end.

If you see 'schedule@2465' versus 'schedule@2555' that kind of 'where in 
the function is the probe, roughly' subjective impression is lost.

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Masami H. <mhi...@re...>]

Masami Hiramatsu wrote:
> Ingo Molnar wrote:
>> For example you might want to probe the point within schedule that calls
>> switch_mm() - this could be done via:
>>
>>    perf probe schedule@switch_mm
>>
>> Or the point where 'next' gets assigned? Sure, you dont need to even
>> open the editor, if you know the rough outline of the function you can
>> probe it via:
>>
>>    perf probe schedule@'next ='
>>
>> Note that i was able to specify both probes without having opened an
>> editor - just based on the general knowledge of the scheduler.
> 
> It may be useful for return probe too :-)
> 
>  perf probe schedule@return

Hmm, IMHO,

>>    perf probe schedule@switch_mm

might be confused as 'probe schedule() called from switch_mm()'.

BTW, there might be several local/inline functions which have
same name.
I think we'd better provide a syntax for solving this issue.
And current syntax uses @ for this purpose as below.

 perf probe localfunc@file

Maybe, we still can use % for special matching,

 perf probe schedule%switch_mm

These can be combined with each other, as below.

 perf probe schedule@kernel/sched.c%switch_mm

Or, supporting lazy string pattern matching
 (reusing glob matching in ftrace?)

 perf probe schedule:'switch_mm(*);'

Just my thought.

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Masami H. <mhi...@re...>]

Ingo Molnar wrote:
> The 'perf probe --list schedule' sub-tool i outlined would display
> relative line numbers for the function - starting at 0:
>
> 000	/*
> 001	 * schedule() is the main scheduler function.
> 002	 */
> 003	asmlinkage void __sched schedule(void)
> 004	{
> 005		struct task_struct *prev, *next;
> 006		unsigned long *switch_count;
> 007		struct rq *rq;
> 008		int cpu;
> 009
> 010	need_resched:
> 011		preempt_disable();
> 012		cpu = smp_processor_id();
> 013		rq = cpu_rq(cpu);
> 014		rcu_sched_qs(cpu);
> 015		prev = rq->curr;
> 016		switch_count = &prev->nivcsw;
> 017
> 018		release_kernel_lock(prev);

Ah, I've gotten what you said, and many lines will be optimized out
it would be better to show line numbers which can be probed, as below.

000	asmlinkage void __sched schedule(void)
	{
		struct task_struct *prev, *next;
		unsigned long *switch_count;
		struct rq *rq;
		int cpu;

007	need_resched:
008		preempt_disable();
009		cpu = smp_processor_id();
010		rq = cpu_rq(cpu);
011		rcu_sched_qs(cpu);
012		prev = rq->curr;
013		switch_count = &prev->nivcsw;

015		release_kernel_lock(prev);


> That way the following two are equivalent:
>
>   perf probe schedule
>   perf probe schedule+0
>
> The advantage of relative line numbers is that they are much more
> version invariant than absolute line numbers. Relative line numbers into
> schedule() will only change if the function itself changes.
>
> This means that expressions like 'schedule+16' will have a lot longer
> life-time than absolute line number driven probes. You can quote it in
> an email and chances are that it will still be valid even a few kernel
> releases down the road.

Hmm, I imagines 'schedule() + 16 byte offset' from 'schedule+16', because
many in-kernel(and oops) messages means that. :-)
So I'd like to use 'schedule:16'.

>>> We also want to have functionality that helps people find probe
>>> spots within a function:
>>>
>>>   perf probe --list-lines schedule
>>>
>>> Would list the line numbers and source code of the schedule()
>>> function. (similar to how GDB 'list' works) That way someone can
>>> have an ad-hoc session of deciding what place to probe, and the line
>>> numbers make for an easy ID of the statement to probe.
>>
>> Agreed!
>
> Furthermore - to answer another question you raised above - the
> following syntax:
>
>    perf probe schedule:'switch_count = &prev->nivcsw'
>
> is basically a 'fuzzy string match' based probe to within a function.
>
> For example you might want to probe the point within schedule that calls
> switch_mm() - this could be done via:
>
>    perf probe schedule@switch_mm
>
> Or the point where 'next' gets assigned? Sure, you dont need to even
> open the editor, if you know the rough outline of the function you can
> probe it via:
>
>    perf probe schedule@'next ='
>
> Note that i was able to specify both probes without having opened an
> editor - just based on the general knowledge of the scheduler.

It may be useful for return probe too :-)

 perf probe schedule@return

> The point is to prefer intuitive, non-mechanic, fundamentally human
> expressions of information above mechanic ones (absolute line numbers,
> addresses, ways of probing, etc.) - and to have a rich variety of them.
>
> String based pattern matching and intuitive syntax that reuses existing
> paradigms of arithmetics and pattern-matching is good - limited syntax
> and extra, arbitrary syntactic hoops to jump through is bad.
>
> If we provide all that, people will start using this stuff - and i'd
> only like to merge this upstream once it's clear that people like me
> will (be able to) use this facility for ad-hoc probe insertion.
>
> In other words: this facility has to 'live within' our source code and
> has to be able to interact with it on a very broad basis - for it to be
> maximally useful for everyday development.

Hmm, so you mean perf-probe should work with source-code?
Without source code (but with debuginfo), maybe we can't use
string matching, is that OK?

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Frederic W. <fwe...@gm...>]

On Mon, Oct 19, 2009 at 01:21:25PM +0200, Ingo Molnar wrote:
> 
> * Frederic Weisbecker <fwe...@gm...> wrote:
> > I personally don't imagine common easy usecases that imply relative 
> > line offsets but rather absolute lines.
> 
> Absolute line numbers could be expressed too, via:
> 
>    perf schedule@5396
> 
> There's no immediate need to express 'sched.c'. (you can do it, or you 
> can do it for extra clarity or for the case of local scope functions of 
> which multiple instances exist)



Ok. As long as there is an easy way to do that, like in the above
example, then I think it's fine.

(More argument below)

 
> > I don't understand your point. If your editor is opened and you have 
> > the source code in front of you, why would you cut'n'paste a line 
> > instead of actually write the line number?
> 
> The 'perf probe --list schedule' sub-tool i outlined would display 
> relative line numbers for the function - starting at 0:
> 
> 000	/*
> 001	 * schedule() is the main scheduler function.
> 002	 */
> 003	asmlinkage void __sched schedule(void)
> 004	{
> 005		struct task_struct *prev, *next;
> 006		unsigned long *switch_count;
> 007		struct rq *rq;
> 008		int cpu;
> 009
> 010	need_resched:
> 011		preempt_disable();
> 012		cpu = smp_processor_id();
> 013		rq = cpu_rq(cpu);
> 014		rcu_sched_qs(cpu);
> 015		prev = rq->curr;
> 016		switch_count = &prev->nivcsw;
> 017
> 018		release_kernel_lock(prev);
> 
> That way the following two are equivalent:
> 
>   perf probe schedule
>   perf probe schedule+0
> 
> The advantage of relative line numbers is that they are much more 
> version invariant than absolute line numbers. Relative line numbers into 
> schedule() will only change if the function itself changes.
> 
> This means that expressions like 'schedule+16' will have a lot longer 
> life-time than absolute line number driven probes. You can quote it in 
> an email and chances are that it will still be valid even a few kernel 
> releases down the road.


I haven't thought that relative lines would make it less short-lived.
So yeah that's a good point.

I think absolute and relative line modes are not colliding/contending
at all but actually fit two different needs.

- absolute is nice when you are lonely doing kernel debugging.
  (can be expanded at will once you imagine user probes)
  You are stuck in your code editor, trying to figure out the
  origin of your problem and then you think it would be nice
  to set a probe in branch 1 and in branch 2 inside func_foo().
  Then you already have absolute lines and relying in
  perf probe --list func_foo() to resolve an absolute line into
  a relative one is a very undesired middle step.

- relative is nice in some other cases. When you already have
  the function target in mind, you even don't need to check your
  editor, just a quick check to this command and get the relative
  line. But also when you want to transmit a probe reference
  in a mailing list because of its better lifetime.


Both are important IMO.


> Furthermore - to answer another question you raised above - the 
> following syntax:
> 
>    perf probe schedule:'switch_count = &prev->nivcsw'
> 
> is basically a 'fuzzy string match' based probe to within a function. 
> 
> For example you might want to probe the point within schedule that calls 
> switch_mm() - this could be done via:
> 
>    perf probe schedule@switch_mm
> 
> Or the point where 'next' gets assigned? Sure, you dont need to even 
> open the editor, if you know the rough outline of the function you can 
> probe it via:
> 
>    perf probe schedule@'next ='



Yeah, that's a nice advanced usage.


> 
> Note that i was able to specify both probes without having opened an 
> editor - just based on the general knowledge of the scheduler.


Yeah, that a good workflow for someone who knows well the targeted
code.

That can't cover every needs though, but I guess you agree that
abs_line/rel_line/smart_resolution methods need to coexist anyway,
all of them fit different level of needs.


> The point is to prefer intuitive, non-mechanic, fundamentally human 
> expressions of information above mechanic ones (absolute line numbers, 
> addresses, ways of probing, etc.) - and to have a rich variety of them.


Agreed!

 
> String based pattern matching and intuitive syntax that reuses existing 
> paradigms of arithmetics and pattern-matching is good - limited syntax 
> and extra, arbitrary syntactic hoops to jump through is bad.
>
> If we provide all that, people will start using this stuff - and i'd 
> only like to merge this upstream once it's clear that people like me 
> will (be able to) use this facility for ad-hoc probe insertion.
> 
> In other words: this facility has to 'live within' our source code and 
> has to be able to interact with it on a very broad basis - for it to be 
> maximally useful for everyday development.
> 
> 	Ingo


Ok.

Thanks.

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Masami H. <mhi...@re...>]

Ingo Molnar wrote:
>>> Here are a few syntax suggestions
>>>
>>> The simpest probe syntax should be to add a probe to a single 
>>> function name:
>>>
>>>   perf probe +schedule
>>>
>>> _nothing else_.
>>>
>>> To remove it, the user should just do something like:
>>>
>>>   perf probe -schedule
>>>
>>> (to be symmetric 'perf probe +schedule' should work as well)
>>
>> I think '-<symbol>' syntax doesn't work good with other command-line 
>> options and multiple definitions. (However, it will be good for 
>> input-from-file syntax. :-))
> 
> dash can be used too - perf has the options library from Git and there's 
> a wide spectrum of option parsing available, via 
> tools/perf/util/parse-options.h.
> 
> But yes, it complicates things a bit.

Yeah, what I'm concerning about is that user will confuse when deleting
probe points which starts with other option, like 'k'.
(-kmalloc can mean -k malloc too)

>> So, what would you think about using -D (def) and -U (undef) ?
> 
> The simpest case should be no extra character at all:
> 
>   perf probe schedule
> 
> There's a few well-known command line idioms to add/remove stuff, but -D 
> / -U is not one of them i'm afraid =B-)
> 
> The following ones might work too:
> 
>   perf probe +schedule
>   perf probe -schedule
> 
>   perf probe add schedule
>   perf probe del schedule
> 
>   perf probe --add schedule
>   perf probe --del schedule
> 
> [ Plain 'add/del' has a minor complication as we could have a similar 
>   symbol defined. ]
> 
> + / - is certainly the simplest.
> 
> --add/--del works like routes do, so that's intuitive as well. As long 
> as we have the simple default to add a new probe at a function, without 
> any extra options we can do this too initially.

How about the following syntax?
<adding>
perf probe schedule
perf probe --add schedule

<deleting>
perf probe --del schedule
perf probe --del all	/* delete all probepoints */

So, this doesn't symmetric, but provides simple way to add a probe.

>>> All the other extensions and possibilities - arguments, variables, 
>>> source code lines, etc. should be natural and intuitive extensions 
>>> of this basic, minimal syntax.
>>
>> Don't you like current space(' ') separated arguments? :-) I mean, 
>> what is 'natural' syntax in your opinion?
> 
> Yeah, space separated arguments are nice too. The question is how to 
> specify a more precise coordinate for the bit we want to probe - and how 
> to specify the information we want to extract. Something like:
> 
>   perf schedule+15
> 
> would be a rather intuitive shortcut for '15 lines into the schedule() 
> function' - and it might even be a largely cross-kernel-version 
> compatible way of specifying probe points.

I agreed with the cross-kernel-version issue. I'd rather like

 perf probe symbol:relative-line

and

 perf probe file:absolute-line

since it will be familiar for GDB users.

And I'd like to preserve

 perf probe symbol+offs-byte

for assembly users who might want to trace assembly code with
objdump.

> Or this:
> 
>   perf schedule:'switch_count = &prev->nivcsw'
> 
> would insert the probe to the source code that matches that statement 
> pattern. Rarely will people want to insert a probe to an absolutely line 
> number - that's a usage mode for higher level tools. (so we definitely 
> want to support it - but it should not use up valuable spots in our 
> options space.) Same goes for symbol offsets, etc. - humans will rarely 
> use them.

Hmm, maybe, it's possible. I should investigate dwarf more...


Thank you!

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Arnaldo C. de M. <ac...@re...>]

Em Mon, Oct 19, 2009 at 01:00:57PM +0200, Frederic Weisbecker escreveu:
> On Mon, Oct 19, 2009 at 09:51:03AM +0200, Ingo Molnar wrote:
> > > So, what would you think about using -D (def) and -U (undef) ?
> > The simpest case should be no extra character at all:
> >   perf probe schedule
> Yeah, I really prefer that too.

/me too
 
> If you want more precision, it also means you have you code editor opened
> and want to set a precise point. Since you also have the absolute
> line directly displayed by your editor, you don't want to calculate the relative
> line but rather the absolute one.

Perhaps this should come with vim/emacs key bindings so that all this
gets hidden by just opening the source code and pressing
control+SOMEKEY?

- Arnaldo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Frederic Weisbecker <fwe...@gm...> wrote:

> On Mon, Oct 19, 2009 at 09:51:03AM +0200, Ingo Molnar wrote:
> > > So, what would you think about using -D (def) and -U (undef) ?
> > 
> > The simpest case should be no extra character at all:
> > 
> >   perf probe schedule
> 
> Yeah, I really prefer that too.
> 
> 
>  
> > > > All the other extensions and possibilities - arguments, 
> > > > variables, source code lines, etc. should be natural and 
> > > > intuitive extensions of this basic, minimal syntax.
> > > 
> > > Don't you like current space(' ') separated arguments? :-) I mean, 
> > > what is 'natural' syntax in your opinion?
> > 
> > Yeah, space separated arguments are nice too. The question is how to 
> > specify a more precise coordinate for the bit we want to probe - and 
> > how to specify the information we want to extract. Something like:
> > 
> >   perf schedule+15
> 
> I personally don't imagine common easy usecases that imply relative 
> line offsets but rather absolute lines.

Absolute line numbers could be expressed too, via:

   perf schedule@5396

There's no immediate need to express 'sched.c'. (you can do it, or you 
can do it for extra clarity or for the case of local scope functions of 
which multiple instances exist)

> I guess the most immediate usecase is a direct function probe:
> 
> 	perf probe schedule
> 
> Just to know if a function is matched.

Correct.

> If you want more precision, it also means you have you code editor 
> opened and want to set a precise point. Since you also have the 
> absolute line directly displayed by your editor, you don't want to 
> calculate the relative line but rather the absolute one.

Not necessarily - see below:

> Hmm?
> 
> Hence I rather imagine the following:
> 
> perf probe schedule.c:line
> 
> (Unfortunately, schedule:line is shorter but less intuitive but that 
> could be a shortcut).
> 
> > Or this:
> > 
> >   perf schedule:'switch_count = &prev->nivcsw'
> > 
> > would insert the probe to the source code that matches that statement 
> > pattern. Rarely will people want to insert a probe to an absolutely line 
> > number - that's a usage mode for higher level tools. (so we definitely 
> > want to support it - but it should not use up valuable spots in our 
> > options space.) Same goes for symbol offsets, etc. - humans will rarely 
> > use them.
> 
> I don't understand your point. If your editor is opened and you have 
> the source code in front of you, why would you cut'n'paste a line 
> instead of actually write the line number?

The 'perf probe --list schedule' sub-tool i outlined would display 
relative line numbers for the function - starting at 0:

000	/*
001	 * schedule() is the main scheduler function.
002	 */
003	asmlinkage void __sched schedule(void)
004	{
005		struct task_struct *prev, *next;
006		unsigned long *switch_count;
007		struct rq *rq;
008		int cpu;
009
010	need_resched:
011		preempt_disable();
012		cpu = smp_processor_id();
013		rq = cpu_rq(cpu);
014		rcu_sched_qs(cpu);
015		prev = rq->curr;
016		switch_count = &prev->nivcsw;
017
018		release_kernel_lock(prev);

That way the following two are equivalent:

  perf probe schedule
  perf probe schedule+0

The advantage of relative line numbers is that they are much more 
version invariant than absolute line numbers. Relative line numbers into 
schedule() will only change if the function itself changes.

This means that expressions like 'schedule+16' will have a lot longer 
life-time than absolute line number driven probes. You can quote it in 
an email and chances are that it will still be valid even a few kernel 
releases down the road.

> > We also want to have functionality that helps people find probe 
> > spots within a function:
> > 
> >   perf probe --list-lines schedule
> > 
> > Would list the line numbers and source code of the schedule() 
> > function. (similar to how GDB 'list' works) That way someone can 
> > have an ad-hoc session of deciding what place to probe, and the line 
> > numbers make for an easy ID of the statement to probe.
> 
> Agreed!

Furthermore - to answer another question you raised above - the 
following syntax:

   perf probe schedule:'switch_count = &prev->nivcsw'

is basically a 'fuzzy string match' based probe to within a function. 

For example you might want to probe the point within schedule that calls 
switch_mm() - this could be done via:

   perf probe schedule@switch_mm

Or the point where 'next' gets assigned? Sure, you dont need to even 
open the editor, if you know the rough outline of the function you can 
probe it via:

   perf probe schedule@'next ='

Note that i was able to specify both probes without having opened an 
editor - just based on the general knowledge of the scheduler.

The point is to prefer intuitive, non-mechanic, fundamentally human 
expressions of information above mechanic ones (absolute line numbers, 
addresses, ways of probing, etc.) - and to have a rich variety of them.

String based pattern matching and intuitive syntax that reuses existing 
paradigms of arithmetics and pattern-matching is good - limited syntax 
and extra, arbitrary syntactic hoops to jump through is bad.

If we provide all that, people will start using this stuff - and i'd 
only like to merge this upstream once it's clear that people like me 
will (be able to) use this facility for ad-hoc probe insertion.

In other words: this facility has to 'live within' our source code and 
has to be able to interact with it on a very broad basis - for it to be 
maximally useful for everyday development.

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Frederic W. <fwe...@gm...>]

On Mon, Oct 19, 2009 at 09:51:03AM +0200, Ingo Molnar wrote:
> > So, what would you think about using -D (def) and -U (undef) ?
> 
> The simpest case should be no extra character at all:
> 
>   perf probe schedule


Yeah, I really prefer that too.


 
> > > All the other extensions and possibilities - arguments, variables, 
> > > source code lines, etc. should be natural and intuitive extensions 
> > > of this basic, minimal syntax.
> > 
> > Don't you like current space(' ') separated arguments? :-) I mean, 
> > what is 'natural' syntax in your opinion?
> 
> Yeah, space separated arguments are nice too. The question is how to 
> specify a more precise coordinate for the bit we want to probe - and how 
> to specify the information we want to extract. Something like:
> 
>   perf schedule+15


I personally don't imagine common easy usecases that imply relative line
offsets but rather absolute lines.

I guess the most immediate usecase is a direct function probe:

	perf probe schedule

Just to know if a function is matched.

If you want more precision, it also means you have you code editor opened
and want to set a precise point. Since you also have the absolute
line directly displayed by your editor, you don't want to calculate the relative
line but rather the absolute one.

Hmm?

Hence I rather imagine the following:

perf probe schedule.c:line

(Unfortunately, schedule:line is shorter but less intuitive
but that could be a shortcut).



> Or this:
> 
>   perf schedule:'switch_count = &prev->nivcsw'
> 
> would insert the probe to the source code that matches that statement 
> pattern. Rarely will people want to insert a probe to an absolutely line 
> number - that's a usage mode for higher level tools. (so we definitely 
> want to support it - but it should not use up valuable spots in our 
> options space.) Same goes for symbol offsets, etc. - humans will rarely 
> use them.



I don't understand your point. If your editor is opened and you have
the source code in front of you, why would you cut'n'paste a line instead
of actually write the line number?



> 
> We also want to have functionality that helps people find probe spots 
> within a function:
> 
>   perf probe --list-lines schedule
> 
> Would list the line numbers and source code of the schedule() function. 
> (similar to how GDB 'list' works) That way someone can have an ad-hoc 
> session of deciding what place to probe, and the line numbers make for 
> an easy ID of the statement to probe.


Agreed!

Thanks.

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Masami Hiramatsu <mhi...@re...> wrote:

> Ingo Molnar wrote:
> > 
> > I took a good look at the current bits, and there are a few more things 
> > that need to be fixed before we can consider 'perf probe' for upstream.
> > 
> > Firstly, this decoder bug is still not fixed:
> > 
> >   CHK     include/linux/compile.h
> >   TEST    posttest
> > Error: ffffffff81068fe0:        66 0f 73 fd 04          pslldq $0x4,%xmm5
> > Error: objdump says 5 bytes, but insn_get_length() says 4 (attr:8000)
> > make[1]: *** [posttest] Error 2
> > 
> > 64-bit allyesconfig will trigger this for example, but the attached 
> > smaller config too. This needs to be fixed before we can apply any
> > new commits.
> 
> Absolutely, yes. Thank you for reporting. I'm checking it again.

Thanks!

> > Secondly, the probe syntax is quite non-obvious currently. All the 
> > 'p' and -P gymnastics is just a barrier to the first-time user 
> > getting his first probe inserted successfully.
> 
> Hmm...
> 
> > The user need not worry about whether it's a 'kprobe' or a 
> > 'kretprobe'. The user should _specify_ what it wants to probe, and 
> > _that_ will lead to 'perf probe' picking the most suitable facility 
> > to achieve that kind of probing.
> > 
> > It might be a kprobe, a kretprobe, or an mcount driven function 
> > probe, an existing tracepoint if it happens to be present in that 
> > place already - or some other future mechanism. The driving force 
> > must be a robust specification of 'what', not the mechanics of 
> > 'how'.
> 
> Agreed.
> 
> > Considering that, the current 'perf probe' syntax does not achieve 
> > that goal yet.
> > 
> > Here are a few syntax suggestions
> > 
> > The simpest probe syntax should be to add a probe to a single 
> > function name:
> > 
> >   perf probe +schedule
> > 
> > _nothing else_.
> > 
> > To remove it, the user should just do something like:
> > 
> >   perf probe -schedule
> > 
> > (to be symmetric 'perf probe +schedule' should work as well)
> 
> I think '-<symbol>' syntax doesn't work good with other command-line 
> options and multiple definitions. (However, it will be good for 
> input-from-file syntax. :-))

dash can be used too - perf has the options library from Git and there's 
a wide spectrum of option parsing available, via 
tools/perf/util/parse-options.h.

But yes, it complicates things a bit.

> So, what would you think about using -D (def) and -U (undef) ?

The simpest case should be no extra character at all:

  perf probe schedule

There's a few well-known command line idioms to add/remove stuff, but -D 
/ -U is not one of them i'm afraid =B-)

The following ones might work too:

  perf probe +schedule
  perf probe -schedule

  perf probe add schedule
  perf probe del schedule

  perf probe --add schedule
  perf probe --del schedule

[ Plain 'add/del' has a minor complication as we could have a similar 
  symbol defined. ]

+ / - is certainly the simplest.

--add/--del works like routes do, so that's intuitive as well. As long 
as we have the simple default to add a new probe at a function, without 
any extra options we can do this too initially.

> > perf probe will make up a synthetic probe name for that - probe-1 
> > for example. It will also pick the suitable probe mechanism 
> > (kprobes).
> 
> How about [perfprobe:symbol_offs] ?

Yeah, that's a nice idea - naming it after the symbol keeps the probe 
namespace still very readable.

> > All the other extensions and possibilities - arguments, variables, 
> > source code lines, etc. should be natural and intuitive extensions 
> > of this basic, minimal syntax.
> 
> Don't you like current space(' ') separated arguments? :-) I mean, 
> what is 'natural' syntax in your opinion?

Yeah, space separated arguments are nice too. The question is how to 
specify a more precise coordinate for the bit we want to probe - and how 
to specify the information we want to extract. Something like:

  perf schedule+15

would be a rather intuitive shortcut for '15 lines into the schedule() 
function' - and it might even be a largely cross-kernel-version 
compatible way of specifying probe points.

Or this:

  perf schedule:'switch_count = &prev->nivcsw'

would insert the probe to the source code that matches that statement 
pattern. Rarely will people want to insert a probe to an absolutely line 
number - that's a usage mode for higher level tools. (so we definitely 
want to support it - but it should not use up valuable spots in our 
options space.) Same goes for symbol offsets, etc. - humans will rarely 
use them.

We also want to have functionality that helps people find probe spots 
within a function:

  perf probe --list-lines schedule

Would list the line numbers and source code of the schedule() function. 
(similar to how GDB 'list' works) That way someone can have an ad-hoc 
session of deciding what place to probe, and the line numbers make for 
an easy ID of the statement to probe.

Anyway, these details make or break the actual utility of this tool, so 
lets iterate this some more and we'll see the limitations and the needs 
in practice. As usual, tool design rarely survives first contact with an 
actual user - so we have to show some adaptibility ;-)

	Ingo

[Dle-develop] DID YOU RECEIVE MY EMAIL?

[RAPHEL A. <ric...@ya...>]

新しいメールアドレスをお知らせします新しいメールアドレス： ric...@ya...

I AM AKUBUEZE, THE CHIEF ACCOUNTANT WITH THE F.M.T NIGERIA.We have some funds to be transferred out of my ministry,$21 million.If you are interested to help us out in this DEAL, reply for more details at email:RAK...@Ag... or Tel 2348078554643

- RAPHEL AKUBUEZE

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Masami H. <mhi...@re...>]

Ingo Molnar wrote:
> 
> I took a good look at the current bits, and there are a few more things 
> that need to be fixed before we can consider 'perf probe' for upstream.
> 
> Firstly, this decoder bug is still not fixed:
> 
>   CHK     include/linux/compile.h
>   TEST    posttest
> Error: ffffffff81068fe0:        66 0f 73 fd 04          pslldq $0x4,%xmm5
> Error: objdump says 5 bytes, but insn_get_length() says 4 (attr:8000)
> make[1]: *** [posttest] Error 2
> 
> 64-bit allyesconfig will trigger this for example, but the attached 
> smaller config too. This needs to be fixed before we can apply any
> new commits.

Absolutely, yes. Thank you for reporting. I'm checking it again.

> Secondly, the probe syntax is quite non-obvious currently. All the 'p' 
> and -P gymnastics is just a barrier to the first-time user getting his 
> first probe inserted successfully.

Hmm...

> The user need not worry about whether it's a 'kprobe' or a 'kretprobe'. 
> The user should _specify_ what it wants to probe, and _that_ will lead 
> to 'perf probe' picking the most suitable facility to achieve that kind 
> of probing.
> 
> It might be a kprobe, a kretprobe, or an mcount driven function probe, 
> an existing tracepoint if it happens to be present in that place already 
> - or some other future mechanism. The driving force must be a robust 
> specification of 'what', not the mechanics of 'how'.

Agreed.

> Considering that, the current 'perf probe' syntax does not achieve that 
> goal yet.
> 
> Here are a few syntax suggestions
> 
> The simpest probe syntax should be to add a probe to a single function 
> name:
> 
>   perf probe +schedule
> 
> _nothing else_.
> 
> To remove it, the user should just do something like:
> 
>   perf probe -schedule
> 
> (to be symmetric 'perf probe +schedule' should work as well)

I think '-<symbol>' syntax doesn't work good with other command-line
options and multiple definitions.
(However, it will be good for input-from-file syntax. :-))

So, what would you think about using -D (def) and -U (undef) ?

> perf probe will make up a synthetic probe name for that - probe-1 for 
> example. It will also pick the suitable probe mechanism (kprobes).

How about [perfprobe:symbol_offs] ?

> All the other extensions and possibilities - arguments, variables, 
> source code lines, etc. should be natural and intuitive extensions of 
> this basic, minimal syntax.

Don't you like current space(' ') separated arguments? :-)
I mean, what is 'natural' syntax in your opinion?

> 
> To insert a simple probe no -P should be needed, 'p', no ':' - no probe 
> name even.

Yeah, return-probe and event-name should be optional.

> Furthermore, there should be a way to list existing probes (and only 
> probes), probably via 'perf list --probes' or 'perf probe --list'.

OK. I think perf probe --list will list up all probes including
user-defined ftrace-event via debugfs (not perf-probe), since
perf can use and delete it.

> Plus, 'perf probe --help' should list a few simple examples, beyond the 
> syntax.
> 
> Ok?

Sure.

Thank you!


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Ingo Molnar <mi...@el...> wrote:

> The simpest probe syntax should be to add a probe to a single function 
> name:
> 
>   perf probe +schedule
> 
> _nothing else_.

here i meant the shortcut without the '+':

   perf probe schedule

> To remove it, the user should just do something like:
> 
>   perf probe -schedule
> 
> (to be symmetric 'perf probe +schedule' should work as well)

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

I took a good look at the current bits, and there are a few more things 
that need to be fixed before we can consider 'perf probe' for upstream.

Firstly, this decoder bug is still not fixed:

  CHK     include/linux/compile.h
  TEST    posttest
Error: ffffffff81068fe0:        66 0f 73 fd 04          pslldq $0x4,%xmm5
Error: objdump says 5 bytes, but insn_get_length() says 4 (attr:8000)
make[1]: *** [posttest] Error 2

64-bit allyesconfig will trigger this for example, but the attached 
smaller config too. This needs to be fixed before we can apply any
new commits.

Secondly, the probe syntax is quite non-obvious currently. All the 'p' 
and -P gymnastics is just a barrier to the first-time user getting his 
first probe inserted successfully.

The user need not worry about whether it's a 'kprobe' or a 'kretprobe'. 
The user should _specify_ what it wants to probe, and _that_ will lead 
to 'perf probe' picking the most suitable facility to achieve that kind 
of probing.

It might be a kprobe, a kretprobe, or an mcount driven function probe, 
an existing tracepoint if it happens to be present in that place already 
- or some other future mechanism. The driving force must be a robust 
specification of 'what', not the mechanics of 'how'.

Considering that, the current 'perf probe' syntax does not achieve that 
goal yet.

Here are a few syntax suggestions

The simpest probe syntax should be to add a probe to a single function 
name:

  perf probe +schedule

_nothing else_.

To remove it, the user should just do something like:

  perf probe -schedule

(to be symmetric 'perf probe +schedule' should work as well)

perf probe will make up a synthetic probe name for that - probe-1 for 
example. It will also pick the suitable probe mechanism (kprobes).

All the other extensions and possibilities - arguments, variables, 
source code lines, etc. should be natural and intuitive extensions of 
this basic, minimal syntax.

To insert a simple probe no -P should be needed, 'p', no ':' - no probe 
name even.

Furthermore, there should be a way to list existing probes (and only 
probes), probably via 'perf list --probes' or 'perf probe --list'.

Plus, 'perf probe --help' should list a few simple examples, beyond the 
syntax.

Ok?

	Ingo

Re: [Dle-develop] [PATCH -tip tracing/kprobes 0/9] tracing/kprobes, perf: perf probe and kprobe-tracer bugfixes

[Ingo M. <mi...@el...>]

* Masami Hiramatsu <mhi...@re...> wrote:

> Hi Ingo and Frederic,
> 
> Here are the bugfix and update (mostly cleanup) patches for
> previous patchset.
> 
> > I hope it's part of the last family of instruction set we
> > are missing.
> 
> I added missing SSE opcodes and 3DNow! support too.
> However, near future, x86 decoder may need AVX support.
> (AFAIK, currently, there are no code using it.)
> 
> Thank you,
> 
> ---
> 
> Masami Hiramatsu (9):
>       perf: Add perf-probe document
>       perf: Add DIE_IF() macro for error checking
>       perf: Use eprintf() for debug messages in perf-probe
>       perf: Use die() for error cases in perf-probe
>       perf: Check libdwarf APIs for perf probe
>       x86: Add AMD prefetch and 3DNow! opcodes to opcode map
>       x86: Add MMX/SSE opcode groups to opcode map
>       tracing/kprobes: Add failure messages for debugging
>       tracing/kprobes: Update kprobe-tracer selftest against new syntax
> 
> 
>  arch/x86/lib/x86-opcode-map.txt         |   23 ++++-
>  kernel/trace/trace_kprobe.c             |   39 ++++++--
>  tools/perf/Documentation/perf-probe.txt |   48 ++++++++++
>  tools/perf/Makefile                     |    5 +
>  tools/perf/builtin-probe.c              |   70 ++++++---------
>  tools/perf/command-list.txt             |    1 
>  tools/perf/util/probe-finder.c          |  149 ++++++++++++++-----------------
>  tools/perf/util/probe-finder.h          |   17 ----
>  tools/perf/util/util.h                  |    9 ++
>  9 files changed, 206 insertions(+), 155 deletions(-)
>  create mode 100644 tools/perf/Documentation/perf-probe.txt

Looks really nice, thanks Masami!

Note, i've created a new topic tree for this work: tip:perf/probes, and 
have put all the commits there - since i expect most of the 
enabling/completion work for this feature to happen on the perf events 
side.

I also merged this tree upto v2.6.32-rc5 and resolved a conflict with 
recent tracing fixes. Please base future patches on this new tree.

Thanks,

	Ingo

[Dle-develop] [PATCH -tip tracing/kprobes 9/9] perf: Add perf-probe document

[Masami H. <mhi...@re...>]

Add perf-probe subcommand document and add it to command-list.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
Cc: Thomas Gleixner <tg...@li...>
Cc: Arnaldo Carvalho de Melo <ac...@re...>
Cc: Steven Rostedt <ro...@go...>
Cc: Mike Galbraith <ef...@gm...>
Cc: Paul Mackerras <pa...@sa...>
Cc: Peter Zijlstra <a.p...@ch...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Jim Keniston <jke...@us...>
Cc: Frank Ch. Eigler <fc...@re...>
---

 tools/perf/Documentation/perf-probe.txt |   48 +++++++++++++++++++++++++++++++
 tools/perf/command-list.txt             |    1 +
 2 files changed, 49 insertions(+), 0 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-probe.txt

diff --git a/tools/perf/Documentation/perf-probe.txt b/tools/perf/Documentation/perf-probe.txt
new file mode 100644
index 0000000..6b6c6ae
--- /dev/null
+++ b/tools/perf/Documentation/perf-probe.txt
@@ -0,0 +1,48 @@
+perf-probe(1)
+=============
+
+NAME
+----
+perf-probe - Define new dynamic tracepoints
+
+SYNOPSIS
+--------
+[verse]
+'perf probe' [-k <file>] -P 'PROBE' [-P 'PROBE' ...]
+
+
+DESCRIPTION
+-----------
+This command defines dynamic tracepoint events, by symbol and registers
+without debuginfo, or by C expressions (C line numbers, C function names,
+and C local variables) with debuginfo.
+
+
+OPTIONS
+-------
+-k::
+--vmlinux::
+	Specify vmlinux path which has debuginfo (Dwarf binary).
+
+-v::
+--verbose::
+        Be more verbose (show parsed arguments, etc).
+
+-P::
+--probe::
+	Define a probe point (see PROBE SYNTAX for detail)
+
+PROBE SYNTAX
+------------
+Probe points are defined by following syntax.
+
+ "TYPE:[GRP/]NAME FUNC[+OFFS][@SRC]|@SRC:LINE [ARG ...]"
+
+'TYPE' specifies the type of probe point, you can use either "p" (kprobe) or "r" (kretprobe) for 'TYPE'. 'GRP' specifies the group name of this probe, and 'NAME' specifies the event name. If 'GRP' is omitted, "kprobes" is used for its group name.
+'FUNC' and 'OFFS' specifies function and offset (in byte) where probe will be put. In addition, 'SRC' specifies a source file which has that function (this is mainly for inline functions).
+You can specify a probe point by the source line number by using '@SRC:LINE' syntax, where 'SRC' is the source file path and 'LINE' is the line number.
+'ARG' specifies the arguments of this probe point. You can use the name of local variable, or kprobe-tracer argument format (e.g. $retval, %ax, etc).
+
+SEE ALSO
+--------
+linkperf:perf-trace[1], linkperf:perf-record[1]
diff --git a/tools/perf/command-list.txt b/tools/perf/command-list.txt
index 00326e2..6475db4 100644
--- a/tools/perf/command-list.txt
+++ b/tools/perf/command-list.txt
@@ -11,3 +11,4 @@ perf-stat			mainporcelain common
 perf-timechart			mainporcelain common
 perf-top			mainporcelain common
 perf-trace			mainporcelain common
+perf-probe			mainporcelain common


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...

[Dle-develop] [PATCH -tip tracing/kprobes 8/9] perf: Add DIE_IF() macro for error checking

[Masami H. <mhi...@re...>]

Add DIE_IF() macro and replace ERR_IF() with it, and use
linux/stringify.h.

Signed-off-by: Masami Hiramatsu <mhi...@re...>
Cc: Frederic Weisbecker <fwe...@gm...>
Cc: Ingo Molnar <mi...@el...>
Cc: Thomas Gleixner <tg...@li...>
Cc: Arnaldo Carvalho de Melo <ac...@re...>
Cc: Steven Rostedt <ro...@go...>
Cc: Mike Galbraith <ef...@gm...>
Cc: Paul Mackerras <pa...@sa...>
Cc: Peter Zijlstra <a.p...@ch...>
Cc: Christoph Hellwig <hc...@in...>
Cc: Ananth N Mavinakayanahalli <an...@in...>
Cc: Jim Keniston <jke...@us...>
Cc: Frank Ch. Eigler <fc...@re...>
---

 tools/perf/Makefile            |    1 
 tools/perf/util/probe-finder.c |   82 ++++++++++++++++++++--------------------
 tools/perf/util/probe-finder.h |   10 -----
 tools/perf/util/util.h         |    9 ++++
 4 files changed, 51 insertions(+), 51 deletions(-)

diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index 03c27b9..1abbf9a 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -321,6 +321,7 @@ LIB_FILE=libperf.a
 LIB_H += ../../include/linux/perf_event.h
 LIB_H += ../../include/linux/rbtree.h
 LIB_H += ../../include/linux/list.h
+LIB_H += ../../include/linux/stringify.h
 LIB_H += util/include/linux/list.h
 LIB_H += perf.h
 LIB_H += util/types.h
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index db24c91..be997ab 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -146,7 +146,7 @@ static int die_compare_name(Dwarf_Die dw_die, const char *tname)
 	char *name;
 	int ret;
 	ret = dwarf_diename(dw_die, &name, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (ret == DW_DLV_OK) {
 		ret = strcmp(tname, name);
 		dwarf_dealloc(__dw_debug, name, DW_DLA_STRING);
@@ -164,11 +164,11 @@ static int die_within_subprogram(Dwarf_Die sp_die, Dwarf_Addr addr,
 
 	/* TODO: check ranges */
 	ret = dwarf_lowpc(sp_die, &lopc, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (ret == DW_DLV_NO_ENTRY)
 		return 0;
 	ret = dwarf_highpc(sp_die, &hipc, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	if (lopc <= addr && addr < hipc) {
 		*offs = addr - lopc;
 		return 1;
@@ -184,7 +184,7 @@ static Dwarf_Bool die_inlined_subprogram(Dwarf_Die dw_die)
 	int ret;
 
 	ret = dwarf_hasattr(dw_die, DW_AT_inline, &inl, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	return inl;
 }
 
@@ -196,9 +196,9 @@ static Dwarf_Off die_get_abstract_origin(Dwarf_Die dw_die)
 	int ret;
 
 	ret = dwarf_attr(dw_die, DW_AT_abstract_origin, &attr, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	ret = dwarf_formref(attr, &cu_offs, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	dwarf_dealloc(__dw_debug, attr, DW_DLA_ATTR);
 	return cu_offs;
 }
@@ -215,28 +215,28 @@ static Dwarf_Addr die_get_entrypc(Dwarf_Die dw_die)
 
 	/* Try to get entry pc */
 	ret = dwarf_attr(dw_die, DW_AT_entry_pc, &attr, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (ret == DW_DLV_OK) {
 		ret = dwarf_formaddr(attr, &addr, &__dw_error);
-		ERR_IF(ret != DW_DLV_OK);
+		DIE_IF(ret != DW_DLV_OK);
 		dwarf_dealloc(__dw_debug, attr, DW_DLA_ATTR);
 		return addr;
 	}
 
 	/* Try to get low pc */
 	ret = dwarf_lowpc(dw_die, &addr, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (ret == DW_DLV_OK)
 		return addr;
 
 	/* Try to get ranges */
 	ret = dwarf_attr(dw_die, DW_AT_ranges, &attr, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	ret = dwarf_formref(attr, &offs, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	ret = dwarf_get_ranges(__dw_debug, offs, &ranges, &cnt, NULL,
 				&__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	addr = ranges[0].dwr_addr1;
 	dwarf_ranges_dealloc(__dw_debug, ranges, cnt);
 	return addr;
@@ -261,7 +261,7 @@ static int __search_die_tree(struct die_link *cur_link,
 	while (!(ret = die_cb(cur_link, data))) {
 		/* Check child die */
 		ret = dwarf_child(cur_link->die, &new_die, &__dw_error);
-		ERR_IF(ret == DW_DLV_ERROR);
+		DIE_IF(ret == DW_DLV_ERROR);
 		if (ret == DW_DLV_OK) {
 			new_link.parent = cur_link;
 			new_link.die = new_die;
@@ -273,7 +273,7 @@ static int __search_die_tree(struct die_link *cur_link,
 		/* Move to next sibling */
 		ret = dwarf_siblingof(__dw_debug, cur_link->die, &new_die,
 				      &__dw_error);
-		ERR_IF(ret == DW_DLV_ERROR);
+		DIE_IF(ret == DW_DLV_ERROR);
 		dwarf_dealloc(__dw_debug, cur_link->die, DW_DLA_DIE);
 		cur_link->die = new_die;
 		if (ret == DW_DLV_NO_ENTRY)
@@ -293,7 +293,7 @@ static int search_die_from_children(Dwarf_Die parent_die,
 
 	new_link.parent = NULL;
 	ret = dwarf_child(parent_die, &new_link.die, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (ret == DW_DLV_OK)
 		return __search_die_tree(&new_link, die_cb, data);
 	else
@@ -309,7 +309,7 @@ static int attr_get_locdesc(Dwarf_Attribute attr, Dwarf_Locdesc *desc,
 	int ret, i;
 
 	ret = dwarf_loclist_n(attr, &llbuf, &lcnt, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	ret = DW_DLV_NO_ENTRY;
 	for (i = 0; i < lcnt; ++i) {
 		if (llbuf[i]->ld_lopc <= addr &&
@@ -317,7 +317,7 @@ static int attr_get_locdesc(Dwarf_Attribute attr, Dwarf_Locdesc *desc,
 			memcpy(desc, llbuf[i], sizeof(Dwarf_Locdesc));
 			desc->ld_s =
 				malloc(sizeof(Dwarf_Loc) * llbuf[i]->ld_cents);
-			ERR_IF(desc->ld_s == NULL);
+			DIE_IF(desc->ld_s == NULL);
 			memcpy(desc->ld_s, llbuf[i]->ld_s,
 				sizeof(Dwarf_Loc) * llbuf[i]->ld_cents);
 			ret = DW_DLV_OK;
@@ -383,8 +383,8 @@ static void show_location(Dwarf_Loc *loc, struct probe_finder *pf)
 				 " %s=%+lld(%s)", pf->var, offs, regs);
 	else
 		ret = snprintf(pf->buf, pf->len, " %s=%s", pf->var, regs);
-	ERR_IF(ret < 0);
-	ERR_IF(ret >= pf->len);
+	DIE_IF(ret < 0);
+	DIE_IF(ret >= pf->len);
 }
 
 /* Show a variables in kprobe event format */
@@ -401,7 +401,7 @@ static void show_variable(Dwarf_Die vr_die, struct probe_finder *pf)
 	if (ret != DW_DLV_OK)
 		goto error;
 	/* TODO? */
-	ERR_IF(ld.ld_cents != 1);
+	DIE_IF(ld.ld_cents != 1);
 	show_location(&ld.ld_s[0], pf);
 	free(ld.ld_s);
 	dwarf_dealloc(__dw_debug, attr, DW_DLA_ATTR);
@@ -418,7 +418,7 @@ static int variable_callback(struct die_link *dlink, void *data)
 	int ret;
 
 	ret = dwarf_tag(dlink->die, &tag, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if ((tag == DW_TAG_formal_parameter ||
 	     tag == DW_TAG_variable) &&
 	    (die_compare_name(dlink->die, pf->var) == 0)) {
@@ -437,8 +437,8 @@ static void find_variable(Dwarf_Die sp_die, struct probe_finder *pf)
 	if (!is_c_varname(pf->var)) {
 		/* Output raw parameters */
 		ret = snprintf(pf->buf, pf->len, " %s", pf->var);
-		ERR_IF(ret < 0);
-		ERR_IF(ret >= pf->len);
+		DIE_IF(ret < 0);
+		DIE_IF(ret >= pf->len);
 		return ;
 	}
 
@@ -456,9 +456,9 @@ static void get_current_frame_base(Dwarf_Die sp_die, struct probe_finder *pf)
 	int ret;
 
 	ret = dwarf_attr(sp_die, DW_AT_frame_base, &attr, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	ret = attr_get_locdesc(attr, &pf->fbloc, (pf->addr - pf->cu_base));
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 	dwarf_dealloc(__dw_debug, attr, DW_DLA_ATTR);
 }
 
@@ -479,7 +479,7 @@ static void show_probepoint(Dwarf_Die sp_die, Dwarf_Signed offs,
 
 	/* Output name of probe point */
 	ret = dwarf_diename(sp_die, &name, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (ret == DW_DLV_OK) {
 		ret = snprintf(tmp, MAX_PROBE_BUFFER, "%s+%u", name,
 				(unsigned int)offs);
@@ -488,8 +488,8 @@ static void show_probepoint(Dwarf_Die sp_die, Dwarf_Signed offs,
 		/* This function has no name. */
 		ret = snprintf(tmp, MAX_PROBE_BUFFER, "0x%llx", pf->addr);
 	}
-	ERR_IF(ret < 0);
-	ERR_IF(ret >= MAX_PROBE_BUFFER);
+	DIE_IF(ret < 0);
+	DIE_IF(ret >= MAX_PROBE_BUFFER);
 	len = ret;
 
 	/* Find each argument */
@@ -515,7 +515,7 @@ static int probeaddr_callback(struct die_link *dlink, void *data)
 	int ret;
 
 	ret = dwarf_tag(dlink->die, &tag, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	/* Check the address is in this subprogram */
 	if (tag == DW_TAG_subprogram &&
 	    die_within_subprogram(dlink->die, pf->addr, &offs)) {
@@ -537,21 +537,21 @@ static void find_by_line(Dwarf_Die cu_die, struct probe_finder *pf)
 	int ret;
 
 	ret = dwarf_srclines(cu_die, &lines, &cnt, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 
 	for (i = 0; i < cnt; i++) {
 		ret = dwarf_line_srcfileno(lines[i], &fno, &__dw_error);
-		ERR_IF(ret != DW_DLV_OK);
+		DIE_IF(ret != DW_DLV_OK);
 		if (fno != pf->fno)
 			continue;
 
 		ret = dwarf_lineno(lines[i], &lineno, &__dw_error);
-		ERR_IF(ret != DW_DLV_OK);
+		DIE_IF(ret != DW_DLV_OK);
 		if (lineno != (Dwarf_Unsigned)pp->line)
 			continue;
 
 		ret = dwarf_lineaddr(lines[i], &addr, &__dw_error);
-		ERR_IF(ret != DW_DLV_OK);
+		DIE_IF(ret != DW_DLV_OK);
 		eprintf("Probe point found: 0x%llx\n", addr);
 		pf->addr = addr;
 		/* Search a real subprogram including this line, */
@@ -574,7 +574,7 @@ static int probefunc_callback(struct die_link *dlink, void *data)
 	int ret;
 
 	ret = dwarf_tag(dlink->die, &tag, &__dw_error);
-	ERR_IF(ret == DW_DLV_ERROR);
+	DIE_IF(ret == DW_DLV_ERROR);
 	if (tag == DW_TAG_subprogram) {
 		if (die_compare_name(dlink->die, pp->function) == 0) {
 			if (die_inlined_subprogram(dlink->die)) {
@@ -582,7 +582,7 @@ static int probefunc_callback(struct die_link *dlink, void *data)
 				ret = dwarf_die_CU_offset(dlink->die,
 							  &pf->inl_offs,
 							  &__dw_error);
-				ERR_IF(ret != DW_DLV_OK);
+				DIE_IF(ret != DW_DLV_OK);
 				eprintf("inline definition offset %lld\n",
 					pf->inl_offs);
 				return 0;
@@ -604,7 +604,7 @@ static int probefunc_callback(struct die_link *dlink, void *data)
 			for (lk = dlink->parent; lk != NULL; lk = lk->parent) {
 				tag = 0;
 				dwarf_tag(lk->die, &tag, &__dw_error);
-				ERR_IF(ret == DW_DLV_ERROR);
+				DIE_IF(ret == DW_DLV_ERROR);
 				if (tag == DW_TAG_subprogram &&
 				    !die_inlined_subprogram(lk->die))
 					goto found;
@@ -613,7 +613,7 @@ static int probefunc_callback(struct die_link *dlink, void *data)
 found:
 			/* Get offset from subprogram */
 			ret = die_within_subprogram(lk->die, pf->addr, &offs);
-			ERR_IF(!ret);
+			DIE_IF(!ret);
 			show_probepoint(lk->die, offs, pf);
 			/* Continue to search */
 		}
@@ -644,13 +644,13 @@ int find_probepoint(int fd, struct probe_point *pp)
 		/* Search CU (Compilation Unit) */
 		ret = dwarf_next_cu_header(__dw_debug, NULL, NULL, NULL,
 			&addr_size, &next_cuh, &__dw_error);
-		ERR_IF(ret == DW_DLV_ERROR);
+		DIE_IF(ret == DW_DLV_ERROR);
 		if (ret == DW_DLV_NO_ENTRY)
 			break;
 
 		/* Get the DIE(Debugging Information Entry) of this CU */
 		ret = dwarf_siblingof(__dw_debug, 0, &cu_die, &__dw_error);
-		ERR_IF(ret != DW_DLV_OK);
+		DIE_IF(ret != DW_DLV_OK);
 
 		/* Check if target file is included. */
 		if (pp->file)
@@ -659,7 +659,7 @@ int find_probepoint(int fd, struct probe_point *pp)
 		if (!pp->file || pf.fno) {
 			/* Save CU base address (for frame_base) */
 			ret = dwarf_lowpc(cu_die, &pf.cu_base, &__dw_error);
-			ERR_IF(ret == DW_DLV_ERROR);
+			DIE_IF(ret == DW_DLV_ERROR);
 			if (ret == DW_DLV_NO_ENTRY)
 				pf.cu_base = 0;
 			if (pp->line)
@@ -670,7 +670,7 @@ int find_probepoint(int fd, struct probe_point *pp)
 		dwarf_dealloc(__dw_debug, cu_die, DW_DLA_DIE);
 	}
 	ret = dwarf_finish(__dw_debug, &__dw_error);
-	ERR_IF(ret != DW_DLV_OK);
+	DIE_IF(ret != DW_DLV_OK);
 
 	return pp->found;
 }
diff --git a/tools/perf/util/probe-finder.h b/tools/perf/util/probe-finder.h
index 6a7cb0c..d17fafc 100644
--- a/tools/perf/util/probe-finder.h
+++ b/tools/perf/util/probe-finder.h
@@ -1,16 +1,6 @@
 #ifndef _PROBE_FINDER_H
 #define _PROBE_FINDER_H
 
-#define _stringify(n)	#n
-#define stringify(n)	_stringify(n)
-
-#define ERR_IF(cnd)	\
-	do { if (cnd) {	\
-		fprintf(stderr, "Error (" __FILE__ ":" stringify(__LINE__) \
-			"): " stringify(cnd) "\n");			\
-		exit(1);						\
-	} } while (0)
-
 #define MAX_PATH_LEN 256
 #define MAX_PROBE_BUFFER 1024
 #define MAX_PROBES 128
diff --git a/tools/perf/util/util.h b/tools/perf/util/util.h
index 9de2329..0daa341 100644
--- a/tools/perf/util/util.h
+++ b/tools/perf/util/util.h
@@ -134,6 +134,15 @@ extern void die(const char *err, ...) NORETURN __attribute__((format (printf, 1,
 extern int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 extern void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 
+#include "../../../include/linux/stringify.h"
+
+#define DIE_IF(cnd)	\
+	do { if (cnd)	\
+		die(" at (" __FILE__ ":" __stringify(__LINE__) "): "	\
+		    __stringify(cnd) "\n");				\
+	} while (0)
+
+
 extern void set_die_routine(void (*routine)(const char *err, va_list params) NORETURN);
 
 extern int prefixcmp(const char *str, const char *prefix);


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhi...@re...