oprofile Log


Commit Date  
[44b633] by Maynard Johnson Maynard Johnson

Cleanup verbose variable usage in operf for easier debugging

This patch removes the "perf_events" verbose option which had
been used to enable debugging for both the recording phase of
profiling (where profile data from the kernel is being read by
the operf-record process) and the conversion phase. Instead,
two new distinct verbose options are introduced: "record" and
"convert".

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-08-06 00:45:54 Tree
[021390] by Maynard Johnson Maynard Johnson

Add a new option to operf to do conversion to oprofile format after profiling is done

The profile data read from the perf_events kernel has to be converted to OProfile
format for the post-processing tools (e.g., opreport). By default, this conversion
is done on-the-fly during profling using a pipe. This method involves starting an
additional operf process that reads the data fed into the pipe by the operf-record
process. This additional process interprets the perf_events data and does the
conversion to OProfile format, persisting the ususal oprofile sample data files in
<session-dir>/samples/current. When operf is run in single application mode, the
extra overhead of the conversion process running during profiling time is not even
recorded, and thus is not easily noticeable. However, when doing a system-wide
profile, operf overhead can be substantial (easily over 6% on a busy system), and
the samples for operf *are* noticeable in the system-wide report.

This patch introduces the "--lazy-conversion" option which directs operf to write
the perf_events profile data to a temporary file during profiling. Then, after
profiling is done, the data in the temporary file is converted to OProfile format.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-08-06 00:44:22 Tree
[2af280] by Maynard Johnson Maynard Johnson

Fix operf profiling of forked processes

When an application being profiled by operf does a fork, and where
that forked process does *not* do an exec after the fork, any work
performed by the forked process was not being profied. This patch
addresses that problem.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-08-06 00:42:50 Tree
[3250f8] by Carl Love Carl Love , pushed by Maynard Johnson Maynard Johnson

operf: print lost sample warning only if lost samples exceeds threshold

Currently a warning is printed if there are any lost samples when collecting
data using the operf tool. This patch changes the code to print the warning
when the total number of lost samples, for whatever reason, exceeds a
threshold of the total number of samples. Initially, the threshold has been
set to 0.01% of the total number of samples.

Signed-off-by: Carl Love <cel@us.ibm.com>

2012-08-03 14:38:13 Tree
[b12f64] by Maynard Johnson Maynard Johnson

Change pp tools to abort if <cur_dir>/oprofile_data exists, but no samples found

In commit 01c7ce945cfe835b908c5f4f8c1125e65302b000 on July 3, 2012,
a change was made to the post-processing tools to look first in
<cur_dir>/oprofile_data; then, if no samples are found there,
the tool should fall back to the "standard" /var/lib/oprofile
session-dir. I've come the conclusion that this is not proper
behavior. For example, a user may run operf using some event that
never occurs during the profiling run, and, thus, no samples would
be stored in <cur_dir>/oprofile_data. But if that user then ran
opreport and there happened to be some old profile data in
/var/lib/oprofile, that old profile data would be displayed in
the report.

This new patch changes the behavior as follows:
The post-processing analysis tools will search for samples in
<current_dir>/oprofile_data first. If that directory exists
but has no samples, then the usual "No sample file found"
message is displayed. On the other hand, if that directory
does not exist, the post-processing tools use the standard
session-dir of /var/lib/oprofile. Also, the message that
indicates the session-dir being used will be displayed
whether or not samples are found so that users can see
clearly where the pp tool was looking for samples.

This patch also makes a small change in the "No sample file found"
message relating to the use of 'opcontrol --dump', which is only
appropriate when using legacy profiling versus operf profiling.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-31 17:05:56 Tree
[0cb1a9] by Maynard Johnson Maynard Johnson

Remove extraneous debugging cerr from previous commit

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-26 16:12:13 Tree
[d3c656] by Maynard Johnson Maynard Johnson

Fix operf to read unit mask in hex and document unit mask spec requirementes

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-26 16:02:39 Tree
[4747a7] by Maynard Johnson Maynard Johnson

Fix errors found by coverity in new and changed code in perf_events port

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-25 21:27:39 Tree
[2530d6] by Maynard Johnson Maynard Johnson

Fix warning messages when building on Ubuntu 11.10

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-25 16:20:40 Tree
[b27671] by Maynard Johnson Maynard Johnson

Fix build warning on Fedora 17 with gcc 4.7.0

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-24 19:18:30 Tree
[67e6e2] by Maynard Johnson Maynard Johnson

Fix previous commit that broke non-ppc64 arch builds

The previous commit that added the capability for ppc64
architecture to use either version 3 or version 4 libpfm
had a bug in it that caused non-ppc64 builds to break.
This patch fixes that problem.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-24 18:43:34 Tree
[2a51bc] by Maynard Johnson Maynard Johnson

Allow libpfm4 to be used by ppc64 for obtaining event code

The ppc64 architecture oprofile event codes are not the same
as are needed for perf_events, so there is ppc64-specific
code to use libpfm to obtain the correct code to pass to
perf_event_open. The libpfm API has undergone a lot of
changes from version 3 to version 4, and some of the version 3
functions have been removed, replaced, deprecated, etc.
This patch detects which version of libpfm is installed
and #define's some macros so that the ppc64-specific code
can be built with the appropriate functions.

Note that at this time, this patch affects *only* the
ppc64 architecture.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-24 18:06:13 Tree
[05b1fb] by Maynard Johnson Maynard Johnson

Fix 'make check' warning treated as error when using newer gcc

A gcc 4.7.1 reports the following error when running 'make check':

../../../oprofile.git/libutil++/tests/cached_value_tests.cpp: In function \u2018bool {anonymous}::check_throw(const cached_value<bool>&)\u2019:
../../../oprofile.git/libutil++/tests/cached_value_tests.cpp:25:8: error: variable \u2018foo\u2019 set but not used [-Werror=unused-but-set-variable]

This patch fixes this problem.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-24 12:28:07 Tree
[152be3] by Maynard Johnson Maynard Johnson

Make git builds use -Werror and fix resulting warning message in operf.cpp

When I merged the perf-events branch, I forgot to change the
"AM_INIT_AUTOMAKE(oprofile, 0.9.8_perf_events)" to
"AM_INIT_AUTOMAKE(oprofile, 0.9.8git)" so that git builds
would use -Werror. Changing this resulted in one warning message
treated as an error due to an unused variable in pe_profiling/operf.cpp.
This patch fixes this warning, along with changing configure.in
to use "0.9.8git".

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-24 11:41:17 Tree
[f63504] by Maynard Johnson Maynard Johnson

Bug #3386923: Document that 'opannoate -a' needs symbol info

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-23 21:13:06 Tree
[54fe9d] by Maynard Johnson Maynard Johnson

Add new operf.log file to what oparchive will archive

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-20 15:21:07 Tree
[9ea425] by Maynard Johnson Maynard Johnson

Fix oparchive to use absolute path for copying abi and oprofiled.log

If oparchive is called with the --session-dir argument using a relative path,
the command will fail at the end if invoked using:

And the result was:
can't copy from oprof_sessions//abi to /media/stick/appbarstatic-2oprof_sessions//abi cause: Success
can't copy from oprof_sessions//samples//oprofiled.log to /media/stick/appbarstatic-2oprof_sessions//samples//oprofiled.log cause: Success

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-20 15:20:05 Tree
[c85362] by Maynard Johnson Maynard Johnson

Change "Nr. event lost ..." field in the oprofiled.log to "Nr. samples lost ..."

The "Nr. event lost due to buffer overflow" field in the oprofiled.log is
mis-named. It should be "Nr. samples lost due to buffer overflow" in spite
of the fact that the filename in oprofilefs is called 'event_lost_overflow'.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-20 15:19:17 Tree
[91919f] by Maynard Johnson Maynard Johnson

[PATCH] Fix how "Nr. non-backtrace samples" from oprofiled.log is calculated

The "Nr. non-backtrace samples" is tracked in the daemon code with
opd_stats[OPD_SAMPLES]. This value is incremented every time sfile_find()
is called from opd_put_sample(). However, if we're processing samples for
which the sfile was previously found and the trans->current value is valid,
we will not call sfile_find() and, thus, the opd_stats[OPD_SAMPLES] stat
will not be incremented.

This problem is most easily seen by starting oprofile on a fairly quiesced
system, then run several instances of the same test program (ideally, at
least as many instances as you have processors). For example, on a laptop
with two processors, I ran 4 copies of a testcase and below is what the
oprofiled.log statistics looked like with this bug:

-----------------------------
-- OProfile Statistics --
Nr. sample dumps: 48
Nr. non-backtrace samples: 329952
Nr. kernel samples: 79505
Nr. lost samples (no kernel/user): 0
. . .
---- Statistics for cpu : 1
Nr. samples lost cpu buffer overflow: 0
Nr. samples received: 944623
Nr. backtrace aborted: 0
Nr. samples lost invalid pc: 0

---- Statistics for cpu : 0
Nr. samples lost cpu buffer overflow: 0
Nr. samples received: 925436
Nr. backtrace aborted: 0
Nr. samples lost invalid pc: 0
oprofiled stopped Thu Jul 19 08:51:47 2012
---------------------------------

As can be seen above, the individual CPU stats show 925436
samples collected, where as the Nr. non-backtrace samples
is 329952. With the fix in this path, the total of the CPU
stats and the Nr. non-backtrace samples are roughly equal.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-20 15:17:06 Tree
[a30a67] by Maynard Johnson Maynard Johnson

[PATCH] Add some helpful messages for various operf usage scenarios

Added the folowing helpful messages:
- Display message in opreport (and other pp tools) to show session dir used.
- When using operf with "-p" or "-s", print message that user can do "ctrl-C"
or "kill SIGINT <operf_PID> " to stop profiling. The "kill" command is mainly
needed in the case where operf is made a background job via the '&'.
- Display a message when a permissions error is encountered when trying to mmap
the kernel profile data. This can happen when a given user tries to run multiple
instances of operf simultaneously. This problem is fixed on newer kernels (e.g., 3.0).
- For 'operf --system-wide', print a message that either the user needs to be root or
the perf_event_paranoid system value must be 0 or -1. Of course, the message is
printed only if neither of these are true.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-19 12:23:03 Tree
[b862e4] by Maynard Johnson Maynard Johnson

Fix up oprofile user manual to add info about new operf program

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-18 15:07:04 Tree
[8f12d5] by Maynard Johnson Maynard Johnson

Fix configure to not alter user variables and remove non-working --with-gcc option

The configure script for oprofile had a few places
where user variables were being altered in contradiction
to the GNU Automake manual. See the following URL for details:

http://www.gnu.org/software/automake/manual/html_node/Flag-Variables-Ordering.html

In brief, user variables are those listed under "Some influential
environment variables" section of the output from 'configure --help'.
These are variables which the user may set prior to invoking configure,
and thus, the configure script should not alter them. This patch
makes use of new internal variables (e.g., OP_LDFLAGS and OP_CPPFLAGS),
which did unfortunately create quite a ripple effect with the Makefile.am
files. But this was unavoidable in order to correct this error.

Additionally, the '--with-gcc' option does not work anymore because
the generated configure script runs the AC_PROG_CC before processing the
"--with-gcc" option, so it fails to find a compiler. Unsure of when
this stopped working (assume it must have worked in 2005 when it was
initially added), but even fairly old distros (e.g., SLES 10) with
autoconf version 2.59 do not process this option as we would want.
If users wish to configure oprofile to use an alternate compiler,
they should pass the compiler pathnames on the configure command line, thusly:
./configure CC=/my-gcc-install/bin/gcc CXX=/my-gcc-instal/g++

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-13 17:44:49 Tree
[ce716e] by Maynard Johnson Maynard Johnson

Fix error when using 'operf --callgraph --events=blah'

When executing the following command:
'operf -g -e CPU_CLK_UNHALTED:500000 ls'

the following error message occurs:
"Invalid count for event --callgraph=1"

I tracked this down to the switch-over I made on June 11 to
change operf from using popt to use getopt_long instead. The
error message is coming from where operf invokes the ophelp
command as follows:
'ophelp --check-events CPU_CLK_UNHALTED:500000 --callgraph=1'

This ophelp command works fine if you execute it from the command
line. It also works fine when executed by opcontrol. So the only
situation it doesn't work correctly is when executed by operf.
I put some debug printf's into ophelp and found that when it calls
poptGetArgs(), the returned value is '2' in the above example when
operf is used, but is '1' when ophelp is called from the command line
or opcontrol. My initial thought was that either operf
or ophelp might be making improper use of some memory, but running
both programs under Valgrind revealed no problems. So now,
my theory is that because popt uses getopt under the
covers, there must be some global variable being used in operf's use
of getopt that later calls something to go awry when ophelp uses
the poptGetArgs function.

The fix for this problem is to have operf pass the "--callgraph=1"
option to ophelp before passing the event spec.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-13 15:37:02 Tree
[1be0be] by Maynard Johnson Maynard Johnson

Fix for bug 3309794: Change type for sample header mtime field to u64

See bug #3309794 (https://sourceforge.net/tracker/?func=detail&aid=3309794&group_id=16191&atid=116191)

The size of the mtime field in the op_header is based on whether
oprofile is built 32-bit or 64-bit. So if you have an oparchive
from a system where oprofile was built, say, as 64-bit and try to
run reports on that profile data on a different system where oprofile
was built as 32-bit, you're likely to see strange results and/or error
messages.

The typical error seen when running a 32-bit opreport on sample data from
a 64-bit oprofile is:
"opreport error: Attempt to process a Cell Broadband Engine SPU
profile withoutproper BFD support"

This is because the 32-bit opreport is looking at the "wrong" offset in the
header for the spu_profile field and finding a non-zero number there. Note
that to reproduce this error, I needed to pass an image spec to opreport.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-11 20:38:53 Tree
[6d6705] by Maynard Johnson Maynard Johnson

Fixes to earlier commit that changed operf to use getopt vs popt

In my June 11 patch posting to the oprofile-list ("[PATCH] Switch
operf from popt to getopt"), I incorrectly removed the configure check
for popt. This is needed because other oprofile programs still
use popt. Also, I neglected to remove a new function in libutil/op_popt
that I had previously added in my failed attempt to have popt support
operf. This patch addresses both of these concerns.

Signed-off-by: Maynard Johnson <maynardj@us.ibm.com>

2012-07-11 20:36:19 Tree
Older >