Menu
▾
▴
[oprof-cvs] CVS: oprofile/doc oprofile.xml,1.164.2.2,1.164.2.3
|
From: Maynard J. <may...@us...> - 2007-12-04 21:35:47
|
Update of /cvsroot/oprofile/oprofile/doc In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv1030/doc Modified Files: Tag: JIT_SUPPORT oprofile.xml Log Message: Add JIT support documentation Index: oprofile.xml =================================================================== RCS file: /cvsroot/oprofile/oprofile/doc/oprofile.xml,v retrieving revision 1.164.2.2 retrieving revision 1.164.2.3 diff -u -p -d -r1.164.2.2 -r1.164.2.3 --- oprofile.xml 15 Nov 2007 20:57:29 -0000 1.164.2.2 +++ oprofile.xml 4 Dec 2007 21:35:44 -0000 1.164.2.3 @@ -52,6 +52,7 @@ OProfile is useful in a number of situat <listitem><para>cannot use highly intrusive profiling methods</para></listitem> <listitem><para>need to profile interrupt handlers</para></listitem> <listitem><para>need to profile an application and its shared libraries</para></listitem> +<listitem><para>need to profile dynamically compiled code of supported virtual machines (see <xref linkend="jitsupport"/>)</para></listitem> <listitem><para>need to capture the performance behaviour of entire system</para></listitem> <listitem><para>want to examine hardware effects such as cache misses</para></listitem> <listitem><para>want detailed source annotation</para></listitem> @@ -67,8 +68,28 @@ OProfile is not a panacea. OProfile migh <listitem><para>require 100% instruction-accurate profiles</para></listitem> <listitem><para>need function call counts or an interstitial profiling API</para></listitem> <listitem><para>cannot tolerate any disturbance to the system whatsoever</para></listitem> -<listitem><para>need to profile interpreted or dynamically compiled code such as Java or Python</para></listitem> +<listitem><para>need to profile interpreted or dynamically compiled code of non-supported virtual machines</para></listitem> +</itemizedlist> +<sect2 id="jitsupport"> +<title>Support for dynamically compiled (JIT) code</title> +<para> +Older versions of OProfile were not capable of attributing samples to symbols from dynamically +compiled code, i.e. "just-in-time (JIT) code". Typical JIT compilers load the JIT code into +anonymous memory regions. OProfile reported the samples from such code, but the attribution +provided was simply: + <screen>"anon: <tgid><address range>" </screen> +Due to this limitation, it wasn't possible to profile applications executed by virtual machines (VMs) +like the Java Virtual Machine. OProfile now contains an infrastructure to support JITed code. +A development library (<xref linkend="developing"/>) is provided to allow developers +to add support for any VM that produces dynamically compiled code. +In addition, built-in support is included for the following:</para> +<itemizedlist><listitem>JVMTI agent library for Java (1.5 and higher)</listitem> +<listitem>JVMPI agent library for Java (1.5 and lower)</listitem> </itemizedlist> +<para> +For information on how to use OProfile's JIT support, see <xref linkend="setup-jit"/>. +</para> +</sect2> </sect1> <sect1 id="requirements"> @@ -228,6 +249,35 @@ is often all you need, but note these ar </para></listitem> </varlistentry> <varlistentry> + <term><option>--with-java</option></term> + <listitem> + <para> + Use this option if you need to profile Java applications. This option is used + to specify the location of the Java Development Kit (JDK) + source tree you wish to use. This is necessary to get the interface description + of the JVMPI (or JVMTI) interface to compile the JIT support code successfully. + </para> + <note> + <para> + The Java Runtime Environment (JRE) does not include the development + files that are required to compile the JIT support code, so the full + JDK must be installed in order to use this option. + </para> + </note> + <para> + By default, the Oprofile JIT support libraries will be installed in + <filename><oprof_install_dir>/lib/oprofile</filename>. To build + and install OProfile and the JIT support libraries as 64-bit, you can + do something like the following: + <screen> + # CFLAGS="-m64" CXXFLAGS="-m64" ./configure \ + --with-kernel-support --with-java={my_jdk_installdir} \ + --libdir=/usr/local/lib64 + </screen> + </para> + </listitem> + </varlistentry> + <varlistentry> <term><option>--with-kernel-support</option></term> <listitem><para> Use this option with 2.6 and above kernels to indicate the @@ -263,7 +313,7 @@ You'll need to have a configured kernel to build the module for 2.4 kernels. Since all distributions provide different kernels it's unlikely the running kernel match the configured source you installed. The safest way is to recompile your own kernel, run it and compile oprofile. It is also recommended that if you have a uniprocessor machine, you enable the local APIC / IO_APIC support for -your kernel (this is automatically enabled for SMP kernels). With many BIOS, kernel >= 2.6.9 and UP kernel it's not sufficient to enable the local APIC you must also turn it on explicitely at boot time by providing "lapic" option to the kernel. On +your kernel (this is automatically enabled for SMP kernels). With many BIOS, kernel >= 2.6.9 and UP kernel it's not sufficient to enable the local APIC you must also turn it on explicitly at boot time by providing "lapic" option to the kernel. On machines with power management, such as laptops, the power management must be turned off when using OProfile with 2.4 kernels. The power management software in the BIOS cannot handle the non-maskable interrupts (NMIs) used by @@ -373,6 +423,13 @@ This section gives a brief description o </varlistentry> <varlistentry> + <term><filename>agent libraries</filename></term> + <listitem><para> + Used by virtual machines (like the Java VM) to record information about JITed code being profiled. See <xref linkend="setup-jit" />. + </para></listitem> +</varlistentry> + +<varlistentry> <term><filename>opreport</filename></term> <listitem><para> This is the main tool for retrieving useful profile data, described in @@ -834,6 +891,58 @@ The table below lists the events selecte </sect1> +<sect1 id="setup-jit"> + <title>Setting up the JIT profiling feature</title> + <para> + To gather information about JITed code from a virtual machine, + it needs to be instrumented with an agent library. We use the + agent libraries for Java in the following example. To use the + Java profiling feature, you must build OProfile with the "--with-java" option + (<xref linkend="install" />). + + </para> + + <sect2 id="setup-jit-jvm"> + <title>JVM instrumentation</title> + <para> + Add this to the startup parameters of the JVM (for JVMTI): + <screen><option>-agentpath:<libdir>/libjvmti_oprofile.so[=<options>]</option> </screen> + or + <screen><option>-agentlib:jvmti_oprofile[=<options>]</option> </screen> + </para> + <para> + The agent library (installed in <filename><oprof_install_dir>/lib/oprofile</filename>) + needs to be in the library search path (e.g. add the library directory + to <constant>LD_LIBRARY_PATH</constant>). If the command line of + the JVM is not accessible, it may be buried within shell scripts or a + launcher program. It may also be possible to set an environment variable to add + the instrumentation. + For Sun JVMs this is <constant>JAVA_TOOL_OPTIONS</constant>. Please check + your JVM documentation for + further information on the agent startup options. + </para> + <para> + The JVMPI agent implementation is enabled with the command line option + <screen><option>-Xrunjvmpi_oprofile[:<options>]</option> </screen> + </para> + <para> +<!-- FIXME-JIT: make sure this statement below is correct once the JIT support is merged into mainline --> + By default, the JIT support for Java does not dump the assembly code of + dynamically compiled Java methods. To force this action, you must add + the "include-code" option with the agent library as follows: + </para> + <para> For JVMTI + <screen><option>-agentpath:<libdir>/libjvmti_oprofile.so=include-code</option> </screen> + or + <screen><option>-agentlib:jvmti_oprofile=include-code</option> </screen> + </para> + <para> For JVMPI + <screen><option>-Xrunjvmpi_oprofile:include-code=yes</option> </screen> + </para> + + </sect2> +</sect1> + <sect1 id="oprofile-gui"> <title>Using <command>oprof_start</command></title> <para> @@ -1494,6 +1603,12 @@ You can restrict which events to list by <sect2 id="opreport-callgraph"> <title>Callgraph output</title> +<!-- FIXME-JIT: test callgraph with JIT support to see if we need to add a caveat here --> +<para> +This section provides details on how to use the OProfile callgraph feature. +</para> +<sect3 id="op-cg1"> +<title>Callgraph details</title> <para> When using the <option>opcontrol --callgraph</option> option, you can see what functions are calling other functions in the output. Consider the @@ -1582,6 +1697,39 @@ as calling <function>strfry()</function> that this doesn't actually happen. See <xref linkend="interpreting-callgraph" /> for an explanation. </para> +</sect3> +<sect3 id="cg-with-jitsupport"> +<title>Callgraph and JIT support</title> +<para> +Callgraph output where anonymously mapped code is in the callstack can sometimes be misleading. +For all such code, the samples for the anonymously mapped code are stored in a samples subdirectory +named <filename>{anon:anon}/<tgid>.<begin_addr>.<end_addr></filename>. +As stated earlier, if this anonymously mapped code is JITed code from a supported VM like Java, +OProfile creates an ELF file to provide a (somewhat) permanent backing file for the code. +However, when viewing callgraph output, any anonymously mapped code in the callstack +will be attributed to <filename>{anon:anon}/<tgid>.<begin_addr>.<end_addr></filename>, +even if a <filename>.jo</filename> ELF file had been created for it. See the example below. +</para> +<screen> +------------------------------------------------------------------------------- + 1 2.2727 libj9ute23.so java.bin traceV + 2 4.5455 libj9ute23.so java.bin utsTraceV + 4 9.0909 libj9trc23.so java.bin fillInUTInterfaces + 37 84.0909 libj9trc23.so java.bin twGetSequenceCounter +8 0.0154 libj9prt23.so java.bin j9time_hires_clock + 27 61.3636 anon (tgid:10014 range:0x100000-0x103000) java.bin (no symbols) + 9 20.4545 libc-2.4.so java.bin gettimeofday + 8 18.1818 libj9prt23.so java.bin j9time_hires_clock [self] +------------------------------------------------------------------------------- +</screen> +<para> +The output shows that "anon (tgid:10014 range:0x100000-0x103000)" was a callee of +<code>j9time_hires_clock</code>, even though the ELF file <filename>10014.jo</filename> was +created for this profile run. Unfortunately, there is currently no way to correlate +that anonymous callgraph entry with its corresponding <filename>.jo</filename> file. +</para> +</sect3> + </sect2> <!-- opreport-callgraph --> @@ -1672,32 +1820,36 @@ considered different. <title>Anonymous executable mappings</title> <para> Many applications, typically ones involving dynamic compilation into -machine code, have executable mappings that are not backed by an ELF -file. <command>opreport</command> has basic support for showing the +machine code (just-in-time, or "JIT", compilation), have executable mappings that +are not backed by an ELF file. <command>opreport</command> has basic support for showing the samples taken in these regions; for example: -</para> <screen> -$ opreport /usr/jre1.5.0/bin/java -CPU: PIII, speed 863.195 MHz (estimated) -Counted CPU_CLK_UNHALTED events (clocks processor is not halted) with a unit mask of 0x00 (No unit mask) count 100000 -CPU_CLK_UNHALT...| - samples| %| ------------------- - 27344 100.000 java - CPU_CLK_UNHALT...| - samples| %| - ------------------ - 27236 99.605 anon (tgid:12135 range:0xb2cb8000-0xb2e80000) - 108 0.3949 java +$ opreport /usr/bin/mono -l +CPU: ppc64 POWER5, speed 1654.34 MHz (estimated) +Counted CYCLES events (Processor Cycles using continuous sampling) with a unit mask of 0x00 (No unit mask) count 100000 +samples % image name symbol name +47 58.7500 mono (no symbols) +14 17.5000 anon (tgid:3189 range:0xf72aa000-0xf72fa000) (no symbols) +9 11.2500 anon (tgid:3189 range:0xf6cca000-0xf6dd9000) (no symbols) +. . . . </screen> +</para> <para> -Currently, there is no support for getting symbol-based summaries for -such regions. Note that, since such mappings are dependent upon -individual invocations of a binary, these mappings are always listed as -a dependent image, even when using <option>--separate=none</option>. +Note that, since such mappings are dependent upon individual invocations of +a binary, these mappings are always listed as a dependent image, +even when using <option>--separate=none</option>. Equally, the results are not affected by the <option>--merge</option> option. </para> +<para> +As shown in the opreport output above, OProfile is unable to attribute the samples to any +symbol(s) because there is no ELF file for this code. +Enhanced support for JITed code is now available for some virtual machines; +e.g., the Java Virtual Machine. For details about OProfile output for +JITed code, see <xref linkend="getting-jit-reports" />. +</para> +<para>For more information about JIT support in OProfile, see <xref linkend="jitsupport"/>. +</para> </sect2> <!-- opreport-anon --> <sect2 id="opreport-xml"> @@ -1985,6 +2137,67 @@ Show version. </sect1> <!-- opannotate --> +<sect1 id="getting-jit-reports"> + <title>OProfile results with JIT samples</title> + <para> + After profiling a Java (or other supported VM) application, the command + <screen><command>"opcontrol --dump"</command> </screen> + flushes the sample buffers and creates ELF binaries from the + intermediate files that were written by the agent library. + The ELF binaries are named <filename><tgid>.jo</filename>. + With the symbol information stored in these ELF files, it is + possible to map samples to the appropriate symbols. + </para> + <para> + The usual analysis tools (<command>opreport</command> and/or + <command>opannotate</command>) can now be used + to get symbols and assembly code for the instrumented VM processes. + (To get annotated assembly for an instrumented Java application, + you must use the "include-code" option with the agent + library. See <xref linkend="setup-jit-jvm"/> + </para> +<para> +Below is an example of a profile report of a Java application that has been +instrumented with the provided agent library. +<!-- FIXME-JIT: re-run the test app with new code that does mangling; then fix the +the opreport output --> +<screen> +$ opreport -l /usr/lib/jvm/jre-1.5.0-ibm/bin/java +CPU: Core Solo / Duo, speed 2167 MHz (estimated) +Counted CPU_CLK_UNHALTED events (Unhalted clock cycles) with a unit mask of 0x00 (Unhalted core cycles) count 100000 +samples % image name symbol name +281 59.9147 [vdso] (tgid:9723 range:0x760000-0x761000) (no symbols) +76 16.2047 9723.jo java/lang/StringBuilder/append˜3 +31 6.6098 9723.jo java/lang/Integer/getChars˜1 +31 6.6098 9723.jo java/lang/StringBuilder/append˜1 +24 5.1173 9723.jo TestApp01/main˜3 +14 2.9851 9723.jo java/lang/StringBuilder/toString˜1 +5 1.0661 9723.jo java/lang/StringBuilder/<init>˜1 +3 0.6397 9723.jo java/lang/String/length˜1 +3 0.6397 java .plt +1 0.2132 java ibmFindJVMDLL +</screen> +</para> +<note><para> + Depending on the JVM that is used, certain options of opreport and opannotate + do NOT work since they rely on debug information (e.g. source code line number) + that is not always available. The Sun JVM does provide the necessary debug + information via the JVMTI[PI] interface, + but other JVMs do not. + </para></note> + <para> + The JIT support agent for Java generates symbols to include the class + and method signature. + A symbol with the suffix ˜<n> (e.g. + <code>java.lang.String test.run_test()˜1</code>) means that this is + the <n>th occurrence of the identical name. This happens if a method is re-JITed. + A symbol with the suffix %<n>, means that the address space of this symbol + was reused during the sample session (see <xref linkend="overlapping-symbols" />). + The value <n> is the percentage of time that this symbol/code was present in + relation to the total lifetime of all overlapping other symbols. + </para> +</sect1> + <sect1 id="opgprof"> <title><command>gprof</command>-compatible output (<command>opgprof</command>)</title> <para> @@ -2550,6 +2763,32 @@ show such example </para> </sect2> --> + +<sect1 id="overlapping-symbols"> + <title>Overlapping symbols in JITed code</title> + <para> + Some virtual machines (e.g., Java) may re-JIT a method, resulting in previously + allocated space for a piece of compiled code to be reused. This means that, at one distinct + code address, multiple symbols/methods may be present during the run time of the application. + </para> + <para> + Since OProfile samples are buffered and don′t have timing information, there is no way + to correlate samples with the (possibly) varying address ranges in which the code for a symbol + may reside. + An alternative would be flushing the OProfile sampling buffer when we get an unload event, + but this could result in high overhead. + </para> + <para> + To moderate the problem of overlapping symbols, OProfile tries to select the symbol that was + present at this address range most of the time. Additionally, other overlapping symbols + are truncated in the overlapping area. + This gives reasonable results, because in reality, address reuse typically takes place + during phase changes of the application -- in particular, during application startup. + Thus, for optimum profiling results, start the sampling session after application startup + and burn in. + </para> +</sect1> + <sect1 id="hidden-cost"> <title>Other discrepancies</title> <para> @@ -2566,6 +2805,68 @@ and <ulink url="http://developer.amd.com </sect1> </chapter> +<chapter id="developing"> + <title>Developing a new JIT agent</title> + <para> + OProfile includes a header file and library that are intended to be used by + developers who wish to extend OProfile's JIT support to other non-supported + virtual machines. This section describes these development files and how + to use them. + </para> +<sect1 id="jit-devel-overview"> +<title>Overview</title> + <para> + OProfile already includes some implementations that use the JIT support, + e.g., the Java Virtual Machine Toolkit Interface (JVMTI) library, + libjvmti_oprofile.so. In developing a new implementation, you will + likely follow a similar (if not identical) procedure as was used in + developing the JVMTI library. Following are the high level steps to follow: + <orderedlist> + <listitem>Ensure your virtual machine provides an API that, at minimum, + can provide the following information about dynamically compiled code: + <itemizedlist> + <listitem>Notification when compilation occurs</listitem> + <listitem>Name of the symbol (i.e., function or class/method, etc.)</listitem> + <listitem>Address in anonymous memory where the compiled code was loaded</listitem> + <listitem>Length of the compiled code segment</listitem> + </itemizedlist> + </listitem> + <listitem>Write an agent library that communicates with your VM to obtain + compiled code notifications. Implement required functions from libopagent.h + (<xref linkend="jit-interface" />) and link your library with libopagent.so + (installed at <filename><oprofile_install_dir>/lib/oprofile</filename>). + </listitem> + </orderedlist> + + </para> +</sect1> +<sect1 id="jit-interface"> +<title>JIT support interface</title> + <para> + The JIT support infrastructure for OProfile defines the following interface + in <filename><oprofile-install-dir>/include/libopagent.h</filename>: + </para> +<screen> +/** This function must be called by agents before any other function */ +int op_open_agent(); + +/** Frees all resources and close open file handles */ +void op_close_agent(); + +/** Signal the dynamic generation of native code from a JIT */ +int op_write_native_code( + char const * symbol_name, + const void * code, + const unsigned int code_size); + +/** Signal the invalidation of native code from a JIT */ +int op_unload_native_code( + const void * code); + +</screen> +</sect1> +</chapter> + <chapter id="ack"> <title>Acknowledgments</title> <para> |