#241 JIT dump file is failed to be loaded in opreport.

None
closed-fixed
None
5
2013-07-29
2013-04-11
Ghacker
No

"opreport" is failed to load JIT dump file in session directory when "--root" option is given and the "root directory" doesn't contain the session directory or the JIT dump file(That means "root directory" and "session directory" is totally separated.)

If '--root' is missed or given root directory contains session directory or JIT dump file, it works fine.

Discussion

  • Ghacker

    Ghacker - 2013-04-11
    • summary: JIT dump file is failed to load on opreport when "--root=pat --> JIT dump file is failed to be loaded in opreport.
     
  • Ghacker

    Ghacker - 2013-04-11

    extra_images::locate_image() is patched to load JIT dump object in session directory.

     
  • Ghacker

    Ghacker - 2013-04-11

    I made and attached a patch for this.
    Please check it out.
    It works fine with mine.

     
  • Maynard Johnson

    Maynard Johnson - 2013-04-11

    The opreport command does not handle jitdump files at all, so your problem description is not making sense to me. The jitdump files are converted to ELF files with filenames of <pid>.jo, and these .jo files are moved into the samples directory for opreport to process later. Please give a more detailed description of the problem, including the exact sequence of commands to reproduce the problem, as well as what version (or git checkout date) of oprofile you are using. Thanks.

     
  • Maynard Johnson

    Maynard Johnson - 2013-04-11
    • assigned_to: nobody --> maynardj
     
  • Ghacker

    Ghacker - 2013-04-12

    There must be misunderstanding about terms..
    I understand the procedure about JIT dump conversion.
    The terms that I used are not suitable for the situation.
    I'm sorry that makes you confusing.
    If you change terms as below from my description, it'll make sense.
    1. Jit dump file --> <pid>.jo ELF file, converted from jit dump
    2. session directory --> samples directory

    I don't have the samples for now. It's on my working sight.
    I'll attach it few days later, if you still want it then.

    For now, let me explain the situation more detailed.
    I using latest version of Oprofile from your git repository for Android kernel based environment.
    Obviously, I need to make some patches to build it with NDK toolchain.
    (BTW, I hope to contribute with these patches later)

    I profiled a process with operf on Android based target device and extract session(sample) directory.
    And there's a (root) directory contains bunch of ELF files which is loaded into the target as stripped.
    Now, consider those directories are listed as below.
    / -+- (root_directory)
    | +- sub-directories and files those are loaded into target device as stripped
    | +- .....
    +- (session_directory)
    +- samples
    | +- ........
    | +- <pid>.jo <-- ELF file from JIT dump
    +- ........
    Then, use opreport to view the result as below.
    1. opreport <options> --session session_directory
    2. opreport <options> --session session_directory --root root_directory
    First case above works fine.
    But, in second case, opreport try to find <absolute path of root_directory>/<absolute path of ".jo" file above>, and it fails obviously.

    I hope these description is enough for you to understand the situation.
    If there's any unclear, let me know.
    Thanks

     
  • Maynard Johnson

    Maynard Johnson - 2013-04-12

    When you say you "extract session(sample) directory", are you meaning to say you pulled the sample data over to a host system and ran opreport there? If so, then the procedure you're following is not recommended or supported. See the man page for the oparchive command for details. This archiving procedure is also documented in http://oprofile.sourceforge.net/doc/oparchive.html. NOTE: If you've collected the profile data into a session directory other that the "standard" /var/lib/oprofile, the oparchive command completion message tells you that you must pass that directory to opreport using the --session-dir option. So, after running oparchive, you tar up the output directory created by oparchive, pull it over to your host, untar it and run opreport something like the following:
    opreport archive:<archive_path> [--session-dir=<my-non-standard-sessiondir-from-target> [other options]

    If your host is a different architecture than your Android device, then you will also need to employ the opimport command (see http://oprofile.sourceforge.net/doc/opimport.html\) before running opreport.

     
  • Ghacker

    Ghacker - 2013-04-15

    Yes. "extract session(sample) directory" means, "pulled the sample data over to a host system" like you said.
    I understand what's wrong with my situation with your comment.
    I should use oparchive before pull the data. (opimport is not needed in my case.)
    Now, it works mostly fine. I'll expain why it's "mostly" later.

    But, I think it's still a kind of issue (i.e. not a bug) which is not a critical but inconvenience.
    So, I suggest 3 solutions to make better.

    1. Change terms on manual
    Refer to http://oprofile.sourceforge.net/doc/oparchive.html, it's not easy to know it should be used to move sample data to other system.
    I thought that I can use oparchive whenever I want to (literally) archive sample and related data.
    That's why I thought, I don't need it in my situation.
    So, if my procedure was not recommended or supported as you told, it's better to be written that oparchive should be used before move sample data.

    2. Add some more warning message when such a case.
    opreport never gave me any kind of message except "warning: <path-to>/<pid>.jo could not be found. file is not loaded"
    I hardly understood that because, it's exactly located in directory described in the message.
    Because, I offered all necessary information with '--root' and '--session-dir' options to opreport. (It makes the situation after all.)
    Moreover, it works fine only with --session-dir option or with --root options when "root" directory contains "session" directory.
    In my expirence, the problematic situation is occured only when "session" directory contains "<pid>.jo" within its sample
    and 2 options above are used at the same time and "session" directory is not included in "root" directory.
    So, opreport (or libop whatever) can detect and warn such a case is not recommended or supported.

    3. Make it just work.
    I think you already know target device in embedded system has not enought resources.
    So, while I port to oprofile for Android platform, I wanted to post every possible process to be processed on host rather than target.
    If I have to use oparchive all the times when I want to get profile data, it'll be a kind of annying procedure and waist of resources.
    Because, executables from target device is useless since those are usually stripped, and their debug information is already on host.
    So, I have to remove those executables from archive and offer new path that contains executables with debug information with "--root" option.
    I know embedded device situation is not that common. But I thought, it's considerable.
    As I told, it already works almost fine. So, (IMHO), it's able to extend its ability without not that much effort.

    I hope this is not bothering you and please don't get me wrong about this "issue".
    If you guys think that is not a issue, I'll get it.
    I just want to help the project to be get more valuable as a fan.

    Thanks.

     
  • Maynard Johnson

    Maynard Johnson - 2013-04-15

    I absolutely agree that the documentation for oparchive lacked sufficient detail for users such as yourself. I've attached a patch to update the oprofile user manual HTML that adds more detail to both the oparchive and opimport sections. The patch also renames the oparchive section from "6. Archiving measurements (oparchive)" to "6. Analyzing profile data on another system (oparchive)". I tried attaching the built HTML file, but it was too big. If you have the required doc creation packages installed on your system, 'make' will build the oprofile.html file, and 'make install' installs it under <op-install-dir>/share/doc/oprofile.

    If, after reviewing the doc changes, you still feel some code changes are needed, please update this bug with specific suggestions. I may not immediately accept suggestions for code changes. I may ask that you post the issue to the oprofile list and get input from other members of the oprofile community who work in similar host/target environments as you.

     
  • Ghacker

    Ghacker - 2013-04-17

    I think, your documentation is great.
    Now, I think, I'll be more clear that use of oparchive is necessary when host/target situation.

    But, I still think that there's some more way to make it a little bit more complete.

    1. Documentation
    I think your patch for documentation of oparchive is already enough and great.
    But, if description for session option is mentioned in opreport and opannotate, it'll be better.
    Because, I think not every users are read manual carefully at first sight.
    May be, 'operf' and 'opreport' will be read even at first sight.
    2. Efficiency
    As you told before and written on modified manual, in host/target situation, oparchive still could collect unnecessary data before tar it.
    In other words, oparchive still need more storage and computational resources for unncessary informations.
    Usually, in host/target situation, target has not enough resources.
    (In my case, I'll not tar archive. Because, tar is not ported on android native and I can just pull directories using adb withou tar.)
    If there's an option to control or choose archive target for oparchive, it can reduce such an inefficiency.
    Of couse it should not effect existing behavior when it's not used explicitly.
    3. noticeability
    opreport still can run with wrong information with not enough warning (I thought).
    If wrong information can be detected, it's better to be warned to guide user to read oparchive manual.

    As I said earlier, I don't want to bother you with this.
    But, I just want you to consider my private opinions.

    Again, I thank you very much for your great efforts.

     
  • Maynard Johnson

    Maynard Johnson - 2013-04-17
    • status: open --> open-fixed
     
  • Maynard Johnson

    Maynard Johnson - 2013-04-17

    I committed the doc patch upstream, but first, I removed references to 'tar' (didn't know that utility wasn't available on Android).

    As for description of session-dir, I believe the man pages for opreport and opannotate give a good description of that option.

    As for opreport being executed in such a way that it uses incorrect information or insufficient information and does not give warnings . . . it's very hard to protect against every kind of mis-use of this command, since it has to handle so many different scenarios. But if you find a particular common scenario that you think it should handle, please submit a patch to the oprofile-list, and I'll be happy to review it.

    For now, I'm marking this bug as "Fixed", but it will remain open until the next official release. If you still have issues, you can still add comments to the bug.

     
  • Ghacker

    Ghacker - 2013-04-18

    My concern about documentation and notice might be too much.
    It already seems good enough.
    But, I just didn't sure that it's enough or not for beginners. Because You and I already know that functionality.

    When I'm finding "common scenario" that makes the situation, I'll comment it to this.
    For now, I don't think my secnario is "common scenario"

    Anyways.. Thanks for what you've done with passion.
    As a fan of oprofile, I hope it'll be getting better always.

     
  • Ghacker

    Ghacker - 2013-04-18
    • status: open-fixed --> open
     
  • Maynard Johnson

    Maynard Johnson - 2013-04-18
    • status: open --> open-fixed
     
  • Maynard Johnson

    Maynard Johnson - 2013-07-29
    • status: open-fixed --> closed-fixed
    • Group: -->
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks