From: Gabor L. <lo...@in...> - 2010-02-25 11:12:16
Attachments:
0001-record-anon.patch
|
Hi all, I would like to use OProfile to measure/profile an application which generates and executes JIT codes. Although OProfile performs well, it skips lots of sample. I have looked into daemon/opd_anon.c and I was socked. Is there really no other way to identify a anonymous memory region? Just parsing /proc/%d/maps? Well, it is very possible that a short JIT sequence could only lives between two sample processing. In this case no anon entry will be recorded. I have created a patch which adds a non-identified anon sample to a general anon region. Although it would be nice to avoid parsing /proc/%d/maps, currently this patch is enough to show all anon sample. What is your opinion about it? --Gabor |
From: Maynard J. <may...@us...> - 2010-02-26 14:35:56
|
Gabor Loki wrote: > Hi all, > > I would like to use OProfile to measure/profile an application which > generates and executes JIT codes. > > Although OProfile performs well, it skips lots of sample. I have looked > into daemon/opd_anon.c and I was socked. Is there really no other way to > identify a anonymous memory region? Just parsing /proc/%d/maps? > > Well, it is very possible that a short JIT sequence could only lives > between two sample processing. In this case no anon entry will be > recorded. I'm no JIT expert, but I'm certain that frequently used code blocks are saved so they don't need to be re-compiled again. Don't such code blocks remain mmap'ed into the process' address space? What you're describing here would seem to be a piece of code that, once compiled, does not get used for long, such that the JIT compiler eventually kicks it out of the cache (and nummap'ing the anon memory). In that case, why do you care if a few samples get discarded? Or am I missing something. I've cc'ed Daniel, since he's the oprofile-JIT sub-maintainer. -Maynard > > I have created a patch which adds a non-identified anon sample to > a general anon region. Although it would be nice to avoid parsing > /proc/%d/maps, currently this patch is enough to show all anon sample. > > What is your opinion about it? > > --Gabor > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > > > ------------------------------------------------------------------------ > > _______________________________________________ > oprofile-list mailing list > opr...@li... > https://lists.sourceforge.net/lists/listinfo/oprofile-list |
From: Gabor L. <lo...@in...> - 2010-02-26 17:20:42
|
Maynard Johnson wrote: > I'm no JIT expert, but I'm certain that frequently used code blocks are saved so they don't need to be re-compiled again. Don't such code blocks remain mmap'ed into the process' address space? What you're describing here would seem to be a piece of code that, once compiled, does not get used for long, such that the JIT compiler eventually kicks it out of the cache (and nummap'ing the anon memory). In that case, why do you care if a few samples get discarded? Or am I missing something. Just to make sure of missing sample: with cvs head: $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' 8757 $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' 0 with the patch: $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' 12747 $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' 4105 Well, I do not say this is negligible difference. The JIT codes consume 30% from total time. This is what I expect from this app. Unfortunately the statistics in oprofiled.log does not show any differences. --- HEAD ---- -- OProfile Statistics -- Nr. sample dumps: 6 Nr. non-backtrace samples: 15397 Nr. kernel samples: 2488 Nr. lost samples (no kernel/user): 0 Nr. lost kernel samples: 0 Nr. incomplete code structs: 0 Nr. samples lost due to sample file open failure: 0 Nr. samples lost due to no permanent mapping: 4030 Nr. event lost due to buffer overflow: 0 Nr. samples lost due to no mapping: 107 Nr. backtraces skipped due to no file mapping: 0 Nr. samples lost due to no mm: 0 ---- Statistics for cpu : 0 Nr. samples lost cpu buffer overflow: 0 Nr. samples received: 23073 Nr. backtrace aborted: 0 Nr. samples lost invalid pc: 0 --- PATCHED --- -- OProfile Statistics -- Nr. sample dumps: 6 Nr. non-backtrace samples: 15924 Nr. kernel samples: 2698 Nr. lost samples (no kernel/user): 0 Nr. lost kernel samples: 0 Nr. incomplete code structs: 0 Nr. samples lost due to sample file open failure: 0 Nr. samples lost due to no permanent mapping: 4103 Nr. event lost due to buffer overflow: 0 Nr. samples lost due to no mapping: 121 Nr. backtraces skipped due to no file mapping: 0 Nr. samples lost due to no mm: 0 ---- Statistics for cpu : 0 Nr. samples lost cpu buffer overflow: 0 Nr. samples received: 23725 Nr. backtrace aborted: 0 Nr. samples lost invalid pc: 0 --Gabor |
From: Daniel H. <dan...@li...> - 2010-03-03 16:18:12
|
Hi Gabor, due to much work in this and last week I could not look into details of that problem and your patch. I will look into that in the next days and will come back next week. Sorry for that. Kind regards, Daniel Gabor Loki wrote: > Maynard Johnson wrote: >> I'm no JIT expert, but I'm certain that frequently used code blocks are saved so they don't need to be re-compiled again. Don't such code blocks remain mmap'ed into the process' address space? What you're describing here would seem to be a piece of code that, once compiled, does not get used for long, such that the JIT compiler eventually kicks it out of the cache (and nummap'ing the anon memory). In that case, why do you care if a few samples get discarded? Or am I missing something. > > Just to make sure of missing sample: > with cvs head: > $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > 8757 > $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > 0 > > with the patch: > $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > 12747 > $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > 4105 > > Well, I do not say this is negligible difference. The JIT codes consume > 30% from total time. This is what I expect from this app. > > Unfortunately the statistics in oprofiled.log does not show any > differences. > > --- HEAD ---- > > -- OProfile Statistics -- > Nr. sample dumps: 6 > Nr. non-backtrace samples: 15397 > Nr. kernel samples: 2488 > Nr. lost samples (no kernel/user): 0 > Nr. lost kernel samples: 0 > Nr. incomplete code structs: 0 > Nr. samples lost due to sample file open failure: 0 > Nr. samples lost due to no permanent mapping: 4030 > Nr. event lost due to buffer overflow: 0 > Nr. samples lost due to no mapping: 107 > Nr. backtraces skipped due to no file mapping: 0 > Nr. samples lost due to no mm: 0 > > ---- Statistics for cpu : 0 > Nr. samples lost cpu buffer overflow: 0 > Nr. samples received: 23073 > Nr. backtrace aborted: 0 > Nr. samples lost invalid pc: 0 > > > --- PATCHED --- > > -- OProfile Statistics -- > Nr. sample dumps: 6 > Nr. non-backtrace samples: 15924 > Nr. kernel samples: 2698 > Nr. lost samples (no kernel/user): 0 > Nr. lost kernel samples: 0 > Nr. incomplete code structs: 0 > Nr. samples lost due to sample file open failure: 0 > Nr. samples lost due to no permanent mapping: 4103 > Nr. event lost due to buffer overflow: 0 > Nr. samples lost due to no mapping: 121 > Nr. backtraces skipped due to no file mapping: 0 > Nr. samples lost due to no mm: 0 > > ---- Statistics for cpu : 0 > Nr. samples lost cpu buffer overflow: 0 > Nr. samples received: 23725 > Nr. backtrace aborted: 0 > Nr. samples lost invalid pc: 0 > > > > --Gabor > > |
From: Gabor L. <lo...@in...> - 2010-03-23 17:37:44
|
Hi Daniel, did you have some time to look into the proposed patch? --Gabor Daniel Hansel wrote: > due to much work in this and last week I could not look into details of that problem \ > and your patch. I will look into that in the next days and will come back next week. > Sorry for that. > > Kind regards, > Daniel > > Gabor Loki wrote: > > Just to make sure of missing sample: > > with cvs head: > > $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > > 8757 > > $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > > 0 > > > > with the patch: > > $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > > 12747 > > $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' > > 4105 > > > > Well, I do not say this is negligible difference. The JIT codes consume > > 30% from total time. This is what I expect from this app. > > > > Unfortunately the statistics in oprofiled.log does not show any > > differences. |
From: Daniel H. <dan...@li...> - 2010-03-23 22:40:31
|
Hi Gabor, Maynard and me have discussed / reviewed your patch. Your patch looks good so far. But I have to do some extensive testing of Java profiling to see if all is ok or if there are other problems we did not see. Hopefully I can find some time during this week. I will come back as soon as possible. Kind regards, Daniel Gabor Loki wrote: > Hi Daniel, > > did you have some time to look into the proposed patch? > > --Gabor > > Daniel Hansel wrote: >> due to much work in this and last week I could not look into details of that problem \ >> and your patch. I will look into that in the next days and will come back next week. >> Sorry for that. >> >> Kind regards, >> Daniel >> >> Gabor Loki wrote: >>> Just to make sure of missing sample: >>> with cvs head: >>> $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' >>> 8757 >>> $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' >>> 0 >>> >>> with the patch: >>> $> opreport -l|grep "test-app"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' >>> 12747 >>> $> opreport -l|grep "anon\|jit"|awk 'BEGIN{o=0;}{o+=$1;}END{print o;}' >>> 4105 >>> >>> Well, I do not say this is negligible difference. The JIT codes consume >>> 30% from total time. This is what I expect from this app. >>> >>> Unfortunately the statistics in oprofiled.log does not show any >>> differences. > > |