From: Tom H. <tha...@ad...> - 2009-09-15 23:37:28
|
Hi all, I'm trying to profile TLB misses, specifically, but my Beagleboard (with Siarhei Siamashka's watchdog patch) records zero samples on any event I've tried other than the default CPU_CYCLES. I have not exhaustively tried all event types yet, but a number of them, e.g. INSTR_EXECUTED, DCACHE_ACCESS. There is one interesting metric in the log file, "backtraces skipped," below. I think that's due to JIT'd code running in the application, but it does suggest that oprofile is active. Any help would be much appreciated, thanks in advance! -- OProfile Statistics -- Nr. sample dumps: 2 Nr. non-backtrace samples: 0 Nr. kernel samples: 0 Nr. lost samples (no kernel/user): 0 Nr. lost kernel samples: 0 Nr. incomplete code structs: 0 Nr. samples lost due to sample file open failure: 0 Nr. samples lost due to no permanent mapping: 0 Nr. event lost due to buffer overflow: 0 Nr. samples lost due to no mapping: 0 Nr. backtraces skipped due to no file mapping: 61787 Nr. samples lost due to no mm: 0 ---- Statistics for cpu : 0 Nr. samples lost cpu buffer overflow: 0 Nr. samples received: 0 Nr. backtrace aborted: 0 Nr. samples lost invalid pc: 0 |
From: Maynard J. <may...@us...> - 2009-09-16 14:54:43
|
Tom Harwood wrote: > Hi all, > > I'm trying to profile TLB misses, specifically, but my Beagleboard (with Siarhei Siamashka's watchdog patch) records zero samples on any event I've tried other than the default CPU_CYCLES. I have not exhaustively tried all event types yet, but a number of them, e.g. INSTR_EXECUTED, DCACHE_ACCESS. Sorry, no ideas. Hopefully some folks on the list who speak "ARM" can help. > > There is one interesting metric in the log file, "backtraces skipped," below. I think that's due to JIT'd code running in the application, but it does suggest that oprofile is active. The bt_lost_no_mapping stat should only come into play when you are doing callgraph profiling AND the oprofile kernel code fails to log a backtrace sample (implying that it can't find a mapping for the instruction address it was trying to record). when you're doing callgraph profiling, you get all the "normal" samples, plus backtraces recorded for each of those. The "normal" samples are the non-backtrace samples, which in your case is zero. Since you're not getting any normal samples, you certainly should not see any backtrace records recorded for them, and therefore, should not ever be seeing any backtraces skipped counts. The fact that you are is a bug -- which I reported to the oprofile kernel driver maintainer on May 27, 2009. This is fixed upstream now, but evidently not in your running kernel. But this is just a nit and you can ignore it. -Maynard > > Any help would be much appreciated, thanks in advance! > > > -- OProfile Statistics -- > Nr. sample dumps: 2 > Nr. non-backtrace samples: 0 > Nr. kernel samples: 0 > Nr. lost samples (no kernel/user): 0 > Nr. lost kernel samples: 0 > Nr. incomplete code structs: 0 > Nr. samples lost due to sample file open failure: 0 > Nr. samples lost due to no permanent mapping: 0 > Nr. event lost due to buffer overflow: 0 > Nr. samples lost due to no mapping: 0 > Nr. backtraces skipped due to no file mapping: 61787 > Nr. samples lost due to no mm: 0 > > ---- Statistics for cpu : 0 > Nr. samples lost cpu buffer overflow: 0 > Nr. samples received: 0 > Nr. backtrace aborted: 0 > Nr. samples lost invalid pc: 0 > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > oprofile-list mailing list > opr...@li... > https://lists.sourceforge.net/lists/listinfo/oprofile-list |
From: Siarhei S. <sia...@no...> - 2009-09-18 13:08:54
|
On Wednesday 16 September 2009 01:25:31 ext Tom Harwood wrote: > Hi all, > > I'm trying to profile TLB misses, specifically, but my Beagleboard (with > Siarhei Siamashka's watchdog patch) records zero samples on any event I've > tried other than the default CPU_CYCLES. I have not exhaustively tried all > event types yet, but a number of them, e.g. INSTR_EXECUTED, DCACHE_ACCESS. Looks like you have spotted a (fatal?) flaw in this workaround (which was by itself not a final working solutions, but a proof of concept intended for testing). Now I think that if some events are naturally very rare (TLB misses), they may be all considered "noise" and filtered out. This can be probably fixed, but we will be still not sure whether we can trust the results and how much they are skewed. You can look for more infomation here: http://marc.info/?t=124628595500016&r=1&w=2 On the other hand, for such kind of events you can run oprofile driver without any kind of workarounds and it will probably work good enough. Actually in all my old experiments, the counters other than CCNT had been quite reliable. Right now here at Nokia we are using an OMAP-specific GPTIMER based oprofile driver which provides 2KiHz sampling rate. It is quite simple, low overhead and robust. Though it is useless for you as you want some more advanced statistics like DCACHE_ACCESS, etc.. -- Best regards, Siarhei Siamashka |
From: Tom H. <tha...@ad...> - 2009-09-19 16:05:31
|
Thanks, Siarhei and Maynard, for the suggestions. We have reverted to a kernel without the PMU patch, but we still see no data. Console log follows for the curious. This is on a freshly powered up Beagleboard -- the "avm" line runs our virtual machine to generate some load. The only thing that sticks out ls the "/usr/bin/objdump" output to opcontrol --setup, but the setup appears to be correct. Has anyone been able to use oprofile events other than CPU_CYCLES on the Beagleboard? Thanks again! -- Tom Harwood Adobe Systems $ opcontrol --deinit Daemon not running $ sudo rm ~root/.oprofile/daemonrc $ sudo ls ~root/.oprofile/ $ opcontrol --init $ opcontrol --setup -e ITLB_MISS:750 -e DTLB_REFILL:750 --vmlinux=/boot/vmlinux /usr/bin/objdump $ opcontrol --status Daemon not running Event 0: ITLB_MISS:750:0:1:1 Event 1: DTLB_REFILL:750:0:1:1 Separate options: none vmlinux file: /boot/vmlinux Image filter: none Call-graph depth: 0 $ opcontrol --start Using 2.6+ OProfile kernel interface. Reading module info. Using log file /var/lib/oprofile/samples/oprofiled.log Daemon started. Profiler running. $ avm sandbox_as/abcdump.abc -- sandbox_as/parseInt.abc > sandbox_as/parseInt.abcdump $ opcontrol --dump $ opcontrol --shutdown Stopping profiling. Killing daemon. $ opreport error: no sample files found: profile specification too strict ? $ tail /var/lib/oprofile/samples/oprofiled.log oprofiled started Sat Sep 19 08:35:28 2009 kernel pointer size: 4 Sat Sep 19 08:36:12 2009 -- OProfile Statistics -- Nr. sample dumps: 3 Nr. non-backtrace samples: 0 Nr. kernel samples: 0 Nr. lost samples (no kernel/user): 0 Nr. lost kernel samples: 0 Nr. incomplete code structs: 0 Nr. samples lost due to sample file open failure: 0 Nr. samples lost due to no permanent mapping: 0 Nr. event lost due to buffer overflow: 0 Nr. samples lost due to no mapping: 0 Nr. backtraces skipped due to no file mapping: 0 Nr. samples lost due to no mm: 0 ---- Statistics for cpu : 0 Nr. samples lost cpu buffer overflow: 0 Nr. samples received: 0 Nr. backtrace aborted: 0 Nr. samples lost invalid pc: 0 oprofiled stopped Sat Sep 19 08:36:12 2009 $ sudo cat ~root/.oprofile/daemonrc SESSION_DIR=/var/lib/oprofile CHOSEN_EVENTS_0=ITLB_MISS:750:0:1:1 CHOSEN_EVENTS_1=DTLB_REFILL:750:0:1:1 NR_CHOSEN=2 SEPARATE_LIB=0 SEPARATE_KERNEL=0 SEPARATE_THREAD=0 SEPARATE_CPU=0 VMLINUX=/boot/vmlinux IMAGE_FILTER= CPU_BUF_SIZE=0 CALLGRAPH=0 KERNEL_RANGE=00000000,00252804 XENIMAGE=none |
From: Robert R. <rob...@am...> - 2009-09-22 10:08:32
|
On 19.09.09 09:04:58, Tom Harwood wrote: > Thanks, Siarhei and Maynard, for the suggestions. We have reverted to a kernel without the PMU patch, but we still see no data. Console log follows for the curious. This is on a freshly powered up Beagleboard -- the "avm" line runs our virtual machine to generate some load. The only thing that sticks out ls the "/usr/bin/objdump" output to opcontrol --setup, but the setup appears to be correct. > > Has anyone been able to use oprofile events other than CPU_CYCLES on the Beagleboard? Thanks again! If your kernel includes this commit: 483e3cd Merge branch 'tracing-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip you also need this one: 1218259 Merge branch 'tracing-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip There was a bug in the ringbuffer. But I am not sure if this is valid for you. -Robert -- Advanced Micro Devices, Inc. Operating System Research Center email: rob...@am... |
From: Tom H. <tha...@ad...> - 2009-09-22 21:54:56
|
Thanks, Robert. We have been using a kernel built per the procedure detailed in http://elinux.org/BeagleBoardLinuxKernel -- with and without Siarhei Siamashka's patch. The merge you referenced looks like it's much too recent to affect us, but moving to a more current kernel is a good idea, which we'll try. Re: the oprofile-watchdog patch, we could also simply disable the noise-filtering code, since we don't typically profile idle systems. Thanks again! -- Tom Harwood, Adobe |