|
From: Bryan M. <om...@br...> - 2006-08-06 13:21:54
|
(oops - messed up my sender alias - sorry) Josef, thanks for the feedback. I have added comments inline to preserve the context. Bryan "Brain Murders" Meredith Josef Weidendorfer wrote: > On Saturday 05 August 2006 13:52, Bryan Meredith wrote: >> COVERGRIND FILES START >> 0:/home/bryan/covergrind/coregrind/m_trampoline.S >> 1:/home/bryan/c++/qt-x11-opensource-src-4.1.4/examples/widgets/analogclock/../../../include/QtCore/../../src/corelib/global/qglobal.h >> 2:/home/bryan/c++/qt-x11-opensource-src-4.1.4/examples/widgets/analogclock/../../../include/QtCore/../../src/corelib/thread/qatomic.h > >> . >> 0:155-155 > > It actually would be very simple to change this to convert this to > cachegrinds/callgrind format. Callgrinds format has an extension to > exactly provide your file/ID number mapping. And you can load multiple > files to get merging done. However, instead of a range you currently > have to add every touched line seperatly (actually, KCachegrinds > parser and data model knows about ranges, but can not visualize them). > You would use an event type like "lineTouched", and give a 1 for every > line touched in execution: > > events: lineTouched > fl=(0) /home/bryan/covergrind/coregrind/m_trampoline.S > fl=(1) /.../qglobal.h > fl=(2) /.../qatomic.h > ... > fn=Unknown > fl=(0) > 155 1 > 161 1 > ... > fl=(1) > 754 1 > ... I changed Covergrind around to output this format (yes, it was simple) then passed it into kcachegrind. I was kind of expecting individual source files with the event marked as 1 next to all the lines that were hit but that didn't happen - I got a single concatenated file that had all the source regions in it and no way that I could find to show individual files or indeed which line belonged to which source file. I also could not stop it from ignoring some of the files. This will be a problem at work (where I intend to use this) as our codebase is huge and we need to know exactly what is and isn't being tested. This could well all be user error - I have seen kcachegrind in operation several times but never used it in anger myself. > > However: if you stripped the cache simulator from cachegrind (ie. only > do instruction fetches), wouldn't this more or less exactly do what > covergrind is doing now? > It never occurred to me to even look at cachegrind so you may well be correct. >>> If >>> you run a program multiple times, are the coverage results merged? >> In the interests of keeping things simple (and hence lightweight and >> fast) merging of results would be done by the GUI (long way of saying No). > > At least I plan to write a command line merge tool for Callgrinds (and thus, > also Cachegrinds) format; so this would be another advantage to output > this format. See above. I would love to use kcachegrind for the display. It helps me because users are more likely to have that around rather than want to download and build yet another tool (with everything that entails, including me having to hack one together in the first place :P) and it would help the users because they get to use something already familiar to them. > >>> Also, does it give percentage coverage? >> Again, in the GUI, with both the source file and line coverage >> information available, it will be simpler to generate any required >> metrics (another long way of saying No). > > Hmmm... Then your GUI has to read debug info yourself. This could indeed be the best way forward as source parsing can be fraught (although I was only planning on C/C++ support - the source would be available for others to extend if they wish). Again, because of the much less restricted environment compared to executing within Valgrind, this could well be a simpler task to accomplish. The output from "readelf -wl" doesn't seem to be too bad - an awk/perl script would make short work of it, outputting a list of lines that actually had code on them. > Otherwise you have > no way to see whether a non-executed line influences the cover percentage: > it could be actual code or not. > If you want to try parsing the code, keep in mind that this is not only > about empty lines or comments (also for Fortran, ADA, GAS ...), but also > about dead code via #ifdefs). > > Another metric a coverage tool should provide is branch/path coverage. > That is possible, but not easy either. I always considered that to be more of a profiling output than a coverage metric - for pure coverage, you simply hit the line or you don't. I do agree that determining this figure from post analysis would not be trivial. Whereas a user would see the file and notice the difference in counts for the conditional areas, it would not be so easy to pick that up with code. > > Josef > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Valgrind-developers mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-developers > |