|
From: Bryan M. <om...@br...> - 2006-08-05 11:52:53
|
Nick, thanks as ever for the quick reply. I have inserted some comments below so that the context is intact. Bryan "Brain Murders" Meredith Nicholas Nethercote wrote: > On Fri, 4 Aug 2006, Bryan Meredith wrote: > >> please check my small web page for the details: >> http://www.brainmurders.eclipse.co.uk/covergrind.html >> >>> From the web page: >> ------------------ >> Covergrind is a code coverage tool for the excellent Valgrind framework. >> Rather than having to link against strange libraries or compile in >> various support, Covergrind uses run time instrumentation in order to >> work out which parts of your source code are traversed. There have been >> comments made about how trivial it would be to use the Valgrind >> framework to construct such a tool but I wasn't able to find one so here >> is mine to share. >> ------------------ > > You don't explain how to use it -- what does the output look like? You need to follow the link in order to download it. Details are available there of how to run it. I admit there is no sample output on the web page - I shall add that on the next revision although I had hoped that the omission was an incentive for people to try it out... :D It looks like this: ==4464== Covergrind-beta-01, A code coverage tool.. ==4464== Copyright (C) 2006, and GNU GPL'd, by Bryan Meredith. ==4464== Using LibVEX rev 1631, a library for dynamic binary translation. ==4464== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==4464== Using valgrind-3.3.0.SVN, a dynamic binary instrumentation framework. ==4464== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==4464== For more details, rerun with: -v ==4464== ==4464== COVERGRIND FILES START 0:/home/bryan/covergrind/coregrind/m_trampoline.S 1:/home/bryan/c++/qt-x11-opensource-src-4.1.4/examples/widgets/analogclock/../../../include/QtCore/../../src/corelib/global/qglobal.h 2:/home/bryan/c++/qt-x11-opensource-src-4.1.4/examples/widgets/analogclock/../../../include/QtCore/../../src/corelib/thread/qatomic.h . . (snipped) . 302:/usr/src/packages/BUILD/glibc-2.4/cc-nptl/csu/crti.S 303:/usr/src/packages/BUILD/glibc-2.4/cc-nptl/csu/crtn.S COVERGRIND FILES END COVERGRIND LINES START 0:155-155 0:161-161 1:754-754 1:1538-1538 2:74-75 2:75-75 3:77-77 4:93-94 . . (snipped) . 302:11-14 302:17-18 302:25-26 303:9-10 303:16-17 COVERGRIND LINES END As you can see, each file is given an identifier. These identifiers are then used to show the line coverage information in order to help keep the file size down. I hope you will agree that it is easily parsable by man or machine. The above is the output from running the QT "analogclock" example application (C++). We (at work) are not too interested in how many times a line range has been hit but there is partial support in the Covergrind source for it and I will probably end up implementing it fully. I would envisage adding the hit count to the end of the line range. Actually, I just noticed that it might be a good idea to output the name of the executable under test as well. > If > you run a program multiple times, are the coverage results merged? In the interests of keeping things simple (and hence lightweight and fast) merging of results would be done by the GUI (long way of saying No). > Also, does it give percentage coverage? Again, in the GUI, with both the source file and line coverage information available, it will be simpler to generate any required metrics (another long way of saying No). > Those latter two things are the > tricky part of writing a coverage tool with Valgrind. I appear to have side-stepped both of them. :P Callgrind has a wonderful GUI for post processing and combining run data so this is not setting any precedent. > I wrote one a > while ago ("vcov") which I never fully tested and released properly. I > attach the code in case you find it useful. Thanks. I shall have a look. > > vcov does the merging from multiple runs ok. It also does the > percentage coverage calculation, but it's not perfect -- because you > have to know how many lines of executable code each file has, and that's > only available from the debug info, and I found that some versions of > GCC produced debug info that was unreliable. If I understand this properly, my method differs from this. By outputting source code line number ranges, the source itself can be annotated by the GUI. With suitable parsing of the source to ignore comment only and whitespace only lines, the resulting statistics should be quite accurate for the more common coding styles. Like gcov, one statement per line styles will produce better results. I felt that the best approach was to just output source file line ranges in a fairly dumb fashion and let another tool post process the data. Because the post processing tool (my proposed GUI) would not be in such a constrained environment, there are many more implementation options available, working with the original source file rather than just the debug information being the most important IMO. > > Benoit Peccatte wrote a similar tool ("cover"), I'm not sure how > advanced it is, you can get it here > http://b.peccatte.free.fr/cover-0.03.tgz. Is it still active? I have no problem picking up someone else's work and submitting patches - I am old enough to have gotten over "not written here" syndrome more than two decades ago - but I will only do that if there are signs of active development and I can't say that I have seen anything on the list for a long time. This will be alive for some time yet as I have written it to use at work. It is also simple enough that others can get right in and hack on it if they wish (and I would like to at least hear about it if they do). > > Nick > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > ------------------------------------------------------------------------ > > _______________________________________________ > Valgrind-developers mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-developers |