|
From: He f. <fig...@gm...> - 2009-07-27 18:29:01
|
Hi All, My current project works on a unit testing with Valgrind. In order to better test the code, a code coverage output with Valgrind is desired. I've looked into Cachegrind and Callgrind and I believe callgrind is more close to what I need. But after running callgrind, I was dissapointted as it doesn't provide a general code percentile summary (like how many lines have been covered), but rather how many instruction covered. I googled online and found no clue info regarding this. The most possible approach will be http://www.brainmurders.eclipse.co.uk/covergrind.html, which will require extra effort in compiling and add-on. I wonder if there's option or combined method within current Valgrind framework that this can be done. Best regards, -- Figo |
|
From: Josef W. <Jos...@gm...> - 2009-07-27 19:44:32
|
On Monday 27 July 2009, He fei wrote: > Hi All, > My current project works on a unit testing with Valgrind. In order to better > test the code, a code coverage output with Valgrind is desired. > I've looked into Cachegrind and Callgrind and I believe callgrind is more > close to what I need. > > But after running callgrind, I was dissapointted as it doesn't provide a > general code percentile summary (like how many lines have been covered), but > rather how many instruction covered. Because Cachegrind/Callgrind never touches code which was not executed, you can not calculate a percentile summary of covered code. There needs to be a detection phase of what source code lines exist even if no code was executed from them. This could then be combined with Callgrinds/Cachegrinds data to provide the summary you want. > I googled online and found no clue info regarding this. The most possible > approach will be http://www.brainmurders.eclipse.co.uk/covergrind.html, > which will require extra effort in compiling and add-on. In Covergrind, I only see the metric "LinesTouched". No idea if it provides "LinesNotTouched" which would be required. Josef > > I wonder if there's option or combined method within current Valgrind > framework that this can be done. > > Best regards, > |
|
From: He f. <fig...@gm...> - 2009-07-27 20:54:11
|
Hi Josef, Thanks for your input~ So you mean there doesn't actually exist a real code coverage tool within Valgrind that can act like gconv? Then what about execute valgrind with the target program and then gconv it (with gconv option turned on during gcc compilation)? Will this does the trick? Regards, -Fei On Mon, Jul 27, 2009 at 3:43 PM, Josef Weidendorfer < Jos...@gm...> wrote: > On Monday 27 July 2009, He fei wrote: > > Hi All, > > My current project works on a unit testing with Valgrind. In order to > better > > test the code, a code coverage output with Valgrind is desired. > > I've looked into Cachegrind and Callgrind and I believe callgrind is more > > close to what I need. > > > > But after running callgrind, I was dissapointted as it doesn't provide a > > general code percentile summary (like how many lines have been covered), > but > > rather how many instruction covered. > > Because Cachegrind/Callgrind never touches code which was not executed, you > can not > calculate a percentile summary of covered code. There needs to be a > detection phase > of what source code lines exist even if no code was executed from them. > This could > then be combined with Callgrinds/Cachegrinds data to provide the summary > you want. > > > I googled online and found no clue info regarding this. The most possible > > approach will be http://www.brainmurders.eclipse.co.uk/covergrind.html, > > which will require extra effort in compiling and add-on. > > In Covergrind, I only see the metric "LinesTouched". No idea if it provides > "LinesNotTouched" which would be required. > > Josef > > > > > I wonder if there's option or combined method within current Valgrind > > framework that this can be done. > > > > Best regards, > > > > > -- Fei He Graduate Student Information Networking Institute Carnegie Mellon University fh...@cm... +1 412 330 8051 |
|
From: Josef W. <Jos...@gm...> - 2009-07-27 21:12:35
|
On Monday 27 July 2009, He fei wrote: > Hi Josef, > Thanks for your input~ > > So you mean there doesn't actually exist a real code coverage tool within > Valgrind that can act like gconv? I suppose you talk about "gcov". Actually, AFAIK, no. > Then what about execute valgrind with the target program and then gconv it > (with gconv option turned on during gcc compilation)? If you recompile your code for use with gcov, why would you use a Valgrind tool at all? What is the expected benefit? Perhaps I misunderstood your request: I never used gcov myself. Josef > > Will this does the trick? > > Regards, > > -Fei > > On Mon, Jul 27, 2009 at 3:43 PM, Josef Weidendorfer < > Jos...@gm...> wrote: > > > On Monday 27 July 2009, He fei wrote: > > > Hi All, > > > My current project works on a unit testing with Valgrind. In order to > > better > > > test the code, a code coverage output with Valgrind is desired. > > > I've looked into Cachegrind and Callgrind and I believe callgrind is more > > > close to what I need. > > > > > > But after running callgrind, I was dissapointted as it doesn't provide a > > > general code percentile summary (like how many lines have been covered), > > but > > > rather how many instruction covered. > > > > Because Cachegrind/Callgrind never touches code which was not executed, you > > can not > > calculate a percentile summary of covered code. There needs to be a > > detection phase > > of what source code lines exist even if no code was executed from them. > > This could > > then be combined with Callgrinds/Cachegrinds data to provide the summary > > you want. > > > > > I googled online and found no clue info regarding this. The most possible > > > approach will be http://www.brainmurders.eclipse.co.uk/covergrind.html, > > > which will require extra effort in compiling and add-on. > > > > In Covergrind, I only see the metric "LinesTouched". No idea if it provides > > "LinesNotTouched" which would be required. > > > > Josef > > > > > > > > I wonder if there's option or combined method within current Valgrind > > > framework that this can be done. > > > > > > Best regards, > > > > > > > > > > > |
|
From: Nicholas N. <n.n...@gm...> - 2009-07-27 22:28:06
|
On Tue, Jul 28, 2009 at 6:53 AM, He fei<fig...@gm...> wrote: > Hi Josef, > Thanks for your input~ > So you mean there doesn't actually exist a real code coverage tool within > Valgrind that can act like gconv? > Then what about execute valgrind with the target program and then gconv it > (with gconv option turned on during gcc compilation)? > Will this does the trick? > Regards, svn://svn.valgrind.org/valgrind/branches/VCOV has an attempt at exactly this. It uses debug info to compute the executed/not-executed percentages. It's probably broken, though. Nick |
|
From: Josef W. <Jos...@gm...> - 2009-07-28 07:29:49
|
On Tuesday 28 July 2009, Nicholas Nethercote wrote: > On Tue, Jul 28, 2009 at 6:53 AM, He fei<fig...@gm...> wrote: > > Hi Josef, > > Thanks for your input~ > > So you mean there doesn't actually exist a real code coverage tool within > > Valgrind that can act like gconv? > > Then what about execute valgrind with the target program and then gconv it > > (with gconv option turned on during gcc compilation)? > > Will this does the trick? > > Regards, > > svn://svn.valgrind.org/valgrind/branches/VCOV has an attempt at Ah, sorry, forgot about this. Josef > exactly this. It uses debug info to compute the executed/not-executed > percentages. It's probably broken, though. > > Nick > |
|
From: He f. <fig...@gm...> - 2009-07-28 05:25:46
|
Hi Josef, Gcov is not available under my current system, and currently not supported yet. I've been able to tried out Covergrind, with Valgrind version 3.2.1. There's a guy asked about this before: http://www.nabble.com/code-coverage-with-Valgrind-%2B-Covergrind-td15308871.html . It turned out that though the output of Covergrind is similar to Callgrind, but rather at line level: fl=(10) /program-path ... fn=10 fl=(10) 52 1 53 1 ... As above, each line that has been "touched" will be maked at 1 (and the number can only be 1). I've written a little test program and result is quite inspiring. So I think the only thing I need to do now is to write a script to count the total line within a program (better to remove #include file lines and comment lines). And then compare the count with the sum that can be calculated based on the above output. Am I right about this? Thanks~ -Fei On Mon, Jul 27, 2009 at 5:11 PM, Josef Weidendorfer < Jos...@gm...> wrote: > On Monday 27 July 2009, He fei wrote: > > Hi Josef, > > Thanks for your input~ > > > > So you mean there doesn't actually exist a real code coverage tool within > > Valgrind that can act like gconv? > > I suppose you talk about "gcov". Actually, AFAIK, no. > > > Then what about execute valgrind with the target program and then gconv > it > > (with gconv option turned on during gcc compilation)? > > If you recompile your code for use with gcov, why would you use a Valgrind > tool at all? What is the expected benefit? > > Perhaps I misunderstood your request: I never used gcov myself. > > Josef > > > > > Will this does the trick? > > > > Regards, > > > > -Fei > > > > On Mon, Jul 27, 2009 at 3:43 PM, Josef Weidendorfer < > > Jos...@gm...> wrote: > > > > > On Monday 27 July 2009, He fei wrote: > > > > Hi All, > > > > My current project works on a unit testing with Valgrind. In order to > > > better > > > > test the code, a code coverage output with Valgrind is desired. > > > > I've looked into Cachegrind and Callgrind and I believe callgrind is > more > > > > close to what I need. > > > > > > > > But after running callgrind, I was dissapointted as it doesn't > provide a > > > > general code percentile summary (like how many lines have been > covered), > > > but > > > > rather how many instruction covered. > > > > > > Because Cachegrind/Callgrind never touches code which was not executed, > you > > > can not > > > calculate a percentile summary of covered code. There needs to be a > > > detection phase > > > of what source code lines exist even if no code was executed from them. > > > This could > > > then be combined with Callgrinds/Cachegrinds data to provide the > summary > > > you want. > > > > > > > I googled online and found no clue info regarding this. The most > possible > > > > approach will be > http://www.brainmurders.eclipse.co.uk/covergrind.html, > > > > which will require extra effort in compiling and add-on. > > > > > > In Covergrind, I only see the metric "LinesTouched". No idea if it > provides > > > "LinesNotTouched" which would be required. > > > > > > Josef > > > > > > > > > > > I wonder if there's option or combined method within current Valgrind > > > > framework that this can be done. > > > > > > > > Best regards, > > > > > > > > > > > > > > > > > > > > -- Fei He Graduate Student Information Networking Institute Carnegie Mellon University fh...@cm... +1 412 330 8051 |
|
From: Nicholas N. <n.n...@gm...> - 2009-07-28 05:41:21
|
On Tue, Jul 28, 2009 at 3:25 PM, He fei<fig...@gm...> wrote: > I've been able to tried out Covergrind, with Valgrind version 3.2.1. There's > a guy asked about this > before: http://www.nabble.com/code-coverage-with-Valgrind-%2B-Covergrind-td15308871.html. > It turned out that though the output of Covergrind is similar to Callgrind, > but rather at line level: > fl=(10) /program-path > ... > fn=10 > fl=(10) > 52 1 > 53 1 > ... > As above, each line that has been "touched" will be maked at 1 (and the > number can only be 1). > I've written a little test program and result is quite inspiring. So I think > the only thing I need to do now is to write a script > to count the total line within a program (better to remove #include file > lines and comment lines). And then compare the count with the sum that can > be calculated based on the above output. > Am I right about this? No. The tricky part is knowing which source code lines are executable and which are not. Just removing #include lines and comment lines isn't enough. Eg, in this snippet: if (c) { foo; } else { bar; } lines 1, 2 and 4 are executable. Ultimately, only the compiler knows which lines are executable. VCov uses the debug info to get that knowledge from the compiler. Furthermore, I updated VCov a little this morning and fixed a bug and it seems to be basically working. I suggest you try it. Nick |
|
From: He f. <fig...@gm...> - 2009-07-30 04:43:33
|
Thanks.
Regarding exp-vcov,
I've checkout the lastest code with rev:10644. It still abort saying “no
such line” during bootstrap. I'll take a look.
Regarding Covergrind,
it seems to work fine with detection of the if-else branch (below). Input 3
is given at prompt.
Note that line 14 is not taken into line counting. But the problem with
Covergrind is that it has taken line 6-8 and line 21 into consideration,
which introduce inaccuracy. Gcov will smartly ignore them.
Given that, it seems still possible to improve Covergrind via some
post-processing scripting to eliminate those in-effective lines to reach
accuracy.
Cheers,
-Fei
<<<<<<<<<<<<<<<<<<<<<<
5 int main () {
6
7 int x;
8
9 scanf("%d", &x);
10
11
12 if (x==1)
13 printf("BINGO!: x = %d\n", x);
14 else
15 printf("ELSE: x = %d\n", x);
16
17
18
19 return 1;
20
21 }
>>>>>>>>>>>>>>>>>>>>>>>>>>
events: lineTouched
fl=(0) /usr/cv_test.c
fn=0
fl=(0)
5 1
6 1
7 1
8 1
9 1
12 1
15 1
19 1
20 1
21 1
On Wed, Jul 29, 2009 at 6:31 PM, Nicholas Nethercote <n.n...@gm...
> wrote:
> On Thu, Jul 30, 2009 at 1:46 AM, He fei<fig...@gm...> wrote:
> > Hi Nicholas,
> > I've checkout your branch and tried exp-vcov. I encounter the following
> > valgrind-abort when running it:
> > exp-VCov: vc_main.c:385 (doOneInstr): the 'impossible' happened.
> > exp-VCov: didn't find 288 in lineCCs
> >
> (/eng/btools/arch/i386-redhat-linux8.0/compilers/x86_64-rhel4-linux-gcc-4.2.3/bin/../lib/gcc/x86_64-rhel4-linux/4.2.3/../../../../x86_64-rhel4-linux/include/c++/4.2.3/bits/basic_string.h)
> > ==25477== at 0x380024AC: ??? (in
> > /usr/VCOV/inst/lib/valgrind/exp-vcov-amd64-linux)
> > ==25477== by 0x403FFE0DF: ???
> > ==25477== by 0x403FFE0DF: ???
> > ==25477== by 0x403FFFFDF: ???
> > sched status:
> > running_tid=1
> > Thread 1: status = VgTs_Runnable
> > ==25477== at 0xB220072:
> mgmt_logger::log_manager::_get_logger(std::string
> > const&) (basic_string.h:288)
> > ==25477== by 0xB2330BA: __static_initialization_and_destruction_0(int,
> > int) (mgmt_logger.h:372)
> > ...
> > In short, it seems to me that exp-vcov thinks the basic_string.h, which
> is
> > called by part of my code, doesn't have line 288. While in fact it does
> > (I've looked into the file).
> > The other thing I'm not sure is the question mark coming out like
> "==25477==
> > by 0x403FFFFDF: ???". Does it mean the compilation doesn't meet with
> > requirement?
>
> The "???" isn't important, ignore that.
>
> As for the crash, make sure you have the most recent SVN version, as I
> fixed a problem like that the other day. If you still have a problem,
> you might like to try fixing it yourself as I have very little time to
> work on VCov right now.
>
> Nick
>
--
Fei He
Graduate Student
Information Networking Institute
Carnegie Mellon University
fh...@cm...
+1 412 330 8051
|