|
From: Brad H. <br...@fr...> - 2004-11-06 00:44:43
|
I'm trying to do coverage testing for my unittests.
The target is Qt Cryptographic Architecture - it is a Qt based library that
supports plugins (essentially just more shared libraries). I'm most interes=
ted
in the plugins, although it is all (except for my unit test harness) of
interest in a coverage sense. All I really need is a post-run output for ea=
ch
line, like gcov produces.
I thought valgrind could do coverage testing, but a bit more research led me
to believe that I need callgrind to do so.=20
At this stage, I can't make callgrind 0.9.9 configure against the current
CVS (valgrind --version says valgrind-2.3.0.CVS).
The configure run ends with:
checking for valgrind installation... in /usr/local
checking if vg_skin.h is available... configure: error: No. Needs header fi=
les from Valgrind (2.0.x/2.2.x).
Perhaps a valgrind-dev package is missing in your installation?
The applicable part of config.log appears to be:
configure:3353: checking for valgrind installation
configure:3365: result: in /usr/local
configure:3387: checking if vg_skin.h is available
configure:3410: gcc -c -I /usr/local/include/valgrind conftest.c >&5
In file included from /usr/local/include/valgrind/vg_skin.h:3,
from conftest.c:19:
/usr/local/include/valgrind/tool.h:39:60: tool_arch.h: No such file or dire=
ctory
/usr/local/include/valgrind/tool.h:40:17: vki.h: No such file or directory
In file included from /usr/local/include/valgrind/vg_skin.h:3,
from conftest.c:19:
/usr/local/include/valgrind/tool.h: In function `main':
/usr/local/include/valgrind/tool.h:374: warning: `struct vki_rlimit' declar=
ed inside parameter list
/usr/local/include/valgrind/tool.h:374: warning: its scope is only this def=
inition or declaration, which is pr
obably not what you want
/usr/local/include/valgrind/tool.h:377: warning: `struct vki_rlimit' declar=
ed inside parameter list
/usr/local/include/valgrind/tool.h:441: warning: `struct vki_dirent' declar=
ed inside parameter list
I looked at /usr/local/include/valgrind/tool.h, and it contains:
#include "basic_types.h"
#include "tool_asm.h" // asm stuff
#include "tool_arch.h" // arch-specific tool stuff
#include "vki.h"
however it appears that tool_arch is in/usr/local/include/valgrind/x86/tool=
_arch.h
and vki.h is in /usr/local/include/valgrind/linux/vki.h
So I guess my questions are:
1. Do I need callgrind, or is there a Better Way?
2. If I need callgrind, how am I supposed to make it compile?
3. Does /usr/local/include/valgrind/tool.h make sense with those includes?
Brad
|
|
From: Tom H. <th...@cy...> - 2004-11-06 11:06:28
|
In message <200...@fr...>
Brad Hards <br...@fr...> wrote:
> I thought valgrind could do coverage testing, but a bit more research led
> me to believe that I need callgrind to do so.
I don't believe there is any valgrind tool for coverage testing at
the moment - it has been suggested but I don't think anybody has
written one. I don't think callgrind will help will it?
> At this stage, I can't make callgrind 0.9.9 configure against the current
> CVS (valgrind --version says valgrind-2.3.0.CVS).
I'd suggest sticking with 2.2.0 if you want to use callgrind. There
are a lot of changes going on in CVS at the moment so it's quite likely
that a third party tool like callgrind will have problems building
against it.
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Nicholas N. <nj...@ca...> - 2004-11-06 12:54:03
|
On Sat, 6 Nov 2004, Tom Hughes wrote: >> I thought valgrind could do coverage testing, but a bit more research led >> me to believe that I need callgrind to do so. > > I don't believe there is any valgrind tool for coverage testing at > the moment - it has been suggested but I don't think anybody has > written one. You can sort of use Cachegrind, and ignore everything except the "Ir" (instruction read) events -- the annotations will tell you, for every line, how many instructions were executed for it. Use --show=Ir option to the cg_annotate script. The downsides of this are that it doesn't save coverage information in a file, so you only get the coverage for a single run of the program. Also, it doesn't give percentage coverage figures. Callgrind might be able to do percentages, I can't remember. N |
|
From: Josef W. <Jos...@gm...> - 2004-11-08 12:43:50
|
On Saturday 06 November 2004 13:53, Nicholas Nethercote wrote: > On Sat, 6 Nov 2004, Tom Hughes wrote: > >> I thought valgrind could do coverage testing, but a bit more research > >> led me to believe that I need callgrind to do so. > .... > You can sort of use Cachegrind, and ignore everything except the "Ir" > (instruction read) events -- the annotations will tell you, for every > line, how many instructions were executed for it. Use --show=Ir option to > the cg_annotate script. > > The downsides of this are that it doesn't save coverage information in a > file, so you only get the coverage for a single run of the program. Also, > it doesn't give percentage coverage figures. Callgrind might be able to > do percentages, I can't remember. These are visualization issues: KCachegrind can load multiple files produced by Cachegrind or Callgrind, and adds them up. It also can show percentages, but probably the wrong ones: You want executed instructions related to all instructions for coverage, yes? Josef |
|
From: Josef W. <Jos...@gm...> - 2004-11-08 12:39:11
|
On Saturday 06 November 2004 12:06, Tom Hughes wrote: > In message <200...@fr...> > > Brad Hards <br...@fr...> wrote: > > I thought valgrind could do coverage testing, but a bit more research led > > me to believe that I need callgrind to do so. > > I don't believe there is any valgrind tool for coverage testing at > the moment - it has been suggested but I don't think anybody has > written one. I don't think callgrind will help will it? Cachegrind and Callgrind both only give you some information that some code has been executed, but not how much not-executed code with debug-info exists, i.e. code which should be covered by tests. I thought that gcov could deliver this information (?), but still there is a format conflict then. I once wrote a VG skin outputing static information of executables: The first time it touches a new executable/library, it dumps out a file with all instructions for which debug info exists. I did it this way as it was the simplest for me. See kcachegrind.sf.net/stagrind-0.0.1.tar.gz It is a little hacky, because it uses some function not officially available for VG tools. This could be loaded together with data from cachegrind/callgrind into KCachegrind. To get coverage info, you still need a formula to combine the information, like "percentage of executed to all instructions with debug info". This needs a formula with operator ">", which is unfortunately not supported in current KCachegrind (but should be easy to add). > > At this stage, I can't make callgrind 0.9.9 configure against the current > > CVS (valgrind --version says valgrind-2.3.0.CVS). Everything needed is adding a include path for compilation. But better stay with VG 2.2.0. Josef > I'd suggest sticking with 2.2.0 if you want to use callgrind. There > are a lot of changes going on in CVS at the moment so it's quite likely > that a third party tool like callgrind will have problems building > against it. > > Tom |