|
From: Stefan K. <en...@ho...> - 2005-03-04 16:17:37
|
hi, sorry if this has been answered before ... I look for a way to integrate valgrind into my unit-test suite. The unit-tests are written using check (http://check.sf.net). The project uses the autotools. Thus 'make check' runns the binaries and scripts listed in the TESTS variable of the Makefile.am. Now it would be nice to run the tests under valgrind, use the macros from e.g. memcheck.h at the end of each test and make the test fail if there are leaks. The problem is that the program is started without valgrind. One hack to solve it would be something like if(!RUNNING_ON_VALGRIND) { char cmd[FILENAME_MAX]; sprintf(cmd,"valgrind %s %s",valgrind_opts,argv[0]); argv[0]=cmd; execv(cmd, argv); } I've not yet tried it. Anyone a better idea? Commants? Ciao Stefan |
|
From: Nicholas N. <nj...@cs...> - 2005-03-05 01:23:24
|
On Fri, 4 Mar 2005, Stefan Kost wrote: > sorry if this has been answered before ... > I look for a way to integrate valgrind into my unit-test suite. > The unit-tests are written using check (http://check.sf.net). The project > uses the autotools. > Thus 'make check' runns the binaries and scripts listed in the TESTS variable > of the Makefile.am. > > Now it would be nice to run the tests under valgrind, use the macros from > e.g. memcheck.h at the end of each test and make the test fail if there are > leaks. > The problem is that the program is started without valgrind. Which program -- the test framework, or the individual tests? > One hack to solve it would be something like > > if(!RUNNING_ON_VALGRIND) { > char cmd[FILENAME_MAX]; > > sprintf(cmd,"valgrind %s %s",valgrind_opts,argv[0]); > argv[0]=cmd; > execv(cmd, argv); > } > > I've not yet tried it. Anyone a better idea? Commants? It's unclear where that code goes -- in the programs being tested? In the test framework? I also don't really see how it works. I would think that you need to find some way within the test framework to augment the command lines used to run the individual tests, and prefix them with "valgrind ...". N |
|
From: Stefan K. <en...@ho...> - 2005-03-05 17:54:45
|
Hi Nicholas, Some more explanation ;) the tests are written using the testing framework. One test-binary contains test suites, that in turn contains tests cases and these finally contain the tests. What I need is to make the test binary to be run under valgrind. The invokation of the test-binary can't be changed, as this is generated by the automake from Makefile.am. It just starts the binaries and depending on the return value, record a pass or a fail for the test. Thus I need a mechanism, that restarts the binary whne it not runs under valgrind. Thats what the code snipped should do. I'll try this out next week. Stefan > On Fri, 4 Mar 2005, Stefan Kost wrote: > >> sorry if this has been answered before ... >> I look for a way to integrate valgrind into my unit-test suite. >> The unit-tests are written using check (http://check.sf.net). The >> project uses the autotools. >> Thus 'make check' runns the binaries and scripts listed in the TESTS >> variable of the Makefile.am. >> >> Now it would be nice to run the tests under valgrind, use the macros >> from e.g. memcheck.h at the end of each test and make the test fail if >> there are leaks. >> The problem is that the program is started without valgrind. > > > Which program -- the test framework, or the individual tests? > >> One hack to solve it would be something like >> >> if(!RUNNING_ON_VALGRIND) { >> char cmd[FILENAME_MAX]; >> >> sprintf(cmd,"valgrind %s %s",valgrind_opts,argv[0]); >> argv[0]=cmd; >> execv(cmd, argv); >> } >> >> I've not yet tried it. Anyone a better idea? Commants? > > > It's unclear where that code goes -- in the programs being tested? In > the test framework? I also don't really see how it works. > > I would think that you need to find some way within the test framework > to augment the command lines used to run the individual tests, and > prefix them with "valgrind ...". > > N |
|
From: Nicholas N. <nj...@cs...> - 2005-03-06 06:38:36
|
On Sat, 5 Mar 2005, Stefan Kost wrote: > Thus I need a mechanism, that restarts the binary whne it not runs under > valgrind. Thats what the code snipped should do. I'll try this out next week. Ok. Valgrind has to run a program from the very start -- it cannot attach itself to an already running program -- so I guess you'll have to do something like what you suggest. Good luck. N |
|
From: Jeroen N. W. <jn...@xs...> - 2005-03-06 13:58:38
|
> hi, > > sorry if this has been answered before ... > I look for a way to integrate valgrind into my unit-test suite. > The unit-tests are written using check (http://check.sf.net). The > project uses the autotools. > Thus 'make check' runns the binaries and scripts listed in the TESTS > variable of the Makefile.am. > > Now it would be nice to run the tests under valgrind, use the macros > from e.g. memcheck.h at the end of each test and make the test fail if > there are leaks. > The problem is that the program is started without valgrind. > FWIW, here is how I integrate valgrind into my unit-test suite. I leave `make check` as it is, but add a new target `make valgrind` which memchecks all programs in variable TESTS in file Makefile.am. 1. In file configure.in, create the option to use valgrind: AC_ARG_WITH([valgrind], AC_HELP_STRING([--with-valgrind], [where Valgrind is installed on your system (default is no)]), [ac_cv_use_valgrind=$withval], [ac_cv_use_valgrind=no])dnl VALGRIND=$ac_cv_use_valgrind AC_SUBST(VALGRIND)dnl AM_CONDITIONAL(HAVEVALGRIND, test x$VALGRIND != xno)dnl VALGRIND_CPPFLAGS= if test $ac_cv_use_valgrind != no; then AC_DEFINE(HAVE_VALGRIND, 1, [Define if valgrind is to be used.]) if test -f $ac_cv_use_valgrind/valgrind.h; then VALGRIND_CPPFLAGS=-I$ac_cv_use_valgrind elif test -f $ac_cv_use_valgrind/include/valgrind.h; then VALGRIND_CPPFLAGS=-I$ac_cv_use_valgrind/include elif test -f $ac_cv_use_valgrind/valgrind/valgrind.h; then VALGRIND_CPPFLAGS=-I$ac_cv_use_valgrind/valgrind elif test -f $ac_cv_use_valgrind/include/valgrind/valgrind.h; then VALGRIND_CPPFLAGS=-I$ac_cv_use_valgrind/include/valgrind elif test -f $ac_cv_use_valgrind/valgrind/include/valgrind.h; then VALGRIND_CPPFLAGS=-I$ac_cv_use_valgrind/valgrind/include fi fi AC_SUBST(VALGRIND_CPPFLAGS)dnl 2a. Add some genaral stuff for valgrind in file Makefile.am: AM_CPPFLAGS = @VALGRIND_CPPFLAGS@ # and -Wall -W -pedantic etc. 2b. And specific stuff for the valgrind target: if HAVEVALGRIND VALPREFIX = $(TESTS_ENVIRONMENT) GLIBCPP_FORCE_NEW= VALDEFAULT = @VALGRIND@/bin/valgrind VALOPTIONS = --tool=memcheck -q --show-reachable=yes --leak-check=yes --num-callers=20 VALCMD = $(VALPREFIX) $(VALDEFAULT) $(VALOPTIONS) valgrind: $(TESTS) for i in $^; do $(VALCMD) ./$$i; done endif 3. In file Makefile.am, I also have TESTS_ENVIRONMENT = ulimit -St20; to prevent endless loops in both `make check` and `make valgrind`. The semicolon at the end of this statement is required. 4. You can use the following script to invoke `make valgrind` if and only if it is present in file Makefile: #!/bin/bash # Target valgrind need not be present and need not succeed. if grep -qE "^valgrind: " Makefile; then make valgrind || true fi Regards, Jeroen. |