Menu
▾
▴
[Valgrind-developers] patch - enable testing of xml output within the reg tests.
|
From: Ashley P. <as...@qu...> - 2006-10-24 17:50:04
|
Hi,
I've attached a patch I use for testing the xml output of valgrind. The
concept is simple enough, if enabled it inserts adds --xml=yes to the
valgrind command line a "prefilter" command to convert from xml back to
the raw valgrind output. This is then checked for errors in the normal
way.
To go along with this I'm developing a stand-alone perl utility which
converts from xml output back into raw output. I'll be submitting this
shortly.
Currently I see a number of extra failures when testing this way, a
number of which represent bugs in the xml output of memcheck, mostly of
the form where memcheck prints a error message but doesn't put the
correct xml tags around it.
Ashley,
Errors I get normally:
memcheck/tests/badjump (stderr)
memcheck/tests/badpoll (stderr)
memcheck/tests/buflen_check (stderr)
memcheck/tests/describe-block (stderr)
memcheck/tests/execve (stderr)
memcheck/tests/execve2 (stderr)
memcheck/tests/fwrite (stderr)
memcheck/tests/match-overrun (stderr)
memcheck/tests/pointer-trace (stderr)
memcheck/tests/stack_switch (stderr)
memcheck/tests/supp_unknown (stderr)
memcheck/tests/writev (stderr)
memcheck/tests/x86/scalar (stderr)
memcheck/tests/x86/scalar_exit_group (stderr)
memcheck/tests/x86/scalar_supp (stderr)
memcheck/tests/xml1 (stderr)
Errors I get using xml output.
memcheck/tests/addressable (stderr)
memcheck/tests/badjump (stderr)
memcheck/tests/badjump2 (stderr)
memcheck/tests/badpoll (stderr)
memcheck/tests/brk (stderr)
memcheck/tests/brk2 (stderr)
memcheck/tests/buflen_check (stderr)
memcheck/tests/describe-block (stderr)
memcheck/tests/erringfds (stderr)
memcheck/tests/execve (stderr)
memcheck/tests/execve2 (stderr)
memcheck/tests/fwrite (stderr)
memcheck/tests/leak-0 (stderr)
memcheck/tests/leak-cycle (stderr)
memcheck/tests/leak-pool-0 (stderr)
memcheck/tests/leak-pool-1 (stderr)
memcheck/tests/leak-pool-2 (stderr)
memcheck/tests/leak-pool-3 (stderr)
memcheck/tests/leak-pool-4 (stderr)
memcheck/tests/leak-pool-5 (stderr)
memcheck/tests/leak-regroot (stderr)
memcheck/tests/leak-tree (stderr)
memcheck/tests/malloc3 (stderr)
memcheck/tests/match-overrun (stderr)
memcheck/tests/mempool (stderr)
memcheck/tests/nanoleak (stderr)
memcheck/tests/new_override (stderr)
memcheck/tests/partial_load_dflt (stderr)
memcheck/tests/partial_load_ok (stderr)
memcheck/tests/partiallydefinedeq (stderr)
memcheck/tests/pointer-trace (stderr)
memcheck/tests/post-syscall (stderr)
memcheck/tests/sh-mem (stderr)
memcheck/tests/sigaltstack (stderr)
memcheck/tests/sigkill (stderr)
memcheck/tests/sigprocmask (stderr)
memcheck/tests/stack_switch (stderr)
memcheck/tests/supp_unknown (stderr)
memcheck/tests/toobig-allocs (stderr)
memcheck/tests/trivialleak (stderr)
memcheck/tests/writev (stderr)
memcheck/tests/x86/scalar (stderr)
memcheck/tests/x86/scalar_exit_group (stderr)
memcheck/tests/x86/scalar_fork (stderr)
memcheck/tests/x86/scalar_supp (stderr)
memcheck/tests/x86/scalar_vfork (stderr)
memcheck/tests/xml1 (stderr)
ashley:valgrind> svn diff Makefile.am tests/vg_regtest.in
Index: Makefile.am
===================================================================
--- Makefile.am (revision 6340)
+++ Makefile.am (working copy)
@@ -64,6 +64,9 @@
regtest: check
@PERL@ tests/vg_regtest $(TOOLS)
+regtestxml: check
+ @PERL@ tests/vg_regtest --xml-tool=xml2raw.pl $(TOOLS)
+
## Preprend @PERL@ because tests/vg_per isn't executable
perf: check
@PERL@ perf/vg_perf perf
Index: tests/vg_regtest.in
===================================================================
--- tests/vg_regtest.in (revision 6340)
+++ tests/vg_regtest.in (working copy)
@@ -93,11 +93,13 @@
my $tmp="vg_regtest.tmp.$$";
# Test variables
-my $vgopts; # valgrind options
+my $vgopts; # valgrind options (per test)
+my @vgextraopts; # valgrind options (global)
my $prog; # test prog
my $args; # test prog args
-my $stdout_filter; # filter program to run stdout results file through
-my $stderr_filter; # filter program to run stderr results file through
+my $stdout_filter; # filter program to run stdout results file through (per test)
+my $stderr_filter; # filter program to run stderr results file through (per test)
+my $stderr_prefilter; # filter program to run strerr results file through (global)
my $prereq; # prerequisite test to satisfy before running test
my $posttest; # check command after running test
my $cleanup; # cleanup command to run
@@ -161,6 +163,9 @@
$valgrind = $1;
} elsif ($arg =~ /^--valgrind-lib=(.*)$/) {
$valgrind_lib = $1;
+ } elsif ($arg =~ /^--xml-tool=(.*)$/) {
+ $stderr_prefilter = validate_program("$tests_dir/auxprogs" , $1, 1, 1);
+ push(@vgextraopts,"--xml=yes");
} else {
die $usage;
}
@@ -308,6 +313,9 @@
# VALGRIND_LIB_INNER in case this Valgrind was configured with
# --enable-inner.
my $tool=determine_tool();
+ if ( $tool eq "memcheck" ) {
+ $vgopts .= " @vgextraopts";
+ }
mysystem("VALGRIND_LIB=$valgrind_lib VALGRIND_LIB_INNER=$valgrind_lib "
. "$valgrind --command-line-only=yes --memcheck:leak-check=no "
. "--tool=$tool $extraopts $vgopts "
@@ -323,6 +331,12 @@
@stdout_exps = ( "/dev/null" ) if (0 == scalar @stdout_exps);
do_diffs($fullname, $name, "stdout", \@stdout_exps);
+ # Run the pre-filter.
+ if ( $tool eq "memcheck" and defined $stderr_prefilter ) {
+ mysystem("$stderr_prefilter $name.stderr.out > $tmp");
+ rename($tmp, "$name.stderr.out");
+ }
+
# Filter stderr
mysystem("$stderr_filter < $name.stderr.out > $tmp");
rename($tmp, "$name.stderr.out");
|
