|
From: Tom H. <th...@cy...> - 2008-12-26 03:49:34
|
Nightly build on vauxhall ( x86_64, Fedora 10 ) started at 2008-12-26 03:20:05 GMT Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 481 tests, 13 stderr failures, 0 stdout failures, 0 post failures == cachegrind/tests/chdir (stderr) cachegrind/tests/clreq (stderr) cachegrind/tests/dlclose (stderr) cachegrind/tests/wrap5 (stderr) cachegrind/tests/x86/fpu-28-108 (stderr) callgrind/tests/simwork1 (stderr) callgrind/tests/simwork2 (stderr) callgrind/tests/simwork3 (stderr) exp-ptrcheck/tests/base (stderr) exp-ptrcheck/tests/preen_invars (stderr) memcheck/tests/linux-syscalls-2007 (stderr) memcheck/tests/x86/scalar (stderr) none/tests/blockfault (stderr) |
|
From: Josef W. <Jos...@gm...> - 2008-12-29 16:24:46
|
On Friday 26 December 2008, Tom Hughes wrote: > > Nightly build on vauxhall ( x86_64, Fedora 10 ) started at 2008-12-26 03:20:05 GMT > Results unchanged from 24 hours ago > > Checking out valgrind source tree ... done > Configuring valgrind ... done > Building valgrind ... done > Running regression tests ... failed > > Regression test results follow > > == 481 tests, 13 stderr failures, 0 stdout failures, 0 post failures == > cachegrind/tests/chdir (stderr) > cachegrind/tests/clreq (stderr) > cachegrind/tests/dlclose (stderr) > cachegrind/tests/wrap5 (stderr) > cachegrind/tests/x86/fpu-28-108 (stderr) > callgrind/tests/simwork1 (stderr) > callgrind/tests/simwork2 (stderr) > callgrind/tests/simwork3 (stderr) Hmm... what does go wrong with these 8 tests? Is this something about the cpuid check? Josef > exp-ptrcheck/tests/base (stderr) > exp-ptrcheck/tests/preen_invars (stderr) > memcheck/tests/linux-syscalls-2007 (stderr) > memcheck/tests/x86/scalar (stderr) > none/tests/blockfault (stderr) > > > ------------------------------------------------------------------------------ > _______________________________________________ > Valgrind-developers mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-developers > |
|
From: Tom H. <to...@co...> - 2009-01-02 09:47:34
|
Josef Weidendorfer wrote: > Hmm... what does go wrong with these 8 tests? Is this something about > the cpuid check? Yep: warning: Unknown Intel cache config value (0x4e), ignoring warning: L2 cache not installed, ignore L2 results. The CPU is: Intel(R) Core(TM)2 Quad CPU Q9450 @ 2.66GHz It has a 6Mb cache, which is a problem I think as valgrind can only do powers of two can't it? Tom -- Tom Hughes (to...@co...) http://www.compton.nu/ |
|
From: Nicholas N. <nj...@cs...> - 2008-12-30 00:31:03
|
On Mon, 29 Dec 2008, Josef Weidendorfer wrote: >> == 481 tests, 13 stderr failures, 0 stdout failures, 0 post failures == >> cachegrind/tests/chdir (stderr) >> cachegrind/tests/clreq (stderr) >> cachegrind/tests/dlclose (stderr) >> cachegrind/tests/wrap5 (stderr) >> cachegrind/tests/x86/fpu-28-108 (stderr) >> callgrind/tests/simwork1 (stderr) >> callgrind/tests/simwork2 (stderr) >> callgrind/tests/simwork3 (stderr) > > Hmm... what does go wrong with these 8 tests? Is this something about > the cpuid check? I'm thinking about trying to improve the regtests, they would be much more useful if more reliable and less noisy. One thing I want to do is work out a good way to attach the diffs from the failing tests so that people can investigate problems without having to ask the usual "what went wrong with this test?" Because the diffs can be quite large, I'd probably do something like take the first N lines (for N=100, or N=1000, say) for each .diff file, then tar and bzip them, and attach them to the email. I just have to work out how to do attachments without requiring external programs... Nick |
|
From: Bart V. A. <bar...@gm...> - 2008-12-30 11:05:46
|
On Tue, Dec 30, 2008 at 1:30 AM, Nicholas Nethercote <nj...@cs...> wrote: > I'm thinking about trying to improve the regtests, they would be much more > useful if more reliable and less noisy. > > One thing I want to do is work out a good way to attach the diffs from the > failing tests so that people can investigate problems without having to ask > the usual "what went wrong with this test?" Because the diffs can be quite > large, I'd probably do something like take the first N lines (for N=100, or > N=1000, say) for each .diff file, then tar and bzip them, and attach them to > the email. I just have to work out how to do attachments without requiring > external programs... Being able to look at the diff output would be a great help. One possible approach is to include the output of head -n 100 */tests/*diff* in the e-mail sent by the nightly/bin/nightly script. Bart. |
|
From: Josef W. <Jos...@gm...> - 2008-12-30 11:07:15
|
> > Hmm... what does go wrong with these 8 tests? Is this something about > > the cpuid check? > > I'm thinking about trying to improve the regtests, they would be much more > useful if more reliable and less noisy. Yes. For the profiling tools, I just do not know about the best way. For callgrind, it would be very good to just check for the call graph detected in a given short program, but that very much depends on the compiler and the runtime system. > One thing I want to do is work out a good way to attach the diffs from the > failing tests so that people can investigate problems without having to ask > the usual "what went wrong with this test?" Because the diffs can be quite > large, I'd probably do something like take the first N lines (for N=100, or > N=1000, say) for each .diff file, then tar and bzip them, and attach them to > the email. I just have to work out how to do attachments without requiring > external programs... An alternative to attachments would be, if the script copies the results of the last regression failures to some web space, and provide a link in the mail. Josef |
|
From: Julian S. <js...@ac...> - 2008-12-30 09:46:04
|
On Tuesday 30 December 2008 00:30:48 Nicholas Nethercote wrote: > I'm thinking about trying to improve the regtests, they would be much more > useful if more reliable and less noisy. Yes. That would be great. > One thing I want to do is work out a good way to attach the diffs from the > failing tests so that people can investigate problems without having to ask > the usual "what went wrong with this test?" That would also be very helpful. I've noticed that the most common reason for regtests to fail on different distros is due to trivial differences in stack tracebacks, particularly the addition/loss of intermediate frames due to differences in inlining. So the test appears to fail for a completely spurious reason, when in fact the functionality it was trying to test actually works correctly. What would help greatly is a more flexible way of specifying the acceptable (expected) tracebacks. Something exactly like the current suppression syntax, allowing frame-level wildcards and function/object-name-level-wildcards would be ideal. So I guess that would mean writing a custom C program to do the check, rather than just using diff. J |
|
From: Bart V. A. <bar...@gm...> - 2008-12-30 10:55:32
|
On Tue, Dec 30, 2008 at 10:18 AM, Julian Seward <js...@ac...> wrote: > I've noticed that the most common reason for regtests to fail on different > distros is due to trivial differences in stack tracebacks, particularly the > addition/loss of intermediate frames due to differences in inlining. So the > test appears to fail for a completely spurious reason, when in fact the > functionality it was trying to test actually works correctly. > > What would help greatly is a more flexible way of specifying the acceptable > (expected) tracebacks. Something exactly like the current suppression syntax, > allowing frame-level wildcards and function/object-name-level-wildcards would > be ideal. So I guess that would mean writing a custom C program to do the > check, rather than just using diff. Maybe it's a good idea to extend syntax of the .vgtest files with a keyword that allows to specify a custom diff program instead of /usr/bin/diff -- not all regression tests produce output that contains stack traces. Bart. |
|
From: Julian S. <js...@ac...> - 2008-12-31 09:17:44
|
> What would help greatly is a more flexible way of specifying the acceptable > (expected) tracebacks. Something exactly like the current suppression > syntax, allowing frame-level wildcards and > function/object-name-level-wildcards would be ideal. So I guess that would > mean writing a custom C program to do the check, rather than just using > diff. Possible it would be useful to have a format for the expected output files that has more structure than at present: * sections that must match literally, and * sections which are stack traces, and therefore are written using the .supp wildcard syntax in 3.4.0 J |