|
From: Nicholas N. <nj...@cs...> - 2005-02-11 14:49:36
|
On Fri, 11 Feb 2005, Tom Hughes wrote: > Some of the boxes that I run the tests on every night have the > debuginfo packages for glibc installed which means that some tests > fail because they produce output that names source files in the stack > trace instead of just libc-x.x.x.so or whatever. > > I was discussing this with Jeremy the other and he suggested adding > a --ignore-debuginfo=yes switch that the test suite could use to stop > valgrind looking at separated debug information. He also suggested > adding a --ignore-configfiles=yes switch to ignore .valgrindrc which > can also cause problems with the tests. > > What do people think? In one way these sound like good suggestions > and they would resolve real problems with the test suite, but I'm > always wary of anything which means that the tests use a different > code path to normal use... > > If people do think it's a good idea then I'll have a look at putting > it in. The other question is whether to have separate switches or a > single switch to enable "test mode" or something. The tests are too brittle now; making them more robust (but no less rigourous) would be great. But I'm uneasy about this suggestion. Stack traces are tricky. We shouldn't just ignore them, because they're an important part of the test -- we want them to fail if they eg. don't get the line number right. Basically, we want a certain level of fuzziness in the stack trace comparison, probably more than we have now. I would suggest reworking the filter_stderr_basic script to strip out more of the unimportant info from traces. But first, we should look at representative examples of test failures caused by insufficient fuzziness, so we can see exactly what we should strip out. Can Tom or Jeremy provide these? For the --ignore-config suggestion, it might be better to be more explicit about which options are used in the vg_regtest file. N |