|
From: J. B. <jb...@gm...> - 2008-10-19 22:05:44
|
Hi Valgrind-Developers, I've been working quite a bit on the Valgrind port for ARM, and I am at the stage where I pass most of the none/memcheck/massif/cache/call regression tests. Is there any ongoing effort to try and update the test cases? I know there is a request about this at the homepage, but I haven't seen if anyone is actively working on it. Some are failing for plain silly reasons (cmdline2, a new commandline option was added to valgrind, but testcase was not updated) Others are a bit more tricky to know if they are correct, especially since the output for all "supported" platforms are not available. the TLS test for example, I'm seeing the output lines in the "wrong" order, which may or may not be a real problem. Other examples would be the memcheck tests, on ARM/linux I sometime get a better stacktrace then the .exp files (exp file for badjump* reports lowest frame as "below main", whereas ARM/linux reports the actual correct position. (- by 0x........: (below main) (in /...libc...) + by 0x........: main (badjump.c:16) ) As I don't have that many machines available, if you have a PPC32/64, or even an x86-64 or x86-32 I would love to receive the the regression tests results for that machine, so I can make appropriate patches. Feel free to either send me the .out/.diff files by email, or send a link to the list. Thanks in advance /Johan Björk |
|
From: Bart V. A. <bar...@gm...> - 2008-10-20 18:10:18
|
On Mon, Oct 20, 2008 at 12:05 AM, Johan Björk <jb...@gm...> wrote: > I've been working quite a bit on the Valgrind port for ARM, and I am > at the stage where I pass most of the none/memcheck/massif/cache/call > regression tests. > > Is there any ongoing effort to try and update the test cases? I know > there is a request about this at the homepage, but I haven't seen if > anyone is actively working on it. There are indeed many false positives reported when running the Valgrind regression tests, especially on non-x86 platforms. I learned the following by reducing the number of false positives triggered by the DRD regression tests: * Instead of adding expected output variants, it is a lot better to modify the test itself and/or the output filter such that the output becomes platform independent. * Most of the times it is possible with a moderate effort to make a test platform independent. Sometimes this takes a significant effort. Bart. |
|
From: Julian S. <js...@ac...> - 2008-10-20 23:03:14
|
> As I don't have that many machines available, if you have a PPC32/64, > or even an x86-64 or x86-32 I would love to receive the the regression > tests results for that machine, It's essentially impossible to make all the tests pass on any specific platform. Some tests require functionality which doesn't work so well on platforms different from x86/x86_64: * recognition/handling of atomic instructions and atomic instruction sequences * tracking function entry/exit (I think callgrind has difficulties on ppc). exp-ptrcheck too. * relating stack addresses to source level variables (memcheck/tests/varinfo*) Additionally the entire test suite is very sensitive to changes in glibc, since many of the backtraces it produces go into glibc. It is also sensitive to different gcc versions, since different gccs make different inlining decisions (particularly in glibc), and so that also affects the backtraces. I wish we had a test system that was more robust, but it's not obvious how to do that. I suggest you make tests work in the following order: none memcheck cachegrind massif and then maybe drd,callgrind The results below are what I get for a run on suse 11.0 (amd64). What this means is, for all the 469-11-3 tests that didn't fail, the results should match the .stdout.exp and at least one of the .stderr.exp* files, for each test. J == 469 tests, 11 stderr failures, 3 stdout failures, 1 post failure == memcheck/tests/file_locking (stderr) memcheck/tests/fprw (stderr) memcheck/tests/malloc_free_fill (stderr) memcheck/tests/origin5-bz2 (stderr) memcheck/tests/pointer-trace (stderr) memcheck/tests/vcpu_fnfns (stdout) memcheck/tests/x86/scalar (stderr) memcheck/tests/x86/scalar_exit_group (stderr) memcheck/tests/x86/scalar_supp (stderr) massif/tests/overloaded-new (post) none/tests/cmdline2 (stdout) none/tests/mremap2 (stdout) none/tests/shell (stderr) helgrind/tests/tc18_semabuse (stderr) helgrind/tests/tc20_verifywrap (stderr) |
|
From: Bart V. A. <bar...@gm...> - 2008-10-25 08:12:54
|
On Mon, Oct 20, 2008 at 12:05 AM, Johan Björk <jb...@gm...> wrote: > Is there any ongoing effort to try and update the test cases? I know > there is a request about this at the homepage, but I haven't seen if > anyone is actively working on it. All DRD tests pass on i386 and on x86_64 systems, and all but one (pth_create_chain) pass on PPC. so I'm interested to hear about any DRD regression test that does not pass on ARM. Note: the nightly build output shows that two out of the 66 tests sometimes fail, but I have not yet been able to reproduce this behavior on any of my (virtual) machines. Bart. |
|
From: J. B. <jb...@gm...> - 2008-10-26 07:34:37
|
Hi Bart, Thanks for the information. I actually haven't yet gotten around to look at the testcases for helgrind and drd. I know it fails pretty much all of them currently, but I suspect many are because of not having correct suppression files. I'll let you know the results once I start digging into drd. /Johan On Sat, Oct 25, 2008 at 1:12 AM, Bart Van Assche <bar...@gm...> wrote: > On Mon, Oct 20, 2008 at 12:05 AM, Johan Björk <jb...@gm...> wrote: >> Is there any ongoing effort to try and update the test cases? I know >> there is a request about this at the homepage, but I haven't seen if >> anyone is actively working on it. > > All DRD tests pass on i386 and on x86_64 systems, and all but one > (pth_create_chain) pass on PPC. so I'm interested to hear about any > DRD regression test that does not pass on ARM. Note: the nightly build > output shows that two out of the 66 tests sometimes fail, but I have > not yet been able to reproduce this behavior on any of my (virtual) > machines. > > Bart. > |
|
From: Bart V. A. <bar...@gm...> - 2008-10-27 06:01:11
|
On Sun, Oct 26, 2008 at 8:34 AM, Johan Björk <jb...@gm...> wrote: > Thanks for the information. I actually haven't yet gotten around to > look at the testcases for helgrind and drd. I know it fails pretty > much all of them currently, but I suspect many are because of not > having correct suppression files. > I'll let you know the results once I start digging into drd. When I ported DRD from x86 to ppc, I only had to add a limited number of suppression patterns to DRD's suppression file. In this context it's important to know that DRD does not only rely on the suppression file to ensure false positives are not reported to the user. For executables in the ELF format DRD also ignores all accesses to .plt and .got.plt sections because resolving addresses of shared library symbols causes conflicting accesses in these sections. If DRD reports a lot of false positives on ARM, this might indicate an issue regarding shared library address resolution. More information about how shared library loading works on Linux can be found here: http://people.redhat.com/drepper/dsohowto.pdf. Bart. |