|
From: Florian K. <br...@ac...> - 2011-10-21 21:44:18
|
Hello, I've begun writing a script that sucks valgrind-developer mail from news.gmane.org and analyzes those that appear to be regression test results. The data is written to a file that can be easily disected by perl or awk later on. For last night's runs we get this summary: Total: 0 platforms pass 12 platforms fail 37 failing testcases (multiple failures counted as 1) 73 failing testcases accumulated Per platform: Regrun: 2011-10-21 ppc openSUSE 11.1 18 failures Regrun: 2011-10-21 s390x RHEL 4 9 failures Regrun: 2011-10-21 x86_64 Fedora 16 9 failures Regrun: 2011-10-21 x86_64 openSUSE 11.4 8 failures Regrun: 2011-10-21 s390x UNKNOWN 5 failures Regrun: 2011-10-21 x86_64 Fedora 15 5 failures Regrun: 2011-10-21 x86_64 Fedora 14 5 failures Regrun: 2011-10-21 s390x SLES 11 SP1 4 failures Regrun: 2011-10-21 x86_64 openSUSE 12.1 4 failures Regrun: 2011-10-21 x86_64 Fedora 11 3 failures Regrun: 2011-10-21 x86_64 Fedora 13 2 failures Regrun: 2011-10-21 x86_64 Fedora 9 1 failures Per testcase (3rd column = how often it fails over all platforms): Test: 2011-10-21 5 memcheck/tests/origin5-bz2 Test: 2011-10-21 4 memcheck/tests/linux/stack_switch Test: 2011-10-21 4 helgrind/tests/tc20_verifywrap Test: 2011-10-21 4 helgrind/tests/tc18_semabuse Test: 2011-10-21 3 memcheck/tests/overlap Test: 2011-10-21 3 gdbserver_tests/mssnapshot Test: 2011-10-21 3 gdbserver_tests/mcvabits Test: 2011-10-21 3 gdbserver_tests/mcinfcallWSRU Test: 2011-10-21 3 drd/tests/tc09_bad_unlock Test: 2011-10-21 3 drd/tests/tc04_free_lock Test: 2011-10-21 3 drd/tests/annotate_barrier_xml Test: 2011-10-21 2 none/tests/amd64/sse4-64 Test: 2011-10-21 2 memcheck/tests/varinfo6 Test: 2011-10-21 2 helgrind/tests/cond_timedwait_invalid Test: 2011-10-21 2 gdbserver_tests/nlpasssigalrm Test: 2011-10-21 2 gdbserver_tests/nlcontrolc Test: 2011-10-21 2 gdbserver_tests/mcmain_pic Test: 2011-10-21 2 gdbserver_tests/mcleak Test: 2011-10-21 2 gdbserver_tests/mcclean_after_fork Test: 2011-10-21 2 gdbserver_tests/mcbreak Test: 2011-10-21 1 none/tests/ppc32/test_gx Test: 2011-10-21 1 none/tests/ppc32/test_fx Test: 2011-10-21 1 none/tests/ppc32/round Test: 2011-10-21 1 none/tests/ppc32/power5+_round Test: 2011-10-21 1 none/tests/ppc32/jm-fp Test: 2011-10-21 1 memcheck/tests/supp_unknown Test: 2011-10-21 1 memcheck/tests/partial_load_ok Test: 2011-10-21 1 memcheck/tests/manuel3 Test: 2011-10-21 1 memcheck/tests/long_namespace_xml Test: 2011-10-21 1 memcheck/tests/linux/stack_changes Test: 2011-10-21 1 massif/tests/overloaded-new Test: 2011-10-21 1 massif/tests/deep-D Test: 2011-10-21 1 massif/tests/big-alloc Test: 2011-10-21 1 helgrind/tests/tc23_bogus_condwait Test: 2011-10-21 1 helgrind/tests/tc09_bad_unlock Test: 2011-10-21 1 helgrind/tests/pth_barrier3 Test: 2011-10-21 1 helgrind/tests/hg05_race2 I took a quick look at individual testcases and noticed that gdbserver testcases fail due to some additional output on some platforms. Often extra blank lines. Perhaps this is a simple filtering issue. I'll look some more tomorrow. Florian |