|
From: Philippe W. <phi...@sk...> - 2013-09-08 19:09:51
|
Every night, Valgrind is compiled and regression tested on a bunch of platform. However, there is no continuous tracking of the performance. => it has been suggested to add in the nightly script a run of the performance tests. This mail discusses how we could set that up (in particular to decide how much additional cpu these perf tests can take). On a slow computer (pentium 4): time perl perf/vg_perf --tools=none,memcheck,cachegrind,callgrind,helgrind,drd,exp-sgcheck,exp-dhat perf takes 32 minutes. On a faster computers such as gcc20, and on gcc110 (but without exp-sgcheck, crashing), it takes about 10 minutes. See details below. If we want more reproducable nrs, we have to give a --reps=... arg. Then to have a comparison with previous day, it further doubles the nr of run. So if we use e.g. --reps=3, this would mean about 3 hours of cpu on a slow computer, and 1 hour on faster computers. So, a few questions: 1. how much --reps ? Is 3 ok ? 2. do we run perf for all tools ? or only for non-experiment tools ? or for even less tools (only none and memcheck ?) 3. I think we should we output the perf nrs in the short log (sent by mail) (after the reg tests results). Unclear how to make a shorter summary of these nrs. 4. How to decrease the variations ? Idea is to use taskset to force all these tests to run on a single cpu, to avoid cpu frequency scaling and process migration to badly interact. 5. Any other idea/suggestion/... ? Philippe Slow computer (pentium 4) -- Running tests in perf ---------------------------------------------- bigcode1 trunk_untouched:0.19s no: 3.9s (20.3x, -----) me: 7.0s (36.8x, -----) ca:22.1s (116.2x, -----) ca:37.5s (197.4x, -----) he: 4.8s (25.2x, -----) dr: 4.7s (24.7x, -----) ex:13.2s (69.6x, -----) ex: 6.3s (33.3x, -----) bigcode2 trunk_untouched:0.20s no: 9.2s (46.1x, -----) me:18.7s (93.5x, -----) ca:31.6s (158.2x, -----) ca:62.8s (313.9x, -----) he:12.7s (63.7x, -----) dr:11.8s (58.9x, -----) ex:21.8s (108.8x, -----) ex:13.7s (68.5x, -----) bz2 trunk_untouched:1.25s no: 5.1s ( 4.1x, -----) me:17.8s (14.2x, -----) ca:77.9s (62.3x, -----) ca:51.5s (41.2x, -----) he:38.1s (30.5x, -----) dr:34.7s (27.8x, -----) ex:110.2s (88.1x, -----) ex:23.0s (18.4x, -----) fbench trunk_untouched:0.66s no: 6.0s ( 9.0x, -----) me:13.2s (19.9x, -----) ca:27.6s (41.8x, -----) ca:21.0s (31.8x, -----) he:14.9s (22.6x, -----) dr:11.2s (17.0x, -----) ex:48.0s (72.7x, -----) ex:10.3s (15.7x, -----) ffbench trunk_untouched:2.00s no: 5.6s ( 2.8x, -----) me: 9.8s ( 4.9x, -----) ca:27.3s (13.7x, -----) ca: 8.6s ( 4.3x, -----) he:33.7s (16.9x, -----) dr:13.8s ( 6.9x, -----) ex:38.4s (19.2x, -----) ex: 9.6s ( 4.8x, -----) heap trunk_untouched:0.23s no: 2.3s (10.0x, -----) me:12.8s (55.8x, -----) ca:25.2s (109.7x, -----) ca:24.7s (107.3x, -----) he:31.3s (136.1x, -----) dr:12.7s (55.1x, -----) ex:54.9s (238.5x, -----) ex:31.6s (137.3x, -----) heap_pdb4 trunk_untouched:0.27s no: 2.5s ( 9.2x, -----) me:23.9s (88.6x, -----) ca:27.4s (101.5x, -----) ca:26.6s (98.5x, -----) he:34.9s (129.4x, -----) dr:14.0s (51.9x, -----) ex:55.8s (206.7x, -----) ex:36.4s (134.9x, -----) many-loss-records trunk_untouched:0.03s no: 0.8s (25.0x, -----) me: 3.2s (106.0x, -----) ca: 4.3s (142.7x, -----) ca: 3.5s (116.3x, -----) he: 4.6s (153.0x, -----) dr: 2.5s (83.0x, -----) ex:13.7s (456.3x, -----) ex: 5.2s (171.7x, -----) many-xpts trunk_untouched:0.09s no: 1.1s (11.7x, -----) me: 4.9s (54.3x, -----) ca:11.1s (122.9x, -----) ca: 9.6s (106.7x, -----) he:11.2s (124.7x, -----) dr: 4.5s (50.6x, -----) ex:23.9s (265.6x, -----) ex: 4.8s (52.9x, -----) sarp trunk_untouched:0.06s no: 0.9s (14.8x, -----) me: 6.2s (103.8x, -----) ca: 5.8s (96.8x, -----) ca: 6.0s (100.7x, -----) he:30.0s (500.2x, -----) dr: 2.9s (48.8x, -----) ex:24.0s (400.2x, -----) ex: 1.3s (21.0x, -----) tinycc trunk_untouched:0.43s no: 3.7s ( 8.6x, -----) me:26.5s (61.6x, -----) ca:56.8s (132.2x, -----) ca:35.8s (83.1x, -----) he:43.7s (101.7x, -----) dr:28.0s (65.1x, -----) ex:101.6s (236.2x, -----) ex:31.2s (72.5x, -----) -- Finished tests in perf gcc110 - Running tests in perf ---------------------------------------------- bigcode1 trunk_untouched:0.23s no: 1.9s ( 8.3x, -----) me: 3.1s (13.6x, -----) ca: 6.3s (27.5x, -----) ca:19.1s (83.0x, -----) he: 1.9s ( 8.3x, -----) dr: 1.8s ( 7.6x, -----) ex: 2.4s (10.2x, -----) bigcode2 trunk_untouched:0.23s no: 1.6s ( 7.2x, -----) me: 3.0s (13.1x, -----) ca: 5.7s (24.8x, -----) ca:18.5s (80.3x, -----) he: 2.1s ( 9.0x, -----) dr: 1.7s ( 7.3x, -----) ex: 2.3s ( 9.9x, -----) bz2 trunk_untouched:0.73s no: 4.5s ( 6.2x, -----) me:12.0s (16.5x, -----) ca:26.4s (36.1x, -----) ca:26.8s (36.6x, -----) he:14.8s (20.2x, -----) dr:20.5s (28.1x, -----) ex:13.1s (17.9x, -----) fbench trunk_untouched:0.34s no: 2.2s ( 6.5x, -----) me: 5.3s (15.7x, -----) ca: 8.2s (24.0x, -----) ca: 8.8s (25.7x, -----) he: 5.4s (16.0x, -----) dr: 5.1s (14.9x, -----) ex: 4.8s (14.0x, -----) ffbench trunk_untouched:0.45s no: 1.1s ( 2.4x, -----) me: 2.8s ( 6.2x, -----) ca: 7.7s (17.0x, -----) ca: 2.6s ( 5.8x, -----) he: 7.2s (16.0x, -----) dr: 5.1s (11.4x, -----) ex: 3.4s ( 7.6x, -----) heap trunk_untouched:0.41s no: 2.4s ( 5.8x, -----) me: 9.9s (24.1x, -----) ca:12.9s (31.5x, -----) ca:13.8s (33.8x, -----) he:13.3s (32.6x, -----) dr: 8.1s (19.7x, -----) ex:17.1s (41.8x, -----) heap_pdb4 trunk_untouched:0.42s no: 2.8s ( 6.6x, -----) me:14.3s (34.0x, -----) ca:13.4s (32.0x, -----) ca:14.7s (34.9x, -----) he:14.6s (34.9x, -----) dr: 9.0s (21.5x, -----) ex:20.4s (48.5x, -----) many-loss-records trunk_untouched:0.03s no: 0.6s (18.3x, -----) me: 2.2s (74.3x, -----) ca: 1.9s (65.0x, -----) ca: 1.9s (64.7x, -----) he: 2.1s (70.0x, -----) dr: 1.5s (49.7x, -----) ex: 3.3s (109.7x, -----) many-xpts trunk_untouched:0.07s no: 0.8s (11.1x, -----) me: 3.5s (49.3x, -----) ca: 3.0s (43.4x, -----) ca: 4.9s (69.6x, -----) he: 5.2s (74.7x, -----) dr: 2.3s (32.3x, -----) ex: 2.8s (39.9x, -----) sarp trunk_untouched:0.02s no: 0.4s (21.0x, -----) me: 3.9s (192.5x, -----) ca: 2.0s (101.0x, -----) ca: 3.0s (151.0x, -----) he:11.3s (565.5x, -----) dr: 1.0s (52.0x, -----) ex: 0.6s (31.5x, -----) tinycc trunk_untouched:0.28s no: 3.1s (11.0x, -----) me:15.9s (56.7x, -----) ca:16.5s (59.0x, -----) ca:17.3s (61.7x, -----) he:18.9s (67.6x, -----) dr:12.1s (43.0x, -----) ex:18.6s (66.4x, -----) -- Finished tests in perf ---------------------------------------------- == 11 programs, 77 timings ================= gcc20 -- Running tests in perf ---------------------------------------------- bigcode1 trunk_untouched:0.12s no: 1.9s (16.2x, -----) me: 3.7s (30.7x, -----) ca: 4.8s (39.8x, -----) ca:16.5s (137.8x, -----) he: 2.2s (18.4x, -----) dr: 2.1s (17.4x, -----) ex: 4.9s (40.9x, -----) ex: 2.4s (19.8x, -----) bigcode2 trunk_untouched:0.12s no: 4.6s (38.5x, -----) me: 9.5s (79.5x, -----) ca: 8.8s (73.3x, -----) ca:34.3s (285.6x, -----) he: 5.6s (46.8x, -----) dr: 5.4s (44.7x, -----) ex: 8.5s (70.8x, -----) ex: 6.0s (49.9x, -----) bz2 trunk_untouched:0.70s no: 2.2s ( 3.2x, -----) me: 7.0s (10.0x, -----) ca:13.0s (18.6x, -----) ca:21.1s (30.1x, -----) he: 9.2s (13.2x, -----) dr:14.0s (20.0x, -----) ex:24.9s (35.5x, -----) ex: 7.3s (10.5x, -----) fbench trunk_untouched:0.28s no: 1.3s ( 4.6x, -----) me: 4.0s (14.3x, -----) ca: 3.9s (13.9x, -----) ca: 6.3s (22.4x, -----) he: 3.4s (12.1x, -----) dr: 2.9s (10.2x, -----) ex:15.1s (53.9x, -----) ex: 2.8s ( 9.9x, -----) ffbench trunk_untouched:0.25s no: 1.2s ( 4.6x, -----) me: 3.3s (13.3x, -----) ca: 5.1s (20.3x, -----) ca: 2.4s ( 9.4x, -----) he: 5.4s (21.5x, -----) dr: 3.8s (15.3x, -----) ex: 9.7s (38.8x, -----) ex: 2.6s (10.4x, -----) heap trunk_untouched:0.10s no: 0.7s ( 7.4x, -----) me: 5.8s (58.1x, -----) ca: 4.3s (42.6x, -----) ca: 7.3s (73.4x, -----) he: 8.4s (83.6x, -----) dr: 5.0s (49.5x, -----) ex:17.5s (174.6x, -----) ex:10.6s (105.8x, -----) heap_pdb4 trunk_untouched:0.12s no: 0.8s ( 6.8x, -----) me: 9.8s (81.9x, -----) ca: 4.4s (36.3x, -----) ca: 7.8s (65.2x, -----) he:10.5s (87.3x, -----) dr: 5.4s (44.8x, -----) ex:16.1s (133.8x, -----) ex:10.9s (91.2x, -----) many-loss-records trunk_untouched:0.02s no: 0.3s (14.0x, -----) me: 1.4s (71.0x, -----) ca: 0.8s (39.0x, -----) ca: 1.1s (57.0x, -----) he: 1.3s (63.5x, -----) dr: 1.0s (50.0x, -----) ex: 4.7s (237.0x, -----) ex: 1.9s (93.0x, -----) many-xpts trunk_untouched:0.04s no: 0.3s ( 8.0x, -----) me: 2.0s (49.5x, -----) ca: 1.1s (27.8x, -----) ca: 3.0s (76.0x, -----) he: 2.5s (61.5x, -----) dr: 1.4s (35.2x, -----) ex: 7.6s (190.0x, -----) ex: 1.6s (40.5x, -----) sarp trunk_untouched:0.03s no: 0.3s ( 9.3x, -----) me: 2.4s (81.0x, -----) ca: 1.0s (34.0x, -----) ca: 2.0s (66.0x, -----) he: 7.4s (247.3x, -----) dr: 0.9s (29.0x, -----) ex: 6.7s (222.7x, -----) ex: 0.4s (14.7x, -----) tinycc trunk_untouched:0.26s no: 1.6s ( 6.0x, -----) me:10.2s (39.3x, -----) ca: 9.2s (35.3x, -----) ca:12.6s (48.3x, -----) he:11.1s (42.8x, -----) dr: 8.9s (34.3x, -----) ex:38.5s (147.9x, -----) ex:12.1s (46.4x, -----) -- Finished tests in perf ---------------------------------------------- == 11 programs, 88 timings ================= |
|
From: Florian K. <fl...@ei...> - 2013-09-08 21:35:06
|
On 09/08/2013 09:09 PM, Philippe Waroquiers wrote: > Every night, Valgrind is compiled and regression tested on a bunch of > platform. However, there is no continuous tracking of the performance. > => it has been suggested to add in the nightly script a run of > the performance tests. > This mail discusses how we could set that up (in particular to decide > how much additional cpu these perf tests can take). > > On a slow computer (pentium 4): > time perl perf/vg_perf --tools=none,memcheck,cachegrind,callgrind,helgrind,drd,exp-sgcheck,exp-dhat perf > takes 32 minutes. > On a faster computers such as gcc20, and on gcc110 (but without exp-sgcheck, crashing), > it takes about 10 minutes. > See details below. > > If we want more reproducable nrs, we have to give a --reps=... arg. > Then to have a comparison with previous day, it further doubles the nr of run. > So if we use e.g. --reps=3, this would mean about 3 hours of cpu > on a slow computer, and 1 hour on faster computers. The problem with this approach is that you don't notice performance degradation that creeps in on you. Say 1% a day for several days in a row. 1% degradation would not get any attention but when it accumulates over time, it should. What if we run perf every day and send the results to e.g pe...@va...? Whenever mail is received at that address a little script runs that reads the perf results and collects them in some light weight "data base" that we could look at at valgrind.org/perf. It could be as simple as a HTML table (per platform) with one row per day that shows, for each tool, the difference to the previous run and the difference to some base line run. Yes, this is more work to set up than just sending the results to the developers list. But the results might get more attention that way. > So, a few questions: > 1. how much --reps ? Is 3 ok ? I typically use --reps=5 but if I had a better handle on variations or a better tool to measure runtime than "time" less repetition mught be possible. > 2. do we run perf for all tools ? > or only for non-experiment tools ? > or for even less tools (only none and memcheck ?) Non-experimental tools should be enough. Florian |
|
From: Philippe W. <phi...@sk...> - 2013-09-09 21:32:22
|
On Sun, 2013-09-08 at 23:34 +0200, Florian Krohm wrote: > The problem with this approach is that you don't notice performance > degradation that creeps in on you. Say 1% a day for several days in a > row. 1% degradation would not get any attention but when it accumulates > over time, it should. > What if we run perf every day and send the results to e.g > pe...@va...? Whenever mail is received at that address a little > script runs that reads the perf results and collects them in some light > weight "data base" that we could look at at valgrind.org/perf. It could > be as simple as a HTML table (per platform) with one row per day that > shows, for each tool, the difference to the previous run and the > difference to some base line run. > Yes, this is more work to set up than just sending the results to the > developers list. But the results might get more attention that way. Yes, effectively, gradual small performance degradation is not easy to detect with a daily difference. So, maybe the valdev archive can be seen as the DB. I think you started to write a script to handle archived test results and make an html page from it. A colleague has started enhancing this script (e.g. taking some other ideas from what wine does for their night tests results). So, handling the perf results is another enhancement to do. (that is a hint for my colleague :). > > > > So, a few questions: > > 1. how much --reps ? Is 3 ok ? > > I typically use --reps=5 but if I had a better handle on variations or a > better tool to measure runtime than "time" less repetition mught be > possible. > > > 2. do we run perf for all tools ? > > or only for non-experiment tools ? > > or for even less tools (only none and memcheck ?) > > Non-experimental tools should be enough. Skipping the exp tools does not change a lot the time it takes. So, --reps=3 => 1 hour on a fast computer, 3 hours on a slow pentium. If we take --reps=5, this is respectively 1h40 and 5 hours. Maybe we could have the "conf" allowing to specify: to run perf test y/n the --reps= value with comparison with the previous day y/n Just having the first 2 configured allows the "valdev" scanner to produce the evolution of performance, the 3rd allows to quickly see a daily perf difference Philippe |
|
From: Philippe W. <phi...@sk...> - 2013-09-10 19:44:58
|
On Tue, 2013-09-10 at 16:37 +0200, Florian Krohm wrote: > On 09/09/2013 11:32 PM, Philippe Waroquiers wrote: > > On Sun, 2013-09-08 at 23:34 +0200, Florian Krohm wrote: > > >> What if we run perf every day and send the results to e.g > >> pe...@va...? Whenever mail is received at that address a little > >> script runs that reads the perf results and collects them in some light > >> weight "data base" that we could look at at valgrind.org/perf. > > > So, maybe the valdev archive can be seen as the DB. > > I think you started to write a script to handle archived test results > > and make an html page from it. > > We could use the mailing list archive as the "data base". But accessing > it is slow. I ran that script you mentioned and it took minutes to > process 100 mails. Which means that you probably don't want to run it > yourself to figure out whether last night's build regressed performancewise. > Also, accessing the mailing list archive is "by number". E.g. mail > #20000 is a certain build that happened on Sep 8, 2012. But you don't > know that relation. So if you want to know how performance evolved since > a particular date, you either need to do some kind of bisection to > figure out the corresponding email number forat that date or you need to > store this information somewhere. > I guess a nightly cronjob on valgrind.org should do this: > (1) process the valdev emails that came in today (which implies that we > keep the largest email number that was processed the previous night) > (2) extract the performance data and write it somewhere for later query > (3) create some web page to visualise the performance data (it will use > the data collected in #2) > > > A colleague has started enhancing > > this script (e.g. taking some other ideas from what wine does for > > their night tests results). > > Let's see it :) Some significant work was done some months ago (the script is now about 1200 lines), including I think performance optimisation as it maintains a local directory with the messages, and produces an html page. It might not be very easy to summarise the performance (ie. many tools/many tests for many archs) > > > > > Maybe we could have the "conf" allowing to specify: > > to run perf test y/n > > the --reps= value > > with comparison with the previous day y/n > > That would be sufficient to do what we envision. Ok, will look at this to at least have the nrs started to be captured. Script and web page will follow :). Philippe |
|
From: marc b. <mar...@gm...> - 2013-09-10 21:29:07
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Valgrind test runs</title>
<link rel="stylesheet" href="../summary.css" type="text/css">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div class="navbar">
<a href="all_tests.index.html">all tests</a>
</div>
<div class="main">
<h2>Valgrind test runs</h2>
<table class="report"><thead><tr>
<th class="test">Revision Valgrind.Vex</th>
<th class="test">Valgrind commit date</th>
<th class="test">Vex commit date</th>
<th class="test"><a href="machine.bristolx8664Fedora11.html">bristol ( x86_64, Fedora 11 )</a></th><th class="test"><a href="machine.bristolx8664Fedora13.html">bristol ( x86_64, Fedora 13 )</a></th><th class="test"><a href="machine.bristolx8664Fedora14.html">bristol ( x86_64, Fedora 14 )</a></th><th class="test"><a href="machine.bristolx8664Fedora15.html">bristol ( x86_64, Fedora 15 )</a></th><th class="test"><a href="machine.bristolx8664Fedora16.html">bristol ( x86_64, Fedora 16 )</a></th><th class="test"><a href="machine.bristolx8664Fedora17BeefyMiracle.html">bristol ( x86_64, Fedora 17 (Beefy Miracle) )</a></th><th class="test"><a href="machine.bristolx8664Fedora18SphericalCow.html">bristol ( x86_64, Fedora 18 (Spherical Cow) )</a></th><th class="test"><a href="machine.bristolx8664Fedora19SchrdingersCat.html">bristol ( x86_64, Fedora 19 (Schrödingerâs Cat) )</a></th><th class="test"><a href="machine.bristolx8664Fedora21.html">bristol ( x86_64, Fedora 21 )</a></th><th class="test"><a href="machine.bristolx8664Fedora9.html">bristol ( x86_64, Fedora 9 )</a></th><th class="test"><a href="machine.fedoras390Fedora17withgcc470onz196s390x.html">fedoras390 ( Fedora 17 with gcc 4.7.0 on z196 (s390x) )</a></th><th class="test"><a href="machine.gcc110Fedorarelease18SphericalCowppc64.html">gcc110 ( Fedora release 18 (Spherical Cow), ppc64 )</a></th><th class="test"><a href="machine.macamd64Darwin1080i386.html">macamd64 ( Darwin 10.8.0 i386 )</a></th><th class="test"><a href="machine.rhels390RHEL64withgcc447onzEC12s390x.html">rhels390 ( RHEL 6.4 with gcc 4.4.7 on zEC12 (s390x) )</a></th><th class="test"><a href="machine.sless390SUSELinuxEnterpriseServer11SP1gcc434onz196s390x.html">sless390 ( SUSE Linux Enterprise Server 11 SP1 gcc 4.3.4 on z196 (s390x) )</a></th><th class="test"><a href="machine.ultragcc451Linux37911desktopx8664.html">ultra ( gcc 4.5.1 Linux 3.7.9-1.1-desktop x86_64 )</a></th> </tr>
<tr><td class="result more_failures"><a href="revision.13537.2755.html">13537.2755</a></td><td class="result more_failures">2013-09-06</td><td></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24252"/>644,2,0</a></td><td class="result more_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24251"/>644,3,2</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24250"/>663,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24248"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24247"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24246"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24245"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24242"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24241"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24253"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24243"/>639,4,0</a></td><td class="result same_failures"><a title="563 tests run,nb failures,delta failure with previous revision" href="../24254"/>563,40,0</a></td><td class="note"></td><td class="note"></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24244"/>639,0,0</a></td><td class="result same_failures"><a title="658 tests run,nb failures,delta failure with previous revision" href="../24249"/>658,2,0</a></td></tr>
<tr><td class="result more_failures"><a href="revision.13535.2751.html">13535.2751</a></td><td class="result more_failures"><a href=../24209>2013-09-05</a></td><td class="result more_failures"><a href=../24206>2013-09-05</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24224"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24223"/>644,1,0</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24222"/>663,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24220"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24219"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24218"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24216"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24214"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24212"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24225"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24215"/>639,4,0</a></td><td class="result more_failures"><a title="563 tests run,nb failures,delta failure with previous revision" href="../24226"/>563,40,4</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24213"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24217"/>639,0,0</a></td><td class="result same_failures"><a title="658 tests run,nb failures,delta failure with previous revision" href="../24221"/>658,2,0</a></td></tr>
<tr><td class="result not_run"><a href="revision.13532.2750.html">13532.2750</a></td><td class="result not_run"><a href=../24227>2013-09-06</a></td><td class="result not_run"><a href=../24175>2013-09-03</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24201"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24200"/>644,1,0</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24199"/>663,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24197"/>665,1,0</a></td><td class="result less_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24196"/>665,1,-2</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24195"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24194"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24191"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24189"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24202"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24192"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24203"/>561,36,0</a></td><td class="note"></td><td class="result not_run"><a title="failure building valgrind" href="../24190"/>no test</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24193"/>639,0,0</a></td><td class="result same_failures"><a title="658 tests run,nb failures,delta failure with previous revision" href="../24198"/>658,2,0</a></td></tr>
<tr><td class="result same_failures"><a href="revision.13531.2750.html">13531.2750</a></td><td class="result same_failures"><a href=../24182>2013-09-04</a></td><td class="result same_failures"><a href=../24175>2013-09-03</a></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="result same_failures"><a title="658 tests run,nb failures,delta failure with previous revision" href="../24184"/>658,2,0</a></td></tr>
<tr><td class="result not_run"><a href="revision.13528.2750.html">13528.2750</a></td><td class="result not_run"><a href=../24170>2013-09-03</a></td><td class="result not_run"><a href=../24175>2013-09-03</a></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24177"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24179"/>561,36,0</a></td><td class="note"></td><td class="result not_run"><a title="failure building valgrind" href="../24176"/>no test</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24178"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13527.2749.html">13527.2749</a></td><td class="result more_failures"><a href=../24167>2013-09-03</a></td><td class="result more_failures"><a href=../24152>2013-09-02</a></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="note"></td><td class="result more_failures"><a title="658 tests run,nb failures,delta failure with previous revision" href="../24168"/>658,2,2</a></td></tr>
<tr><td class="result more_failures"><a href="revision.13526.2749.html">13526.2749</a></td><td class="result more_failures"><a href=../24151>2013-09-02</a></td><td class="result more_failures"><a href=../24152>2013-09-02</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24162"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24164"/>644,1,0</a></td><td class="result less_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24166"/>663,1,-2</a></td><td class="result less_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24165"/>665,1,-1</a></td><td class="result more_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24161"/>665,3,2</a></td><td class="result less_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24159"/>665,6,-1</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24157"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24155"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24153"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24163"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24156"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24160"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24154"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24158"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13520.2747.html">13520.2747</a></td><td class="result more_failures"><a href=../24114>2013-08-31</a></td><td class="result more_failures"><a href=../24130>2013-09-01</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24139"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24141"/>644,1,0</a></td><td class="result more_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24143"/>663,3,1</a></td><td class="result more_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24142"/>665,2,1</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24144"/>665,1,0</a></td><td class="result more_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24138"/>665,7,1</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24136"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24133"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24131"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24140"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24134"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24137"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24132"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24135"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13520.2745.html">13520.2745</a></td><td class="result more_failures"><a href=../24114>2013-08-31</a></td><td class="result more_failures"><a href=../24050>2013-08-27</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24126"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24125"/>644,1,0</a></td><td class="result more_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24124"/>663,2,1</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24123"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24122"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24121"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24120"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24117"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24115"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24127"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24118"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24128"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24116"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24119"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result less_failures"><a href="revision.13519.2745.html">13519.2745</a></td><td class="result less_failures"><a href=../24099>2013-08-30</a></td><td class="result less_failures"><a href=../24050>2013-08-27</a></td><td class="result less_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24111"/>644,2,-1</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24110"/>644,1,0</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24109"/>663,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24108"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24107"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24106"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24104"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24102"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24100"/>665,7,0</a></td><td class="result less_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24112"/>642,2,-2</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24103"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24113"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24101"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24105"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13518.2745.html">13518.2745</a></td><td class="result more_failures"><a href=../24083>2013-08-29</a></td><td class="result more_failures"><a href=../24050>2013-08-27</a></td><td class="result more_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24093"/>644,3,1</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24095"/>644,1,0</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24096"/>663,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24097"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24098"/>665,1,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24092"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24090"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24089"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24085"/>665,7,0</a></td><td class="result more_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24094"/>642,4,2</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24087"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24091"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24086"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24088"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result less_failures"><a href="revision.13517.2745.html">13517.2745</a></td><td class="result less_failures"><a href=../24067>2013-08-28</a></td><td class="result less_failures"><a href=../24050>2013-08-27</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24079"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24078"/>644,1,0</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24077"/>663,1,0</a></td><td class="result less_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24076"/>665,1,-1</a></td><td class="result less_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24075"/>665,1,-1</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24074"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24072"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24070"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24068"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24080"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24071"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24081"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24069"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24073"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13516.2745.html">13516.2745</a></td><td class="result more_failures"><a href=../24052>2013-08-27</a></td><td class="result more_failures"><a href=../24050>2013-08-27</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24064"/>644,2,0</a></td><td class="result same_failures"><a title="644 tests run,nb failures,delta failure with previous revision" href="../24063"/>644,1,0</a></td><td class="result same_failures"><a title="663 tests run,nb failures,delta failure with previous revision" href="../24062"/>663,1,0</a></td><td class="result more_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24061"/>665,2,1</a></td><td class="result more_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24060"/>665,2,1</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24059"/>665,6,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24057"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24055"/>665,3,0</a></td><td class="result same_failures"><a title="665 tests run,nb failures,delta failure with previous revision" href="../24053"/>665,7,0</a></td><td class="result same_failures"><a title="642 tests run,nb failures,delta failure with previous revision" href="../24065"/>642,2,0</a></td><td class="result same_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24056"/>639,4,0</a></td><td class="result same_failures"><a title="561 tests run,nb failures,delta failure with previous revision" href="../24066"/>561,36,0</a></td><td class="note"></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24054"/>636,0,0</a></td><td class="result pass"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24058"/>639,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13513.2744.html">13513.2744</a></td><td class="result more_failures"><a href=../24033>2013-08-25</a></td><td></td><td class="result same_failures"><a title="640 tests run,nb failures,delta failure with previous revision" href="../24045"/>640,2,0</a></td><td class="result same_failures"><a title="640 tests run,nb failures,delta failure with previous revision" href="../24044"/>640,1,0</a></td><td class="result same_failures"><a title="659 tests run,nb failures,delta failure with previous revision" href="../24043"/>659,1,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24042"/>661,1,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24041"/>661,1,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24040"/>661,6,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24038"/>661,3,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24036"/>661,3,0</a></td><td class="result more_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24034"/>661,7,7</a></td><td class="result same_failures"><a title="638 tests run,nb failures,delta failure with previous revision" href="../24046"/>638,2,0</a></td><td class="result same_failures"><a title="637 tests run,nb failures,delta failure with previous revision" href="../24037"/>637,4,0</a></td><td class="result same_failures"><a title="559 tests run,nb failures,delta failure with previous revision" href="../24047"/>559,36,0</a></td><td class="note"></td><td class="result pass"><a title="634 tests run,nb failures,delta failure with previous revision" href="../24035"/>634,0,0</a></td><td class="result pass"><a title="637 tests run,nb failures,delta failure with previous revision" href="../24039"/>637,0,0</a></td><td class="note"></td></tr>
<tr><td class="result less_failures"><a href="revision.13512.2744.html">13512.2744</a></td><td class="result less_failures"><a href=../24019>2013-08-24</a></td><td></td><td class="result same_failures"><a title="640 tests run,nb failures,delta failure with previous revision" href="../24030"/>640,2,0</a></td><td class="result less_failures"><a title="640 tests run,nb failures,delta failure with previous revision" href="../24029"/>640,1,-2</a></td><td class="result same_failures"><a title="659 tests run,nb failures,delta failure with previous revision" href="../24028"/>659,1,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24027"/>661,1,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24026"/>661,1,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24025"/>661,6,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24024"/>661,3,0</a></td><td class="result same_failures"><a title="661 tests run,nb failures,delta failure with previous revision" href="../24021"/>661,3,0</a></td><td class="note"></td><td class="result same_failures"><a title="638 tests run,nb failures,delta failure with previous revision" href="../24031"/>638,2,0</a></td><td class="result same_failures"><a title="637 tests run,nb failures,delta failure with previous revision" href="../24022"/>637,4,0</a></td><td class="result same_failures"><a title="559 tests run,nb failures,delta failure with previous revision" href="../24032"/>559,36,0</a></td><td class="note"></td><td class="result pass"><a title="634 tests run,nb failures,delta failure with previous revision" href="../24020"/>634,0,0</a></td><td class="result pass"><a title="637 tests run,nb failures,delta failure with previous revision" href="../24023"/>637,0,0</a></td><td class="note"></td></tr>
<tr><td class="result more_failures"><a href="revision.13507.2744.html">13507.2744</a></td><td class="result more_failures">2013-08-22</td><td></td><td class="result more_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24009"/>639,2,2</a></td><td class="result more_failures"><a title="639 tests run,nb failures,delta failure with previous revision" href="../24010"/>639,3,3</a></td><td class="result more_failures"><a title="658 tests run,nb failures,delta failure with previous revision" href="../24008"/>658,1,1</a></td><td class="result more_failures"><a title="660 tests run,nb failures,delta failure with previous revision" href="../24007"/>660,1,1</a></td><td class="result more_failures"><a title="660 tests run,nb failures,delta failure with previous revision" href="../24006"/>660,1,1</a></td><td class="result more_failures"><a title="660 tests run,nb failures,delta failure with previous revision" href="../24005"/>660,6,6</a></td><td class="result more_failures"><a title="660 tests run,nb failures,delta failure with previous revision" href="../24003"/>660,3,3</a></td><td class="result more_failures"><a title="660 tests run,nb failures,delta failure with previous revision" href="../24001"/>660,3,3</a></td><td class="note"></td><td class="result more_failures"><a title="637 tests run,nb failures,delta failure with previous revision" href="../24011"/>637,2,2</a></td><td class="result more_failures"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24002"/>636,4,4</a></td><td class="result more_failures"><a title="558 tests run,nb failures,delta failure with previous revision" href="../24012"/>558,36,36</a></td><td class="result more_failures"><a title="566 tests run,nb failures,delta failure with previous revision" href="../24013"/>566,116,116</a></td><td class="result pass"><a title="633 tests run,nb failures,delta failure with previous revision" href="../24000"/>633,0,0</a></td><td class="result pass"><a title="636 tests run,nb failures,delta failure with previous revision" href="../24004"/>636,0,0</a></td><td class="note"></td></tr>
</table></div>
<div class="legend">
<h3>Legend</h3>
<p class="legend">All tests <span class="result pass">passed</span>.</p>
<p class="legend">The same tests as in previous report
<span class="result same_failures">failed</span>.</p>
<p class="legend">More tests than in the previous report
<span class="result more_failures">failed</span>.</p>
<p class="legend">Less tests than in the previous report
<span class="result less_failures">failed</span>.</p>
<p class="legend">No test run, build was
<span class="not_run">broken</span>.</p>
</div>
</body></html> |
|
From: Florian K. <fl...@ei...> - 2013-09-10 14:38:15
|
On 09/09/2013 11:32 PM, Philippe Waroquiers wrote: > On Sun, 2013-09-08 at 23:34 +0200, Florian Krohm wrote: >> What if we run perf every day and send the results to e.g >> pe...@va...? Whenever mail is received at that address a little >> script runs that reads the perf results and collects them in some light >> weight "data base" that we could look at at valgrind.org/perf. > So, maybe the valdev archive can be seen as the DB. > I think you started to write a script to handle archived test results > and make an html page from it. We could use the mailing list archive as the "data base". But accessing it is slow. I ran that script you mentioned and it took minutes to process 100 mails. Which means that you probably don't want to run it yourself to figure out whether last night's build regressed performancewise. Also, accessing the mailing list archive is "by number". E.g. mail #20000 is a certain build that happened on Sep 8, 2012. But you don't know that relation. So if you want to know how performance evolved since a particular date, you either need to do some kind of bisection to figure out the corresponding email number forat that date or you need to store this information somewhere. I guess a nightly cronjob on valgrind.org should do this: (1) process the valdev emails that came in today (which implies that we keep the largest email number that was processed the previous night) (2) extract the performance data and write it somewhere for later query (3) create some web page to visualise the performance data (it will use the data collected in #2) > A colleague has started enhancing > this script (e.g. taking some other ideas from what wine does for > their night tests results). Let's see it :) > > Maybe we could have the "conf" allowing to specify: > to run perf test y/n > the --reps= value > with comparison with the previous day y/n That would be sufficient to do what we envision. Florian |
|
From: Philippe W. <phi...@sk...> - 2013-09-11 20:15:06
|
On Wed, 2013-09-11 at 18:42 +0200, Florian Krohm wrote: > On 09/10/2013 11:28 PM, marc bessieres wrote: > > > > If this seems useful, I will resume working on it, I've postponed that work > > too much. > > I just gave this a spin and it is very useful! I can imagine a tweak > here and there but, as is, it is quite handy to maneuver the nightly > builds. Definitely a great starting point. > > I'm attaching the regtest results for August and September 2013 so > people can see for themselves and we can gather some feedback. > - tar xjf TOPDIR.tar.bz2 > - firefox TOPDIR/generated/index.html > > > Florian Attachment was too big, so it did not reach valdev. => I have created bug https://bugs.kde.org/show_bug.cgi?id=324809 with the script and the example result produced by Florian. Further suggestions/comments can be added in the bugzilla entry. Thanks to Mark for looking at this, I also think it is good work. Philippe |