Re: [Valgrind-developers] performances of trunk compared to 3.8.1 ?

[Florian K. <fl...@ei...>]

On 09/05/2013 12:32 AM, Philippe Waroquiers wrote:
> At least on x86 (old pentium, and on gcc20), I observe some
> performance regression (e.g. on perf/bz2) between valgrind 3.8.1
> and trunk.
> Also, I have the feeling that regression tests are slower.
> 
> Does anybody share such feelings ?
> 
> And better, some measurements e.g. on mips or s390 or arm,
> as I have no access (anymore) to such platforms ?
> 

As Maran already reported there is performance regression on s390. It
was guessed by Julian that r13278 might be to blame (at least
partially). That was a good guess. Here is what I see on a z10-EC:
tr-13278 is trunk with r13278 (only) backed out.

perl perf/vg_perf --vg=../3.8.1 --vg=../trunk --reps=5 perf
-- Running  tests in perf ----------------------------------------------
-- bigcode1 --
bigcode1 3.8.1    :0.49s  no: 4.6s ( 9.3x, -----)  me: 8.1s (16.6x, -----)
bigcode1 trunk    :0.50s  no: 4.8s ( 9.5x, -3.5%)  me: 8.4s (16.8x, -3.7%)
bigcode1 tr-13278 :0.49s  no: 4.8s ( 9.7x, -3.7%)  me: 8.3s (16.9x, -2.1%)
-- bigcode2 --
bigcode2 3.8.1    :0.49s  no: 8.9s (18.2x, -----)  me:17.1s (34.9x, -----)
bigcode2 trunk    :0.51s  no: 9.0s (17.7x, -1.5%)  me:17.8s (34.9x, -4.0%)
bigcode2 tr-13278 :0.49s  no: 9.1s (18.5x, -1.9%)  me:17.4s (35.5x, -1.9%)
-- bz2 --
bz2      3.8.1    :1.07s  no: 5.5s ( 5.1x, -----)  me:24.2s (22.6x, -----)
bz2      trunk    :1.07s  no: 5.7s ( 5.3x, -2.7%)  me:29.0s (27.1x,-19.7%)
bz2      tr-13278 :1.07s  no: 5.5s ( 5.2x, -1.1%)  me:21.5s (20.1x, 11.1%)
-- fbench --
fbench   3.8.1    :0.80s  no: 2.8s ( 3.5x, -----)  me:10.6s (13.3x, -----)
fbench   trunk    :0.80s  no: 3.6s ( 4.5x,-26.2%)  me:12.2s (15.3x,-15.2%)
fbench   tr-13278 :0.80s  no: 3.5s ( 4.3x,-22.7%)  me:10.5s (13.1x,  0.8%)
-- ffbench --
ffbench  3.8.1    :0.52s  no: 1.5s ( 2.8x, -----)  me: 5.7s (11.0x, -----)
ffbench  trunk    :0.52s  no: 2.3s ( 4.4x,-57.5%)  me: 7.7s (14.9x,-35.6%)
ffbench  tr-13278 :0.52s  no: 2.3s ( 4.4x,-54.4%)  me: 6.2s (11.9x, -8.1%)
-- heap --
heap     3.8.1    :0.33s  no: 1.8s ( 5.5x, -----)  me:14.4s (43.7x, -----)
heap     trunk    :0.33s  no: 2.2s ( 6.7x,-20.2%)  me:16.3s (49.5x,-13.2%)
heap     tr-13278 :0.33s  no: 1.9s ( 5.7x, -3.3%)  me:14.4s (43.7x,  0.1%)
-- heap_pdb4 --
heap_pdb 3.8.1    :0.33s  no: 2.0s ( 6.1x, -----)  me:21.2s (64.2x, -----)
heap_pdb trunk    :0.32s  no: 2.3s ( 7.2x,-11.7%)  me:23.0s (71.8x, -8.6%)
heap_pdb tr-13278 :0.33s  no: 2.1s ( 6.4x, -5.0%)  me:21.2s (64.2x,  0.0%)
-- many-loss-records --
many-los 3.8.1    :0.03s  no: 0.4s (14.7x, -----)  me: 3.7s (122.0x, ----)
many-los trunk    :0.04s  no: 0.5s (13.0x,-15.6%)  me: 3.9s (96.2x, -5.2%)
many-los tr-13278 :0.03s  no: 0.5s (17.3x,-18.2%)  me: 3.6s (119.7x, 1.9%)
-- many-xpts --
many-xpt 3.8.1    :0.06s  no: 0.7s (11.5x, -----)  me: 5.0s (83.5x, -----)
many-xpt trunk    :0.08s  no: 0.8s (10.1x,-20.9%)  me: 5.4s (67.9x, -9.7%)
many-xpt tr-13278 :0.06s  no: 0.8s (12.5x, -8.7%)  me: 5.0s (83.0x,  0.6%)
-- sarp --
sarp     3.8.1    :0.04s  no: 0.5s (13.0x, -----)  me: 5.7s (142.8x, -----)
sarp     trunk    :0.04s  no: 0.7s (16.8x,-26.4%)  me: 7.2s (179.8x,-26.1%)
sarp     tr-13278 :0.04s  no: 0.6s (14.0x, -7.7%)  me: 6.3s (158.0x,-10.7%)
-- tinycc --
tinycc   3.8.1    :0.35s  no: 3.9s (11.1x, -----)  me:26.3s (75.1x, -----)
tinycc   trunk    :0.35s  no: 4.1s (11.8x, -6.7%)  me:30.3s (86.7x,-15.5%)
tinycc   tr-13278 :0.35s  no: 3.9s (11.2x, -1.0%)  me:25.3s (72.3x,  3.7%)
-- Finished tests in perf ------------------------------------

sarp and ffbench still regress notably.

  Florian