|
From: David <dg...@iu...> - 2006-09-18 01:13:28
|
Any keys on how much repeatability should we expect from callgrind results? We are intending to execute callgrind to detect efficiency regression on th= e=20 execution of (deterministic) algorithms. But when we executed several times= =20 the same binary, the results we got for tested methods were different. I expected callgrind/valgrind to simulate cache and system calls so that th= e=20 instrumentation always would get the same results for successive executions= =20 of the same compilation of the same program. =46or some simple test cases (math ops+just few cache misses) the results s= eems=20 pretty repetable. But when dealing with real test cases the results vary=20 (getting random speedups among +-2%). Some test cases variation may be due = to=20 system calls for file access and memory handling. But some others test case= s=20 don't. So my only suspicious is that the cache initial state is not=20 deterministic. Note: This is on the context of developing Efficiency Guardian, an efficien= cy=20 testing framework.=20 See https://sourceforge.net/projects/efficiencyguard =2D-=20 David Garc=C3=ADa Garz=C3=B3n (Work) dgarcia at iua dot upf anotherdot es (Home) vokimon at telefonica adot net http://www.iua.upf.edu/~dgarcia |