|
From: 齐玉华 <qi...@12...> - 2012-06-01 10:34:16
|
Thanks for your detail account. In the manual the program degradation is relatively 5 times slower than native execution with the "--tool=none" flag. And "valgrind --tool=none -q date" takes about 0.370 seconds. Thus, i assume that the valgrind start time is no more than 0.370 seconds. But I am confused to the following program degradation(~about 10 times): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ grace@grace:~/software/WAutoRepair/experiment/php/php_null$ time /home/grace/software/WAutoRepair/experiment/php/php_null/php-src-5.3/sapi/cli/php good2.php 6 real 0m4.764s user 0m4.756s sys 0m0.004s grace@grace:~/software/WAutoRepair/experiment/php/php_null$ time valgrind --tool=none -q /home/grace/software/WAutoRepair/experiment/php/php_null/php-src-5.3/sapi/cli/php good2.php 6 real 0m44.110s user 0m43.971s sys 0m0.140s ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ than, what has happened? At 2012-06-01 17:20:33,"Dallman, John" <joh...@si...> wrote: > Because the recompilation time of one patched php is nearly 20 seconds, so if the > sum of degradation time is more than 20 seconds, then our method of indirecting > function will perorm worse than native regression testing. You must expect performance under Valgrind to be worse than any normal kind of test harness for regression testing. Valgrind is not a substitute for normal kinds of testing, or is it a general-purpose testing framework. Its value is that it can find errors that normal testing won't, errors that leave the code producing correct results most of the time, but wasting memory, using uninitialized variables whose contents are unpredictable, or other errors depending on the tool that you use. For example, the products I work on have slowdowns of x20 to x30 under Valgrind. I don't attempt to run daily testing with Valgrind, because it would be far too slow. Instead, as the development phase of each release comes (there are two per year) to an end, and pre-release maintenance gets under way, I run all of the appropriate testing once under Valgrind. This takes several weeks, and finds (some types of) coding errors made by the developers during the development of the release. Those get fixed in the release. This method is not perfect — running all the testing every day under Valgrind would be preferable. But the amount of hardware it would require (perhaps 50 modern Linux machines) compared to the number of problem it would find (perhaps one per moth) makes the perfect solution quite impractical. Using Valgrind has considerably reduced the numbers of errors reported by users that we can't reproduce, which is well worthwhile. That has been achieved using one or two machines. -- John Dallman |