From: Jeff H. <je...@ac...> - 2011-03-25 16:32:58
|
On 25/03/2011 9:06 AM, tpo...@gm... wrote: > I for one would be in favor of adding a set of application benchmarks > to tclbench. How optimization X or tweak Y really effects *my* > application is always the most important factor for *me*. Having a > set of representative applications is probably the next best thing, > as long as enough problem domains are covered. I would probably want > benchmark apps that cover: tclbench is a mish-mash of tests that do cover some aspects of the above, with a lot of micro-tests as well. I think for the larger tests you get truer estimation of impact, but I would guess you are looking for even larger app runs. Some examples in tclbench: > - computationally intensive base64, gccont*, md5 and sha all count here (all pure Tcl in tclbench) > - i/o intensive File IO? That would be READ, but maybe not sufficient to represent larger app usage. tclbench does no socket tests. > - symbol manipulation > - data parsing Aren't these the same? gccont may be the best rep for the former, which is an extract of real code where someone was biotech algorithms with pure Tcl, and it is compute intensive for playing around with lots of variety to achieve the same result. This is another aspect of tclbench that can be very informative to learning Tcl'ers about how to achieve the best results using a different algorithmic style in Tcl. Data parsing ... could cover a large range, but the PARSE html form upload is a "real world" example (though again, just prepares the data, doesn't do any networking). > Of course developing and verifying a set of representative applications > will take a fair amount of time (which is why > I'm proposing this, but don't have any concrete examples.) I think This might actually be a good GSoC project, if it included creating a better base of larger test apps. Overall tclbench is over 8000 loc for just the benchmark tests, and I'd say it really only scratches the surface of the operations that are "real world". That said, while tclbench isn't exact enough for some performance testing, it is best to look at it like the canary in the coalmine. Sometimes the twitches are no big deal, but it will also keel over, and we shouldn't ignore that. It's been more useful than not, and does highlight major regressions sometimes. Jeff |