#63 Revisiting statistics of method/block/line execution

[Kurt]

I LOVE EMMA! Keep up the great work!

I noticed the following Item was closed:

Summary:
Number fo times method/block/line is executed
Could you please add statistic about number of
times method/block or line is executed.

Date: 2004-09-03 08:49
Sender: vlad_r

Hmm. In fact, I had such stats in what was
effectively EMMA 1.0 and none of my users
seemed to use that data. Since collecting stats
(vs just boolean "hit/not hit" flags) causes higher
runtime overhead, I removed this.

Although I am aware that other coverage tools
report such stats, I believe that doing performance
analysis based on runs of functional test suites is a
fundamentally wrong approach. (Functional/unit
tests are written for functional coverage and do not
represent the true runtime/performance scenarios).

As far as the runtime overhead is concerned, why not
leave this functionality turned OFF by default, but allow
the users who actually want it to turn it ON in their ANT
Build file? You can caveat away in the documentation
about how turning ON this feature will significantly
impact performance and should never be used to
analyze performance type test suites.

I don't really get your argument above
about "performance analysis based on runs of functional
test suites". EMMA is purposefully NOT limited to
evaluating the execution of just Unit Test Suites.
Right? I can set up EMMA to evaluate anything that I
can execute, right? There might be a certain test that I
want to perform by hand and find out how many times a
certain block of code was executed vs. another block of
code.

I don't see how this feature would do any harm as long
as it was turned OFF by default and warnings about
performance were included in the documentation. That
way only the hard core users would actually ever use it,
although probably not very often.

Thanks for listening,
Kurt from Florida