#45 semantic code exclusion

[Vlad Roubtsov]

A very desirable feature for the end users would be
being able to exclude code from coverage requirements
at levels that differ from basic blocks or method bodies.
Some examples include:

- all methods with names fitting certain patterns
- all catch blocks
- assert checks
- if-statements conditioned on calls into certain classes
(e.g., log4j logging guards)

There is also demand for capabilities that are manifestly
for a source code-based tool, e.g., the ability to exclude
certain *source* code patterns.

There are several approaches possible here:

(a) [easiest] limit ourselves to a fixed set of exclusions
possible via pure bytecode analysis (e.g., assertions,
catch blocks, method names could be handled this way)

(b) [hardest] infer "semantic" intent from opcode stream
(decompilation, essentially) and use pattern matching
against new type of metadata built from opcodes to
trigger exclusion. This needs further research, but
sounds very interesting.

(c) [medium] cave in on the processing speed side and
start doing source code analysis. This is further
subdivided into two routes:

(c.1) parse and *instrument* source code. This
direction was what I explicitly didn't want to choose at
the outset, but it might be worth providing as an option
for users that are ok with slower instrumentation.

(c.2) parse bytecode and sourcecode at the same time.
The source code is used only for mapping ranges of
bytecode to particular sections of source code, the
instrumentation is still done at the bytecode level. With
some clever thinking here it might be possible to minimize
source code parsing overhead and the tool usability does
not suffer (no full recompile just for coverage)

Related discussion:
http://sourceforge.net/forum/forum.php?
thread_id=1104854&forum_id=373865

Related feature requests:

- http://sourceforge.net/tracker/index.php?
func=detail&aid=974836&group_id=108932&atid=651900

- http://sourceforge.net/tracker/index.php?
func=detail&aid=962010&group_id=108932&atid=651900

[Nobody/Anonymous]

Logged In: NO

Possible extension of assert checks: omit any block that
ends with a throw of static type in a list of excluded
Throwable types. List could be explicit types only, or
could include derived classes of throwables (so listing
Error would also exclude AssertionError, and listing both
Error and RuntimeException would exclude all Throwable types
that don't require declarations).

Low hanging fruit: exclude only the leaf block. This will
cover most cases such as assertions and 'should not reach
here' default-cases/catches.

Complete coverage would require some flow analysis to find
all block paths that necessarily end with the throw (such as
if there is a branch or loop while computing the throw
message string).

[Tom Parker]

Logged In: YES
user_id=1037926

I support the previous comment. Even being able to simply
ignore objects of type Error (or just InternalError) would
being me a lot closer to 100% coverage and provides a simple
step vs. the much more complicated analysis of the code.

[gcey]

Logged In: YES
user_id=1169208

I very much support this feature request.
And I think, it should be implemented very soon.
Therefore the simple approach (a) should be taken.
Then we will see, whether this is enough.
If not, the much more difficult approaches (b) or (c) could be
taken later.

[Ian Pilcher]

Logged In: YES
user_id=1442067

As a user of EMMA, I'd like to see this get more attention.

As things currently stand, adding assertions causes the
coverage reported by EMMA to fall.

[hsuob1]

Logged In: YES
user_id=1491483

I'm in need of this feature, and actually only need
something that's quite simple. If the block is surrounded
by a pair of comments like this:

// BEGIN INFEASIBLE
...
// END INFEASIBLE

then count this block as covered.

This can be done as a post processing step. That is,
provide a simple routine that go over the source file and
scan for these commented blocks, then generate coverage data
pretending that these blocks are commented in the existing
format. Before generating the coverage report, the end user
would merge this with the actual coverage data.

This has no runtime penality, and scanning the source code
for such comments is much easier than parsing.

I think this is a simpler variation of option (c.2)
mentioned above, and I sure hope this is implemented soon.