#28 Ant task for unit test coverage enforcement

[Andriy Palamarchuk]

Create an Ant task to insure test coverage is not lower
than specified threshold.

Usage is like:

<coveragecheck coverage="0.8"
datadir="${coverage-data}" logdir="${coverage-log}">
<fileset dir="${dirs.classes}"/>
</fileset>
</coveragecheck>

Fails build if coverage goes below 0.8 for the
specified classes.

Such task would be *very* useful for introduction unit
testing in the team. It would help to develop unit
tests class-after-class and to force any changes to
covered classes to be provided with unit tests.

Thanks,
Andriy

[Andriy Palamarchuk]

Logged In: YES
user_id=437388

Good name candidates for this task are coverageenforce and
enforcecoverage.

[Matt Albrecht]

Logged In: YES
user_id=171822

Already posted as another feature (901586:
https://sourceforge.net/tracker/?func=detail&aid=901586&group_id=22594&atid=375592\),
but there it's defined as a part of the report task.

This will definitely be in the next release, but I'm still
working on the precise symantics. I'll leave both feature
requests open, since they have different levels of detail.

[Matt Albrecht]

Logged In: YES
user_id=171822

Added to GroboCoverage 1.1.0.

The implementation allows for any number of <failon> tags
inside the reporting tag. The build script can have these
tags optionally set an Ant property if the failure condition
fails. Also, the owning report task can optionally fail on
these errors, or keep running. The build script, then, can
be arranged such that the reporting task can check for an
error but not fail, then at a future point a <fail
if="myprop"> task can run to check if those marker
properties were set.

[Andriy Palamarchuk]

Logged In: YES
user_id=437388

Matt, thank you. Look forward for the new release.
Andriy

[Andriy Palamarchuk]

Logged In: YES
user_id=437388

Matt,
while working more with the coverage tool on a project
with quickly groing code base I realized that there is a
reason to implement the enforcement in a separate task
instead being a part of report task - it takes quite a bit
of time by report task to generate report. I run coverage
report only from time to time but I'd like to icorporate
coverage level enforcement right into test target together
with unit tests and have reports generation in a separate
task. I guess enforcement would run much faster than report
generation, right? Having the same task to be called with
for completely different purposes does not look nice.

Another idea along the same lines - I'd like to have
enformcement report integrated into JUnit report. Would it
be better to implement enformcement as JUnit extension
instead as Ant task?
What do you think?

[Matt Albrecht]

Logged In: YES
user_id=171822

I would imagine that a failure due to insufficient code
coverage would mean that someone would want to see where
there was insufficient code coverage.

Right now, the failure checking fits very well into the
report generation tool, as both reports and failure checks
share much of the same data. However, this is a design
detail, and so we shouldn't alter the use case (that is, Ant
usability) to meet the design. At the moment, however, it
was the easiest way to add this functionality, and I'm
strapped for time.

Perhaps a future version will separate the two.

[Andriy Palamarchuk]

Logged In: YES
user_id=437388

Agree.
Andriy

[Matt Albrecht]

Logged In: YES
user_id=171822

This has been implemented.