[ cobertura-Feature Requests-1443252 ] Mark untestable code

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Feature Requests item #1443252, was opened at 2006-03-05 00:02
Message generated for change (Comment added) made by maisonobe
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=720018&aid=1443252&group_id=130558

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jalbert (jalbert631)
Assigned to: Nobody/Anonymous (nobody)
Summary: Mark untestable code

Initial Comment:
I would like to be able to mark small pieces of
"untestable" code (e.g. "catch" clauses for exceptions
that cannot be readily simulated), and have it marked
differently in the reports.

I would like to be able to mark code in some way. I
would suggest a comment flag at the start of each line
- say "/*--*/" (probably it should be configurable). If
a piece of unexecuted code is marked, it would be
flagged in the source listings in yellow, rather than
red, and the summary reports would show the percentage
of "untestable" code in yellow, in addition to the
current red and green bars.

The reason is that I cannot readily tell, when looking
at the reports, whether a lower-than-wanted percentage
of testing is because there is code that should be
tested but isn't, or code that cannot be tested. At
present, I can't tell it a few lines have been added
without testing, or it is still just the
knowingly-untested code that is being reported. If, in
a report, the green and yellow add up  to 100%, I know
there is no accidentally untested code.

The marks I am suggesting would look a little ugly, but
as far as I am concerned, that is fine - it  is nothing
something to be encouraged. This idea could be extended
in many ways - e.g. mark "code I will test later", etc.
- but I don't seen any need to get carried away.

----------------------------------------------------------------------

Comment By: Luc Maisonobe (maisonobe)
Date: 2008-04-10 20:18

Message:
Logged In: YES 
user_id=729699
Originator: NO

I also really miss this feature.
Currently, popular programming rules like adding private constructors to
utility methods or adding a default case in switch statement even if all
cases are already covered reduce code coverage despite they are enforced by
other tools (like chekstyle).
It is currently impossible to have the two tools happy at the same time,
and one must continuously manually check the output to make sure the
non-covered part are not due to bad testing.
This is important for some critical applications where the coverage
objectives are really high.

----------------------------------------------------------------------

Comment By: Gary Levin (gmlevin)
Date: 2008-02-27 22:57

Message:
Logged In: YES 
user_id=304909
Originator: NO

Another vote for this.  Some checked exceptions (like
InterruptedException) are difficult to meaningfully trigger, particularly
in Unit Test.  

The notation can be whatever fits your model.  

Is this scheduled for development?  Still marked as unassigned.

Cobertura works very well.  Thanks for a useful tool.

----------------------------------------------------------------------

Comment By: Rodrigo Ruiz (nodens2k)
Date: 2008-02-14 10:21

Message:
Logged In: YES 
user_id=1326652
Originator: NO

Both Javadoc tags and annotations would be perfect for me. I don't see any
reason to choose only one of them.

Private constructors are the most common ones in my code, but I agree with
jalbert631. There are some cases where a unit test is not the right way to
go. Simulating hardware errors and some invalid data conditions fit better
in the integration test phase. Sometimes, it is just impossible to automate
the test, and it must be tested manually.

----------------------------------------------------------------------

Comment By: Jiri Mares (jirimares)
Date: 2007-12-12 19:36

Message:
Logged In: YES 
user_id=819956
Originator: NO

Annotation? I don't think so ... 

----------------------------------------------------------------------

Comment By: Michael (michael6666)
Date: 2007-12-12 15:39

Message:
Logged In: YES 
user_id=1958403
Originator: NO

I'd like to vote for this feature. It would make cobertura *much* more
convenient to use, which is stated as one of cobertura's main strengths on
its home page.

Did you already consider using annotations to mark methods as "not to be
tested"?

----------------------------------------------------------------------

Comment By: Jiri Mares (jirimares)
Date: 2007-09-13 07:36

Message:
Logged In: YES 
user_id=819956
Originator: NO

We have been speaking about something like you are describing and the
semi-decision we made was to reuse the clover comments, maybe to introduce
some cobertura's one. So we know about this problem and want to solve it.

----------------------------------------------------------------------

Comment By: jalbert (jalbert631)
Date: 2007-09-12 14:51

Message:
Logged In: YES 
user_id=1467776
Originator: YES

The problem is that there are pieces of code that are difficult or
impossible to test in unit test. A typical example, as I mentioned in my
original post, is the catch clause for an exception that cannot easily be
simulated, such as a library that triggers an exception for a hardware
problem, or an invalid data condition that cannot be established.

Another example is code that is required but never intended to be
executed, such as a private constructor for a static class.

In some development methodologies, "trivial" methods, such as getters and
setters, are not required to be tested. Even though trivial, or generated
automatically by IDEs, they can form many lines of code in some classes,
and result in Cobertura reporting very low levels of coverage - low enough
that it is questionable whether the reports really have any value. (I do
not support this approach, and have written a JUnit extension to
automatically test most getters and setters, to eliminate this issue, but I
have worked in places where this was the case.)

----------------------------------------------------------------------

Comment By: Kasper B. Graversen (kbg)
Date: 2007-09-12 06:48

Message:
Logged In: YES 
user_id=118125
Originator: NO

Why are you not interested in testing your error handling code? or testing
that your units actually handless erroneous situations?

----------------------------------------------------------------------

Comment By: hasko (hasko)
Date: 2007-01-11 13:43

Message:
Logged In: YES 
user_id=99663
Originator: NO

It would also be nice to mark boilerplate code (e.g. toString) as not to
be tested. As an additional idea: the markup could be based on
annotations...

----------------------------------------------------------------------

Comment By: Bas Cancrinus (bascan)
Date: 2006-05-22 16:08

Message:
Logged In: YES 
user_id=437864

This would be a great solution to suppress coverage markers
on private constructors. They are usually used only to avoid
instantiation of utility classes, so they aren't covered by
definition.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=720018&aid=1443252&group_id=130558