Thread: [Cobertura-devel] switch coverage inconsistencies

Cobertura is not consistent with how it is doing conditional coverage for switch statements.   Piotr's branch handles them much better, but I want to go over it so there are no surprises with the next release.

First of all, there are two ways to look at the coverage of a switch statement.   Consider the following statement:


                                switch (e) {
                                                case FOO:
                                                case BAR:
                                                case GONK:
                                                                return true;

                                                default:
                                                                return false;
                                }

One way of thinking of this is that you have 4 different cases that you may want to cover in your tests.   If your tests only set 'e' to GONK, then the coverage would be 1/4.  The other way of thinking of this is that there are 2 different code blocks.   If your tests only set 'e' to GONK, then the coverage would be 1/2 since the first block (return true) was covered.

Since Cobertura does not look at the source, we are at the mercy of how the compiler handles switch statements.  Generally, if you have gaps in your cases, the compiler puts extra cases in your class files.   Consider the following switch statement that contains gaps:

                                switch (i) {
                                case 15:
                                                System.out.println("15");
                                                break;
                                case 16:
                                                System.out.println("16");
                                                break;
                                case 17:
                                                System.out.println("17");
                                                break;
                                case 18:
                                                System.out.println("18");
                                                break;
                                // intentionally skip 19 and 20
                                case 21:
                                                System.out.println("21");
                                                break;
default:
                                                System.out.println("default");
}

The compiler will interpret this as if you had written the following (note that case 19 and 20 are with the default):

                                switch (i) {
                                case 15:
                                                System.out.println("15");
                                                break;
                                case 16:
                                                System.out.println("16");
                                                break;
                                case 17:
                                                System.out.println("17");
                                                break;
                                case 18:
                                                System.out.println("18");
                                                break;
                                case 21:
                                                System.out.println("21");
                                                break;

                                case 19:
                                case 20:
default:
                                                System.out.println("default");
}

So, because Cobertura only looks at the classes, we can only keep track of the coverage of the blocks - not the cases.   Currently, Cobertura does not do a good job of this.  This bug has a good example of the problem:

http://sourceforge.net/tracker/?func=detail&aid=2998305&group_id=130558&atid=720015

Next, I want to show a few examples and show how Piotr's branch is handling them.

Back to the first example presented above:

                                switch (e) {
                                                case FOO:
                                                case BAR:
                                                case GONK:
                                                                return true;

                                                default:
                                                                return false;
                                }

If FOO, BAR, and GONK are all covered, but the default is not, then the coverage will be reported as 1/2.   This is because the compiler combines FOO, BAR, and GONK into the same code block.

Similarly, if FOO is the only thing covered, the coverage will be reported as 1/2.

Next example:

switch (i) {
case 1:
System.out.println("1");
break;
case 2:
System.out.println("2");
break;
}

Assuming i==2 is covered, the coverage is 1/3.   The third is the default.

Next example:

switch (i) {
case 2:
System.out.println("2");
break;
case 3:
System.out.println("3");
break;
case 1:
default:
System.out.println("1 or default");
}

Assuming i==5 and i==1 are covered, the coverage will be reported as 1/3.   Since case 1 falls through to the default, it is considered as a single condition of the switch statement.   The compiler considers it such.

Next example:

                                switch (e) {
                                                case FOO:
                                                                System.out.println("FOO");

                                                case BAR:
                                                                System.out.println("BAR");
                                                                return true;

                                                case GONK:
                                                                return true;

                                                default:
                                                                return false;
                                }

Note that FOO falls through to BAR.  If FOO is the only thing covered, the coverage will be 1/4.    BAR is not considered covered although execution fell through to it.   (the line coverage will show BAR to be covered, but the conditional coverage will not.)

If FOO and BAR are exercised during testing, then the coverage will be 2/4.

Note that this is a little confusing because if FOO and BAR shared a block like this:

                                switch (e) {
                                                case FOO:
                                                case BAR:
                                                                System.out.println("FOO or BAR");
                                                                return true;

                                                case GONK:
                                                                return true;

                                                default:
                                                                return false;
                                }

Then, if FOO and BAR are covered, the coverage is reported as 1/3 because FOO and BAR share the same block.

Generally speaking, a case is not considered a case of its own unless it has a block of its own.   Also, if multiple cases share the same block they are combined as one case.

This method seems to be reporting coverage in a much less confusing way overall.

John