[Jxcl-devel] RE: So what's the status of JXCL

[King D. <dal...@th...>]

> -----Original Message-----
> From: Jim Dixon
> 
> On Sat, 28 Feb 2004, Dale King wrote:
> 
> > Just pinging to find out what the status of JXCL. Doesn't
> > seem to be any activity recently. I'm interested in seeing a
> > good OS coverage tool. Clover sucks. It tells me most of my
> > class was not executed when it was. JCoverage/GPL is OK,
> > but not great. I think JXCL can be better.
> >
> > You'll see that I submitted a patch that fixes the class path
> > handling to work on Windoze.
> 
> ? I haven't seen any patch.  Where did you send it?

I uploaded it as a patch on Sourceforge:

http://sourceforge.net/tracker/?group_id=92728&atid=601791

> > I have some ideas about JXCL.
> >
> > One is that there are tools like Jcoverage that instrument the
> > code separately from testing it and JXCL which instruments it
> > on the fly. Both have advantages and disadvantages. But
> > there is no reason why JXCL couldn't do both. In both cases
> > you have to read a class, instrument it, and produce data in
> > class file format. That could be called from either a
> > classloader or from a tool that wrote it to disk.
> 
> As you describe it, this is a fairly minor change.  All that you
> have to do is write the byte array containing the instrumented code
> to disk.

Sure its easy, but you aren't exposing that functionality.

> > I also want to be able to work in what I think is an absolute
> > must for any coverage tool, which is the ability to declare
> > certain code that isn't supposed to be executed so that it is
> > not included in the total for calculating percentage. But it is
> > absolutely vital that it also report it as an error if it is
> > executed. I think it may be possible to implement that in JXCL
> > even though it is only working from byte code.
> 
> Can you define exactly what you mean here?  Are you talking about
> marking certain ranges of source code?  Or would you be content with
> excluding code at the method level?

I mean marking ranges of source code, not at the method level! I actually
tried to make the case to Joshua Bloch that the new metadata facility in 1.5
should include annotating within a method for this very kind of thing, but
was turned down.

As I understand your code, you compute a control flow graph of the method so
basically you could determine where the end of a block is. If we could
insert a harmless statement into the code that could be easily identified by
the instrumentation code then we should be able to mark code as unreachable.
The first idea is simply a static method call on a certain class. So I could
have:

	if( param == null ) // Should never happen
	{
		JXCL.unreachableCode();
		throw new NullPointerException();
	}
	doSomething();

The method does not even have to do anything. It is just a signal to the
instrumentation code, which simply looks for an invokestatic on that method.
The tough part is that you have to figure out where the end of the block is
because none of the rest of the code should be reachable from after that
point until the end of the block. I think that should be possible from your
control flow analysis.

I also want to make this configurable because you might have different types
of testing like unit testing, integration testing, system testing, etc.
Whether this statement is reachable may vary based on that. For instance, in
unit testing you might be able to pass a null param, but in integration and
functional testing that should never happen. So it is reachable in unit
testing but not integration testing.

Under most coverage tools that will actually decrease your code coverage
totals when in reality it is a good thing that you never pass a bad
parameter.

So I am thinking of having an overload of that method that takes a parameter
to say when it is unreachable. Either an int which is a bit mask or a
string. The assigning of bits or string tokens would be up to the user and
not dictated by JXCL. So for this scenario we might have:

	JXCL.unreachableCode( TESTING.Integration | TESTING.System );

or

	JXCL.unreachableCode( "integration, system" );

There are probably several other ways to do it, but this is what I am
contemplating. It just has to be something that the instrumentation code can
easily identify and that is configurable to allow different testing
scenarios.

Another way I just thought of that also eliminates any dependencies on JXCL
in the regular code would be allow the user to specify a method name that is
used during instrumentation to flag unreachable code. The type of testing is
then controlled by the class name upon which it is invoked. So the above
could be done as:

	IntegrationTesting.unreachableCode();
	SystemTesting.unreachableCode();

The definition of the IntegrationTesting and SystemTesting are not dictated
by JXCL. It is just looking for a static method call of a given name with no
parameters.

I think this new idea might be better because it removes the need to figure
out what the parameter is to the method call. You just have to look for
certain invoke static calls. I like the idea of not having any dependence on

All the instrumentation code has to do is include data in about what lines
are marked unreachable in its output. The work of looking for executions of
these lines is simply part of the reporting code.

This kind of feature has not been implemented by anybody that I know of and
to me seem like a basic necessity in code coverage. Without it you will
never be able to reach 100% coverage. But remember that the reporting has to
signal an error if one of these lines is executed. This prevents someone
from marking a whole bunch of code as unreachable to inflate the coverage
numbers. The report should also indicate the percent of code marked
unreachable so someone can't mark most of the code unreachable and not
exercise it.

> > But then again I have never actually seen JXCL in action. I've
> > had to work to even get it to build and run on Windows.
> 
> The examples don't work?

I haven't gotten them to work. I am trying from Windows, which is obviously
something you had not tested (see my patch). The issue may be the Ant
classpath bug you talked about.

You report that you were geting several test failures with Maven. After my
patch I only got the ones to do with finally clauses. I started looking at
those. TestFinally fails because the code generated by the compiler is not
how the code thinks it is. You expect a try finally to have 3 null handlers
when it only has two and you get an array bounds exception. Perhaps the code
generated for finally has changed from what you expected.

-- 
 Dale King