#23 Increased speed of instrumented classes

[Grzegorz Lukasik]

Right now instrumeted classes can be very slow. With
each line of code in instrumented class there are
additional operations on maps executed. This patchs
fixes this behaviour, by adding to each line of code
just one instruction that increments value in an array.
It fixes also problems with multi-threaded applications
(RFE #1212146).

Attached is simple project that can be used to
check/benchmark changes. For benchmarks see below.

---------------- Patch description
When instrumenting class, two fields that reference
integer arrays are added to the class. First array
holds line numbers, and second holds values that
indicate times each line was executed. When instrumeted
class is first time used during tests, in class init
block these arrays are "registered" in ProjectData.
When serializing data after the tests, data from
registered arrays is collected and written to
serialized file.
To each line of code in instrumented class
incrementation of one cell inside one of the arrays is
added.

---------------- About multithread applications
It fixes problems reported in RFE #1212146. There
should be no exceptions thrown. But becouse access to
arrays fields is not synchronized (it would kill
efficiency) in rare cases lower numbers can be reported
for number of times line codes were executed. However
if line was executed at least once, it will be marked
as executed.

---------------- Benchmark results
Benchmark made on:
Celeron 2.80Ghz, 512MB RAM, Windows 2000, Sun J2RE
1.5.0-b64

without instrumentation: 3.1 sec
cobertura 1.4 : 61 sec
cobertura 1.3 : 707 sec (really!)

1.4 with patch : 4.1 sec

[Grzegorz Lukasik]

Patch

[Grzegorz Lukasik]

Sample application

[Mark Doliner]

Logged In: YES
user_id=20979

I like the speed increases (although I think the real-world
increase won't be quite as much as in your example program).

The main reason I don't feel comfortable applying this right
now is because I think it makes some things more
complicated... I'm afraid that the changes will make
Cobertura a little harder to maintain and a little easier to
break. Also, does merging work correctly after your
changes? And do you need to increment serialVersionUID in
ClassData, LineData and ProjectData?

I think this patch could be very beneficial, but I would
like to take care of some of Cobertura bigger bugs first.
Like having merging working with some good unit tests, and
instrumenting and reporting working with multiple directories.

[Grzegorz Lukasik]

Logged In: YES
user_id=1216999

I will change this patch a little. I will eliminate one of
the tables.

This patch does not change things related with how coverage
data is stored - it just postpones using touch() methods to
virtual machine shutdown. Merging should work exactly the
same as before, and serialVersionUID's should remain unchanged.

[Grzegorz Lukasik]

Logged In: YES
user_id=1216999

Bigger bugs from Cobertura were eliminated, so it is time
for some improvements ;)

Second version of this patch contains only one table but
give the same speed increase as previous one. Of course
results that are presented in initial submission are not
common for normal applications - but for programs that make
many computations it can be serious gain.

Later I will add some test cases for instrumentation process.

Here are some technical details:

To each instrumented class special table called
CoberturaHits is added. Cells in this table corespond to
source lines, and when some line is executed value in
corresponding cell is increased. These tables are registed
in ProjectData at class load. At JVM shutdown all tables are
visited and coverage data is collected.

Currently with execution of each line some operations on
maps are performed. After this patch only single ++
operation is executed with each original line.

[Grzegorz Lukasik]

Patch, version 2

[Lasse Koskela]

Logged In: YES
user_id=845101

What's the news on this patch?
Any chance of getting it in?

[Mark Doliner]

Logged In: YES
user_id=20979

Hey Grzegorz, I'm perfectly ok with having these changes
merged in now, if you're still interested in doing it. I
imagine the patch would need to be updated some. And it
might be good to wait to see if Jiri is going to make his
changes for better branch coverage, as I think that might
conflict some.

[Grzegorz Lukasik]

Logged In: YES
user_id=1216999

OK, i will add it after changed you have mentioned.

[Marcel]

Logged In: YES
user_id=1938395
Originator: NO

Is there any news on this? I would be very interested in such a speed increase. I'm currently comparing EMMA and Cobertura to decide which one to integrate into our nightly tests. Emma seems to be much faster, but somehow dead (long time no updates) and the output now really nice. Cobertura instead is much slower than Emma, and about 4x slower than without instrumentation, but has the much better output. Now if it would be possible to get Cobertura a bit faster, say that it may use afterwards only twice the time compared to no instrumentation, that would be very welcome.

[Jiri Mares]

Logged In: YES
user_id=819956
Originator: NO

I know about this patch ... I'm prepared to apply it after small changes in coverage data classes. The problem of the patch is, that it can be used only as inspiration because the cobertura instrumentation has been changed very much ...