Re: [Simpletest-support] Code coverage

[Douglas H. <dh...@gm...>]

Jakob Ketterl <jketterl@...> writes:
> On Tuesday 18 November 2008 06:13:16 Douglas Hubler wrote:
> > Jakob Ketterl <jketterl@...> writes:
> > > I found two little bugs in the reporting templates (it's just missing
> > > echo statements). i'm including a patch
> >
> > thanks, i committed these
> >
> > Any progress on general review?
> 
> yes, i've even managed to integrate it with phpundercontrol (using a special 
> reporter class i've hacked up myself).
> 
> however, i didn't use the concept of an external shell script to separate 
> coverage setup/collection from the actual test run

that's great, my goal was to give folks APIs for custom setups, i almost didn't
create shell script to encourage folks to do just what you did, but then i
thought we'd loose the folks that just wanted to follow a simple pattern.

> i've read in the source that uncovered files are scanned for from getcwd() 
> downwards (iirc) - so maybe there's a way to make that configurable?

yes, that directory really should be made configurable


> i've also had a look into what's inside the sqlite file, and from my point of 
> view the structure doesn't look that complicated, imho that's nothing a 
> serialized array couldn't handle (if it really needs to be stored to disk - 
> for my case it's not even necessary).

the tests i run take about 2+ hours and sqlite helps me perform data selection
without loading the entire coverage into memory.  Having said this, all this is
handled by CoverageDataHandler and so this could have alternate implementations


> i think it would be better to integrate this deeper into the simpletest main 
> code, but that's beyond the scope of "extension", if i'm correct...

For reporting, I see someone developing some unit test reports that integrate
the coverage data, i think someone even suggested that.  I like the fact that
they can run independently, but i think there are a lot of ways folks can create
optional ways to make them work together.  

For data collection, in my situation, i run simple tests from CLI, that in turn
make http requests to a local apache server, that i need to accumulate coverage
data for.  This request knows nothing about simpletest.


On another note

I'm starting the process of using this branch for generating
 http://acquia.com/files/test-results/index.html
This is a report that is generated on a nightly basis.