Re: [Simpletest-support] Code coverage

[Mark R. <ma...@co...>]

Have been letting this loose, and so far it seems to be generating all
the results perfectly - I'm assuming that it's as accurate as xdebug
can be, and I like the way that the source code is presented with
green/red illustrating which parts of the code were executed and which
were not.

However, I had a few teething problems with thousands of E_NOTICE
errors being thrown from the coverage run:

Notice: Undefined offset: in /simpletest/trunk/coverage.php on line 290

which is this line:

$lineCoverage = self::lineCoverageCodeToStyleClass($coverage[$i]);

I'm reviewing the code now, so expect to have some more technical
feedback soon... but in general, it looks good - I do think we need to
look at whether this is best packaged as an extension (especially the
reporting and static file generation part of it)... I don't have a
problem with coverage hooks going in to the main simpletest core
(again, subject to code review), but I think this needs to be more
closely integrated with the existing paradigm we have for reporters
(which is subject to change as we move forward - more on that in a
later post).

I would like to see more abstraction at the level of generating the
coverage reports - by default I would prefer to see the data being fed
directly back to a TextReporter decorator or something similar, so
that it could be displayed directly in the stdout rather than dumped
to HTML files. I think it would be good to give users the option of
where they want to export the results, and keep things as minimal and
close to the existing way of doing things as possible.

The dependency on Sqlite is understandable, not sure sure about the
Smarty - again this would be more appropriate as an extension, not in
core. We want people to be able to download simpletest and run it on
any PHP server, without needing to stitch together dependencies, but
having said that, anyone who is using pear will have absolutely no
problems.

I've never been a fan of code coverage personally, but now I'm
starting to see the possibilities. I think one of the potential
benefits is simply having greater high level visibility of the spread
of a code-base. It help you to understand the bigger picture when you
can browse the source code through the perspective of what the tests
are doing.

Another thing to bear in mind, is that the mileage of this feature may
vary - there are a lot of simpletest users who make heavy use of the
web tester, which peeks at responses over HTTP, so doesn't operate on
code in the local process scope and can't be xdebugged.

But good work, thanks for that... I'm enjoying playing with it...

Regards,
Mark