Menu
▾
▴
simpletest-support
|
From: Lars O. <lso...@gm...> - 2008-04-23 14:58:29
|
Is it possible to do a report on code coverage using simpletest? If it is, it would be possible to use simpletest with phpUnderControl and cruisecontrol. It is pretty neat that you can get both a code coverage report and a codesniffer report. -- Lars Olesen Gratis backup https://mozy.com/?code=9MYJPF |
|
From: tarjei <ta...@nu...> - 2008-10-03 08:58:41
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, does anyone have any tips on how to get coverage data out of simpletest? Tarjei -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI5dlmYVRKCnSvzfIRAqKQAJ4knchkPUS69GSzPOoStNbScKUwUwCgsv0G bH5HR337pS963RzkzmeAzMQ= =n6sh -----END PGP SIGNATURE----- |
|
From: Elijah I. <tm...@gm...> - 2008-10-05 04:06:59
|
I'm pretty certain there is no coverage data being collected. -- Elijah On Fri, Oct 3, 2008 at 1:35 AM, tarjei <ta...@nu...> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, does anyone have any tips on how to get coverage data out of > simpletest? > Tarjei > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFI5dlmYVRKCnSvzfIRAqKQAJ4knchkPUS69GSzPOoStNbScKUwUwCgsv0G > bH5HR337pS963RzkzmeAzMQ= > =n6sh > -----END PGP SIGNATURE----- > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Simpletest-support mailing list > Sim...@li... > https://lists.sourceforge.net/lists/listinfo/simpletest-support > |
|
From: Douglas H. <dh...@gm...> - 2008-10-05 04:09:08
|
tarjei <tarjei@...> writes: > > > Hi, does anyone have any tips on how to get coverage data out of simpletest? > Tarjei > I had some luck with this http://developer.spikesource.com/projects/phpcoverage/ The xmlrpc solution to combine coverage results did not perform well on the scale of tests I needed so a colleague of mine started a rewrite and I finished it. http://acquia.com/blog/drupal-cli-utils Sample results http://acquia.com/files/test-results/index.html Don't worry, it's not only for drupal. I would like to create an official project somewhere and link to it from simpletest's website as a part of a php test solution. |
|
From: Marcus B. <ma...@wo...> - 2008-10-05 18:40:08
|
Hi... Douglas Hubler wrote: > tarjei <tarjei@...> writes: > I would like to create an official project somewhere and link to it from > simpletest's website as a part of a php test solution. We can easily link to it from simpletest.org. What state is the project in right now? I am looking to bundle a code coverage solution as long as it passes the "it just works" test. I am happy to modify the SimpleTest core to accommodate it. yours, Marcus |
|
From: Douglas H. <dh...@gm...> - 2008-10-15 18:43:11
|
Marcus Baker <marcus@...> writes: > > Hi... > > Douglas Hubler wrote: > > tarjei <tarjei@...> writes: > > I would like to create an official project somewhere and link to it from > > simpletest's website as a part of a php test solution. > > We can easily link to it from simpletest.org. What state is the project > in right now? I am looking to bundle a code coverage solution as long as > it passes the "it just works" test. So sorry I'm so late on this post!!! I would consider it in the category, "it just works" and I would very much like to add it to simpletest. I will submit a patch to trunk for review as it's probably the best way to get this started. |
|
From: Douglas H. <dh...@gm...> - 2008-10-16 03:47:21
|
Douglas Hubler <dhubler@...> writes: > I would consider it in the category, "it just works" and I would very much like > to add it to simpletest. I will submit a patch to trunk for review as it's > probably the best way to get this started. As promised: http://sourceforge.net/tracker/index.php?func=detail&aid=2170375&group_id=76550&atid=547457 |
|
From: Marcus B. <ma...@wo...> - 2008-10-16 15:01:28
|
Hi... Douglas Hubler wrote: > Douglas Hubler <dhubler@...> writes: > As promised: > > http://sourceforge.net/tracker/index.php?func=detail&aid=2170375&group_id=76550&atid=547457 Cheers! I am not yet sure what we will be able to do for the upcoming 1.1 release, as I don't want to derail this. It really needs a few people to try it out first. yours, Marcus |
|
From: tarjei <ta...@nu...> - 2008-10-16 15:12:11
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Would it be possible to merge the patch to a branch of 1.1 ? I'm just thinking that would help with regard to testing and fixing errors. regards, Tarjei Marcus Baker wrote: > Hi... > > Douglas Hubler wrote: >> Douglas Hubler <dhubler@...> writes: >> As promised: >> >> http://sourceforge.net/tracker/index.php?func=detail&aid=2170375&group_id=76550&atid=547457 > > Cheers! > > I am not yet sure what we will be able to do for the upcoming 1.1 > release, as I don't want to derail this. It really needs a few people to > try it out first. > > yours, Marcus > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Simpletest-support mailing list > Sim...@li... > https://lists.sourceforge.net/lists/listinfo/simpletest-support -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI91m3YVRKCnSvzfIRAh+uAJ0V584c4A3Scswx1JgdYasbwd2D5gCfc870 rCUoXzB7MYTUCpdSdIPUcmM= =c3rL -----END PGP SIGNATURE----- |
|
From: Marcus B. <ma...@wo...> - 2008-10-16 16:16:44
|
Hi... tarjei wrote: > Would it be possible to merge the patch to a branch of 1.1 ? > > I'm just thinking that would help with regard to testing and fixing errors. If Douglas is happy to do that, but you'll have to ask him. Let's try it out first. > > regards, > Tarjei yours, Marcus |
|
From: Douglas H. <dh...@gm...> - 2008-10-16 20:23:26
|
Marcus Baker <marcus@...> writes: > > Hi... > > tarjei wrote: > > Would it be possible to merge the patch to a branch of 1.1 ? > > > > I'm just thinking that would help with regard to testing and fixing errors. > > If Douglas is happy to do that, but you'll have to ask him. Let's try it > out first. I'd be happy to merge it, because the single 1-line change i made to existing code was because I need advice on how to solve it (calling exit on shutdown hook) Reason it is so independent of simpletest is because I had to design it so web processes produce coverage. So the flow goes 1.) init coverage 2.) run all your code/tests/etc with hook to autocoverage.php in as many processes commands as you need to. 3.) close coverage 4.) run report One comment I had, I now think coverage reporter should be in separate php file so autocoverage.php doesn't load more than it has to. |
|
From: Mark R. <ma...@co...> - 2008-10-17 23:50:29
|
I have a question about the change to the return value of the cli exit
routine...
if (SimpleReporter::inCli()) {
- exit($result ? 0 : 1);
+ //exit($result ? 0 : 1);
}
}
what was the issue that led to you commenting this out?
regards,
Mark
|
|
From: Mark R. <ma...@co...> - 2008-10-18 00:46:30
|
Have been letting this loose, and so far it seems to be generating all the results perfectly - I'm assuming that it's as accurate as xdebug can be, and I like the way that the source code is presented with green/red illustrating which parts of the code were executed and which were not. However, I had a few teething problems with thousands of E_NOTICE errors being thrown from the coverage run: Notice: Undefined offset: in /simpletest/trunk/coverage.php on line 290 which is this line: $lineCoverage = self::lineCoverageCodeToStyleClass($coverage[$i]); I'm reviewing the code now, so expect to have some more technical feedback soon... but in general, it looks good - I do think we need to look at whether this is best packaged as an extension (especially the reporting and static file generation part of it)... I don't have a problem with coverage hooks going in to the main simpletest core (again, subject to code review), but I think this needs to be more closely integrated with the existing paradigm we have for reporters (which is subject to change as we move forward - more on that in a later post). I would like to see more abstraction at the level of generating the coverage reports - by default I would prefer to see the data being fed directly back to a TextReporter decorator or something similar, so that it could be displayed directly in the stdout rather than dumped to HTML files. I think it would be good to give users the option of where they want to export the results, and keep things as minimal and close to the existing way of doing things as possible. The dependency on Sqlite is understandable, not sure sure about the Smarty - again this would be more appropriate as an extension, not in core. We want people to be able to download simpletest and run it on any PHP server, without needing to stitch together dependencies, but having said that, anyone who is using pear will have absolutely no problems. I've never been a fan of code coverage personally, but now I'm starting to see the possibilities. I think one of the potential benefits is simply having greater high level visibility of the spread of a code-base. It help you to understand the bigger picture when you can browse the source code through the perspective of what the tests are doing. Another thing to bear in mind, is that the mileage of this feature may vary - there are a lot of simpletest users who make heavy use of the web tester, which peeks at responses over HTTP, so doesn't operate on code in the local process scope and can't be xdebugged. But good work, thanks for that... I'm enjoying playing with it... Regards, Mark |
|
From: Douglas H. <dh...@gm...> - 2008-10-19 04:50:07
|
Mark Rickerby <maetl@...> writes: > I have a question about the change to the return value of the cli exit > routine... > what was the issue that led to you commenting this out? yes, this is the big item i could use help on - autorun uses shutdown hook to run tests - coverage also uses a shutdown hook to know when to save coverage - coverage has to happen after tests run, but tests call exit so coverage never happens. |
|
From: Marcus B. <ma...@wo...> - 2008-10-20 12:53:06
|
Hi... Douglas Hubler wrote: > yes, this is the big item i could use help on > > - autorun uses shutdown hook to run tests > - coverage also uses a shutdown hook to know when to save coverage > - coverage has to happen after tests run, but tests call exit so coverage > never happens. OK, so some kind of post test hook might be needed. I'd rather look at alternatives first. If you had a reporter decorator, and autorun understood the concept, then you could just tell auto run to inject it. Then your post test stuff could all be in the paintFooter() of the decorator. I suggest that I we fix autorun sharpish, you try it out, then we regroup. How near are you to getting this stuff into a branch? yours, Marcus |
|
From: Douglas H. <dh...@gm...> - 2008-10-21 04:34:04
|
Marcus Baker <marcus@...> writes: > I suggest that I we fix autorun sharpish, you try it out, then we regroup. i decided to separate function simpletest_autorun in autorun.php into 2 parts and I was able to avoid exit call. So autorun functionality is intact as it was. Let me know if this is ok. > > How near are you to getting this stuff into a branch? i put it in /simpletest/simpletest/branches/code_coverage/ 2 open issues - package as extension? - make dependency on smarty optional |
|
From: Douglas H. <dh...@gm...> - 2008-10-28 04:43:42
|
status update Marcus Baker <marcus@...> writes: > > - package as extension? > Extension at first if it;s possible. I've finished this work, no hooks in core required. > > - make dependency on smarty optional > Better to remove it altogether. this is coming along, not quite ready yet but almost there. |
|
From: Douglas H. <dh...@gm...> - 2008-10-21 18:03:37
|
Marcus Baker <marcus@...> writes: > Extension at first if it;s possible. If we need hooks or extra code > we'll put them into the core, and even into HEAD. Can describe this process or point me details. I am not familiar with simpletest extensions at all. > > > - make dependency on smarty optional > > Better to remove it altogether. Have something simple (HTML tables and a > bit of styling) and add the pretty version as an extension. It must, > must work out of the box for "Simple"Test. Not much of a UI style guide > I admit, but I'm afraid I'm quite rigid about it :(. I plan to implement something useful in just php and it can be the default implementation. I will investigate the reporters in simpletest. re:branch I branched from trunk, I could not find 1.1 branch, which for me is good because I don't have to merge twice. |
|
From: Mark R. <ma...@co...> - 2008-10-21 18:30:04
|
Hi Douglas, > Can describe this process or point me details. I am not familiar with > simpletest extensions at all. There's no 'formal' extension mechanism, basically it's just loosely coupled classes that interact with the extensible parts of the core. If you look in /extensions/ you'll see the existing code that is bundled - the format is: /ext_name.php <-- hook class to include /ext_name/ <-- supporting package files /ext_name/tests/ <-- unit tests for the individual package some of the older and smaller extensions are just a single PHP file though. Regards, Mark |
|
From: Mark R. <ma...@co...> - 2008-10-18 01:27:06
|
Sorry, I was referring to the code coverage patch... One of the only changes it makes to core code is to comment out that return exit value... |
|
From: Mark R. <ma...@co...> - 2008-10-18 12:28:44
|
Here's a spiral chart illustrating the levels of test coverage within simpletest itself, starting with 100% coverage (invoker.php apparently), and working its way down to the reporter.php (29% coverage, due to the paint methods that don't get called). This may not be the best way to visualize this, but I'm just getting started with it :) |
|
From: Marcus B. <ma...@wo...> - 2008-10-18 01:23:09
|
Hi... Mark Rickerby wrote: > what was the issue that led to you commenting this out? Which file? I don't remember commenting it out. Strange. Maybe the functionality was already covered by autorun, but then I'd have deleted the old line. I don't know. > > regards, > Mark yours, Marcus |
|
From: Sebastian S. <ssc...@ch...> - 2008-10-06 07:07:57
|
<3 Code coverage in Simpletest -----Ursprüngliche Nachricht----- Von: Douglas Hubler [mailto:dh...@gm...] Gesendet: Sonntag, 5. Oktober 2008 03:42 An: sim...@li... Betreff: Re: [Simpletest-support] Code coverage tarjei <tarjei@...> writes: > > > Hi, does anyone have any tips on how to get coverage data out of simpletest? > Tarjei > I had some luck with this http://developer.spikesource.com/projects/phpcoverage/ The xmlrpc solution to combine coverage results did not perform well on the scale of tests I needed so a colleague of mine started a rewrite and I finished it. http://acquia.com/blog/drupal-cli-utils Sample results http://acquia.com/files/test-results/index.html Don't worry, it's not only for drupal. I would like to create an official project somewhere and link to it from simpletest's website as a part of a php test solution. ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Simpletest-support mailing list Sim...@li... https://lists.sourceforge.net/lists/listinfo/simpletest-support |
|
From: Douglas H. <dh...@gm...> - 2008-10-19 04:41:27
|
Mark Rickerby <maetl@...> writes: > However, I had a few teething problems with thousands of E_NOTICE > errors being thrown from the coverage run: y will fix this, I didn't have the logging level right when developing this. > I do think we need to > look at whether this is best packaged as an extension (especially the > reporting and static file generation part of it)... I don't have a let me know if you decide that, and how to go about this > I would like to see more abstraction at the level of generating the > coverage reports makes sense > > The dependency on Sqlite is understandable, not sure sure about the > Smarty - again this would be more appropriate as an extension, not in > core. also makes sense > there are a lot of simpletest users who make heavy use of the > web tester, which peeks at responses over HTTP, so doesn't operate on > code in the local process scope and can't be xdebugged. you can use this in http env., depending on your php setup. If you can include autocoverage.php somewhere that will get called, it will collect coverage. This is how i use it. Permission and current working directory also have to be conducive > > But good work, thanks for that... I'm enjoying playing with it... thanks |
|
From: Mark R. <ma...@co...> - 2008-10-16 04:14:11
|
Cool, I'll take a look at it and give you some feedback. Thanks! Mark |
Thanks for helping keep SourceForge clean.
X