[Simpletest-devel] test suites

[Blake D <dan...@gm...>]

hello,

I just incorporated simpletest into my projects. I thought I would
contribute some things regarding test suites.

First, something for your documentation if you wish. On the page:
http://simpletest.org/en/group_test_documentation.html , the example
code is given:
$this->collect(dirname(__FILE__) . '/unit',
   new SimplePatternCollector('/_test.php/'));

The above example matches and collects all filenames that end with
"_test".  However, if you want to match filenames that -begin- with
test, it's not so easy at all.  I designed a regular expression that
will match any path, containing a filename portion that begins with
"test_". It works properly with both UNIX and Windows directory
separators. It took forever to get it right (including proper escape
sequences). So I'd like to offer the following snippet to include in
the documentation:

$file_begins_with_test = '~(\\\\|/)test_[^\\\\/]+\.php$~';
$this->collect(__DIR__,
   new SimplePatternCollector($file_begins_with_test));

Here are some test strings that show that this pattern works as intended:
$pattern = '~(\\\\|/)test_[^\\\\/]+\.php$~';
echo "<br>\n" .preg_match($pattern,
   '/Projects/Project/test_1.php');       // match
echo "<br>\n" .preg_match($pattern,
   'P:\Projects\Project\test_1.php');     // match
echo "<br>\n" .preg_match($pattern,
   'P:\Projects\Project');                // not match
echo "<br>\n" .preg_match($pattern,
   'P:\Projects\Project/test_1\hi.php');  // not match
echo "<br>\n" .preg_match($pattern,
   'P:\Projects\Project\test_1/hi.php');  // not match
echo "<br>\n" .preg_match($pattern,
   'P:\Projects\Project\test_1.php1');    // not match
echo "<br>\n" .preg_match($pattern,
   'P:\Projects\Project\try_test_.php');  // not match

Bug:
The other thing I wanted to mention is that I found a bug.
On the same documentation page:
http://simpletest.org/en/group_test_documentation.html ,
it says that you can add a test file to a test suite more than once,
but files will only be run once per file: "If you want to see lower
level components fail first in the test suite, and this will make
diagnosis a lot easier, then you should manually call addFile() for
these. Tests cases are only loaded once, so it's fine to have these
included again by a directory scan."

This did not prove to be the case with my project. Here is some code
that demonstrates the bug:

all_tests.php:
<?php
require_once($simpletest_directory . 'autorun.php');

class all_tests extends TestSuite {
   function __construct() {
      parent::__construct();

      $this->addFile('some_test.php');
      $this->collect(__DIR__,
         new SimplePatternCollector('/_test.php/'));
   }
}
?>

some_test.php:
<?php
require_once($simpletest_directory . 'autorun.php');

class some_test extends UnitTestCase {

   function test_example1() {
      echo "I am running!<br>\n";
      $this->assertTrue(true);
   }
}
?>

This outputs:
I am running!<br>\n
I am running!<br>\n
(it runs twice)

I changed the simpletest code to fix this bug for my project, and my
fix works for me. However, it's a really bad kludge, and since I'm not
familiar with the simpletest source code, there is also a chance that
my solution broke other intended functionality. Rather than using my
patch, it would be better to have the bug fixed by someone familiar
with the code, or at minimum to have my solution reviewed for
potential problems.

Thanks:
Lastly, thank you so much for helping to maintain such a nice testing
library. I tried PHPunit first because it seems to be more popular,
however I liked simpletest better mostly because I could drop the
required files right into my project, without having to do any
installations into my web server or use Pear or anything like that. I
like my libraries to be as simple and understandable as possible.

thank you,
Blake