Re: [Simpletest-support] Fail fast?

[Christoph B. <ber...@ne...>]

I'd prefer to have a configuration switch and if that's not possible to keep
the current behavior.

We've been using Simpletest very happily for three years to test a
moderately complex data collection website providing a nested set of forms
for each user.

The argument against fast fail: yesterday, I made some underlying changes in
the database - implementing InnoDB foreign key constraints, where before
we'd relied just on PHP logic. I ran the test suite, and saw a handful of
failures in one of the tests run "in the middle" of the suite. That was
annoying, but it was helpful for my time planning to see that it was just
part of one test file, rather than massive (or scattered) failures
throughout.

With a "fast fail" constraint, I wouldn't have been able to distinguish, up
front, between one failure that in effect hides dozens more (as I work
through the code fixing bugs) or, as in this case, a single failure that
really is the only failure in the test suite.

A "fast fail" option would be useful once I know the extent of the damage,
in that it perhaps speeds up each test run and certainly reduces clutter.
But my workaround is to to have separate test files, each relatively short,
and a "run-all.php" in the directory as well. Once I see the extent of the
problem via run-all, I can focus in on getting each individual test file to
pass.

Chris Berendes

On Wed, Jan 5, 2011 at 3:33 PM, Marcus Baker <ma...@la...> wrote:

> Hi...
>
> Solwitz, Benjamin wrote:
> > Is it possible to configure simpletest to fail fast, that is, after the
> first assertion failure, the rest of the test is skipped?
>
> You can throw an exception instead, but no, there is no fail fast option
> right now. I'm seriously thinking of switching to that behaviour as
> default though unless I get a lot of objections.
>
> >
> > Ben
>
> yours, Marcus
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment,
> and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Simpletest-support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpletest-support
>