Re: [glean-dev] Scissor test

[Allen A. <ak...@po...>]

On Sat, Feb 03, 2001 at 11:41:10AM +1100, Gareth Hughes wrote:
| 
| In terms of floating-point color, Brian Paul made a pretty strong point
| that black should be black and white should be white (ie 0.0 and 1.0
| exactly), so I followed his test implementations and used those values
| as well.

He's right, they really should be 0 and 1 exactly, but if they're not
then all the tests depending on that behavior will fail.  Sometimes
this allows the true cause of a failure to hide behind a bunch of
spurious failures.  Particularly when the suite grows to include
hundreds of tests, it's hard for the driver developer to know which
failure to debug first.

I've made some provisions for dealing with this, but haven't discussed
them with anyone.  My theory was that we could write some simple tests
(e.g.  for readback capability).  The more sophisticated tests would
check the results from the simpler tests to verify that they passed
rather than making assumptions about which functions really work.

Basically each test would check the hasRun flag in the tests on which
it depends.  If hasRun is false, invoke the prerequisite test's run()
function.  This will propagate recursively until all the necessary
results have been accumulated, and the hasRun flag will be used to
make sure no test is run more than once.  Then check the result
structures (which have been carefully left in memory for just this
purpose) to make sure all the prerequisite functions actually work. 
Assuming everything is OK, then go ahead and run the requested test.

Putting together an example of how to do this is on my to-do list, but
it seemed low priority.  Maybe it should be higher.

Allen