Re: [Firebird-test] Testing FB1

[John B. <bel...@cs...>]

Sean,

On Friday, August 3, 2001, at 07:14 AM, Leyne, Sean wrote:

> John,
>
>> -----Original Message-----
>> From: John Bellardo [mailto:bel...@cs...]
>> Sent: Friday, August 03, 2001 1:36 AM
>
> ...
>
>>>
>>> An alternative track that also deserves consideration, is (as Sean
>>> points out) adjusting the TCS test tool parser to run from
>> text files.
>>
>> The hardest part will be handling the test hierarchy that TCS has.
>> There are global tests, platform tests...
>
> I think the solution is in extending the TCS suite to allow for scripts
> to point to (include?) other scripts by name, which could point ot other
> scripts...  We could call these scripts groups (with an extension GRP).
> That way we could use the actual test scripts (we need a unique
> extension for those - TST?) in any combination we wanted --
> Nightly_test.grp, Full_test.grp, etc.

I was talking about the tests themselves.  Currently there are two 
levels in TCS, global and local.  The global database (gtcs.gdb) 
contains tests and other configuration information that is common across 
all platforms.  The local database (ltcs.gdb, a sym. link to the correct 
platform db) contains tests and configurations that override those in 
the global database.

In order to move TCS out of the database and into flat files (and 
maintain correct functionality) we will need to duplicate this 
functionality.

Which brings me to my next question.  How much of the existing TCS 
functionality to we want to maintain?  There are a number of features 
that I have never used, but are there.  For example all the test 
failures are stored back into the local test database.

Do you want to move the boiler plates and environments out of TCS too?  
What is the motivation for moving to flat files?  They are easier to 
track in CVS and to change.  But moving all the test files and test 
names to flat file would address that.  We don't have to move 
*everything* out of the databases.

TCS is combersome to use, and a replacement that allowed us (unix 
guys :-) to just run "make check" or "make certify" would be great.  But 
where do we draw the line between "improving" the existing, functional 
test program and working on the engine?

-John