RE: [Firebird-test] BBT: Automated QA and TCS

[Pavel C. <pc...@us...>]

Sean,

On 3 Jul 2002 at 22:08, Leyne, Sean wrote:

> Just as a reminder of a historical posting, Reed Mideke wrote:
> 
> > Emm, actually, embedded GDML (i.e. GPRE) is the gold standard.
> > isql only lets you test DSQL. 
> 
> Accordingly, perhaps we need to balance the use of GPRE based tests and
> DSQL/isql based test.
>
> Ideally, the GPRE test should cover the very basic elements of the
> database/engine and those features which can't be tested any other
> way. Once the basics have been tested then other tools (like isql) can
> be used to perform the reminder (majority) of the tests.  In the long
> run, these basic tests should change less frequently than the need to
> create new test to address DSQL issues. 

I agree with you that we'll need (at least some) ESQL tests, because not 
all functionality is exposed by DSQL, and significant engine parts and 
tools are written in it. We should tests these "bits" that cannot be 
tested by other ways.

> To address the deployment issues, installing/configuring a C/C++
> compiler, we could build binary packages which could be installed by
> users/testers (this is in addition to the test package -- no CVS access
> should be required).  Thus, only 1 person would need to compile and
> build the binary package, and another to build/post the package of
> testing scripts. 
>
> Then, the thousands of users can contribute to the overall testing
> process. 

I'm not sure if 1000+1 users running _exactly_ the same tests would 
assure better quality than 1-3 (per platform). I'd prefer if those 1000+1 
users would test FB in real-world applications.

Best regards
Pavel Cisar
http://www.ibphoenix.com
For all your upto date Firebird and
InterBase information