Re: [Firebird-test] BBT: Automated QA and TCS

[Pavel C. <pc...@us...>]

On 3 Jul 2002 at 15:31, Paul Reeves wrote:

> How would you see us testing DSQL? I'd trust the results of an ESQL test 
> over a DSQL test anyday. Writing good (I mean correct) DSQL tests is not 
> easy. At least gpre knows what it is doing.

AFAIK any db access layer that's in use today works over DSQL (IBO, IBX, 
ODBC, dbExpress, kinterbasdb, JDBC? etc.). So use of DSQL means that we 
can simply use one such library (in case of QMTest it's kinterbasdb). 
 
> This is not a reason to ignore testing DSQL, especially as gpre 
> generates undocumented api calls in some cases. But having played a 
> little with writing gpre scripts I think the learning curve is more 
> shallow. Test writers can concentrate on writing code that will 
> accomplish a test. Writing DSQL will turn into a test of the programmers 
> C/C++ abilities - at least in the beginning.

I think the opposite (in favour of DSQL), because with ESQL we have to 
use a C/C++ compiler, GPRE and other tools just to compile test. With 
DSQL, the db access layer could be part of Test harness, and test cases 
can use declarative approach instead procedural. Of course, some test are 
better written in ESQL/GPRE (like tests for events), but core SQL 
compliance tests could be pure declarations: sql command -> expected 
results.

> Have you seen QAT yet? (http://qat.sourceforge.com). It is another OS 
> test harness, this time written in Java. I haven't played with it 
> seriously but the fact that it is in Java makes it more interesting to me.

Java is also nice, as one from top priority requirements for test harness 
is portability, but it depends on working Java FB connectivity. Of 
course, we can use it at least for jca-jdbc driver tests. I'll take a 
look at QAT.
 
> My main objection (in fact my only objection) to QMTest is that it 
> requires Python. It is another thing to install and configure and it 
> quite probably requires learning a bit of python, too. That said, I'm 
> not stuck on this. A harness that is easily configurable and allows easy 
> extension addition to the test 'database' is the most important thing. 
> We need to reduce the barriers to test writing as much as possible.

Python is easy to setup and QMTest as well (just install them and you're 
ready). Many test cases doesn't really need any Python knowledge as I'd 
like provide support for declarative approach for test writers where 
possible. This is also one from main topics for discussion here: How 
common SQL compliance test should be written and what support is expected 
from TCS side to build them quickly without special knowledge and with 
minimum coding.

Best regards
Pavel Cisar
http://www.ibphoenix.com
For all your upto date Firebird and
InterBase information