Re: [Firebird-test] BBT: Automated QA and TCS

[Paul R. <pr...@ib...>]

Pavel Cisar wrote:


> Tests mostly use ESQL for access to the database, 
> instead of DSQL which is prevailing approach today. 

How would you see us testing DSQL? I'd trust the results of an ESQL test 
over a DSQL test anyday. Writing good (I mean correct) DSQL tests is not 
easy. At least gpre knows what it is doing.

This is not a reason to ignore testing DSQL, especially as gpre 
generates undocumented api calls in some cases. But having played a 
little with writing gpre scripts I think the learning curve is more 
shallow. Test writers can concentrate on writing code that will 
accomplish a test. Writing DSQL will turn into a test of the programmers 
C/C++ abilities - at least in the beginning.

I think one of the goals of the QA group is to create an environment 
where tests can be created quickly and easily by many developers. ESQL 
makes this a lot easier.


> We'll need more friendly TCS. Open source QMTest (www.codesourcery.com) 
> is an option, but you can suggest other tools (including home-grown) as 
> well.


Have you seen QAT yet? (http://qat.sourceforge.com). It is another OS 
test harness, this time written in Java. I haven't played with it 
seriously but the fact that it is in Java makes it more interesting to me.

My main objection (in fact my only objection) to QMTest is that it 
requires Python. It is another thing to install and configure and it 
quite probably requires learning a bit of python, too. That said, I'm 
not stuck on this. A harness that is easily configurable and allows easy 
extension addition to the test 'database' is the most important thing. 
We need to reduce the barriers to test writing as much as possible.


Paul
-- 

Paul Reeves
http://www.ibphoenix.com
Supporting users of Firebird and InterBase