[Firebird-test] WBT: Development test cases

[Pavel C. <pc...@us...>]

Hi all,

Developers often use home-made test applications to test basic 
functionality of parts they're working on. They are not always developed 
nor complete, but if they are developed, they are not deleted afterwards 
and could be reused in test system or by QA group. They don't need to be 
so extreme like in eXtreme programming, and developed *before* any real 
development take place (although XP approach would be nice :), but *any* 
such application has its value.  

Pro's and Con's: 

This technique impose significant burden to core developers, if they're 
required to write test cases along their code. Of course, these tests 
could be written by others, but this task require intimate knowledge at 
least tested piece of source code. Tests must be written in certain 
standard way, to assure seamless integration into automatic test system. 
Test applications that would test correctness of various low-level 
functions or modules/subsystems can quickly uncover bugs that are 
otherwise hard to catch at higher level, especially those that are 
related to parameter values and execution flow handling (if/else/switch). 
From my experience, around 50% of all bugs are related to bad 
if/else/switch handling (undetected/badly handled values etc.). These 
tests can also work as a specification for produced code.   

Implementation:

We can initiate a sub-project in Firebird to build such test programs for 
selected libraries and separate subsystems (like memory management, 
optimizer, parser etc.)  and/or declare a policy for core developers 
about when and how write such test cases. The basic prerequisite to start 
it successfully is a good source code documentation (file dependencies, 
list of routines and symbols etc.) to allow a seasoned C/C++ developers 
to get on track quickly. We have at (least basic) such documentation, and 
for rest it's exactly where GLOBAL tagging systems would come handy.  

Current status:

We don't have anything like this for Firebird right now.

Questions for you:
1) Do you think that we should use low-level test cases ? 


2) If yes, what parts of core engine do you suggest to be a subject of 
this testing ?


3) If yes, what procedure and/or toolkit do you recommend ?


Your comments would be greatly appreciated.
Best regards
Pavel Cisar
http://www.ibphoenix.com
For all your upto date Firebird and
InterBase information