Re: [Firebird-test] Firebird Testing - A plan for discussion

[Martijn T. <m.t...@up...>]

Hi all,

I offered testing some time ago and already had a few email with Mark and
Tord.

One of the points I very much dislike (and actually restrain me from
testing) is the need to install a (visual) C++ suite. This is a big no-no
for me... Besides the fact that I have zero C++ (alright, just a little)
knowledge, I do not want to install a complete development suite for testing
IB/Firebird.

I would like to see a set of scripts/application that runs a few
tests/durability tests/volume tests etc...

In general, I think that (mind you, I don't have knowledge about TCS):

- after every change, the programmer should think of what has changed
opposed to the code before the change
- if it was changed because of a bug:
  1) is the normal behaviour still available
  2) does the original 'buggy behaviour' still exists (probably not :))
  3) do the tests that test extraordinary situations still run
- new functionality:
  1) the programmer should make a detailed description of how this should
work
  2) make a test plan, with extraordinary situations, the normal behaviour
and possible buggy situations when changed (in the form: when I do this, the
output should be: <something>)
  3) test everything before declaring it 'working' code

Furthermore I think that:
1) nothing should be installed (except the IB/FB client)
2) there should be detailed instructions for volume testing as well as a
filling procedure
3) there should be detailed instructions considering the output of the tests
4) there should be an easy-to-use application/set of scripts that test all
or parts of the engine
5) there should be ways that the whole testing process can be devided into
parts that can be performed by several people (or groups of people) instead
of a single testbed for all tests
6) there should be applications for durability and load testing

Well, that's about it.

For starters, I can contribute with testing on Win2000, Linux (Suse 7.1)
(both AMD CPUs).


Comments anyone?

Martijn Tonies
InterBase Workbench - the developer tool for InterBase and Firebird
http://www.interbaseworkbench.com

Upscene Productions
http://www.upscene.com

"This is an object-oriented system.
If we change anything, the users object."