Hi,
On 18 Dec 2002 at 23:19, Ignacio J. Ortega wrote:
> I've playing with great success with qmTest suite, and it's is really
> promising, a big step forward, thanks for the effort..
Nice to hear that :)
> I'm struggling testing FB 1.5, and i've found a number of failures
> related to the number of pages a empty databse has.. it seems fb1,5 has
> a little bigger footprint in pages ( maybe related to the new indexes in
> metatadata tables), so it seems this tests has been developed against
> fb1.0..
Yes, it was developed around 1.0, as I need to use stable engine as
reference platform.
> my question is, how it's supposussed the test database will be
> organized to test different fb versions?, or maybe we only consider the
> development branch under QA testing? ( so this finding is really a bug
> :)..
Well, that wasn't yet decided. TCS use different db records for platform
and version specific tests, so TCS will "check-out" tests according to
configuration on run. QMTest haven't (yet) such capability, so we have to
decide on "workaround". We can try to use organizational approach or try
to change QMTest to support platform/version tests/config like TCS. I'd
like go with test db organization rather than QMtest modification from
obvious reasons.
Use of CVS branch feature is not very good, because that impose a burden
to merge them often (both are supposed "stable").
I'd propose the next scenario:
- We can use stable and development trees for tests. Stable would be used
for current stable FB version (1.0 right now), development for current
development FB (1.5 right now).
- Stable tests would be used for maintenance release and as regression
basis for dev engine.
- Development tree would contain only adapted tests and tests for new
features. We can run stable and dev tests against dev engine and use
saved list of expectations for stable tests so test failure would be
treated as pass if it's expected to fail :) Dev suite should pass.
- When dev engine reach stable release, we'll merge dev to stable and
clear out dev for another round.
Of course, current directory structure of QMDB have to be changed, but
we'll still use"natural" test suites - subdirectories. So we should have
"stable" and "dev" subdirectories under QMDB, and "purpose"
subdirectories like "create database", "alter database" etc. beneath
them.
Comments ? Suggestions ?
Best regards
Pavel Cisar
http://www.ibphoenix.com
For all your upto date Firebird and
InterBase information
|
