[Firebird-test] Re: [Firebird-admin] What does this mean?

[Mark O'D. <mar...@lu...>]

Ann Harrison wrote:

> At 07:55 AM 10/25/2000 +1100, Mark O'Donohue wrote:
> 
>>> It appears that we have the nightly test stuff which belonged to 
>>> engineering
>>> and generally worked.
>> 
>> 
>> Thanks for that, so this is kind of a smoke test, So what is in this 
>> other
>> stuff that takes months to run?
> 
> 
> The suite used by the QA group was full of test and result errors -
> bad design and bad initialization.  Each of those tests was run once
> for each server/client platform pair.  Each failure was researched and
> documented.  So, a test which printed 'today' in date format would be
> analyzed a dozen times, and each time reported as an error in test
> design - "test does not return a consistent answer."  Each analysis
> required reading the test, running it manually, reading the expected
> result, comparing it with the actual result both of the automated test
> and the manual test, and writing a little report.
> 
>> I presume a lot of it is manual, which explains the time required to
>> run it, and probably explains Inprises reluctance to run it again.
> 
> 
> There are some manual tests and some load tests that require special
> hardware.  You don't want to know my real opinion about all this.


I presume these are the tests then that require a good cleanup and 
rethink anyway.  I assume the chances of Inprise doing this are about zero. 

Do you think Inprise will ever invest time in rerunning this QA test 
suite again?

It's interesting, if this was one of the main cost items, I wonder how 
they would plan to go in selling it or use of it.  It would appear that 
if it requires a lot of effort to implement then they wouldn't be able 
to sell it for too much.  I suppose if Jim is right and they effectively 
dump IB as soon as their current contracts wind up, then they'll try and 
sell it then.

How would you rate it, I would guess that the TCS suite gets rid of the 
obvious bugs, but how much value does running the QA suite add to say 
running TCS on all client/server platforms, and putting in some 
volume/stress tests.


Cheers

Mark