firebird-test — Firebird engine testing, test flight/suite discussion/information

Re: [Firebird-test] FBT - firebird tests

[Tord H. <th...@ha...>]

Hi,

> I guess I'm a bit confused by the resistance of a database
> development project to using database technology.

The TCS needs a central database server which holds the global
test database. The local database file is (according to the
docs) only needed if the QA person needs to create a new test
and want do try something without screwing the main global
database.

Second I assume the majority of test runs in the borland QA
are tested against a test database server (remote). I dont
think the QA members installed an unstable version on there
workstations, when the TCS needs a database server to access
the test files.

Third maintaining the global test database is now impossible
with the current firebird structure. And no, I dont think
fetch from cvs, restore gdb, change test, backup gdb,
store to cvs is a reasonable way to create and maintain
tests.

Most people who want to help testing dont have multiple
computers or a client-server-setup like the QA office at
borland.

I have monitored the firebird progress since the first days
and in seven months only one (the generator test from FSG) 
test where added to the original TCS.

So there are NO scripts to test the new features of firebird.
NOBODY can test if the latest changes (e.g. metadata issues
from Pattrick Griffin, missing services entry fallback, 64 bit
io, 16k pages) workes, they can only test that the changes
dont break the old behavior.

And there are NO scripts to test the error fixes. The firebird
team (mainly Ann and Skywalker) have fixed a bunch of typing
error (like a == b insteed of a = b, or missing parentheses),
but there are no test scripts which failed before the fix
and pass now. So you just dont know if you fix it correctly
or screw up a side-effect of this typing error.

Last but not least, there are no test scripts which shows the
bug reports at Sourceforge. A metaseries SF_BUGS which
fails at current build but passes with firebird 1.0 would
be a very satisfied moment, wouldn't it.

For me this shows, that either nobody is willing to create
tests or nobody is able to create tests. I hope it's the
second one ;-)

Just thoughts,
Tord

Remark:
English is not my native language (hint: look at TLD), so
if some words sound insulting, it's only because these
words were the first ones in my dictionary <g>

Re: [Firebird-test] FBT - firebird tests

[<Sch...@t-...>]

"Ann W. Harrison" wrote:
> 
> At 03:40 PM 5/12/2001 +0200, Tord Hammer wrote:
> 
> >I suggest a new test systems called FBT ;-) which works
> >like the PostgresQL one. It would be strict script-based,
> >which means all tests and expected results are stored
> >in text files. Groups of tests (calles series in TCS) and
> >group of series (calles meta-series) would be text files
> >too.
> 
> I guess I'm a bit confused by the resistance of a database
> development project to using database technology.

Me too, maybe its just that these scripts from tcs are
too *n*x oriented. Or the front end is a bit outdated.
It's just plain text that is stored in the test db's.

IMHO we need 
1. an easy way to maintain the scripts under cvs control
2. a platform independent (modern) frontend to this stuff
3. a way to use this under MS Win_something 
   without Cygwin et al

And time to do this all :-(

This doesn't mean that I wouldn't be happy about any 
other solution that solves the problem. (especially
for the MS platforms)

So, never mind.


Frank
-- 
"Fascinating creatures, phoenixes. They can carry immensely heavy loads,
 their tears have healing powers and they make highly faithful pets."
         - J.K. Rowling
http://firebird.sourceforge.net

Re: [Firebird-test] FBT - firebird tests

[Ann W. H. <aha...@ib...>]

At 03:40 PM 5/12/2001 +0200, Tord Hammer wrote:

>I suggest a new test systems called FBT ;-) which works
>like the PostgresQL one. It would be strict script-based,
>which means all tests and expected results are stored
>in text files. Groups of tests (calles series in TCS) and
>group of series (calles meta-series) would be text files
>too.

I guess I'm a bit confused by the resistance of a database
development project to using database technology.


Regards,

Ann
www.ibphoenix.com
We have answers.

RE: [Firebird-test] FBT - firebird tests

[Pavel C. <pc...@at...>]

Hi all,

There was a talk about XP-like test suite some time ago. I think 
that it's the right way to go. For details, look at 
http://www.xprogramming.com/
There are numerous XP test frameworks for various languages. The 
only one thing that we have to solve is cross-platform compatibility 
issue. That mean that tests have to be developed in one from next 
languages: C/C++, Java, Python. Personally, I'd vote for Python.

Comments?

Regards
-- Pavel

RE: [Firebird-test] FBT - firebird tests

[Dmitry Y. <di...@ma...>]

Hi Tord,

> up to now the only way to test the changes in firebird
> is the old (summer 2000) test suite from borland (aka TCS).
>
> I suggest a new test systems called FBT ;-) which works
> like the PostgresQL one. It would be strict script-based,
> which means all tests and expected results are stored
> in text files. Groups of tests (calles series in TCS) and
> group of series (calles meta-series) would be text files
> too.
>
> In about two weeks I'll have enough spare time to start
> this project.
>
> Ideas, thoughts ?

Sounds good to me. I think it would be our another great step forward, so
feel free to start a stone rolling.

Cheers,
Dmitry

[Firebird-test] FBT - firebird tests

[Tord H. <th...@ha...>]

Hi,

up to now the only way to test the changes in firebird
is the old (summer 2000) test suite from borland (aka TCS).

I suggest a new test systems called FBT ;-) which works
like the PostgresQL one. It would be strict script-based,
which means all tests and expected results are stored
in text files. Groups of tests (calles series in TCS) and
group of series (calles meta-series) would be text files
too. 

In about two weeks I'll have enough spare time to start
this project.

Ideas, thoughts ?


Tord

Re: [Firebird-test] SQL test suite

[<Sch...@t-...>]

Martijn Tonies wrote:
> 
> Downloading right now :)
> 
> btw, any words on someone having a Win32 version of the TCS test suite for
> Firebird/InterBase?

Have you tried:
http://www.hammer-software.de/users/thammer/fb/tcs/

Frank

-- 
"Fascinating creatures, phoenixes. They can carry immensely heavy loads,
 their tears have healing powers and they make highly faithful pets."
         - J.K. Rowling
http://firebird.sourceforge.net

RE: [Firebird-test] SQL test suite

[Dmitry Y. <di...@ma...>]

Martijn,

> btw, any words on someone having a Win32 version of the TCS
> test suite for
> Firebird/InterBase?

Try to contact Tord Hammer. As far as I remember, he reported in the
fb-devel list that he's got the TCS win32 stuff working under Cygwin.

HTH,
Dmitry

Re: [Firebird-test] SQL test suite

[Martijn T. <m.t...@up...>]

Downloading right now :)


btw, any words on someone having a Win32 version of the TCS test suite for
Firebird/InterBase?

--
Martijn Tonies
InterBase Workbench - the developer tool for InterBase and Firebird
http://www.interbaseworkbench.com

Upscene Productions
http://www.upscene.com

"This is an object-oriented system.
If we change anything, the users object."



""Pavel Císaø"" <pc...@at...> wrote in message
news:3AFBEF9E.12331.149826B@localhost...
> Hi all,
>
> Anyone interested to give a try to NIST SQL test suite ?
> Downloadable at
>
> http://www.itl.nist.gov/div897/ctg/sql_form.htm
>
> Regards
> -- Pavel Cisar
>
> Firebird - The most addictive database
> http://firebird.sourceforge.net
>
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
>

[Firebird-test] SQL test suite

[<pc...@at...>]

Hi all,

Anyone interested to give a try to NIST SQL test suite ?
Downloadable at

http://www.itl.nist.gov/div897/ctg/sql_form.htm

Regards
-- Pavel Cisar

Firebird - The most addictive database
http://firebird.sourceforge.net

[Firebird-test] Fwd: [IB-Architect] Re: Query joins

[John B. <bel...@cs...>]

Someone should contact this guy.

-John

Begin forwarded message:

> From: ye...@ho...
> Date: Sun Apr 29, 2001  07:20:56 PM US/Pacific
> To: IB-...@ya...
> Subject: [IB-Architect] Re: Query joins
>
> Thanks for the replies.  It is a bit disappointing to hear what Mr
> Starkey had to say.  Just have to roll with it, then.  Sorry I do not
> have any experience with rivers, streams and tributraries (other than
> sinking in them), but I'll gladly volunteer as a beta tester for
> Firebird development.
> ....

[Firebird-test] cancel <3AE8C071.9CA7BA76@autoparts.sk>

[Radovan B. <ra...@au...>]

This message was cancelled from within Mozilla.

Re: [Firebird-test] Stress Testing Firebird

[Martijn T. <m.t...@up...>]

Hi,

> I'd one of those too! <grin>
> 
> As I understand it, the instructions are platforms specific (certainly
> Windows vs *nix).
> 
> Which platform are you looking for?

Windows (2000) and Linux...

Martijn  Tonies

> 
> Hi Luis,
> 
> could you send me (detailed) instruction on how to get that TCS thing
> going?
> 
> --
> Martijn Tonies
> InterBase Workbench - the developer tool for InterBase and Firebird
> http://www.interbaseworkbench.com
> 
> Upscene Productions
> http://www.upscene.com
> 
> "This is an object-oriented system.
> If we change anything, the users object."
> 
> 
> 
> "Luis Feliz" <lou...@no...> wrote in message
> news:3AE...@no......
> > I would be interested in participating. I am trying to get TCS
> installed
> > on my Linux box. I would be great is a tool existed written Delphi
> that
> > allows the scripts to be store and executed from along with a way of
> > checking results and timing them.
> >
> > I think what is needed is an overall plan detailing what kings of
> tests
> > are needed or a pre-defined database with wich to test against. That
> way
> > new tests can be created against a known DB.
> >
> > - Lou
> >
> >
> >
> > Mark O'Donohue wrote:
> > >
> > > [ This time for sure :-). again, sorry about multiple mutiple post
> here,
> > > I made a spelling mistake, in the link to atkin newsserver]
> > >
> > > Hi All
> > >
> > > I have been asked quite a few times how people can help firebird,
> well I
> > > think we thought of a small task that everyone/anyone can do to help
> > > Firebird advance to the next level.
> > >
> > > We at Firebird are getting close to producing our first production
> > > release and are advancing into the Beta test stage.  In order to
> > > progress Firebird from beta into a production release  Firebird has
> to
> > > successfully pass through a "stress and volume" testing cycle and
> > > currently we don't have a "stress and volume" test suite.
> > >
> > > Now when I think of "stress and volume" I naturally think of the IB
> > > community in general, so I thought that might be a good place to ask
> for
> > > some example stress and volume tests.
> > >
> > > Since you are all veteran IB/FB RDB users I was thinking we could
> start
> > > the "stress and volume" ball rolling by asking for examples of your
> > > meanest, toughest and dirtiest database scripts plus simple
> supporting
> > > programs where required.
> > >
> > > We also have to think of a simple supporting infrastructure that
> will
> > > run of the stress tests on a number of platforms.
> > >
> > > The aim is to build a simple test suite and simple support program
> that
> > > we can then collectively pass around and run the upcoming beta
> > > version of Firebird on the huge variety of available platforms and
> get
> > > some feedback.
> > >
> > > I think the initially the discussion should be in these newsgroups,
> but
> > > that when we start running or distributing the scripts probably the
> > > fir...@li... (also available as news from
> > > news.atkin.com) will be a good place to focus the testing effort.
> > >
> > > So if you would like to support the Firebird project because your a
> user
> > > or developer that is likely to benefit from the project, your
> > > opportunity to contribute has arrived.
> > >
> > > So who has some "real world scripts" or ideas on what would look
> good in
> > > a stress testing suite :-) ?
> > >
> > > Cheers
> > >
> > > Mark
> > >
> > > --
> > > Your database needs YOU!
> > > http://firebird.sourceforge.net
> > >
> > > _______________________________________________
> > > Firebird-test mailing list
> > > Fir...@li...
> > > http://lists.sourceforge.net/lists/listinfo/firebird-test
> > >
> > > _______________________________________________
> > > Firebird-test mailing list
> > > Fir...@li...
> > > http://lists.sourceforge.net/lists/listinfo/firebird-test
> >
> > _______________________________________________
> > Firebird-test mailing list
> > Fir...@li...
> > http://lists.sourceforge.net/lists/listinfo/firebird-test
> >
> 
> 
> 
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
> 
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
> 

RE: [Firebird-test] Stress Testing Firebird

[Leyne, S. <sl...@at...>]

Martijn,

I'd one of those too! <grin>

As I understand it, the instructions are platforms specific (certainly
Windows vs *nix).

Which platform are you looking for?


Sean

 -----Original Message-----
From: 	Martijn Tonies [mailto:m.t...@up...] 
Sent:	Friday, April 27, 2001 2:07 PM
To:	fir...@li...
Subject:	Re: [Firebird-test] Stress Testing Firebird

Hi Luis,

could you send me (detailed) instruction on how to get that TCS thing
going?

--
Martijn Tonies
InterBase Workbench - the developer tool for InterBase and Firebird
http://www.interbaseworkbench.com

Upscene Productions
http://www.upscene.com

"This is an object-oriented system.
If we change anything, the users object."



"Luis Feliz" <lou...@no...> wrote in message
news:3AE...@no......
> I would be interested in participating. I am trying to get TCS
installed
> on my Linux box. I would be great is a tool existed written Delphi
that
> allows the scripts to be store and executed from along with a way of
> checking results and timing them.
>
> I think what is needed is an overall plan detailing what kings of
tests
> are needed or a pre-defined database with wich to test against. That
way
> new tests can be created against a known DB.
>
> - Lou
>
>
>
> Mark O'Donohue wrote:
> >
> > [ This time for sure :-). again, sorry about multiple mutiple post
here,
> > I made a spelling mistake, in the link to atkin newsserver]
> >
> > Hi All
> >
> > I have been asked quite a few times how people can help firebird,
well I
> > think we thought of a small task that everyone/anyone can do to help
> > Firebird advance to the next level.
> >
> > We at Firebird are getting close to producing our first production
> > release and are advancing into the Beta test stage.  In order to
> > progress Firebird from beta into a production release  Firebird has
to
> > successfully pass through a "stress and volume" testing cycle and
> > currently we don't have a "stress and volume" test suite.
> >
> > Now when I think of "stress and volume" I naturally think of the IB
> > community in general, so I thought that might be a good place to ask
for
> > some example stress and volume tests.
> >
> > Since you are all veteran IB/FB RDB users I was thinking we could
start
> > the "stress and volume" ball rolling by asking for examples of your
> > meanest, toughest and dirtiest database scripts plus simple
supporting
> > programs where required.
> >
> > We also have to think of a simple supporting infrastructure that
will
> > run of the stress tests on a number of platforms.
> >
> > The aim is to build a simple test suite and simple support program
that
> > we can then collectively pass around and run the upcoming beta
> > version of Firebird on the huge variety of available platforms and
get
> > some feedback.
> >
> > I think the initially the discussion should be in these newsgroups,
but
> > that when we start running or distributing the scripts probably the
> > fir...@li... (also available as news from
> > news.atkin.com) will be a good place to focus the testing effort.
> >
> > So if you would like to support the Firebird project because your a
user
> > or developer that is likely to benefit from the project, your
> > opportunity to contribute has arrived.
> >
> > So who has some "real world scripts" or ideas on what would look
good in
> > a stress testing suite :-) ?
> >
> > Cheers
> >
> > Mark
> >
> > --
> > Your database needs YOU!
> > http://firebird.sourceforge.net
> >
> > _______________________________________________
> > Firebird-test mailing list
> > Fir...@li...
> > http://lists.sourceforge.net/lists/listinfo/firebird-test
> >
> > _______________________________________________
> > Firebird-test mailing list
> > Fir...@li...
> > http://lists.sourceforge.net/lists/listinfo/firebird-test
>
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
>



_______________________________________________
Firebird-test mailing list
Fir...@li...
http://lists.sourceforge.net/lists/listinfo/firebird-test

Re: [Firebird-test] Stress Testing Firebird

[Martijn T. <m.t...@up...>]

Hi Luis,

could you send me (detailed) instruction on how to get that TCS thing going?

--
Martijn Tonies
InterBase Workbench - the developer tool for InterBase and Firebird
http://www.interbaseworkbench.com

Upscene Productions
http://www.upscene.com

"This is an object-oriented system.
If we change anything, the users object."



"Luis Feliz" <lou...@no...> wrote in message
news:3AE...@no......
> I would be interested in participating. I am trying to get TCS installed
> on my Linux box. I would be great is a tool existed written Delphi that
> allows the scripts to be store and executed from along with a way of
> checking results and timing them.
>
> I think what is needed is an overall plan detailing what kings of tests
> are needed or a pre-defined database with wich to test against. That way
> new tests can be created against a known DB.
>
> - Lou
>
>
>
> Mark O'Donohue wrote:
> >
> > [ This time for sure :-). again, sorry about multiple mutiple post here,
> > I made a spelling mistake, in the link to atkin newsserver]
> >
> > Hi All
> >
> > I have been asked quite a few times how people can help firebird, well I
> > think we thought of a small task that everyone/anyone can do to help
> > Firebird advance to the next level.
> >
> > We at Firebird are getting close to producing our first production
> > release and are advancing into the Beta test stage.  In order to
> > progress Firebird from beta into a production release  Firebird has to
> > successfully pass through a "stress and volume" testing cycle and
> > currently we don't have a "stress and volume" test suite.
> >
> > Now when I think of "stress and volume" I naturally think of the IB
> > community in general, so I thought that might be a good place to ask for
> > some example stress and volume tests.
> >
> > Since you are all veteran IB/FB RDB users I was thinking we could start
> > the "stress and volume" ball rolling by asking for examples of your
> > meanest, toughest and dirtiest database scripts plus simple supporting
> > programs where required.
> >
> > We also have to think of a simple supporting infrastructure that will
> > run of the stress tests on a number of platforms.
> >
> > The aim is to build a simple test suite and simple support program that
> > we can then collectively pass around and run the upcoming beta
> > version of Firebird on the huge variety of available platforms and get
> > some feedback.
> >
> > I think the initially the discussion should be in these newsgroups, but
> > that when we start running or distributing the scripts probably the
> > fir...@li... (also available as news from
> > news.atkin.com) will be a good place to focus the testing effort.
> >
> > So if you would like to support the Firebird project because your a user
> > or developer that is likely to benefit from the project, your
> > opportunity to contribute has arrived.
> >
> > So who has some "real world scripts" or ideas on what would look good in
> > a stress testing suite :-) ?
> >
> > Cheers
> >
> > Mark
> >
> > --
> > Your database needs YOU!
> > http://firebird.sourceforge.net
> >
> > _______________________________________________
> > Firebird-test mailing list
> > Fir...@li...
> > http://lists.sourceforge.net/lists/listinfo/firebird-test
> >
> > _______________________________________________
> > Firebird-test mailing list
> > Fir...@li...
> > http://lists.sourceforge.net/lists/listinfo/firebird-test
>
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
>

Re: [Firebird-test] Stress Testing Firebird

[Luis F. <lou...@no...>]

I would be interested in participating. I am trying to get TCS installed
on my Linux box. I would be great is a tool existed written Delphi that
allows the scripts to be store and executed from along with a way of
checking results and timing them. 

I think what is needed is an overall plan detailing what kings of tests
are needed or a pre-defined database with wich to test against. That way
new tests can be created against a known DB. 

- Lou



Mark O'Donohue wrote:
> 
> [ This time for sure :-). again, sorry about multiple mutiple post here,
> I made a spelling mistake, in the link to atkin newsserver]
> 
> Hi All
> 
> I have been asked quite a few times how people can help firebird, well I
> think we thought of a small task that everyone/anyone can do to help
> Firebird advance to the next level.
> 
> We at Firebird are getting close to producing our first production
> release and are advancing into the Beta test stage.  In order to
> progress Firebird from beta into a production release  Firebird has to
> successfully pass through a "stress and volume" testing cycle and
> currently we don't have a "stress and volume" test suite.
> 
> Now when I think of "stress and volume" I naturally think of the IB
> community in general, so I thought that might be a good place to ask for
> some example stress and volume tests.
> 
> Since you are all veteran IB/FB RDB users I was thinking we could start
> the "stress and volume" ball rolling by asking for examples of your
> meanest, toughest and dirtiest database scripts plus simple supporting
> programs where required.
> 
> We also have to think of a simple supporting infrastructure that will
> run of the stress tests on a number of platforms.
> 
> The aim is to build a simple test suite and simple support program that
> we can then collectively pass around and run the upcoming beta
> version of Firebird on the huge variety of available platforms and get
> some feedback.
> 
> I think the initially the discussion should be in these newsgroups, but
> that when we start running or distributing the scripts probably the
> fir...@li... (also available as news from
> news.atkin.com) will be a good place to focus the testing effort.
> 
> So if you would like to support the Firebird project because your a user
> or developer that is likely to benefit from the project, your
> opportunity to contribute has arrived.
> 
> So who has some "real world scripts" or ideas on what would look good in
> a stress testing suite :-) ?
> 
> Cheers
> 
> Mark
> 
> --
> Your database needs YOU!
> http://firebird.sourceforge.net
> 
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
> 
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test

Re: [Firebird-test] Stress Testing Firebird

[Radovan B. <ra...@au...>]

Mark O'Donohue wrote:

> So if you would like to support the Firebird project because your a user
> or developer that is likely to benefit from the project, your
> opportunity to contribute has arrived.
>
> So who has some "real world scripts" or ideas on what would look good in
> a stress testing suite :-) ?

Hallo Mark,

I'm not sure if I'm on right place here, but:

I'm trying to port my company's information system (includinig clients,
invoices, stocks, e.t.c)
made in M$ Access foreground on M$ Access background (MDB files, app. 60 MB

of data) to Interbase or Firebird.

I'm trying several months, but I'm always meeting problems that kill me:

- first, it was transliteration errors (already fixed in new ODBC drivers)

now, I usually meet these problems:

- INNER JOINs, taking 10 seconds to run (the same statement with LEFT JOIN
   runs 0.005 sec.)   on IB 6.0.1. and more than 30 minutes on Firebird
0.9.4.
- I have a VIEW made of two tables and VIEW2 made of VIEW1 and another
table.
  On Access, this takes 23 seconds to run. On IB 6.0.1 it takes 20 minutes.

- Some other queries probably put IB in endless loop (I resigned after 30
minutes
   of waiting). And, I was trying statements with 2-4 tables (but, my
program has also
   statements with 10-12 tables)

I can send you these scripts, but probably they will not work on emtpy
tables. so I would send a
piece of my database (dont worry, it would be probably less than 1MB,
packed with
zip, arj, bzip2 or so.) - if you want.

NOTE: I would appreciate some actual beta-build of FB for Windows (newer
that
that from 28.12.2000), so I could make more tests at home. Now I'm working
with
IB 6.0.1 (I stopped using FB 0.9.4, because it was more buggy than IB 6.0.1
for me)

(Please reply me also to ra...@au..., I'm not sure when I come to
read this
mailinglist again)

PS. Also, there are several missing features in IB / FB that are important
for me, that I will
probably discuss later in egoups.ib-priorities.

--

   Radovan Bukoci @ home

Re: [Firebird-test] Stress Testing Firebird

[Konstantin K. <klk...@ns...>]

Hi !
I suggest to use Bill Karwin perl to testing
Test may consist some fork() to emulate multi users envaronment and
all possible in this module database operation.
Some kind of this test I use to testing recentry installed FireBird servers.
Konstantin

[Firebird-test] Stress Testing Firebird

[Mark O'D. <mar...@lu...>]

[ This time for sure :-). again, sorry about multiple mutiple post here, 
I made a spelling mistake, in the link to atkin newsserver]


Hi All

I have been asked quite a few times how people can help firebird, well I
think we thought of a small task that everyone/anyone can do to help
Firebird advance to the next level.

We at Firebird are getting close to producing our first production
release and are advancing into the Beta test stage.  In order to
progress Firebird from beta into a production release  Firebird has to
successfully pass through a "stress and volume" testing cycle and
currently we don't have a "stress and volume" test suite.

Now when I think of "stress and volume" I naturally think of the IB
community in general, so I thought that might be a good place to ask for
some example stress and volume tests.

Since you are all veteran IB/FB RDB users I was thinking we could start
the "stress and volume" ball rolling by asking for examples of your
meanest, toughest and dirtiest database scripts plus simple supporting
programs where required.

We also have to think of a simple supporting infrastructure that will
run of the stress tests on a number of platforms.

The aim is to build a simple test suite and simple support program that
we can then collectively pass around and run the upcoming beta
version of Firebird on the huge variety of available platforms and get
some feedback.

I think the initially the discussion should be in these newsgroups, but
that when we start running or distributing the scripts probably the
fir...@li... (also available as news from
news.atkin.com) will be a good place to focus the testing effort.

So if you would like to support the Firebird project because your a user
or developer that is likely to benefit from the project, your
opportunity to contribute has arrived.

So who has some "real world scripts" or ideas on what would look good in
a stress testing suite :-) ?


Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net



_______________________________________________
Firebird-test mailing list
Fir...@li...
http://lists.sourceforge.net/lists/listinfo/firebird-test

[Firebird-test] Stress Testing Firebird

[Mark O'D. <mar...@lu...>]

Hi All

(sorry about multiple post here, it's the same message as before, I am 
just checking it gets to the atkin newsserver).

I have been asked quite a few times how people can help firebird, well I
think we thought of a small task that everyone/anyone can do to help
Firebird advance to the next level.

We at Firebird are getting close to producing our first production
release and are advancing into the Beta test stage.  In order to
progress Firebird from beta into a production release  Firebird has to
successfully pass through a "stress and volume" testing cycle and
currently we don't have a "stress and volume" test suite.

Now when I think of "stress and volume" I naturally think of the IB
community in general, so I thought that might be a good place to ask for
some example stress and volume tests.

Since you are all veteran IB/FB RDB users I was thinking we could start
the "stress and volume" ball rolling by asking for examples of your
meanest, toughest and dirtiest database scripts plus simple supporting
programs where required.

We also have to think of a simple supporting infrastructure that will
run of the stress tests on a number of platforms.

The aim is to build a simple test suite and simple support program that
we can then collectively pass around and run the upcoming beta
version of Firebird on the huge variety of available platforms and get
some feedback.

I think the initially the discussion should be in these newsgroups, but
that when we start running or distributing the scripts probably the
fir...@li... (also available as news from
news.atkin.com) will be a good place to focus the testing effort.

So if you would like to support the Firebird project because your a user
or developer that is likely to benefit from the project, your
opportunity to contribute has arrived.

So who has some "real world scripts" or ideas on what would look good in
a stress testing suite :-) ?


Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net

[Firebird-test] Stress Testing Firebird

[Mark O'D. <mar...@lu...>]

Hi All

Firstly sorry about the cross post, but I wanted to make sure I reached
the right audience.


I have been asked quite a few times how people can help firebird, well I
think we thought of a small task that everyone/anyone can do to help
Firebird advance to the next level.

We at Firebird are getting close to producing our first production
release and are advancing into the Beta test stage.  In order to
progress Firebird from beta into a production release  Firebird has to
successfully pass through a "stress and volume" testing cycle and
currently we don't have a "stress and volume" test suite.

Now when I think of "stress and volume" I naturally think of the IB
community in general, so I thought that might be a good place to ask for
some example stress and volume tests.

Since you are all veteran IB/FB RDB users I was thinking we could start
the "stress and volume" ball rolling by asking for examples of your
meanest, toughest and dirtiest database scripts plus simple supporting
programs where required.

We also have to think of a simple supporting infrastructure that will
run of the stress tests on a number of platforms.

The aim is to build a simple test suite and simple support program that
we can then collectively pass around and run the upcoming beta
version of Firebird on the huge variety of available platforms and get
some feedback.

I think the initially the discussion should be in these newsgroups, but
that when we start running or distributing the scripts probably the
fir...@li... (also available as news from
news.atkin.com) will be a good place to focus the testing effort.

So if you would like to support the Firebird project because your a user
or developer that is likely to benefit from the project, your
opportunity to contribute has arrived.

So who has some "real world scripts" or ideas on what would look good in
a stress testing suite :-) ?


Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net

[Firebird-test] Stress Testing Firebird

[Mark O'D. <mar...@lu...>]

Hi All

Firstly sorry about the cross post, but I wanted to make sure I reached 
the right audience.


I have been asked quite a few times how people can help firebird, well I 
think we thought of a small task that everyone/anyone can do to help 
Firebird advance to the next level.

We at Firebird are getting close to producing our first production
release and are advancing into the Beta test stage.  In order to
progress Firebird from beta into a production release  Firebird has to
successfully pass through a "stress and volume" testing cycle and
currently we don't have a "stress and volume" test suite.

Now when I think of "stress and volume" I naturally think of the IB 
community in general, so I thought that might be a good place to ask for 
some example stress and volume tests.

Since you are all veteran IB/FB RDB users I was thinking we could start
the "stress and volume" ball rolling by asking for examples of your
meanest, toughest and dirtiest database scripts plus simple supporting
programs where required.

We also have to think of a simple supporting infrastructure that will
run of the stress tests on a number of platforms.

The aim is to build a simple test suite and simple support program that
we can then collectively pass around and run the upcoming beta
version of Firebird on the huge variety of available platforms and get
some feedback.

I think the initially the discussion should be in these newsgroups, but 
that when we start running or distributing the scripts probably the 
fir...@li... (also available as news from 
news.atkin.com) will be a good place to focus the testing effort.

So if you would like to support the Firebird project because your a user 
or developer that is likely to benefit from the project, your 
opportunity to contribute has arrived.

So who has some "real world scripts" or ideas on what would look good in 
a stress testing suite :-) ?


Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net

[Firebird-test] Re: [Firebird-admin] Testing program

[Mark O'D. <mar...@lu...>]

Hi Sean

(it was a good idea to cc, this to the firebird-test group)

How do you feel about you (Sean) taking on the role of keeping abrest of 
and coordinating the feedback we get when we open this discussion up to 
a larger audience.  (by way of a bribe, can I offer you - that if you 
accept, I'll use a 1.0 Beta1 tag, on the release :-).


I think the discussion we are having here is probably better done in 
ibdi/firebird-devel and firebird-test with those we announce it, since 
we're asking them to do the work, we should give them a say in what they 
write it in etc.

It's not much point wasting our breath out here, since we'd have to 
peddle the same arguments again when we talk about it in the wild.  If 
Sean can commit to keeping abrest of how the discussion goes, and we're 
agreed that we want to push ahead.

How about I release my "call to arms" (with changes as recommended) and 
we announce it in firrebird-devel/firebird-test/ibdi,  say we focus 
discussion in firebird-test, (but to get coverage expect people to also 
discuss it in own groups a bit. 

Im happy to start it now, if Sean's willing.

Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net

[Firebird-test] Testing program

[Leyne, S. <sl...@at...>]

All,

I think we need to step back from the C++ / Delphi/Kylix debate (one
which will have no end) and review the requirements for the testing
solution.  (This discussion also really belongs in the testing list)

The basic requirements for the engine/API testing programs should be:

1)	The test scripts and the expected results need to be stored
outside of any database in a text (or xml) format.  The file format
should include a header which explains the nature of the test.  (You
would no longer need to have FB installed in order to test FB!)
 
2)	The testing scripts must support the ability to define groups of
tests (another script file in the scripts subdirectory).  The group
script would contain special tags pointing to the scripts to be
executed.  That way the NIGHTLY.GRP file would be a group test script
which could be run against the nightly builds.

3)	The actual testing program(s) should be able to run from any
client platform against any server platform.

4)	Wherever possible, binaries of all testing programs should be
provided to make the task of running the tests as easy as possible for
all testers. 

5)	Although the testing scripts would become part of the CVS tree,
the scripts and expected results need to be available as a separate ZIP
or TAR download.  This combined with the testing binaries will allow for
anybody to download the program binary of their choice and the scripts
and be testing within minutes.

6)	All testing programs need to be able to produce identical
(byte-level duplicated) results - i.e. a C++ program should produce the
exact same result file as a Delphi/Kylix program.  So, if your "visual
inclined" (like me) and hate command line utilities, you can run the
Windows GUI version of the testing program and still be able to know
that the test were successful.

Other requirements will apply to the other testing programs which will
be required for ODBC, JDBC, the new Java Client etc.  but this should be
the start of something we all agree on.


Sean

[Firebird-test] Stress testing FB builds

[<bel...@cs...>]

Hi,
	Are there any thoughts on how to stress test builds?  I personally 
don't have enough machines at my disposal to do a good job.  Should we 
make a semi-formal process where a "victim" build can be made accessible 
and synchronized stress tests performed?  Or does this fall in the too 
insecure/maybe someday category?

-John