firebird-test — Firebird engine testing, test flight/suite discussion/information

[Firebird-test] Stress testing

[John B. <bel...@cs...>]

FYI,
	About a month ago I put a very basic stress testing program in my 
sandbox (interbase/firebird/bellardo/stress).  I'm sure it is grossly 
inadequate, but it provides something to mess around with.  Just wanted 
to make sure people know it is there, now that we are thinking more 
along those lines.

-John

Re: [Firebird-test] Does anyone have a script to load > 2gig database

[David J. <dav...@di...>]

Hi,
this probably isn't the quickest to setup, however the testUseBlob in
org/firebirdsql/jca/TestBlob.java from client-java inserts a large blob.
(size is bloblength variable)  You could run it in a loop or after the
first record call 

insert into T1 (select t.C1 + x, t.C2 from T1 t where t.C1 < x)
where x is a power of 2

a few times.

If you actually want to try this comment out tearDown so the table doesn't
get dropped, and comment out all but one test.

In the ant script, modify the junit target to run only TestBlob, and run
./build.sh junit

david jencks


On 2001.08.05 23:00:56 -0400 Mark O'Donohue wrote:
> 
> Hi
> 
> I've built a linux 64bit io version of Firebird and want to test it,
> 
> Does anyone have (or can whip up) a small script that will loop round 
> and load up a database with > 2gig of data.  Some sort of blob script 
> that I can stick in a loop comes to mind, but any help welcome.
> 
> Cheers
> 
> Mark
> 
> 
> 
> _______________________________________________
> Firebird-test mailing list
> Fir...@li...
> http://lists.sourceforge.net/lists/listinfo/firebird-test
> 
> 

[Firebird-test] Does anyone have a script to load > 2gig database

[Mark O'D. <mar...@lu...>]

Hi

I've built a linux 64bit io version of Firebird and want to test it,

Does anyone have (or can whip up) a small script that will loop round 
and load up a database with > 2gig of data.  Some sort of blob script 
that I can stick in a loop comes to mind, but any help welcome.

Cheers

Mark

[Firebird-test] Testing limit with dejagnu

[Mark O'D. <mar...@lu...>]

Just to keep you up to date:

experimenting with dejagnu for limit,

Here is how the results of running dejagnu on my still simple test 
script for limit:



golem:~/src/interbase/TCS/new/test1> make check
runtest
WARNING: Couldn't find the global config file.
WARNING: Couldn't find tool init file
Test Run By odonohue on Mon Aug  6 02:09:48 2001
Native configuration is i686-pc-linux-gnu

        === isql tests ===

Schedule of variations:
    unix

Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file 
for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for 
target.
Using ./testsuite/config/unix.exp as tool-and-target-specific interface 
file.
Running ./testsuite/isql.test/isql_sample.exp ...
Running ./testsuite/isql.test/isql_sample2.exp ...
Running ./testsuite/isql.test/limit/isql_sample2.exp ...
FAIL: DSQL_Limit_skip3 (wrong count - 7 records found expected 6 )
FAIL: DSQL_Limit_skipFail2 (bad match)
FAIL: DSQL_Limit_firstFail1 (bad match)

        === isql Summary ===

# of expected passes        15
# of unexpected failures    3
/opt/interbase/bin/isql version ISQL Version: LI-T0.9.5.309 Firebird Beta1




It also leaves an isql.log file with a trace of the transcript as well. 
 And below is the current expect test script:

The way it works is that in a global spot (different for each platform) 
it starts up the sub task (isql basically), then the command

send "select X \r"

will send that string to the sub isql process, then

expect {
    <pattern> { actions }
}

are used to workout if the test succeeded.

The spawned child processes that are run can be anything, and you can 
spawn many of them so you can in an advanced script spawn shells to 
compile test programs or run two seperate tasks at once.

But also having a procedural language (basically tcl/expect) to specify 
the interaction makes it easy to write simple procedures to handle a 
large amount of the tricky pattern matching and error handling.

There are still issues to work out, like where to store the additional 
data files, and suchlike, but nothing really difficult.

Cheers

Mark




#_______________________________________________________________
#
# expectations that clean up in case of error. Note that `$test' is
# a purely local variable.
#
# The first of these is used to match any bad responses, and resynchronise
# things by finding a prompt. The second is a timeout error, and shouldn't
# ever be triggered.
#

expect_after {


    -re "\[^\n\r\]*$prompt$" {
        fail "${moduleName}_$test (bad match)"
        if { $verbose > 0 } {
            regexp ".*\r\n(\[^\r\n\]+)(\[\r\n\])+$prompt$" \
                    $expect_out(buffer) "" output
            send_user "\tUnmatched output: \"$output\"\n"
    }
    }
    timeout {
        fail "${moduleName}_$test (timeout)"
    }
}



# Build the database used for testing limit

proc runBuildLimitDb {} {

    global moduleName
    global prompt

    set test "buildLimitDB"
    send "[exec cat data/builddb.sql]\n"

    expect {
        "$prompt$"   { }
        "Statement failed" { fail "${moduleName}_$test (unable to build 
database)" }
    }
}


# Build the database used for testing limit

proc runConnectLimitDb {} {

    global moduleName
    global prompt
    global databaseName

    set test "connectLimitDB"
    send "connect $databaseName;\n"

    expect {
        "$prompt$"   { }
        "Statement failed" { fail "${moduleName}_$test (unable to 
connect to database)" }
        timeout             { fail "${moduleName}_$test (timeout)"  }
    }
}


#______________________________________________________________________________
# This process runs an sqlStmt and counts the number of results returned
# checking them against the match mattern
# success if the results equal fail if they do not.

proc runBasicLimitTest { test sqlStmt matchPattern expectNum } {
    global moduleName
    global prompt
    global verbose

    set counter 0
       
    send "$sqlStmt\r"

    expect {       
        -re "$matchPattern"  { set counter  [expr $counter + 1]    
            exp_continue
        }
        timeout             { fail "${moduleName}_$test (timeout)"  }
        -re "$prompt$"     { }
    }


# In the following expected counts is one more than real since we also match
# the value as passed in so we decrement it.
# maybe this could be done by setting echo off, or having a bit 
previously that
# sucks up until we the echoed sql stmt, or the first "=====" which is 
the title
# header.

    set counter [ expr $counter - 1 ]

    if { $counter == $expectNum  } {
        pass "${moduleName}_$test"
    } else {
        fail "${moduleName}_$test (wrong count - $counter records found 
expected $expectNum )"
    }

}

#______________________________________________________________________________
# We expect these stmt to get back 'Statment failed' result;

proc runMatchStmtTest { test sqlStmt matchPattern } {
    global moduleName
    global prompt
    global verbose

    set counter 0
       
    send "$sqlStmt\n"

    expect {       
        -re "$matchPattern"  {   
            pass "${moduleName}_$test"
        }
        timeout             { fail "${moduleName}_$test (timeout)"  }
#        -re "$prompt$"     {
#            fail "${moduleName}_$test (statement did not fail)"
#        }
    }

}



# Here are the tests
#


set moduleName DSQL_Limit
set databaseName "work/limits.gdb"

#send_user "$argv0\n"

set timeout 1000
#set verbose 1

runBuildLimitDb


set timeout 3
runConnectLimitDb

runBasicLimitTest "basicSelect" "select 'XXXX' from project;" "XXXX" 11
runBasicLimitTest "basic1" "select first 4 'XXXX' from project;" "XXXX" 4
runBasicLimitTest "basic2" "select first 4 skip 3 'XXXX' from project;" 
"XXXX" 4
runBasicLimitTest "basic4" "select first 5 skip 8 'XXXX' from project;" 
"XXXX" 4

runBasicLimitTest "skip1" "select skip 10 'XXXX' from project;" "XXXX" 2
runBasicLimitTest "skip2" "select skip 1 'XXXX' from project;" "XXXX" 11
runBasicLimitTest "skip3" "select skip 5 'XXXX' from project;" "XXXX" 7


runBasicLimitTest "first1" "select first 5 'XXXX' from project;" "XXXX" 5
runBasicLimitTest "first2" "select first 20 'XXXX' from project;" "XXXX" 11
runBasicLimitTest "first3" "select first 1 'XXXX' from project;" "XXXX" 1


# Some error cases that result in failure

runMatchStmtTest "skipFail1" "select skip -1 'XXXX' from project;" 
"Statement failed"
runMatchStmtTest "skipFail2" "select skip 0 'XXXX' from project;" 
"Statement failed"
runMatchStmtTest "skipFail3" "select skip 1.8 'XXXX' from project;" 
"Statement failed"


runMatchStmtTest "firstFail1" "select first -1 'XXXX' from project;" 
"Statement failed"
runMatchStmtTest "firstFail2" "select first 0 'XXXX' from project;" 
"Statement failed"
runMatchStmtTest "firstFail3" "select first 1.8 'XXXX' from project;" 
"Statement failed"








-- 
Your database needs YOU!
http://firebird.sourceforge.net
GPL: free software today - and still free tomorrow

Re: [Firebird-test] Testing FB1

[Mark O'D. <mar...@lu...>]

Leyne, Sean wrote:

>John,
>
>>-----Original Message-----
>>From: John Bellardo [mailto:bel...@cs...]
>>Sent: Friday, August 03, 2001 1:29 PM
>>
>
>...
>
I think testing will still involve many tools, TCS in some form I think 
we will still use, I'm not against using a database for some of the 
components, but flat files work better for CVS and for individuals 
wanting to look and do specific tests when they fail.  

I think somehting else (perhaps dejagnu) will also have something to 
offer, and finally:

We need something for load and bulk testing. looking for hangups, 
monitoring cpu that sort of thing.

Given we have limited resources I don't think it wise to embark on 
writing a new test tool - but some mods to the existing TCS one to make 
it run test cases from flat files would be useful.


BTW: I've done a few limit tests as a trial with dejagnu, I'll post them 
on the ftp site in a bit, I also want to run them on my NT system 
(vmware of course) to make sure they run, just so I dont upset the PC 
crowd :-).


Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net
GPL: free software today - and still free tomorrow

RE: [Firebird-test] Testing FB1

[Leyne, S. <sl...@at...>]

John,

> -----Original Message-----
> From: John Bellardo [mailto:bel...@cs...]
> Sent: Friday, August 03, 2001 1:29 PM

...

> I was talking about the tests themselves.  Currently there are two 
> levels in TCS, global and local.  The global database (gtcs.gdb) 
> contains tests and other configuration information that is 
> common across 
> all platforms.  The local database (ltcs.gdb, a sym. link to 
> the correct 
> platform db) contains tests and configurations that override those in 
> the global database.

My apologizes, I didn't get that from your original message.

Would storing the data in different directories solve the problem?

I thinking of something along the lines of the new directory structure
for build info for the various ports.

maybe something like:

	TCS\
		Environment\
		Global\
			gpre\
			SQL\
			ODBC\
			Java\
			...
		Local\
			{port}\
				gpre\
				SQL\
				ODBC\
				Java\
				...			
			...\
		Results\
		Expected\
		

Another option, a much better one (now that I think of it), use a single
XML file in which both the Global/Default and Local information is
stored, in that way all the data about a test (or template or
boilerplate) is in one place, while supporting local/port specific
versions as required.  The layout of the file could go something like:

	<XML>
		<Description>
		...
		</Description>
		<Script Platform = "All">
		...
		</Script>
		<Script Platform = "Linux">
		...
		</Script>
	</XML>

> Which brings me to my next question.  How much of the existing TCS 
> functionality to we want to maintain?  There are a number of features 
> that I have never used, but are there.  For example all the test 
> failures are stored back into the local test database.
> Do you want to move the boiler plates and environments out of 
> TCS too?  

Yep!

> What is the motivation for moving to flat files?  They are easier to 
> track in CVS and to change.  But moving all the test files and test 
> names to flat file would address that.  We don't have to move 
> *everything* out of the databases.

Well, I do.  It seems illogical for you to need to have the database
engine running, in order to test the database engine.  If there is a
failure in the test, is it due to the fact that the engine running the
test failed or did the testing program fail when updating the testing
data.

With respect to the templates and environments, etc...  I far as I can
tell, all this information is 'key' to the operation of the TCS system
as such, we need to ensure that all of this data is exposed and easily
available to users to review and change as appropriate.

Finally, in a closed product it's fine for all the testing information
to be hidden, in an open product the testing process/programs need to be
completely open/available for review.  Using the database only hides the
information.

> TCS is combersome to use, and a replacement that allowed us (unix 
> guys :-) to just run "make check" or "make certify" would be 
> great.  But 
> where do we draw the line between "improving" the existing, 
> functional 
> test program and working on the engine?

Unfortunately, when dealing with a very complex program (as FB is today)
its reliability becomes almost more important than the scope of the
functionality.

Think of it.

What would you think if a database that had been running for 2 months
stopped working after you did an update?

If that happended a couple of times, you'd probably look for a new
product and tell everybody who asked, that the product was a real piece
of s**t!

It's one thing to have to have to work around missing functionality but
if you can't depend on the existing functionality!

Bottom line is that we need to spend time working on the testing
programs/functions if we really expect to be able to move forward with
the engine.  

Currently we are working with the existing code and making medium level
(in a few cases) changes to the engine.  

When we start doing 'real' work on v2, it will involve much much more
serious changes, if we can't provide some guarantee about our work (i.e.
this has completely passed our QA testing) -- Who will use the product
in a production environment?

I realize that this will not make some people happy, having to work on
improving/changing TCS, but this is simply the fact of life -- it's not
glamorous, in fact it's kind of tedious.  But it is ABSOLUTELY
necessary.


Sean

[Firebird-test] Updated TCS

[John B. <bel...@cs...>]

Hi,
	I've updated the TCS global test database to reflect the latest 
changes to the source.  There are still a small number (< 5) of 
outstanding issues on Darwin.  At least one of them is platform 
specific.  I've also checked in the Darwin local test database for other 
Darwinites to use.

-John

Re: [Firebird-test] Testing FB1

[John B. <bel...@cs...>]

Sean,

On Friday, August 3, 2001, at 07:14 AM, Leyne, Sean wrote:

> John,
>
>> -----Original Message-----
>> From: John Bellardo [mailto:bel...@cs...]
>> Sent: Friday, August 03, 2001 1:36 AM
>
> ...
>
>>>
>>> An alternative track that also deserves consideration, is (as Sean
>>> points out) adjusting the TCS test tool parser to run from
>> text files.
>>
>> The hardest part will be handling the test hierarchy that TCS has.
>> There are global tests, platform tests...
>
> I think the solution is in extending the TCS suite to allow for scripts
> to point to (include?) other scripts by name, which could point ot other
> scripts...  We could call these scripts groups (with an extension GRP).
> That way we could use the actual test scripts (we need a unique
> extension for those - TST?) in any combination we wanted --
> Nightly_test.grp, Full_test.grp, etc.

I was talking about the tests themselves.  Currently there are two 
levels in TCS, global and local.  The global database (gtcs.gdb) 
contains tests and other configuration information that is common across 
all platforms.  The local database (ltcs.gdb, a sym. link to the correct 
platform db) contains tests and configurations that override those in 
the global database.

In order to move TCS out of the database and into flat files (and 
maintain correct functionality) we will need to duplicate this 
functionality.

Which brings me to my next question.  How much of the existing TCS 
functionality to we want to maintain?  There are a number of features 
that I have never used, but are there.  For example all the test 
failures are stored back into the local test database.

Do you want to move the boiler plates and environments out of TCS too?  
What is the motivation for moving to flat files?  They are easier to 
track in CVS and to change.  But moving all the test files and test 
names to flat file would address that.  We don't have to move 
*everything* out of the databases.

TCS is combersome to use, and a replacement that allowed us (unix 
guys :-) to just run "make check" or "make certify" would be great.  But 
where do we draw the line between "improving" the existing, functional 
test program and working on the engine?

-John

RE: [Firebird-test] Testing FB1

[Leyne, S. <sl...@at...>]

John,

> -----Original Message-----
> From: John Bellardo [mailto:bel...@cs...]
> Sent: Friday, August 03, 2001 1:36 AM

...

> >
> > An alternative track that also deserves consideration, is (as Sean 
> > points out) adjusting the TCS test tool parser to run from 
> text files.
> 
> The hardest part will be handling the test hierarchy that TCS has.  
> There are global tests, platform tests...

I think the solution is in extending the TCS suite to allow for scripts
to point to (include?) other scripts by name, which could point ot other
scripts...  We could call these scripts groups (with an extension GRP).
That way we could use the actual test scripts (we need a unique
extension for those - TST?) in any combination we wanted --
Nightly_test.grp, Full_test.grp, etc.

Just some thoughts


Sean

Re: [Firebird-test] Testing FB1

[John B. <bel...@cs...>]

On Thursday, August 2, 2001, at 09:49 PM, Mark O'Donohue wrote:

> John Bellardo wrote:
>
>>
>> Yup, changes in error message text, removing the extra space in number 
>> converted into strings, and the like.
>
> Yep, go ahead, ocmmit them
>
> Im keen to experiment with dejagnu for limit, (working on a rough 
> framework as we speak) and if we decide it's a good idea, will try for 
> some sort of extract script to parse the "$CREATE" etc out of the TCS 
> files.

I'm interested in seeing how well it works out too.

>
> An alternative track that also deserves consideration, is (as Sean 
> points out) adjusting the TCS test tool parser to run from text files.

The hardest part will be handling the test hierarchy that TCS has.  
There are global tests, platform tests...

>
> Long term dejagnu/expect offers a much richer control language for 
> driving test cases and checking results.  And I think in writing a new 
> test tool, (other than something for a thrasher) we would end up with 
> the same problems as the current TCS program.
> Anyway dejagnu  does run on all our platforms (including win32).

If we choose to go with dejagnu we could still take advantage of a flat 
file modified TCS.  We could make an expect script front end for all 
tests still run under TCS, and still use dejagnu to drive the whole 
thing until we have moved all tests out of TCS control.  From what I 
have seen so far that should be fairly simple to set up given a slightly 
modified TCS.

>
> So I'd like to use it to test limit, to see how we go with it.
>
>
> BTW: I can't remember if I said, but I commited Franks TCS database 
> extract tools into the TCS tree, and added a few simple instructions 
> and script into TCS on how to build TCS.


-John

Re: [Firebird-test] Testing FB1

[Mark O'D. <mar...@lu...>]

John Bellardo wrote:

>
> Yup, changes in error message text, removing the extra space in number 
> converted into strings, and the like.

Yep, go ahead, ocmmit them

Im keen to experiment with dejagnu for limit, (working on a rough 
framework as we speak) and if we decide it's a good idea, will try for 
some sort of extract script to parse the "$CREATE" etc out of the TCS files.

An alternative track that also deserves consideration, is (as Sean 
points out) adjusting the TCS test tool parser to run from text files.

Long term dejagnu/expect offers a much richer control language for 
driving test cases and checking results.  And I think in writing a new 
test tool, (other than something for a thrasher) we would end up with 
the same problems as the current TCS program.  

Anyway dejagnu  does run on all our platforms (including win32).

So I'd like to use it to test limit, to see how we go with it.


BTW: I can't remember if I said, but I commited Franks TCS database 
extract tools into the TCS tree, and added a few simple instructions and 
script into TCS on how to build TCS.


Cheers

Mark

Re: [Firebird-test] Testing FB1

[John B. <bel...@cs...>]

Mark,

On Thursday, August 2, 2001, at 08:58 PM, Mark O'Donohue wrote:

> John Bellardo wrote:
>
>> Hi,
>>     I am running the latest FB1 build through TCS and I'm getting a 
>> very good pass rate.  Most of the failures are attributable to error 
>> message changes that have been made.  The TCS database needs to be 
>> updated to reflect these changes.  Before I do that I have a few 
>> concerns.  First, is it worth my time to update the database before we 
>> have decided on a testing strategy?  Second, does anyone else have any 
>> pending changes to the database?
>
> I assume those changes are generic, and apply to all platforms?

Yup, changes in error message text, removing the extra space in number 
converted into strings, and the like.

-John

Re: [Firebird-test] Testing FB1

[Mark O'D. <mar...@lu...>]

John Bellardo wrote:

> Hi,
>     I am running the latest FB1 build through TCS and I'm getting a 
> very good pass rate.  Most of the failures are attributable to error 
> message changes that have been made.  The TCS database needs to be 
> updated to reflect these changes.  Before I do that I have a few 
> concerns.  First, is it worth my time to update the database before we 
> have decided on a testing strategy?  Second, does anyone else have any 
> pending changes to the database?

I assume those changes are generic, and apply to all platforms?

Mark

Re: [Firebird-test] Testing FB1

[John B. <bel...@cs...>]

Sean,

On Thursday, August 2, 2001, at 08:23 PM, Leyne, Sean wrote:

> John,
>
> [...]
> Accordingly, TCS is still our horse -- we have to ride it.  So I'd
> recommend that you make the necessary changes.

OK.  I'll look into updating the expected test results to reflect the 
recent changes.

>
> [...]

-John

RE: [Firebird-test] Testing FB1

[Leyne, S. <sl...@at...>]

John,

Until a new testing strategy has been completely thoughtout and
implemented, TCS will still be required.

Not that I doubt Mark's investigations but we need to carefully consider
all our testing needs -- engine, client, GPRE, ODBC, JDBC, Java
client...  quick engine/build tests, long term abuse tests... 

Accordingly, TCS is still our horse -- we have to ride it.  So I'd
recommend that you make the necessary changes.

I'd really, though, like to see TCS modified to eliminate the use of the
database to store the scripts and results.

After I've finished my work (it seems to be never ending) on the bug
tracker update, I'll turn my own attention to working with you all on a
testing implementation plan.


Sean

[Firebird-test] Testing FB1

[John B. <bel...@cs...>]

Hi,
	I am running the latest FB1 build through TCS and I'm getting a 
very good pass rate.  Most of the failures are attributable to error 
message changes that have been made.  The TCS database needs to be 
updated to reflect these changes.  Before I do that I have a few 
concerns.  First, is it worth my time to update the database before we 
have decided on a testing strategy?  Second, does anyone else have any 
pending changes to the database?

-John

Re: [Firebird-test] Using dejagnu to test the limit command

[Mark O'D. <mar...@lu...>]

John Bellardo wrote:

> Mark,
>
> On Sunday, July 29, 2001, at 10:27 AM, Mark O'Donohue wrote:
>
>>
>> Hi
>>
>> [....]
>> I've been looking at the dejagnu system 
>> (http://dejagnu.sourceforge.net).
>> It was recommened by a couple of people as suitable for what we are 
>> trying to do, and is fairly simple to code test cases in.
>>
>> So dejagnu is a fairly simple protocol built on top of the NIST 
>> expect tool (http://expect.nist.gov) - I've also written a couple of 
>> programs in expect and Im confident that it has the features that we 
>> needs.
>>
>> [....]
>
>
> What is it going to take to get dejagnu up to speed on the LIMIT 
> tests?  I'd like to familiarize myself with the system.  I've been 
> skimming the dejagnu manual, but I figured I'd ask you how much you 
> had finished.  Is it to the point you can commit it (in 
> interbase/testcases) so others can start fiddling?  What still needs 
> to be done?


Just start writing the test scripts in expect, use autoexpect to 
generate the first one then cull it.

Something like the following, (Im not at my proper desk, so it's all a 
bit off the cuff here and some little changes will be needed - the 
expression set a (b - a) I need to check the syntax for.


#!/usr/bin/expect -f


proc runlimit { testname offset  limit} {
    global prompt

    send "select 'XXX' from rdb$tables limit($offset, $limit)\r"

    set counter 0
    # loop round counting the numer of selects, and when get a prompt 
again continue.
    expect {
       "XXX" {   counter++
                          exp_continue
                      }
       $prompt { }
    }

# check results and print success/failure.

    set expectedAmount  (limit - offset)

    if (counter == expectedAmount) {
        send_user "Passed - $testname "
    }
    else {
        send_user "Success - $testname"
    }
}


# main program
#start isql

spawn isql

set prompt "SQL>"

runlimit ("test1", 0, 50)
runlimit ("test2", 50, 50 )

# you can see that it would be easy to add a function that checks for a 
general error message such as :
#runlimitBad ("test2", -1, 50 )



Cheers

Mark

Re: [Firebird-test] Using dejagnu to test the limit command

[John B. <bel...@cs...>]

Mark,

On Sunday, July 29, 2001, at 10:27 AM, Mark O'Donohue wrote:

>
> Hi
>
> [....]
> I've been looking at the dejagnu system 
> (http://dejagnu.sourceforge.net).
> It was recommened by a couple of people as suitable for what we are 
> trying to do, and is fairly simple to code test cases in.
>
> So dejagnu is a fairly simple protocol built on top of the NIST expect 
> tool (http://expect.nist.gov) - I've also written a couple of programs 
> in expect and Im confident that it has the features that we needs.
>
> [....]

What is it going to take to get dejagnu up to speed on the LIMIT tests?  
I'd like to familiarize myself with the system.  I've been skimming the 
dejagnu manual, but I figured I'd ask you how much you had finished.  Is 
it to the point you can commit it (in interbase/testcases) so others can 
start fiddling?  What still needs to be done?

Thanks.

-John

Re: [Firebird-test] Using dejagnu to test the limit command

[Mark O'D. <mar...@lu...>]

John Bellardo wrote:

> On Sunday, July 29, 2001, at 10:27 AM, Mark O'Donohue wrote:
>
>>
>> Hi
>>
>>
>
> Are you proposing switching all tests to dejagnu starting with LIMIT, 
> or using the LIMIT tests to experiment with dejagnu to determine if we 
> want to move all our tests over?  

I was thinking of using dejagnu (or more particularly expect) to do the 
main driving, it has some wonderful terminal driving capabilities, and 
it has been around quite a while.

Actually just looking at http://expect.nist.gov - there are links to two 
windows ports one cygwin and one native.

But it is a supurb tool every bit as good as what i've seen in 
LoadRunner scripting language.  It's not windows based scripting (which 
is a dog anyway) but then neither is sql.

> Either way I'm sure it would work out better than TCS.

That I think is a certainty, at at least with cygwin the whole dejagnu 
thing can be run.  expect runs the process and it's up to the expect 
script to recognise a successful outcome and write "SUCCESS: test104" 
which is then used in the dejagnu package to workout what passed and 
failed - pretty simple stuff.

As I wrote previously,

$autoexpect

also works like a macro recorder.


Cheers

Mark

-- 
Your database needs YOU!
http://firebird.sourceforge.net
GPL: free software today - and still free tomorrow

[Firebird-test] RE: Testing and stuff.

[Leyne, S. <sl...@at...>]

Mark,

> From: Mark O'Donohue [mailto:mar...@lu...]
> Sent: Monday, July 30, 2001 11:06 AM

...

> I just wanted to say, that Im not intentionally bumping into 
> the testing 
> sphere, it just happend by accident, but the dejagnu I think 
> works well.

It's those types of accidents that I like and are ones that get things
moving along...

> It is cross plaftform and even has a recorder (you will need cygwin 
> presumably on win32).
> 
> $autoexpect
> isql
> select * from fred limit(1,100);
> exit;
>
> And it generates a record script that matched "all" the 
> output.  Usually 
> with a little editing that can be cut down considerably.

Is there anyway that we could eliminate the need for cygwin?

I think that to expect someone to install cygwin in order to run the
testing would place a very high 'bar'.

I would dearly love to come up with a dead stupid EXE which any joe
could run to perform the basic tests.  If someone wanted to run the full
suite of test, then a CYGWIN install could be in order.


Sean

Re: [Firebird-test] Using dejagnu to test the limit command

[John B. <bel...@cs...>]

On Sunday, July 29, 2001, at 10:27 AM, Mark O'Donohue wrote:

>
> Hi
>
> While John (and Ann) have been doing their excellent work on limit, 
> I've been having a play around with the tcs test system, mainly  I got 
> there trying to test the fb2 build.
>
> I've been looking at the dejagnu system 
> (http://dejagnu.sourceforge.net).
> It was recommened by a couple of people as suitable for what we are 
> trying to do, and is fairly simple to code test cases in.
>
> So dejagnu is a fairly simple protocol built on top of the NIST expect 
> tool (http://expect.nist.gov) - I've also written a couple of programs 
> in expect and Im confident that it has the features that we needs.
>
> The expect tool is itself an extension of the tcl language mainly for 
> driving terminal driven applications.  For exercising the engine 
> through isql it's perfect and in addition I've been able to convert two 
> of the exising tcs test cases fairly easily.
>
> So I was thinking we could use it to build a library of test cases for 
> LIMIT.

Are you proposing switching all tests to dejagnu starting with LIMIT, or 
using the LIMIT tests to experiment with dejagnu to determine if we want 
to move all our tests over?  I haven't used dejagnu, but from what I 
briefly read of the manual it seems like it will fit most of our needs.  
I have a concern about win32 compatibility.  The manual said to install 
cygwin to use dejagnu on NT.  Is this true for all our win32 platforms 
(did I mis-read it)?  Is this something we are willing to accept?  I 
know we moved the build process away from using cygwin type tools.  How 
about testing?  Either way I'm sure it would work out better than TCS.

-John

[Firebird-test] Using dejagnu to test the limit command

[Mark O'D. <mar...@lu...>]

Hi

While John (and Ann) have been doing their excellent work on limit, I've 
been having a play around with the tcs test system, mainly  I got there 
trying to test the fb2 build.

I've been looking at the dejagnu system (http://dejagnu.sourceforge.net).  

It was recommened by a couple of people as suitable for what we are 
trying to do, and is fairly simple to code test cases in.

So dejagnu is a fairly simple protocol built on top of the NIST expect 
tool (http://expect.nist.gov) - I've also written a couple of programs 
in expect and Im confident that it has the features that we needs.

The expect tool is itself an extension of the tcl language mainly for 
driving terminal driven applications.  For exercising the engine through 
isql it's perfect and in addition I've been able to convert two of the 
exising tcs test cases fairly easily.

So I was thinking we could use it to build a library of test cases for 
LIMIT.

Any thoughts?

Cheers

Mark

 

-- 
Your database needs YOU!
http://firebird.sourceforge.net
GPL: free software today - and still free tomorrow

Re: [Firebird-test] Can't to send mail to egroups.ib-support

[Frank Schlottmann-G. <sch...@t-...>]

Sandor Szollosi wrote:

> My mails not appears after sending. I' dont know what happened, I used it
> 5 days ago successfully.
> (Outlook Express 5,  news.atkin.com)
> 
> Plese help !

Seems that the sourceforge-groups are working, posts to egroups
via news.atkin.com still seem to show up only at Yahoo.

I don't know when this will get fixed.

Frank 

-- 
"Fascinating creatures, phoenixes. They can carry immensely heavy loads,
 their tears have healing powers and they make highly faithful pets."
         - J.K. Rowling
http://firebirdsql.org

[Firebird-test] Can't to send mail to egroups.ib-support

[Sandor S. <ss...@fr...>]

My mails not appears after sending. I' dont know what happened, I used it 5
days ago successfully.
(Outlook Express 5,  news.atkin.com)

Plese help !

Sandor

[Firebird-test] Test - Please not read

[Sandor S. <ss...@fr...>]