octave-dev

[OctDev] test scripts

[Paul K. <pki...@ja...>]

Hi,

I've been working on exercising the test scripts that already
exist in octave-forge.  I would like for someone to volunteer
to organize the gathering of more tests.  

The simplist solution would be to set up a wiki page with 
three columns:

	directory coordinator missing

where directory is the name of the directory, coordinator is
the sflogin name of someone with octave-forge write privileges
and status is a list of functions which don't have adequate
test coverage.  Once every directory has a coordinator, send
a message to help-octave requesting that users should check 
the wiki, put a star by the function(s) they can write tests 
for, write the tests, then submit patches to
coo...@us.... 

Thanks in advance,

Paul Kienzle
pki...@us...

BTW, tests just got easier, since they are now installed 
with mark_as_command.

Re: [OctDev] test scripts

[Etienne G. <et...@cs...>]

  Hi all,

that's a good idea. However, that would be a huge table until most
tests scripts are written: There are 26 dirs in main, w/ >400 m-files
very few of which have a test.

  Also, how should we name test files? I use test_XYZ.m or
test_XYZ_1.m, test_XYZ_2.m ... to test XYZ_m. Would using test
subdirectories in each dir reduce clutter?

  How should tests behave? I set a 'ok' variable to 1 if success, 0 if
failure. Test produces terminal output if a variable 'verbose' is
set.

  Running the tests automatically? Should probably be a non-octave
script, as some tests may crash octave.

  Looks like a huge task. It may be easier psychologically to add a
test to a function when a bug has been fixed on it.

  Just my 2c,

  Etienne

On Tue, Dec 16, 2003 at 05:29:02PM -0500, Paul Kienzle wrote:
# Hi,
# 
# I've been working on exercising the test scripts that already
# exist in octave-forge.  I would like for someone to volunteer
# to organize the gathering of more tests.  
# 
# The simplist solution would be to set up a wiki page with 
# three columns:
# 
# 	directory coordinator missing
# 
# where directory is the name of the directory, coordinator is
# the sflogin name of someone with octave-forge write privileges
# and status is a list of functions which don't have adequate
# test coverage.  Once every directory has a coordinator, send
# a message to help-octave requesting that users should check 
# the wiki, put a star by the function(s) they can write tests 
# for, write the tests, then submit patches to
# coo...@us.... 
# 
# Thanks in advance,
# 
# Paul Kienzle
# pki...@us...
# 
# BTW, tests just got easier, since they are now installed 
# with mark_as_command.
# 
# 
# -------------------------------------------------------
# This SF.net email is sponsored by: IBM Linux Tutorials.
# Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
# Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
# Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
# _______________________________________________
# Octave-dev mailing list
# Oct...@li...
# https://lists.sourceforge.net/lists/listinfo/octave-dev

-- 
Etienne Grossmann ------ http://www.cs.uky.edu/~etienne

Re: [OctDev] test scripts

[Paul K. <pki...@us...>]

On Dec 16, 2003, at 7:23 PM, Etienne Grossmann wrote:

>
>   Hi all,
>
> that's a good idea. However, that would be a huge table until most
> tests scripts are written: There are 26 dirs in main, w/ >400 m-files
> very few of which have a test.

Maybe a separate page for each directory.  It is up to the
directory coordinator to decide how they want to organize it.

>
>   Also, how should we name test files? I use test_XYZ.m or
> test_XYZ_1.m, test_XYZ_2.m ... to test XYZ_m. Would using test
> subdirectories in each dir reduce clutter?

Embed the tests in the script files.  See 'help test' for the
method.

>
>   How should tests behave? I set a 'ok' variable to 1 if success, 0 if
> failure. Test produces terminal output if a variable 'verbose' is
> set.

I capture the test results to a log file and keep track of how many
passed and how many failed.

>
>   Running the tests automatically? Should probably be a non-octave
> script, as some tests may crash octave.

I thought about this a bit.  Currently if octave crashes, you will need
to comment out the crashing test and restart from scratch.  Admittedly
not the best, but then octave doesn't crash much :-)

Another approach would be to run octave from each directory rather
than from the start.  At least that way, you will only have to rerun the
directory, not the whole tree.

A more sophisticated approach would be to write a line number
before each test script.  Then if octave crashes, we can simply
skip the tests that have already been completed and continue.
That's a project for another day.

>   Looks like a huge task. It may be easier psychologically to add a
> test to a function when a bug has been fixed on it.

Although huge, the project is very distributable.   Just add a
couple of lines to an existing script file and you have a test.
That's why I think we should ask the community for help.  You
never know --- some of them may decide to contribute more!

Paul Kienzle
pki...@us...

Re: [OctDev] test scripts

[Paul K. <pki...@us...>]

On Dec 16, 2003, at 10:28 PM, Paul Kienzle wrote:
>
>>   Looks like a huge task. It may be easier psychologically to add a
>> test to a function when a bug has been fixed on it.
>
> Although huge, the project is very distributable.   Just add a
> couple of lines to an existing script file and you have a test.
> That's why I think we should ask the community for help.  You
> never know --- some of them may decide to contribute more!

Just to make the project bigger, all doc strings should be texinfo-ized,
particularly those for functions which belong in the octave-core.

Paul Kienzle
pki...@us...