Re: [Ocaml-lib-devel] Future of ExtLib

[Bardur A. <oca...@sc...>]

On Sat, Dec 18, 2004 at 10:51:54AM +1100, skaller wrote:

> > I guess for Extlib, you're always assumed to pass *all* tests before 
> > making a release or committing changes.  
> 
> Yup, but when a test fails you need to know which one :)

What about just having a -verbose switch for the ./test_*.ml
program which prints the number of each test before
running it...? Seems simple enough to me...

Of course, stack traces would really be the optimal way to
do this, but OCaml doesn't support stack traces for
exceptions (although the debugger does... go figure).

> 
> > I don't think it would be so hard to find volunteers for this kind of 
> > testing.  In fact I'd be willing to write a few tests for it myself.
> 
> So would I. The main obstacle is just to specify how
> to structure the tests. I'm sure a couple of us could write
> the test harness once that is done, and someone can volunteer
> to collect tests from this mailing list and glue them into CVS.
> 
> How about this:
> 
> (a) Each test is called 'test_*.ml'.

It might be a bit less cluttered to have a directory (or
multiple directories?) exclusively for tests, but I guess
any convention which clearly separates test from non-test
would be fine.

> (b) Each is a program linked against ExtLib.

Yup.

> (c) The test passes if 
> 
>    i) it exits with 0
>   ii) stderr is empty
>   iii) stdout agrees with 'test_*.expected'
> 
> (d) Each test optionally has a file 'test_*.input'
> which is passed via stdin.

Wouldn't it be easier to just hardcode input/expected
values into the test_*.ml files themselves instead of
relying on "external" files? Certainly, all the things
I've thought about writing test cases for would be just as
easy to hardcode input/output values for.

> 
> An alternative is: the first line of the test
> is the checker: eg:
> 
> (* env FRED=something test_298 <somein >tmp.tmp && diff tmp.tmp
> test_298.expected *)
> 
> which allows command line args etc.

Wouldn't this necessitate a UNIX-ish environment?

-- 
Bardur Arantsson
<ba...@im...>
<ba...@sc...>

Friends are like plants. They need attention and they need to
drink.
                                    SPYvSPY | http://kuro5hin.org