Re: [Ocaml-lib-devel] test harness generator

[skaller <sk...@us...>]

On Thu, 2004-12-23 at 04:35, Janne Hellsten wrote:

I have committed 'mktest.ml' to the extlib-test repository.

This program is similar to the one initially posted to this
list, except it is a bit cleaned up and:


> 1. Building in tmp directory (extlib-test/build-tmp). 

Done.

> 2. Remove the .tst extension and just use .ml.

Done.

Also, mktest program accepts options:

	--author=xxxx

If at least one such option is found, the set of tests is
filtered on the set of authors, so you can selectively
build tests from all, one, or some set of test authors.

The program prints the list of modules and tests.
To try out this program do this:

ocamlc -o mktest unix.cma str.cma mktest.ml && \
./mktest --author=js --author=jh && \
./extlib_test

This should create:

(a) a directory 'build-tmp' 
   -- with permissions 0o777 (no idea what to use 
   but it worked on my box, please fix it if you know better
  -- this is the only use of module Unix, I switched to Sys
  everywhere else for portability

(b) stuff in build-tmp

(c) A top level program 'extlib_test'

It shouldn't touch anything else in the extlib-test directory.

I have NOT changed any test names to conform to the required
format:

	test_<author>_<module>_<test>.ml

noting again that the <test> suffix is mandatory.
I suggest authors either number their tests with three digits,
or use a name suggestive of the kind of test. The need for the
suffix could be relaxed but I have not done it.

I have NOT changed anything else: for example the Makefile
and other files are left untouched.

If the harness is acceptable some files will need to be
removed from the repository and others renamed.

I was thinking that if the 'author' option is specified,
the executable should perhaps have a different name ..
but at present it doesn't.

Note that even with the author specified you get a thunk
for every module, even if there are no tests for that
module.

Also note there isn't any proper checking -- tests with
module names that don't match an actual ExtLib module
may cause problems -- I think there should be a warning,
what actually happens is that they're compiled and linked 
but never called.

It probably makes sense to be able to filter on the
module too, I think that can be done in a couple of minutes,
but I thought I would commit it now so people can have a look.

Also, if we agree on the test naming protocol and also
the requirement for a test to be properly called by
the harness, it should be documented in a 

	README.html

file or something, which goes in the repository so people
know how to construct tests.

Finally -- I haven't bothered to add licence to the mktest.ml
file, that probably needs to be done.

BTW: on licencing, it may be worth everyone *assiging copyright*
to Nicolas so any decisions about licencing policy can be
executed by one person. For example I could ask for permission
to include ExtLib in Felix which uses a BSD like licence,
and if most people agreed with that Nicolas could execute the
group decision legally if he was the sole copyright holder.
Otherwise everyone who made a contribution would have
to give permission.. (and one person who has gone away
would thwart the group's choice).

Note INRIA may change the Ocaml licence, which is a scenario
in which the Extlib licence should probably be changed to match.

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net