ocaml-lib-devel — Developer's mailing list

Re: [Ocaml-lib-devel] test harness generator

[skaller <sk...@us...>]

On Thu, 2004-12-23 at 04:35, Janne Hellsten wrote:

> 1. Building in tmp directory (extlib-test/build-tmp).  This means 
> copying all the necessary source to build dir.  This includes tester 
> entrypoint and any additional utility modules (just util.ml currently).

OK, generate in 'build-tmp' -- not sure that copying files
into there is needed, just

	ocamlc -c -o build-tmp/util util.ml

seems to work .. :)

> 2. Remove the .tst extension and just use .ml.  

OK, that will work provided step 1 is done.

> John, it would be nice if you could finish the above two steps before 
> committing your test harness generator.  Do you have the time to do this?

Sure .. it will take longer to test than actually do it :)
Expect commit in next message.

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

[PATCH] BitSet.compare fix (was Re: [Ocaml-lib-devel] a bug in BitSet?)

[Janne H. <ja...@hy...>]

Here is a patch that should fix BitSet.compare bugs.  I made a 
relatively extensive tester for it and couldn't find any problems.

The compare now works as if the bitvector was a BigInt.  I am guessing 
that that has been the original intent as well.

Perhaps someone can take a look before I commit the changes?

ciao,
janne

> Compare seems completely broken to me too, at least if one
> expects it to work like you suggest (which is IMHO the
> correct way). 
> 
> It doesn't seem to properly account for the two bitsets
> possibly being of different lengths, it just starts
> comparing at the MSB even if the difference between the
> lengths of the sets is more than a byte. It gets more
> complicated because you can't even use |t1|>|t2| as an
> indication that t1>t2, because there is AFAICT no
> guarantee that the leading bit (or even just one bit in
> the first byte of the representation!) is set.

Re: [Ocaml-lib-devel] test harness generator

[Janne H. <ja...@hy...>]

> If enough people like the idea, I can just commit it,
> then we can all change it as needed.
> 
> Clearly, to add extra management: statistics, options to
> stop on error or not, or whatever, will require changes.
> It doesn't generate native code either.

Yep.

>>4. Filtering of test cases.  Could you make a command line option for 
>>specifying what tests are included in the test? 
> 
> 
> As an argument to the build of the mainline,
> or an argument to the mainline? Or both?

By mainline do you mean "mktest"?  I mean adding the option to the 
mktest, not the actual test executable.  The reason I want to select 
tests is a) faster compiles and b) faster execution of tests.  This is 
useful for developing the test suite and for debugging test cases.

> The code I posted was mainly intended to be a demo of
> the idea. 

Yes, but it is definitely close to what we want.

So for the next steps:

1. Building in tmp directory (extlib-test/build-tmp).  This means 
copying all the necessary source to build dir.  This includes tester 
entrypoint and any additional utility modules (just util.ml currently).

I suggested making a library out of the test modules, but I don't think 
it's such a good idea anymore.  Let's make it more simple and just copy 
the files.

2. Remove the .tst extension and just use .ml.  I think we can skip file 
patching for now.. This is something we can add later.

* *

John, it would be nice if you could finish the above two steps before 
committing your test harness generator.  Do you have the time to do this?

Everything after that can be added more or less incrementally.

ciao,
janne

Re: [Ocaml-lib-devel] test harness generator

[skaller <sk...@us...>]

On Thu, 2004-12-23 at 00:16, Bardur Arantsson wrote:

> IMHO, using a different extension than .ml is a bit silly.
> You already know it's a test from the test_ prefix.

The reason is that there is a function 'patch_file' which
transforms *.tst to *.ml, meaning, the basename is preserved.

This transformation is currently the identity
and could be dropped entirely.

Alternatively .. or as well .. the generated code,
including a copy of the source, could go into
a tmp directory as Janne suggested.


-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] test harness generator

[skaller <sk...@us...>]

On Thu, 2004-12-23 at 01:36, Janne Hellsten wrote:
> Is there some reason why you run the tests twice?  E.g., once in 
> extlib_test.ml and then in each seprate test with "test ()"?

That's a bug :)

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] test harness generator

[skaller <sk...@us...>]

On Thu, 2004-12-23 at 00:03, Janne Hellsten wrote:
> Hi John,

> 1. Create and build tests in some tmp build directory. 

Yeah, that makes sense.

> 2. Are you sure we need the .tst extension? 

Nope :)

The transformation tst -> ml is just the identity .. at the moment.

The idea was simply to distinguish the tests from other
support code eg util.ml.

>  I'd like to put all the 
> tests into a separate directory and just use .ml extension?  .ml is nice 
> so that Emacs automatically selects tuareg-mode

Ah, ok... I'm a Vim user :)

> 3. More thorough error checking and logging. 

That partly depends on also upgrading the util module.

> I understand there's not 
> much error checking since this is the first version.  Could you print 
> out the command line for each ocamlc call?

If enough people like the idea, I can just commit it,
then we can all change it as needed.

Clearly, to add extra management: statistics, options to
stop on error or not, or whatever, will require changes.
It doesn't generate native code either.

> 4. Filtering of test cases.  Could you make a command line option for 
> specifying what tests are included in the test? 

As an argument to the build of the mainline,
or an argument to the mainline? Or both?

> 5. Add some options for generating ocamlprofs out of the testing suite 
> and ExtLib?  Code coverage reports are really helpful when trying to 
> make the test suite more extensive.

Indeed.

> As for the near future.. Once 1&2 are done/agreed, I'd like to integrate 
> this into the suite.

The code I posted was mainly intended to be a demo of
the idea. 

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] test harness generator

[Janne H. <ja...@hy...>]

Is there some reason why you run the tests twice?  E.g., once in 
extlib_test.ml and then in each seprate test with "test ()"?

ciao,
janne

skaller wrote:
> I have attached first cut of a test harness generator.
> (On my box it all runs, but one of the test cases now segfaults).
> 
> to run it use this:
> 
> ocamlc -o mktest unix.cma
> str.cma mktest.ml && ./mktest && ./extlib_test
> 
> from a COPY (PLEASE!!) of extlib-test directory.
> 
> This program makes a check for ALL extlib modules.
> It searches for files in the current (extlib-test)
> directory with this exact format:
> 
> test_<author>_<module>_<test_name>.tst
> 
> Note the test name is mandatory, and the extension must be .tst.
> 
> It generates files:
> 
> extlib_test.ml
> mtest_<module>.ml
> itest_<author>_<module>_<test_name>.ml
> 
> and compiles and links everything together, including
> the utility util.ml. Bytecode only at the moment.
> 
> To make this work, all individual tests mut be renamed
> to the correct format.
> 
> Note: at present the only thing done to a *.tst file
> is to copy it to a *.ml file without touching anything inside.
> 
> This code is *not supposed* to clobber any existing files.
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> (* This program generates the test harness *)
> (* This program must be executed from the directory
> 
>    extlib-test
> 
>    and extlib itself must be located in the directory
> 
>    extlib-dev
> *)
> 
> (* Step 1: find a list of all ExtLib modules *)
> 
> let extlib_dev_dir = 
>   Filename.concat Filename.parent_dir_name "extlib-dev"
> ;;
> 
> (* get all the files in a directory as a list, the names
>   do not include the directory pathname
> *)
> let find_files dirname =
>   let dir_handle = Unix.opendir dirname in
>   let all_files = 
>     let files = ref [] in
>       let rec loop () =
>         let filename = Unix.readdir dir_handle in
>         files := filename :: !files;
>         loop()
>       in try loop() with End_of_file -> !files
>   in
>   Unix.closedir dir_handle;
>   all_files
> ;;
> 
> (* filter a list of strings with a regular expression *)
> let filter_files rexp files =
>   let crexp = Str.regexp rexp in
>   List.filter
>   (fun s -> Str.string_match crexp s 0)
>   files
> ;;
> 
> (* regexp for finding *.mli files *)
> let mli_re = "^.+\\.mli$"
> ;;
> 
> (* regexp for eliding .mli extension *)
> let mli_rex = "^\\(.+\\)\\.mli$"
> ;;
> 
> (* find the modules in a directory, assuming
> each .mli file represents a module. The names
> are have leading character capitalised.
> *)
> 
> let modules_of dirname =
>   let crexp = Str.regexp mli_rex in
>   List.map
>   (fun s-> 
>     ignore(Str.string_match crexp s 0); 
>     let s = Str.matched_group 1 s in
>     s.[0] <- Char.uppercase s.[0];
>     s
>   )
>   (filter_files mli_re (find_files dirname))
> ;;
> 
> let all_modules = (modules_of extlib_dev_dir)
> ;;
> 
> List.iter
> (fun s-> print_endline s)
> all_modules
> ;;
> 
> (* now make the top level test harness *)
> let mk_top () =
>   let f = open_out "extlib_test.ml" in
> 
>   output_string f "let main() = \n";
>   output_string f "  Util.log \"ExtLib tester started\";\n";
>   List.iter
>   (fun s->
>     output_string f ("  Mtest_" ^ s ^ ".test ();\n")
>   )
>   all_modules
>   ;
>   output_string f "  Util.log \"ExtLib tests completed\"\n";
>   output_string f ";;\n";
>   output_string f "\n";
>   output_string f "main();;\n";
>   close_out f
> ;;
> 
> mk_top()
> ;;
> 
> (* now make the individual module tests *)
> 
> (* regexp for finding *.tst files *)
> let tst_re = "^test_.+_.+_.+\\.tst$"
> ;;
> 
> (* regexp for decoding test names *)
> let tst_rex = "^test_\\(.+\\)_\\(.+\\)_\\(.+\\)\\.tst$"
> ;;
> 
> 
> let tests_of dirname =
>   let h = Hashtbl.create 97 in
>   let crexp = Str.regexp tst_rex in
>   List.iter
>   (fun s-> 
>     ignore(Str.string_match crexp s 0); 
>     let author_key = Str.matched_group 1 s in
>     let module_key = Str.matched_group 2 s in
>     let test_key = Str.matched_group 3 s in
>     Hashtbl.add h module_key (author_key,test_key)
>   )
>   (filter_files tst_re (find_files dirname));
>   h
> ;;
> 
> let all_tests = (tests_of Filename.current_dir_name)
> ;;
> 
> 
> 
> let mtest mname =
>   let f = open_out ("mtest_" ^ mname ^ ".ml") in
> 
>   output_string f "let test() = \n";
> 
>   let tests = Hashtbl.find_all all_tests mname in
>   output_string f ("  Util.log \"Checking module " ^ mname ^ "\";\n");
>   List.iter
>   (fun (author,test) ->
>     output_string f ("  Itest_" ^ author ^ "_" ^mname^"_"^test^".test ();\n")
>   )
>   tests
>   ;
>   output_string f "  ()\n";
>   output_string f ";;\n";
>   output_string f "\n";
>   output_string f "test();;\n";
>   close_out f;
> ;;
> 
> let mk_mtests () = List.iter mtest all_modules
> ;;
> 
> mk_mtests()
> ;;
> 
> let patch_test mname author test =
>   let input_filename = "test_" ^ author ^ "_" ^mname^"_"^test^".tst" in
>   let output_filename = "itest_" ^ author ^ "_" ^mname^"_"^test^".ml" in
>   let fi = open_in input_filename in
>   let fo = open_out output_filename in
>   let rec loop() =
>     let s = input_line fi in
>     output_string fo (s ^ "\n");
>     loop()
>   in try loop() with End_of_file ->
>   output_string fo "\n";
>   close_out fo;
>   close_in fi
> ;;
>   
> let patch_tests () =
>   Hashtbl.iter
>   (fun mname (author,test) -> 
>     patch_test mname author test
>   )
>   all_tests
> ;;
> 
> patch_tests()
> ;;
> 
> let xqt cmd msg =
>   let result = Unix.system(cmd) in
>   let result = 
>     match result with
>     | Unix.WEXITED i -> i == 0
>     | Unix.WSIGNALED _ -> false
>     | Unix.WSTOPPED _ -> false
>   in 
>     if not result 
>     then 
>       failwith ("FAILURE: msg=" ^ msg ^ ", cmd=" ^cmd)
> ;;
> 
> let compile_file filename =
>   let cmd = "ocamlc -I "^extlib_dev_dir^" -c " ^ filename in
>   xqt cmd ("Compilation of " ^ filename)
> ;;
> 
> let compile_tests () =
>   (* compile individual tests *)
>   Hashtbl.iter
>   (fun mname (author,test) ->
>     let filename = "itest_" ^ author ^ "_" ^mname^"_"^test^".ml" in
>     compile_file filename
>   )
>   all_tests
>   ;
>   (* compile generated module level thunks *)
>   List.iter
>   (fun s ->
>     let filename = "mtest_" ^ s ^ ".ml" in
>     compile_file filename
>   )
>   all_modules
>   ;
>   (* compile mainline *)
>   compile_file "extlib_test.ml"
> ;;
> 
> compile_tests()
> ;;
> 
> compile_file "util.ml"
> ;;
> 
> let link_tests () =
>   (* individual tests *)
>   let linkstring = ref (
>     "ocamlc -I "^extlib_dev_dir ^
>     " -o extlib_test extLib.cma util.cmo " 
>   )
>   in 
>   Hashtbl.iter
>   (fun mname (author,test) ->
>     let filename = "itest_" ^ author ^ "_" ^mname^"_"^test^".cmo" in
>     linkstring := !linkstring ^ " " ^ filename
>   )
>   all_tests
>   ;
>   (* compile generated module level thunks *)
>   List.iter
>   (fun s ->
>     let filename = "mtest_" ^ s ^ ".cmo" in
>     linkstring := !linkstring ^ " " ^ filename
>   )
>   all_modules
>   ;
>   (* compile mainline *)
>   linkstring := !linkstring ^ " extlib_test.cmo"
>   ;
>   xqt !linkstring "Linking extlib_test"
> ;;
> 
> link_tests()
> ;;
> 
> print_endline "extlib_test generated"
> ;;
> 

Re: [Ocaml-lib-devel] test harness generator

[Bardur A. <oca...@sc...>]

On Wed, Dec 22, 2004 at 03:03:41PM +0200, Janne Hellsten wrote:

> 2. Are you sure we need the .tst extension?  I'd like to put all the 
> tests into a separate directory and just use .ml extension?  .ml is nice 
> so that Emacs automatically selects tuareg-mode (and I don't want to 
> change my Emacs confs :)). IIRC you also wanted to put the actual tests 
> into a subdir.  If you believe .tst is needed, then how about naming 
> them "test_xyz_*.ml.tst"?

IMHO, using a different extension than .ml is a bit silly.
You already know it's a test from the test_ prefix.

-- 
Bardur Arantsson
<ba...@im...>
<ba...@sc...>

- Blue! No, yellow! AHHHHHHHHHhhhh.....
                                | Monty Python and the Holy Grail

Re: [Ocaml-lib-devel] test harness generator

[Janne H. <ja...@hy...>]

Hi John,

this looks promising.  After thinking about this a little while, I've 
come to the conclusion that the test harness generator is not such a bad 
idea after all.

I have a few suggestions though (roughly in order of priority):

1. Create and build tests in some tmp build directory.  How about 
building a library of the tests and then link that into the main testing 
suite to create an executable?  The main motivation behind is just to 
not clutter the test suite source directory.

2. Are you sure we need the .tst extension?  I'd like to put all the 
tests into a separate directory and just use .ml extension?  .ml is nice 
so that Emacs automatically selects tuareg-mode (and I don't want to 
change my Emacs confs :)). IIRC you also wanted to put the actual tests 
into a subdir.  If you believe .tst is needed, then how about naming 
them "test_xyz_*.ml.tst"?

3. More thorough error checking and logging.  I understand there's not 
much error checking since this is the first version.  Could you print 
out the command line for each ocamlc call?

4. Filtering of test cases.  Could you make a command line option for 
specifying what tests are included in the test?  This would be useful 
when developing the actual tests (e.g., I would only turn on "jh*" tests 
  when writing my own tests).

5. Add some options for generating ocamlprofs out of the testing suite 
and ExtLib?  Code coverage reports are really helpful when trying to 
make the test suite more extensive.

As for the near future.. Once 1&2 are done/agreed, I'd like to integrate 
this into the suite.

ciao,
janne

skaller wrote:
> I have attached first cut of a test harness generator.
> (On my box it all runs, but one of the test cases now segfaults).
> 
> to run it use this:
> 
> ocamlc -o mktest unix.cma
> str.cma mktest.ml && ./mktest && ./extlib_test
> 
> from a COPY (PLEASE!!) of extlib-test directory.
> 
> This program makes a check for ALL extlib modules.
> It searches for files in the current (extlib-test)
> directory with this exact format:
> 
> test_<author>_<module>_<test_name>.tst
> 
> Note the test name is mandatory, and the extension must be .tst.
> 
> It generates files:
> 
> extlib_test.ml
> mtest_<module>.ml
> itest_<author>_<module>_<test_name>.ml
> 
> and compiles and links everything together, including
> the utility util.ml. Bytecode only at the moment.
> 
> To make this work, all individual tests mut be renamed
> to the correct format.
> 
> Note: at present the only thing done to a *.tst file
> is to copy it to a *.ml file without touching anything inside.
> 
> This code is *not supposed* to clobber any existing files.
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> (* This program generates the test harness *)
> (* This program must be executed from the directory
> 
>    extlib-test
> 
>    and extlib itself must be located in the directory
> 
>    extlib-dev
> *)
> 
> (* Step 1: find a list of all ExtLib modules *)
> 
> let extlib_dev_dir = 
>   Filename.concat Filename.parent_dir_name "extlib-dev"
> ;;
> 
> (* get all the files in a directory as a list, the names
>   do not include the directory pathname
> *)
> let find_files dirname =
>   let dir_handle = Unix.opendir dirname in
>   let all_files = 
>     let files = ref [] in
>       let rec loop () =
>         let filename = Unix.readdir dir_handle in
>         files := filename :: !files;
>         loop()
>       in try loop() with End_of_file -> !files
>   in
>   Unix.closedir dir_handle;
>   all_files
> ;;
> 
> (* filter a list of strings with a regular expression *)
> let filter_files rexp files =
>   let crexp = Str.regexp rexp in
>   List.filter
>   (fun s -> Str.string_match crexp s 0)
>   files
> ;;
> 
> (* regexp for finding *.mli files *)
> let mli_re = "^.+\\.mli$"
> ;;
> 
> (* regexp for eliding .mli extension *)
> let mli_rex = "^\\(.+\\)\\.mli$"
> ;;
> 
> (* find the modules in a directory, assuming
> each .mli file represents a module. The names
> are have leading character capitalised.
> *)
> 
> let modules_of dirname =
>   let crexp = Str.regexp mli_rex in
>   List.map
>   (fun s-> 
>     ignore(Str.string_match crexp s 0); 
>     let s = Str.matched_group 1 s in
>     s.[0] <- Char.uppercase s.[0];
>     s
>   )
>   (filter_files mli_re (find_files dirname))
> ;;
> 
> let all_modules = (modules_of extlib_dev_dir)
> ;;
> 
> List.iter
> (fun s-> print_endline s)
> all_modules
> ;;
> 
> (* now make the top level test harness *)
> let mk_top () =
>   let f = open_out "extlib_test.ml" in
> 
>   output_string f "let main() = \n";
>   output_string f "  Util.log \"ExtLib tester started\";\n";
>   List.iter
>   (fun s->
>     output_string f ("  Mtest_" ^ s ^ ".test ();\n")
>   )
>   all_modules
>   ;
>   output_string f "  Util.log \"ExtLib tests completed\"\n";
>   output_string f ";;\n";
>   output_string f "\n";
>   output_string f "main();;\n";
>   close_out f
> ;;
> 
> mk_top()
> ;;
> 
> (* now make the individual module tests *)
> 
> (* regexp for finding *.tst files *)
> let tst_re = "^test_.+_.+_.+\\.tst$"
> ;;
> 
> (* regexp for decoding test names *)
> let tst_rex = "^test_\\(.+\\)_\\(.+\\)_\\(.+\\)\\.tst$"
> ;;
> 
> 
> let tests_of dirname =
>   let h = Hashtbl.create 97 in
>   let crexp = Str.regexp tst_rex in
>   List.iter
>   (fun s-> 
>     ignore(Str.string_match crexp s 0); 
>     let author_key = Str.matched_group 1 s in
>     let module_key = Str.matched_group 2 s in
>     let test_key = Str.matched_group 3 s in
>     Hashtbl.add h module_key (author_key,test_key)
>   )
>   (filter_files tst_re (find_files dirname));
>   h
> ;;
> 
> let all_tests = (tests_of Filename.current_dir_name)
> ;;
> 
> 
> 
> let mtest mname =
>   let f = open_out ("mtest_" ^ mname ^ ".ml") in
> 
>   output_string f "let test() = \n";
> 
>   let tests = Hashtbl.find_all all_tests mname in
>   output_string f ("  Util.log \"Checking module " ^ mname ^ "\";\n");
>   List.iter
>   (fun (author,test) ->
>     output_string f ("  Itest_" ^ author ^ "_" ^mname^"_"^test^".test ();\n")
>   )
>   tests
>   ;
>   output_string f "  ()\n";
>   output_string f ";;\n";
>   output_string f "\n";
>   output_string f "test();;\n";
>   close_out f;
> ;;
> 
> let mk_mtests () = List.iter mtest all_modules
> ;;
> 
> mk_mtests()
> ;;
> 
> let patch_test mname author test =
>   let input_filename = "test_" ^ author ^ "_" ^mname^"_"^test^".tst" in
>   let output_filename = "itest_" ^ author ^ "_" ^mname^"_"^test^".ml" in
>   let fi = open_in input_filename in
>   let fo = open_out output_filename in
>   let rec loop() =
>     let s = input_line fi in
>     output_string fo (s ^ "\n");
>     loop()
>   in try loop() with End_of_file ->
>   output_string fo "\n";
>   close_out fo;
>   close_in fi
> ;;
>   
> let patch_tests () =
>   Hashtbl.iter
>   (fun mname (author,test) -> 
>     patch_test mname author test
>   )
>   all_tests
> ;;
> 
> patch_tests()
> ;;
> 
> let xqt cmd msg =
>   let result = Unix.system(cmd) in
>   let result = 
>     match result with
>     | Unix.WEXITED i -> i == 0
>     | Unix.WSIGNALED _ -> false
>     | Unix.WSTOPPED _ -> false
>   in 
>     if not result 
>     then 
>       failwith ("FAILURE: msg=" ^ msg ^ ", cmd=" ^cmd)
> ;;
> 
> let compile_file filename =
>   let cmd = "ocamlc -I "^extlib_dev_dir^" -c " ^ filename in
>   xqt cmd ("Compilation of " ^ filename)
> ;;
> 
> let compile_tests () =
>   (* compile individual tests *)
>   Hashtbl.iter
>   (fun mname (author,test) ->
>     let filename = "itest_" ^ author ^ "_" ^mname^"_"^test^".ml" in
>     compile_file filename
>   )
>   all_tests
>   ;
>   (* compile generated module level thunks *)
>   List.iter
>   (fun s ->
>     let filename = "mtest_" ^ s ^ ".ml" in
>     compile_file filename
>   )
>   all_modules
>   ;
>   (* compile mainline *)
>   compile_file "extlib_test.ml"
> ;;
> 
> compile_tests()
> ;;
> 
> compile_file "util.ml"
> ;;
> 
> let link_tests () =
>   (* individual tests *)
>   let linkstring = ref (
>     "ocamlc -I "^extlib_dev_dir ^
>     " -o extlib_test extLib.cma util.cmo " 
>   )
>   in 
>   Hashtbl.iter
>   (fun mname (author,test) ->
>     let filename = "itest_" ^ author ^ "_" ^mname^"_"^test^".cmo" in
>     linkstring := !linkstring ^ " " ^ filename
>   )
>   all_tests
>   ;
>   (* compile generated module level thunks *)
>   List.iter
>   (fun s ->
>     let filename = "mtest_" ^ s ^ ".cmo" in
>     linkstring := !linkstring ^ " " ^ filename
>   )
>   all_modules
>   ;
>   (* compile mainline *)
>   linkstring := !linkstring ^ " extlib_test.cmo"
>   ;
>   xqt !linkstring "Linking extlib_test"
> ;;
> 
> link_tests()
> ;;
> 
> print_endline "extlib_test generated"
> ;;
> 

Re: [Ocaml-lib-devel] [PATCH] bitSet.ml/bitSet.mli: added diff, diff_sym, inter and union

[Janne H. <ja...@hy...>]

> Just curious -- you have 'diff' and 'diff_sym'.
> My Ocaml 3.0.8 manual says Set just has 'diff' and describes
> it as 'set difference'.
> 
> Which function is the standard Set,, subtraction or symmetric
> difference?

ExtLib's BitSet had differentiate_sym, so I decided it's better to add 
non-destructive version of that as well (sym_diff).

sym_diff is "set symmetriccal difference", which is corresponds to XOR 
in Boolean logic.

Set.diff is subtraction.

ciao,
janne

Re: [Ocaml-lib-devel] [PATCH] bitSet.ml/bitSet.mli: added diff, diff_sym, inter and union

[Janne H. <ja...@hy...>]

OK, renamed diff_sym to sym_diff and committed.

ciao,
janne

Bardur Arantsson wrote:
> On Tue, Dec 21, 2004 at 11:28:58PM +0200, Janne Hellsten wrote:
> 
> 
>>This patch adds new functionality to BitSet.  These functions were 
>>discussed earlier on this mailing lists.
>>
>>The patch does not modify old functions, just adds new ones.  A careful 
>>reader will note that it is doing a bit of extra work.. I doubt it will 
>>make much of a difference, but perhaps I can optimize these later on.
>>
>>It has been tested with the testing suite.
>>
>>If the diff is OK, I can commit to CVS myself (testers included).
> 
> 
> Just a tiny naming nit: diff_sym should probably be called
> sym_diff since that more closely resembles the full name,
> "symmetric difference".
> 
> The code looks fine to me.
> 

Re: [Ocaml-lib-devel] Future of ExtLib

[Nicolas C. <war...@fr...>]

> > Logging fail/success:  Actually I can easily do this with Util.run_test.
> >   Perhaps I will add this in the near future.
> >
> > Commit.log: I have no idea what that's all about.. I wondered about the
> > same thing, but figured it's not a problem since the commit worked and
> > individual file logs seem to be working OK.
>
> I think one of the two administrators will need to address
> this issue. Probably just a minor permission glitch.

Yes but I don't have access to the file rights on the CVSROOT directory. I
tried several times to disable the log and it failed, and I asked one time
to SF managers but they didn't fix it right. Not so big deal...

Nicolas

[Ocaml-lib-devel] test harness generator

[skaller <sk...@us...>]

I have attached first cut of a test harness generator.
(On my box it all runs, but one of the test cases now segfaults).

to run it use this:

ocamlc -o mktest unix.cma
str.cma mktest.ml && ./mktest && ./extlib_test

from a COPY (PLEASE!!) of extlib-test directory.

This program makes a check for ALL extlib modules.
It searches for files in the current (extlib-test)
directory with this exact format:

test_<author>_<module>_<test_name>.tst

Note the test name is mandatory, and the extension must be .tst.

It generates files:

extlib_test.ml
mtest_<module>.ml
itest_<author>_<module>_<test_name>.ml

and compiles and links everything together, including
the utility util.ml. Bytecode only at the moment.

To make this work, all individual tests mut be renamed
to the correct format.

Note: at present the only thing done to a *.tst file
is to copy it to a *.ml file without touching anything inside.

This code is *not supposed* to clobber any existing files.


-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] Future of ExtLib

[skaller <sk...@us...>]

On Wed, 2004-12-22 at 07:18, Janne Hellsten wrote:
> Hi,
> 
> > which is 'triple handling', in fact worse, since the test name
> > has to be written as well.. :)
> 
> Well, it depends on how many test contributors and how many tests we're 
> expecting.  I'm not expecting that many, so I don't consider adding 
> tests such a big problem.

But you are only one contributor to the project.
Even if you personally don't expect many contributions,
you probably shouldn't make a decision without consulting
list members.

> This might be doable with GNU Make's $(wildcard) and some other tricks. 
>   I never tried, but it seems doable.

> Then again, if I *don't* do this, the Makefile would be more 
> straightforward and would even allow test contributors to write their 
> own small utility modules.  If I want to use wildcards, I will also need 
> a portable way to automatically make the .ml files topologically sorted.

> However, as to your other suggestions, frankly, I think you're making 
> the build process unnecessarily complicated.  I don't see how adding new 
> tests can ever become a bottleneck.  

It already has been for me. It took me 10 times longer than
necessary to add a single 3 line test.

I need to question your assumption that automating the build
of the test system will make it more complicated.

In fact, I believe the opposite is true: we can probably
dispense *completely* with any makefile, and would provide
one only as a convenience.

I have had this reaction before -- automation is one of
my specialities, it really *isn't* that hard to do.

> Nor do I think it can be somehow 
> unsafe (even if a couple of files need to be modified), due to O'Caml's 
> strong static typing.

The static typing does not

(a) guarrantee a test will be called
(b) ensure the test is in the correct category
(c) the test is properly named


However it is worse. Remember we're using CVS. A system which
cleanly separates changes into files is obviously better
than the current situation, where every tester is going
to modify files *other* testers will also have to modify.

Whether CVS can handle simultaneous changes or not,
it is probably better to make it easier than harder :)

> Anyhow.. If someone will make a more sophisticated build system in the 
> future, I'm OK with that.. Just make 100% sure that it ports to Windows.

Indeed this is a strong argument against Makefiles and shell
script. An Ocaml based solution is more likely to be portable.
 
>   I noticed there's been some talk about adding path handling into 
> ExtLib.  These *definitely* need to be tested on Windows to make sure 
> they work as expected!

Yes.

> Currently my main concern though is getting more meaningful tests into 
> the suite..

This will come over time. I wouldn't worry so much about that
as setting up the mechanism so that it is as easy as possible
for people to contribute tests.

> > test_*_<module>_*.ml
> > 
> > Note this implies names like
> > 
> > test_jh_ExtList.ml
> > 
> > would not be allowed, a suffix would be required.

> Surely you can just check if the suffix is missing and if so, 
> automatically name it "untitled".

Of course once could do this -- the question is: do we want to?
I'm in favour of strictness and simplicity, and this format breaks
those guidelines.

> I find it perfectly OK to have more than one test in one file.  In some 
> cases it allows to share some code between individual tests.  See 
> test_jh_BitSet.ml for an example (bitset_of_int, etc).

I have no problem with that. Nor the current 3 level design
I described before. I'm not sure it is the best, but I'm quite
happy to work with it for the moment.

My concern is solely to limit the work of test designers
to making a file containing tests and then submitting it,
perhaps to CVS, and perhaps just posting on the mailing list
so someone else with access can 'cut and paste' the example
into the test suite with minimal effort.


> As for renaming.. Grrr I wish we could use Subversion.. :)

LOL! Whatever, the time to make this kind of change is 
right now, before it is too hard and we're stuck with
a possibly suboptimal design.

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] [PATCH] bitSet.ml/bitSet.mli: added diff, diff_sym, inter and union

[skaller <sk...@us...>]

On Wed, 2004-12-22 at 08:28, Janne Hellsten wrote:
> This patch adds new functionality to BitSet.  

Just curious -- you have 'diff' and 'diff_sym'.
My Ocaml 3.0.8 manual says Set just has 'diff' and describes
it as 'set difference'.

Which function is the standard Set,, subtraction or symmetric
difference?

What does ExtSet have?

Woops there isn't one .. how about pSet .. Woops there isn't one..

Why isn't there a pSet (polymorphic set)?
> 
> ______________________________________________________________________
Otherwise .. I vote for committing your changes.

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] Future of ExtLib

[skaller <sk...@us...>]

On Wed, 2004-12-22 at 05:57, Janne Hellsten wrote:

> Logging fail/success:  Actually I can easily do this with Util.run_test. 
>   Perhaps I will add this in the near future.
> 
> Commit.log: I have no idea what that's all about.. I wondered about the 
> same thing, but figured it's not a problem since the commit worked and 
> individual file logs seem to be working OK.

I think one of the two administrators will need to address
this issue. Probably just a minor permission glitch.

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] [PATCH] bitSet.ml/bitSet.mli: added diff, diff_sym, inter and union

[Bardur A. <oca...@sc...>]

On Tue, Dec 21, 2004 at 11:28:58PM +0200, Janne Hellsten wrote:

> This patch adds new functionality to BitSet.  These functions were 
> discussed earlier on this mailing lists.
> 
> The patch does not modify old functions, just adds new ones.  A careful 
> reader will note that it is doing a bit of extra work.. I doubt it will 
> make much of a difference, but perhaps I can optimize these later on.
> 
> It has been tested with the testing suite.
> 
> If the diff is OK, I can commit to CVS myself (testers included).

Just a tiny naming nit: diff_sym should probably be called
sym_diff since that more closely resembles the full name,
"symmetric difference".

The code looks fine to me.

-- 
Bardur Arantsson
<ba...@im...>
<ba...@sc...>

- I'm sorry I lied to you, Marge, but this gun had a hold on me.
I felt this incredible surge of power, like God must feel when
he's holding a gun.
                                     Homer Simpson | The Simpsons

[Ocaml-lib-devel] [PATCH] bitSet.ml/bitSet.mli: added diff, diff_sym, inter and union

[Janne H. <ja...@hy...>]

This patch adds new functionality to BitSet.  These functions were 
discussed earlier on this mailing lists.

The patch does not modify old functions, just adds new ones.  A careful 
reader will note that it is doing a bit of extra work.. I doubt it will 
make much of a difference, but perhaps I can optimize these later on.

It has been tested with the testing suite.

If the diff is OK, I can commit to CVS myself (testers included).

ciao,
janne

Re: [Ocaml-lib-devel] Future of ExtLib

[Janne H. <ja...@hy...>]

Hi,

> which is 'triple handling', in fact worse, since the test name
> has to be written as well.. :)

Well, it depends on how many test contributors and how many tests we're 
expecting.  I'm not expecting that many, so I don't consider adding 
tests such a big problem.

> Ideally, the test.mak, mainline, and module level
> test function should be generated, not hand coded,
> that is the Makefile should generate the these things
> using a refined equivalent of 'ls test_*.ml'.

This might be doable with GNU Make's $(wildcard) and some other tricks. 
  I never tried, but it seems doable.

Then again, if I *don't* do this, the Makefile would be more 
straightforward and would even allow test contributors to write their 
own small utility modules.  If I want to use wildcards, I will also need 
a portable way to automatically make the .ml files topologically sorted.

However, as to your other suggestions, frankly, I think you're making 
the build process unnecessarily complicated.  I don't see how adding new 
tests can ever become a bottleneck.  Nor do I think it can be somehow 
unsafe (even if a couple of files need to be modified), due to O'Caml's 
strong static typing.

Anyhow.. If someone will make a more sophisticated build system in the 
future, I'm OK with that.. Just make 100% sure that it ports to Windows. 
  I noticed there's been some talk about adding path handling into 
ExtLib.  These *definitely* need to be tested on Windows to make sure 
they work as expected!

Currently my main concern though is getting more meaningful tests into 
the suite..

> test_*_<module>_*.ml
> 
> Note this implies names like
> 
> test_jh_ExtList.ml
> 
> would not be allowed, a suffix would be required.
> That could be fixed by also allowing names of that form.
> I personally think we should mandate the suffix:
> we'd better decide fast, before there are too many tests:
> CVS isn't good at file renaming .. :)

Surely you can just check if the suffix is missing and if so, 
automatically name it "untitled".

I find it perfectly OK to have more than one test in one file.  In some 
cases it allows to share some code between individual tests.  See 
test_jh_BitSet.ml for an example (bitset_of_int, etc).

As for renaming.. Grrr I wish we could use Subversion.. :)

ciao,
janne

Re: [Ocaml-lib-devel] Future of ExtLib

[Janne H. <ja...@hy...>]

Hi,

thank you for taking a peek.

A nice way to checkout whole extlib is just to do

cvs -d :ext:use...@cv...:/cvsroot/ocaml-lib co .

(note the dot)

This will checkout both extlib-dev and extlib-test.  If you then go to 
extlib-test and run "make all" it will build both the ExtLib and the 
testing suite.

While I agree with some people here about making the test suite an 
integral part of extlib, I think the current CVS layout allows people to 
both checkout and build&run the tester without a hassle.

Logging fail/success:  Actually I can easily do this with Util.run_test. 
  Perhaps I will add this in the near future.

Commit.log: I have no idea what that's all about.. I wondered about the 
same thing, but figured it's not a problem since the commit worked and 
individual file logs seem to be working OK.

I will address your (Skaller) other e-mail later (hopefully today).

ciao,
janne

skaller wrote:
> On Tue, 2004-12-21 at 07:24, Janne Hellsten wrote:
> 
>>Just to inform everyone, I have just imported the testing suite into CVS. 
>>The module layout is as proposed by Nicolas.
>>
> 
> 
> OK, I import it like this:
> 
> My system has the structure:
> 
> /usr/local/src/extlib/extlib-dev
> 
> So I did this:
> 
> cd /usr/local/src/extlib
> export CVS_RSH=ssh
> cvs -z3 -d:ext:sk...@cv...:/cvsroot/ocaml-lib co
> extlib-test
> 
> Now I have this:
> 
> [skaller@pelican] /usr/local/src/extlib>ls
> extlib-dev  extlib-test
> 
> 
> [skaller@pelican] /usr/local/src/extlib>ls extlib-test/
> CVS       test_Base64.ml   test_ExtString.ml  test_jh_ExtList.ml   
> test.mak
> Makefile  test_BitSet.ml   test_jh_Base64.ml  test_jh_ExtString.ml 
> util.ml
> README    test_ExtList.ml  test_jh_BitSet.ml  test_main.ml
> 
> 
> 
> [skaller@pelican] /usr/local/src/extlib>cd extlib-test/
> /usr/local/src/extlib/extlib-test
> 
> 
> 
> [skaller@pelican] /usr/local/src/extlib/extlib-test>make
> cd ../extlib-dev && make all
> make[1]: Entering directory `/mnt/local/src/extlib/extlib-dev'
> ocamlc -a -o extLib.cma enum.mli bitSet.mli dbi.mli dynArray.mli
> extHashtbl.mli
> extList.mli extString.mli global.mli IO.mli option.mli pMap.mli std.mli
> uChar.mli uTF8.mli base64.mli unzip.mli refList.mli optParse.mli
> dllist.mli enum.ml bitSet.ml dbi.ml dynArray.ml extHashtbl.ml extList.ml
> extString.ml global.ml IO.ml option.ml pMap.ml std.ml uChar.ml uTF8.ml
> base64.ml unzip.ml refList.ml optParse.ml dllist.ml extLib.ml
> make[1]: Leaving directory `/mnt/local/src/extlib/extlib-dev'
> make -f test.mak all
> make[1]: Entering directory `/mnt/local/src/extlib/extlib-test'
> ocamlc -I ../extlib-dev -g extLib.cma util.ml test_jh_Base64.ml
> test_Base64.ml test_jh_BitSet.ml test_BitSet.ml test_jh_ExtList.ml
> test_ExtList.ml test_jh_ExtString.ml test_ExtString.ml test_main.ml  -o
> extlib_test
> make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test'
> 
> [skaller@pelican] /usr/local/src/extlib/extlib-test>make run
> make -f test.mak run
> make[1]: Entering directory `/mnt/local/src/extlib/extlib-test'
> ./extlib_test
> Extlib tester started..
>  
> run: Base64
> run: jh_Base64.test
> run: BitSet
> run: jh_BitSet.test_intersect
> run: jh_BitSet.test_diff
> run: jh_BitSet.test_rnd_creation ..FAILED
>   reason:
> test_jh_BitSet.ml:60:4 test jh_BitSet.test_rnd_creation failed
>  
> run: jh_BitSet.test_empty
> run: jh_BitSet.test_exceptions ..FAILED
>   reason:
> test_jh_BitSet.ml:116:2 test jh_BitSet.test_exceptions failed
> 
> run: ExtString
> run: jh_ExtString.t_starts_with
> run: jh_ExtString.t_map
> run: jh_ExtString.t_lchop
> run: jh_ExtString.t_rchop
> run: jh_ExtString.t_split
> run: ExtList
> run: jh_ExtList.iteri
> run: jh_ExtList.mapi
> run: jh_ExtList.exceptions
> run: jh_ExtList.find_exc
> All tests completed.
> make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test'
> 
> 
> All this is expected, including the failures (hmm .. it says
> all tests completed, but doesn't report how many faiures .. still
> this can be fixed later, for now the important step seems
> achieved .. there is a place to put tests and a protocol for
> constructing them.
> 
> ------------------------------------------------------------------------
> 
> I added a test for Dynarray (just a stub at the moment).
> I got this:
> 
> ....
> 
> Checking in test_main.ml;
> /cvsroot/ocaml-lib/extlib-test/test_main.ml,v  <--  test_main.ml
> new revision: 1.3; previous revision: 1.2
> done
> sh: line 1: /cvsroot/ocaml-lib/CVSROOT/commit.log: Permission denied
> **********************************************************************
> 
> Looks like the commits worked, but the log wasn't updated??
> 
> [Comments on test harness next post]
> 

Re: [Ocaml-lib-devel] Future of ExtLib

[skaller <sk...@us...>]

On Tue, 2004-12-21 at 07:24, Janne Hellsten wrote:
[Makefile .. test structure]

Janne currently has a three level test structure:

1. The mainline calls the test for each module.
2. The module test calls individual tests for each module
3. Individual tests call subtests 

In all 3 cases the call looks like this:

let test () = 
  Util.run_test ~test_name:"jh_ExtList.iteri" test_iteri;
  Util.run_test ~test_name:"jh_ExtList.mapi" test_mapi;
  Util.run_test ~test_name:"jh_ExtList.exceptions" test_exceptions;
  Util.run_test ~test_name:"jh_ExtList.find_exc" test_find_exc

In addition each file has to be listed in the file

test.mak

which is 'triple handling', in fact worse, since the test name
has to be written as well.. :)

Ideally, the test.mak, mainline, and module level
test function should be generated, not hand coded,
that is the Makefile should generate the these things
using a refined equivalent of 'ls test_*.ml'.

I.e. the to create a new test you just commit the
test file -- you don't change any existing files,
instead the files are constructed.

I suppose I can write an Ocaml program which will create these
files. This would work as follows:

(1) Look in extlib-dev for all mli files, to find the
names of top level modules. [Alternative -- hard code
the list of module names]

(2) Generate the mainline 'test_mail.ml' using these module names.

(3) For each module, generate a test file test_<module>.ml,
which calls each test for that module in the extlib-test directory
found with name

test_*_<module>_*.ml

Note this implies names like

test_jh_ExtList.ml

would not be allowed, a suffix would be required.
That could be fixed by also allowing names of that form.
I personally think we should mandate the suffix:
we'd better decide fast, before there are too many tests:
CVS isn't good at file renaming .. :)

Capitalisation rules would apply (the module names must
be exactly as used in Ocaml code). I'm not sure how this
will work on systems that aren't strict about filenames..

An alternative to an Ocaml program to do this would
be a bash script.. probably easier to write but could
be less portable, considering native Win32 environment
for example.

A BIG advantage of automatically generating the
test call protocol, apart from ease of adding new
tests, is that the actual protocol can be changed
to do things like count success/failure, generate
error reports blah blah .. without manual recoding,
except possibly at the individual test level.

It may actually be possible to build the individual
test() functions too, if the test functions within
the individual test files are annotated with
some special kind of mark eg:

TEST inner_test_name
 ......

which is replaced by

let inner_test_name () =

but which also gets added to the harness at the bottom of the
file like this:

let test () =
  Util.run_test ~test_name:"js_DynArray.inner_test_name" inner_test_name
  ...


Thus fully automating the whole suite construction, requiring
only the special form of declaration. This would imply changing
the CVS filenames to something like

test_jh_ExtLib_123.tst

[to avoid overwriting them when generating the *.ml file]

Comments?

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] Future of ExtLib

[skaller <sk...@us...>]

On Tue, 2004-12-21 at 07:24, Janne Hellsten wrote:
> Just to inform everyone, I have just imported the testing suite into CVS. 
> The module layout is as proposed by Nicolas.
> 

OK, I import it like this:

My system has the structure:

/usr/local/src/extlib/extlib-dev

So I did this:

cd /usr/local/src/extlib
export CVS_RSH=ssh
cvs -z3 -d:ext:sk...@cv...:/cvsroot/ocaml-lib co
extlib-test

Now I have this:

[skaller@pelican] /usr/local/src/extlib>ls
extlib-dev  extlib-test


[skaller@pelican] /usr/local/src/extlib>ls extlib-test/
CVS       test_Base64.ml   test_ExtString.ml  test_jh_ExtList.ml   
test.mak
Makefile  test_BitSet.ml   test_jh_Base64.ml  test_jh_ExtString.ml 
util.ml
README    test_ExtList.ml  test_jh_BitSet.ml  test_main.ml



[skaller@pelican] /usr/local/src/extlib>cd extlib-test/
/usr/local/src/extlib/extlib-test



[skaller@pelican] /usr/local/src/extlib/extlib-test>make
cd ../extlib-dev && make all
make[1]: Entering directory `/mnt/local/src/extlib/extlib-dev'
ocamlc -a -o extLib.cma enum.mli bitSet.mli dbi.mli dynArray.mli
extHashtbl.mli
extList.mli extString.mli global.mli IO.mli option.mli pMap.mli std.mli
uChar.mli uTF8.mli base64.mli unzip.mli refList.mli optParse.mli
dllist.mli enum.ml bitSet.ml dbi.ml dynArray.ml extHashtbl.ml extList.ml
extString.ml global.ml IO.ml option.ml pMap.ml std.ml uChar.ml uTF8.ml
base64.ml unzip.ml refList.ml optParse.ml dllist.ml extLib.ml
make[1]: Leaving directory `/mnt/local/src/extlib/extlib-dev'
make -f test.mak all
make[1]: Entering directory `/mnt/local/src/extlib/extlib-test'
ocamlc -I ../extlib-dev -g extLib.cma util.ml test_jh_Base64.ml
test_Base64.ml test_jh_BitSet.ml test_BitSet.ml test_jh_ExtList.ml
test_ExtList.ml test_jh_ExtString.ml test_ExtString.ml test_main.ml  -o
extlib_test
make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test'

[skaller@pelican] /usr/local/src/extlib/extlib-test>make run
make -f test.mak run
make[1]: Entering directory `/mnt/local/src/extlib/extlib-test'
./extlib_test
Extlib tester started..
 
run: Base64
run: jh_Base64.test
run: BitSet
run: jh_BitSet.test_intersect
run: jh_BitSet.test_diff
run: jh_BitSet.test_rnd_creation ..FAILED
  reason:
test_jh_BitSet.ml:60:4 test jh_BitSet.test_rnd_creation failed
 
run: jh_BitSet.test_empty
run: jh_BitSet.test_exceptions ..FAILED
  reason:
test_jh_BitSet.ml:116:2 test jh_BitSet.test_exceptions failed

run: ExtString
run: jh_ExtString.t_starts_with
run: jh_ExtString.t_map
run: jh_ExtString.t_lchop
run: jh_ExtString.t_rchop
run: jh_ExtString.t_split
run: ExtList
run: jh_ExtList.iteri
run: jh_ExtList.mapi
run: jh_ExtList.exceptions
run: jh_ExtList.find_exc
All tests completed.
make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test'


All this is expected, including the failures (hmm .. it says
all tests completed, but doesn't report how many faiures .. still
this can be fixed later, for now the important step seems
achieved .. there is a place to put tests and a protocol for
constructing them.

------------------------------------------------------------------------

I added a test for Dynarray (just a stub at the moment).
I got this:

....

Checking in test_main.ml;
/cvsroot/ocaml-lib/extlib-test/test_main.ml,v  <--  test_main.ml
new revision: 1.3; previous revision: 1.2
done
sh: line 1: /cvsroot/ocaml-lib/CVSROOT/commit.log: Permission denied
**********************************************************************

Looks like the commits worked, but the log wasn't updated??

[Comments on test harness next post]

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Ocaml-lib-devel] Future of ExtLib

[Blair Z. <bl...@or...>]

Bardur Arantsson wrote:
> On Mon, Dec 20, 2004 at 10:59:27AM +0100, Nicolas Cannasse wrote:
> 
> 
>>>>You can download the source from:
>>>>
>>>>http://www.saunalahti.fi/~jjhellst/extlib/
>>>>
>>>>It doesn't have many tests yet, just a few placeholders to demonstrate
>>
>>the
>>
>>>>file layout.
>>>
>>>Ok this looks great. The most important issue right now IMHO is
>>>not how the test manager works, but where it lives in CVS.
>>>
>>>Nicolas said at one stage that the test suite shouldb't be
>>>part of the distribution. Whether or not this is the case,
>>>I would say it should be easily separable.
>>>
>>>IMHO test code and library code should be siblings,
>>>the tests should be in a subdir of the src (just my opinion).
>>
>>If Janne or anybody else is willing to take over the test suite CVS
>>maintenance, I will give CVS write access.
>>As for the repository, I prefer having it in a separate module, so anybody
>>checking out extlib sources does not automaticaly get test suite too.
>>Something like :
>>
>>/CVSROOT
>>/extlib-dev
>>/extlib-test
>>
> 
> 
> I'm wondering why you'd prefer to keep them separate...
> Does it provide any tangible benefits?
> 
> (It certainly does have the drawback of making it more
> cumbersome to keep the test suite and the library in
> sync.)

I agree that the two should be kept together to support code coding. 
Any additional steps that a developer has to do to check their new code 
  just gets in the way, even if it's done only once.

As practice, I don't think I know of any package that has a test suite 
that doesn't ship with it.  Most GNU software has a make test that you 
can run after you build it.

Regards,
Blair

-- 
Blair Zajac <bl...@or...>
Plots of your system's performance - http://www.orcaware.com/orca/

Re: [Ocaml-lib-devel] Future of ExtLib

[Janne H. <ja...@hy...>]

Just to inform everyone, I have just imported the testing suite into CVS. 
The module layout is as proposed by Nicolas.

I will add copyright notices to the source files.  Currently the tester is 
using Markus Mottl's OCamlMakefile, I'm hoping to replace it by something 
simpler.

Currently the most complete tester exists for BitSet.  Unfortunately the 
best part of it is currently disabled, as it uses the new BitSet API 
(diff, union, et al.)

ciao,
janne

> If Janne or anybody else is willing to take over the test suite CVS
> maintenance, I will give CVS write access.
> As for the repository, I prefer having it in a separate module, so anybody
> checking out extlib sources does not automaticaly get test suite too.
> Something like :
>
> /CVSROOT
> /extlib-dev
> /extlib-test

Re: [Ocaml-lib-devel] Future of ExtLib

[skaller <sk...@us...>]

On Mon, 2004-12-20 at 20:59, Nicolas Cannasse wrote:

> If Janne or anybody else is willing to take over the test suite CVS
> maintenance, I will give CVS write access.

Don't all developers have that anyhow?

> As for the repository, I prefer having it in a separate module, so anybody
> checking out extlib sources does not automaticaly get test suite too.
> Something like :
> 
> /CVSROOT
> /extlib-dev
> /extlib-test

That looks good.

-- 
John Skaller, mailto:sk...@us...
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net