From: Janne H. <ja...@hy...> - 2004-12-18 17:15:25
|
Hello, I made an initial version of the tester. I think something like this should suffice for the time being. I used the module naming scheme proposed by John Max Skaller. Otherwise it's pretty much what everyone else has been talking here on the list. You can download the source from: http://www.saunalahti.fi/~jjhellst/extlib/ It doesn't have many tests yet, just a few placeholders to demonstrate the file layout. I found one issue: BitSet.is_set does not throw Negative_index at all. Other functions tend seem to raise this exception. Is this the expected behaviour? ciao, janne On Sat, 18 Dec 2004, Richard W.M. Jones wrote: > On Sat, Dec 18, 2004 at 11:38:31PM +1100, skaller wrote: >> OK, but the naming 'test999.ml' sux :) > > Yup. > >> Random contributions require a more consistent naming scheme. >> So how about: >> >> test_<author>_<module>_<name>.ml > > Seems sensible. > >> <author> is the authors initials eg rj, js, nc, jh. >> <module> is the Extlib module being tested, or 'multi' if more >> than one. > > Should we have tests which test more than one module? > > Rich. > > -- > Richard Jones. http://www.annexia.org/ http://www.j-london.com/ >>>> http://www.team-notepad.com/ - collaboration tools for teams <<< > Merjis Ltd. http://www.merjis.com/ - improving website return on investment > http://subjectlink.com/ - Lesson plans and source material for teachers > > |
From: skaller <sk...@us...> - 2004-12-20 09:46:08
|
On Sun, 2004-12-19 at 04:15, Janne Hellsten wrote: > You can download the source from: > > http://www.saunalahti.fi/~jjhellst/extlib/ > > It doesn't have many tests yet, just a few placeholders to demonstrate the > file layout. Ok this looks great. The most important issue right now IMHO is not how the test manager works, but where it lives in CVS. Nicolas said at one stage that the test suite shouldb't be part of the distribution. Whether or not this is the case, I would say it should be easily separable. IMHO test code and library code should be siblings, the tests should be in a subdir of the src (just my opinion). -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Nicolas C. <war...@fr...> - 2004-12-20 10:07:57
|
> > You can download the source from: > > > > http://www.saunalahti.fi/~jjhellst/extlib/ > > > > It doesn't have many tests yet, just a few placeholders to demonstrate the > > file layout. > > Ok this looks great. The most important issue right now IMHO is > not how the test manager works, but where it lives in CVS. > > Nicolas said at one stage that the test suite shouldb't be > part of the distribution. Whether or not this is the case, > I would say it should be easily separable. > > IMHO test code and library code should be siblings, > the tests should be in a subdir of the src (just my opinion). If Janne or anybody else is willing to take over the test suite CVS maintenance, I will give CVS write access. As for the repository, I prefer having it in a separate module, so anybody checking out extlib sources does not automaticaly get test suite too. Something like : /CVSROOT /extlib-dev /extlib-test Nicolas Cannasse |
From: Janne H. <ja...@hy...> - 2004-12-20 10:37:21
|
Nicolas Cannasse wrote: > If Janne or anybody else is willing to take over the test suite CVS > maintenance, I will give CVS write access. > As for the repository, I prefer having it in a separate module, so anybody > checking out extlib sources does not automaticaly get test suite too. > Something like : > > /CVSROOT > /extlib-dev > /extlib-test I am willing to take over the suite.. Having it as a separate module is OK for me. The only thing I'm concerned about is building and setting up file paths. E.g., I want it to be easy to checkout both and then be able compile extlib-test out-of-the-box without an explicit "configure". I don't remember my CVS well anymore.. Is it easy to setup the CVS modules so that when I checkout extlib-test, I also get extlib-dev as a sibling? ciao, janne |
From: Bardur A. <oca...@sc...> - 2004-12-20 11:09:40
|
On Mon, Dec 20, 2004 at 10:59:27AM +0100, Nicolas Cannasse wrote: > > > You can download the source from: > > > > > > http://www.saunalahti.fi/~jjhellst/extlib/ > > > > > > It doesn't have many tests yet, just a few placeholders to demonstrate > the > > > file layout. > > > > Ok this looks great. The most important issue right now IMHO is > > not how the test manager works, but where it lives in CVS. > > > > Nicolas said at one stage that the test suite shouldb't be > > part of the distribution. Whether or not this is the case, > > I would say it should be easily separable. > > > > IMHO test code and library code should be siblings, > > the tests should be in a subdir of the src (just my opinion). > > If Janne or anybody else is willing to take over the test suite CVS > maintenance, I will give CVS write access. > As for the repository, I prefer having it in a separate module, so anybody > checking out extlib sources does not automaticaly get test suite too. > Something like : > > /CVSROOT > /extlib-dev > /extlib-test > I'm wondering why you'd prefer to keep them separate... Does it provide any tangible benefits? (It certainly does have the drawback of making it more cumbersome to keep the test suite and the library in sync.) -- Bardur Arantsson <ba...@im...> <ba...@sc...> - We interrupt this programme to annoy you and make things generally irritating. BBC Announcer | Monty Python's Flying Circus |
From: Blair Z. <bl...@or...> - 2004-12-20 21:00:37
|
Bardur Arantsson wrote: > On Mon, Dec 20, 2004 at 10:59:27AM +0100, Nicolas Cannasse wrote: > > >>>>You can download the source from: >>>> >>>>http://www.saunalahti.fi/~jjhellst/extlib/ >>>> >>>>It doesn't have many tests yet, just a few placeholders to demonstrate >> >>the >> >>>>file layout. >>> >>>Ok this looks great. The most important issue right now IMHO is >>>not how the test manager works, but where it lives in CVS. >>> >>>Nicolas said at one stage that the test suite shouldb't be >>>part of the distribution. Whether or not this is the case, >>>I would say it should be easily separable. >>> >>>IMHO test code and library code should be siblings, >>>the tests should be in a subdir of the src (just my opinion). >> >>If Janne or anybody else is willing to take over the test suite CVS >>maintenance, I will give CVS write access. >>As for the repository, I prefer having it in a separate module, so anybody >>checking out extlib sources does not automaticaly get test suite too. >>Something like : >> >>/CVSROOT >>/extlib-dev >>/extlib-test >> > > > I'm wondering why you'd prefer to keep them separate... > Does it provide any tangible benefits? > > (It certainly does have the drawback of making it more > cumbersome to keep the test suite and the library in > sync.) I agree that the two should be kept together to support code coding. Any additional steps that a developer has to do to check their new code just gets in the way, even if it's done only once. As practice, I don't think I know of any package that has a test suite that doesn't ship with it. Most GNU software has a make test that you can run after you build it. Regards, Blair -- Blair Zajac <bl...@or...> Plots of your system's performance - http://www.orcaware.com/orca/ |
From: skaller <sk...@us...> - 2004-12-20 11:15:48
|
On Mon, 2004-12-20 at 20:59, Nicolas Cannasse wrote: > If Janne or anybody else is willing to take over the test suite CVS > maintenance, I will give CVS write access. Don't all developers have that anyhow? > As for the repository, I prefer having it in a separate module, so anybody > checking out extlib sources does not automaticaly get test suite too. > Something like : > > /CVSROOT > /extlib-dev > /extlib-test That looks good. -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Janne H. <ja...@hy...> - 2004-12-20 20:25:03
|
Just to inform everyone, I have just imported the testing suite into CVS. The module layout is as proposed by Nicolas. I will add copyright notices to the source files. Currently the tester is using Markus Mottl's OCamlMakefile, I'm hoping to replace it by something simpler. Currently the most complete tester exists for BitSet. Unfortunately the best part of it is currently disabled, as it uses the new BitSet API (diff, union, et al.) ciao, janne > If Janne or anybody else is willing to take over the test suite CVS > maintenance, I will give CVS write access. > As for the repository, I prefer having it in a separate module, so anybody > checking out extlib sources does not automaticaly get test suite too. > Something like : > > /CVSROOT > /extlib-dev > /extlib-test |
From: skaller <sk...@us...> - 2004-12-21 06:33:45
|
On Tue, 2004-12-21 at 07:24, Janne Hellsten wrote: > Just to inform everyone, I have just imported the testing suite into CVS. > The module layout is as proposed by Nicolas. > OK, I import it like this: My system has the structure: /usr/local/src/extlib/extlib-dev So I did this: cd /usr/local/src/extlib export CVS_RSH=ssh cvs -z3 -d:ext:sk...@cv...:/cvsroot/ocaml-lib co extlib-test Now I have this: [skaller@pelican] /usr/local/src/extlib>ls extlib-dev extlib-test [skaller@pelican] /usr/local/src/extlib>ls extlib-test/ CVS test_Base64.ml test_ExtString.ml test_jh_ExtList.ml test.mak Makefile test_BitSet.ml test_jh_Base64.ml test_jh_ExtString.ml util.ml README test_ExtList.ml test_jh_BitSet.ml test_main.ml [skaller@pelican] /usr/local/src/extlib>cd extlib-test/ /usr/local/src/extlib/extlib-test [skaller@pelican] /usr/local/src/extlib/extlib-test>make cd ../extlib-dev && make all make[1]: Entering directory `/mnt/local/src/extlib/extlib-dev' ocamlc -a -o extLib.cma enum.mli bitSet.mli dbi.mli dynArray.mli extHashtbl.mli extList.mli extString.mli global.mli IO.mli option.mli pMap.mli std.mli uChar.mli uTF8.mli base64.mli unzip.mli refList.mli optParse.mli dllist.mli enum.ml bitSet.ml dbi.ml dynArray.ml extHashtbl.ml extList.ml extString.ml global.ml IO.ml option.ml pMap.ml std.ml uChar.ml uTF8.ml base64.ml unzip.ml refList.ml optParse.ml dllist.ml extLib.ml make[1]: Leaving directory `/mnt/local/src/extlib/extlib-dev' make -f test.mak all make[1]: Entering directory `/mnt/local/src/extlib/extlib-test' ocamlc -I ../extlib-dev -g extLib.cma util.ml test_jh_Base64.ml test_Base64.ml test_jh_BitSet.ml test_BitSet.ml test_jh_ExtList.ml test_ExtList.ml test_jh_ExtString.ml test_ExtString.ml test_main.ml -o extlib_test make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test' [skaller@pelican] /usr/local/src/extlib/extlib-test>make run make -f test.mak run make[1]: Entering directory `/mnt/local/src/extlib/extlib-test' ./extlib_test Extlib tester started.. run: Base64 run: jh_Base64.test run: BitSet run: jh_BitSet.test_intersect run: jh_BitSet.test_diff run: jh_BitSet.test_rnd_creation ..FAILED reason: test_jh_BitSet.ml:60:4 test jh_BitSet.test_rnd_creation failed run: jh_BitSet.test_empty run: jh_BitSet.test_exceptions ..FAILED reason: test_jh_BitSet.ml:116:2 test jh_BitSet.test_exceptions failed run: ExtString run: jh_ExtString.t_starts_with run: jh_ExtString.t_map run: jh_ExtString.t_lchop run: jh_ExtString.t_rchop run: jh_ExtString.t_split run: ExtList run: jh_ExtList.iteri run: jh_ExtList.mapi run: jh_ExtList.exceptions run: jh_ExtList.find_exc All tests completed. make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test' All this is expected, including the failures (hmm .. it says all tests completed, but doesn't report how many faiures .. still this can be fixed later, for now the important step seems achieved .. there is a place to put tests and a protocol for constructing them. ------------------------------------------------------------------------ I added a test for Dynarray (just a stub at the moment). I got this: .... Checking in test_main.ml; /cvsroot/ocaml-lib/extlib-test/test_main.ml,v <-- test_main.ml new revision: 1.3; previous revision: 1.2 done sh: line 1: /cvsroot/ocaml-lib/CVSROOT/commit.log: Permission denied ********************************************************************** Looks like the commits worked, but the log wasn't updated?? [Comments on test harness next post] -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Janne H. <ja...@hy...> - 2004-12-21 18:57:39
|
Hi, thank you for taking a peek. A nice way to checkout whole extlib is just to do cvs -d :ext:use...@cv...:/cvsroot/ocaml-lib co . (note the dot) This will checkout both extlib-dev and extlib-test. If you then go to extlib-test and run "make all" it will build both the ExtLib and the testing suite. While I agree with some people here about making the test suite an integral part of extlib, I think the current CVS layout allows people to both checkout and build&run the tester without a hassle. Logging fail/success: Actually I can easily do this with Util.run_test. Perhaps I will add this in the near future. Commit.log: I have no idea what that's all about.. I wondered about the same thing, but figured it's not a problem since the commit worked and individual file logs seem to be working OK. I will address your (Skaller) other e-mail later (hopefully today). ciao, janne skaller wrote: > On Tue, 2004-12-21 at 07:24, Janne Hellsten wrote: > >>Just to inform everyone, I have just imported the testing suite into CVS. >>The module layout is as proposed by Nicolas. >> > > > OK, I import it like this: > > My system has the structure: > > /usr/local/src/extlib/extlib-dev > > So I did this: > > cd /usr/local/src/extlib > export CVS_RSH=ssh > cvs -z3 -d:ext:sk...@cv...:/cvsroot/ocaml-lib co > extlib-test > > Now I have this: > > [skaller@pelican] /usr/local/src/extlib>ls > extlib-dev extlib-test > > > [skaller@pelican] /usr/local/src/extlib>ls extlib-test/ > CVS test_Base64.ml test_ExtString.ml test_jh_ExtList.ml > test.mak > Makefile test_BitSet.ml test_jh_Base64.ml test_jh_ExtString.ml > util.ml > README test_ExtList.ml test_jh_BitSet.ml test_main.ml > > > > [skaller@pelican] /usr/local/src/extlib>cd extlib-test/ > /usr/local/src/extlib/extlib-test > > > > [skaller@pelican] /usr/local/src/extlib/extlib-test>make > cd ../extlib-dev && make all > make[1]: Entering directory `/mnt/local/src/extlib/extlib-dev' > ocamlc -a -o extLib.cma enum.mli bitSet.mli dbi.mli dynArray.mli > extHashtbl.mli > extList.mli extString.mli global.mli IO.mli option.mli pMap.mli std.mli > uChar.mli uTF8.mli base64.mli unzip.mli refList.mli optParse.mli > dllist.mli enum.ml bitSet.ml dbi.ml dynArray.ml extHashtbl.ml extList.ml > extString.ml global.ml IO.ml option.ml pMap.ml std.ml uChar.ml uTF8.ml > base64.ml unzip.ml refList.ml optParse.ml dllist.ml extLib.ml > make[1]: Leaving directory `/mnt/local/src/extlib/extlib-dev' > make -f test.mak all > make[1]: Entering directory `/mnt/local/src/extlib/extlib-test' > ocamlc -I ../extlib-dev -g extLib.cma util.ml test_jh_Base64.ml > test_Base64.ml test_jh_BitSet.ml test_BitSet.ml test_jh_ExtList.ml > test_ExtList.ml test_jh_ExtString.ml test_ExtString.ml test_main.ml -o > extlib_test > make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test' > > [skaller@pelican] /usr/local/src/extlib/extlib-test>make run > make -f test.mak run > make[1]: Entering directory `/mnt/local/src/extlib/extlib-test' > ./extlib_test > Extlib tester started.. > > run: Base64 > run: jh_Base64.test > run: BitSet > run: jh_BitSet.test_intersect > run: jh_BitSet.test_diff > run: jh_BitSet.test_rnd_creation ..FAILED > reason: > test_jh_BitSet.ml:60:4 test jh_BitSet.test_rnd_creation failed > > run: jh_BitSet.test_empty > run: jh_BitSet.test_exceptions ..FAILED > reason: > test_jh_BitSet.ml:116:2 test jh_BitSet.test_exceptions failed > > run: ExtString > run: jh_ExtString.t_starts_with > run: jh_ExtString.t_map > run: jh_ExtString.t_lchop > run: jh_ExtString.t_rchop > run: jh_ExtString.t_split > run: ExtList > run: jh_ExtList.iteri > run: jh_ExtList.mapi > run: jh_ExtList.exceptions > run: jh_ExtList.find_exc > All tests completed. > make[1]: Leaving directory `/mnt/local/src/extlib/extlib-test' > > > All this is expected, including the failures (hmm .. it says > all tests completed, but doesn't report how many faiures .. still > this can be fixed later, for now the important step seems > achieved .. there is a place to put tests and a protocol for > constructing them. > > ------------------------------------------------------------------------ > > I added a test for Dynarray (just a stub at the moment). > I got this: > > .... > > Checking in test_main.ml; > /cvsroot/ocaml-lib/extlib-test/test_main.ml,v <-- test_main.ml > new revision: 1.3; previous revision: 1.2 > done > sh: line 1: /cvsroot/ocaml-lib/CVSROOT/commit.log: Permission denied > ********************************************************************** > > Looks like the commits worked, but the log wasn't updated?? > > [Comments on test harness next post] > |
From: skaller <sk...@us...> - 2004-12-22 02:22:35
|
On Wed, 2004-12-22 at 05:57, Janne Hellsten wrote: > Logging fail/success: Actually I can easily do this with Util.run_test. > Perhaps I will add this in the near future. > > Commit.log: I have no idea what that's all about.. I wondered about the > same thing, but figured it's not a problem since the commit worked and > individual file logs seem to be working OK. I think one of the two administrators will need to address this issue. Probably just a minor permission glitch. -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Nicolas C. <war...@fr...> - 2004-12-22 06:58:32
|
> > Logging fail/success: Actually I can easily do this with Util.run_test. > > Perhaps I will add this in the near future. > > > > Commit.log: I have no idea what that's all about.. I wondered about the > > same thing, but figured it's not a problem since the commit worked and > > individual file logs seem to be working OK. > > I think one of the two administrators will need to address > this issue. Probably just a minor permission glitch. Yes but I don't have access to the file rights on the CVSROOT directory. I tried several times to disable the log and it failed, and I asked one time to SF managers but they didn't fix it right. Not so big deal... Nicolas |
From: Brian H. <bh...@sp...> - 2005-01-17 00:35:13
|
My apologies for being silent for so long (life intervened). One question on this subject: is there a reason we're not using OUnit? I have some minor annoyances with it (it needs findlib installed is the biggest one), but not enough to write my own. I'm about 1/2 way done with a module implementing Chris Okasaki's random access lists, if people are interested. For those who don't know, these are lists that work mostly like normal linked lists, except that getting the nth element is O(log N), and not O(N). Several other operations are similiarly more efficient. The reason I bring this up is that I'm writting the test suite in OUnit. Brian |
From: skaller <sk...@us...> - 2005-01-17 01:38:37
|
On Mon, 2005-01-17 at 11:37, Brian Hurt wrote: > My apologies for being silent for so long (life intervened). One > question on this subject: is there a reason we're not using OUnit? Dependencies. The test mechanism needs to be independent of everything that isn't either in standard Ocaml distro or Extlib. This problem would go away if GODI was standard but it isn't .. -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Bardur A. <oca...@sc...> - 2005-01-17 08:56:32
|
On Mon, Jan 17, 2005 at 12:38:28PM +1100, skaller wrote: > On Mon, 2005-01-17 at 11:37, Brian Hurt wrote: > > My apologies for being silent for so long (life intervened). One > > question on this subject: is there a reason we're not using OUnit? > > Dependencies. The test mechanism needs to be independent > of everything that isn't either in standard Ocaml > distro or Extlib. It seems that OUnit doesn't actually depend on anything else (well, except findlib for the build procedure, but that's easily remedied -- it's just one module after all), so I'm guessing you mean that you don't want ExtLib to become dependent on OUnit...? What about just bundling a version of OUnit with ExtLib? The code seems to be under a BSD-ish license (w/o advertising clause), so this would be permitted and it shouldn't really cause any maintenance issues as the OUnit code (nor any code using OUnit) shouldn't be installed anyway. Cheers, -- Bardur Arantsson <ba...@im...> <ba...@sc...> Light a man a fire, you warm him for a night. Light a man on fire, he'll be warm the rest of his life. |
From: Janne H. <ja...@hy...> - 2005-01-17 09:16:35
|
Hi all, I am willing to help if you have any trouble with the existing ExtLib tester. I think the assertion mechanism is easy to use and easy to understand, but of course I'm biased since I wrote the suite. extlib-test/readme.html has a some documentation on how to make new tests. When I embarked on writing the test suite, I was more interested in making the actual tests rather than discussing on what kind of a framework to use for the tests. Perhaps someone could clarify what is gained by using OUnit instead of the current suite? I personally feel that the real test cases are what counts, not the testing framework. Best regards, Janne skaller wrote: > On Mon, 2005-01-17 at 11:37, Brian Hurt wrote: > >>My apologies for being silent for so long (life intervened). One >>question on this subject: is there a reason we're not using OUnit? > > > Dependencies. The test mechanism needs to be independent > of everything that isn't either in standard Ocaml > distro or Extlib. > > This problem would go away if GODI was standard > but it isn't .. > |
From: Nicolas C. <war...@fr...> - 2005-01-17 10:36:11
|
I agree here. The one willing to write the tests are free to choose how they want to write them. Nicolas > Hi all, > > I am willing to help if you have any trouble with the existing ExtLib > tester. I think the assertion mechanism is easy to use and easy to > understand, but of course I'm biased since I wrote the suite. > > extlib-test/readme.html has a some documentation on how to make new tests. > > When I embarked on writing the test suite, I was more interested in > making the actual tests rather than discussing on what kind of a > framework to use for the tests. > > Perhaps someone could clarify what is gained by using OUnit instead of > the current suite? I personally feel that the real test cases are what > counts, not the testing framework. > > Best regards, > Janne > > skaller wrote: > > On Mon, 2005-01-17 at 11:37, Brian Hurt wrote: > > > >>My apologies for being silent for so long (life intervened). One > >>question on this subject: is there a reason we're not using OUnit? > > > > > > Dependencies. The test mechanism needs to be independent > > of everything that isn't either in standard Ocaml > > distro or Extlib. > > > > This problem would go away if GODI was standard > > but it isn't .. |
From: skaller <sk...@us...> - 2004-12-21 07:04:28
|
On Tue, 2004-12-21 at 07:24, Janne Hellsten wrote: [Makefile .. test structure] Janne currently has a three level test structure: 1. The mainline calls the test for each module. 2. The module test calls individual tests for each module 3. Individual tests call subtests In all 3 cases the call looks like this: let test () = Util.run_test ~test_name:"jh_ExtList.iteri" test_iteri; Util.run_test ~test_name:"jh_ExtList.mapi" test_mapi; Util.run_test ~test_name:"jh_ExtList.exceptions" test_exceptions; Util.run_test ~test_name:"jh_ExtList.find_exc" test_find_exc In addition each file has to be listed in the file test.mak which is 'triple handling', in fact worse, since the test name has to be written as well.. :) Ideally, the test.mak, mainline, and module level test function should be generated, not hand coded, that is the Makefile should generate the these things using a refined equivalent of 'ls test_*.ml'. I.e. the to create a new test you just commit the test file -- you don't change any existing files, instead the files are constructed. I suppose I can write an Ocaml program which will create these files. This would work as follows: (1) Look in extlib-dev for all mli files, to find the names of top level modules. [Alternative -- hard code the list of module names] (2) Generate the mainline 'test_mail.ml' using these module names. (3) For each module, generate a test file test_<module>.ml, which calls each test for that module in the extlib-test directory found with name test_*_<module>_*.ml Note this implies names like test_jh_ExtList.ml would not be allowed, a suffix would be required. That could be fixed by also allowing names of that form. I personally think we should mandate the suffix: we'd better decide fast, before there are too many tests: CVS isn't good at file renaming .. :) Capitalisation rules would apply (the module names must be exactly as used in Ocaml code). I'm not sure how this will work on systems that aren't strict about filenames.. An alternative to an Ocaml program to do this would be a bash script.. probably easier to write but could be less portable, considering native Win32 environment for example. A BIG advantage of automatically generating the test call protocol, apart from ease of adding new tests, is that the actual protocol can be changed to do things like count success/failure, generate error reports blah blah .. without manual recoding, except possibly at the individual test level. It may actually be possible to build the individual test() functions too, if the test functions within the individual test files are annotated with some special kind of mark eg: TEST inner_test_name ...... which is replaced by let inner_test_name () = but which also gets added to the harness at the bottom of the file like this: let test () = Util.run_test ~test_name:"js_DynArray.inner_test_name" inner_test_name ... Thus fully automating the whole suite construction, requiring only the special form of declaration. This would imply changing the CVS filenames to something like test_jh_ExtLib_123.tst [to avoid overwriting them when generating the *.ml file] Comments? -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Janne H. <ja...@hy...> - 2004-12-20 10:11:50
|
> Ok this looks great. The most important issue right now IMHO is > not how the test manager works, but where it lives in CVS. > > Nicolas said at one stage that the test suite shouldb't be > part of the distribution. Whether or not this is the case, > I would say it should be easily separable. Thank you for taking a look! > IMHO test code and library code should be siblings, > the tests should be in a subdir of the src (just my opinion). I agree. However, I'll have to familiarize myself with the current state of the ExtLib CVS before I can say anything more about this though. BTW: Perhaps I should add the license/copyright notice from ExtLib to test suite files? ciao, janne |
From: skaller <sk...@us...> - 2004-12-18 00:17:26
|
On Sat, 2004-12-18 at 00:40, Bardur Arantsson wrote: > I think we may be discussing different things... I'm only > talking about something which ExtLib developers can use > exclusively for testing ExtLib itself, not about making a > test harness available to ExtLib users. No, we're talking about the same thing. > In general, yes, but it doesn't mean that we can't agree > on an *internal* method for testing ExtLib itself. Well, can you could write a specification? Assume a directory containing tests, people can dump tests in it. The tests need names -- what names? it must be possible to specify expected result, the harness must report which test(s) (if any) fail. You can use an 'assert' test (if the code runs at all it passes). The problem with that is that library function are hard to test like that.. How do you test 'string.split'? This is not so easy! Try: assert (x = combine "/" (split x "/")) Which suggests making the split char vary. How do you test ExtSet? You need stuff to put in it, where does it come from? -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Janne H. <ja...@hy...> - 2004-12-21 20:19:04
|
Hi, > which is 'triple handling', in fact worse, since the test name > has to be written as well.. :) Well, it depends on how many test contributors and how many tests we're expecting. I'm not expecting that many, so I don't consider adding tests such a big problem. > Ideally, the test.mak, mainline, and module level > test function should be generated, not hand coded, > that is the Makefile should generate the these things > using a refined equivalent of 'ls test_*.ml'. This might be doable with GNU Make's $(wildcard) and some other tricks. I never tried, but it seems doable. Then again, if I *don't* do this, the Makefile would be more straightforward and would even allow test contributors to write their own small utility modules. If I want to use wildcards, I will also need a portable way to automatically make the .ml files topologically sorted. However, as to your other suggestions, frankly, I think you're making the build process unnecessarily complicated. I don't see how adding new tests can ever become a bottleneck. Nor do I think it can be somehow unsafe (even if a couple of files need to be modified), due to O'Caml's strong static typing. Anyhow.. If someone will make a more sophisticated build system in the future, I'm OK with that.. Just make 100% sure that it ports to Windows. I noticed there's been some talk about adding path handling into ExtLib. These *definitely* need to be tested on Windows to make sure they work as expected! Currently my main concern though is getting more meaningful tests into the suite.. > test_*_<module>_*.ml > > Note this implies names like > > test_jh_ExtList.ml > > would not be allowed, a suffix would be required. > That could be fixed by also allowing names of that form. > I personally think we should mandate the suffix: > we'd better decide fast, before there are too many tests: > CVS isn't good at file renaming .. :) Surely you can just check if the suffix is missing and if so, automatically name it "untitled". I find it perfectly OK to have more than one test in one file. In some cases it allows to share some code between individual tests. See test_jh_BitSet.ml for an example (bitset_of_int, etc). As for renaming.. Grrr I wish we could use Subversion.. :) ciao, janne |
From: skaller <sk...@us...> - 2004-12-22 02:46:09
|
On Wed, 2004-12-22 at 07:18, Janne Hellsten wrote: > Hi, > > > which is 'triple handling', in fact worse, since the test name > > has to be written as well.. :) > > Well, it depends on how many test contributors and how many tests we're > expecting. I'm not expecting that many, so I don't consider adding > tests such a big problem. But you are only one contributor to the project. Even if you personally don't expect many contributions, you probably shouldn't make a decision without consulting list members. > This might be doable with GNU Make's $(wildcard) and some other tricks. > I never tried, but it seems doable. > Then again, if I *don't* do this, the Makefile would be more > straightforward and would even allow test contributors to write their > own small utility modules. If I want to use wildcards, I will also need > a portable way to automatically make the .ml files topologically sorted. > However, as to your other suggestions, frankly, I think you're making > the build process unnecessarily complicated. I don't see how adding new > tests can ever become a bottleneck. It already has been for me. It took me 10 times longer than necessary to add a single 3 line test. I need to question your assumption that automating the build of the test system will make it more complicated. In fact, I believe the opposite is true: we can probably dispense *completely* with any makefile, and would provide one only as a convenience. I have had this reaction before -- automation is one of my specialities, it really *isn't* that hard to do. > Nor do I think it can be somehow > unsafe (even if a couple of files need to be modified), due to O'Caml's > strong static typing. The static typing does not (a) guarrantee a test will be called (b) ensure the test is in the correct category (c) the test is properly named However it is worse. Remember we're using CVS. A system which cleanly separates changes into files is obviously better than the current situation, where every tester is going to modify files *other* testers will also have to modify. Whether CVS can handle simultaneous changes or not, it is probably better to make it easier than harder :) > Anyhow.. If someone will make a more sophisticated build system in the > future, I'm OK with that.. Just make 100% sure that it ports to Windows. Indeed this is a strong argument against Makefiles and shell script. An Ocaml based solution is more likely to be portable. > I noticed there's been some talk about adding path handling into > ExtLib. These *definitely* need to be tested on Windows to make sure > they work as expected! Yes. > Currently my main concern though is getting more meaningful tests into > the suite.. This will come over time. I wouldn't worry so much about that as setting up the mechanism so that it is as easy as possible for people to contribute tests. > > test_*_<module>_*.ml > > > > Note this implies names like > > > > test_jh_ExtList.ml > > > > would not be allowed, a suffix would be required. > Surely you can just check if the suffix is missing and if so, > automatically name it "untitled". Of course once could do this -- the question is: do we want to? I'm in favour of strictness and simplicity, and this format breaks those guidelines. > I find it perfectly OK to have more than one test in one file. In some > cases it allows to share some code between individual tests. See > test_jh_BitSet.ml for an example (bitset_of_int, etc). I have no problem with that. Nor the current 3 level design I described before. I'm not sure it is the best, but I'm quite happy to work with it for the moment. My concern is solely to limit the work of test designers to making a file containing tests and then submitting it, perhaps to CVS, and perhaps just posting on the mailing list so someone else with access can 'cut and paste' the example into the test suite with minimal effort. > As for renaming.. Grrr I wish we could use Subversion.. :) LOL! Whatever, the time to make this kind of change is right now, before it is too hard and we're stuck with a possibly suboptimal design. -- John Skaller, mailto:sk...@us... voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net |
From: Richard J. <ri...@an...> - 2004-12-16 12:05:18
|
On Thu, Dec 16, 2004 at 10:22:05AM +0100, Nicolas Cannasse wrote: > Hi list, >=20 > It's been long time without activity on this list, I have been quite busy > lately but I keep on fixing bugs when reports are coming sometimes ;). Ri= ght > now the ExtLib acheivements are pretty good : I am a happy ExtLib user an= d I > don't feel anymore when I write Ocaml that I miss all theses functions fr= om > Stdlib. That's good, and I hope you feel same. >=20 > As for the future, I am not certain which path we should follow. I tend to > appreciate the current shape of ExtLib and I'm not wishing to include a l= ot > more modules, but other people might think different. What I would like to > be included in ExtLib would be small Xml and Date modules, but I'm not su= re > we can reach agreement on what should be put inside theses ;). Any insigh= ts > or wishes are welcome ! ExtLib works great for me. The only thing I'd like to see is more of the same. Some ideas: List.takewhile and List.dropwhile still seem to be missing (I posted a patch for this on list some time ago). Simple config file parsing. YAML-compatibility would be awesome. Or take a look at ConfigParser in JG's MissingLib. List.range: let rec range a b =3D if a <=3D b then a :: range (a+1) b else [] String.truncate: let truncate ?(with_dots =3D false) ?(dots =3D " ...") n str =3D if String.length str < n then str else ( if not with_dots then String.sub str 0 n else ( let n =3D n - String.length dots in let str =3D String.sub str 0 n in str ^ dots ) ) Controversial, because they only work on ASCII-compatible string encodings (ie. ASCII, UTF-8, ISO-8859-1, but not UCS-2), but extremely useful: val isspace : char -> bool val isalpha : char -> bool val isdigit : char -> bool val isalnum : char -> bool val islower : char -> bool val isupper : char -> bool val isxdigit : char -> bool =46rom this you can have trimming functions: let triml ?(test =3D isspace) str =3D let i =3D ref 0 in let n =3D ref (String.length str) in while !n > 0 && test str.[!i]; do decr n; incr i done; if !i =3D 0 then str else String.sub str !i !n =20 let trimr ?(test =3D isspace) str =3D let n =3D ref (String.length str) in while !n > 0 && test str.[!n-1]; do decr n done; if !n =3D String.length str then str else String.sub str 0 !n =20 let trim ?test str =3D trimr ?test (triml ?test str) And also String.for_all / String.exists functions: let string_for_all f str =3D let len =3D String.length str in let rec loop i =3D if i =3D len then true else ( let c =3D str.[i] in if not (f c) then false else loop (i+1) ) in loop 0 =20 let string_exists f str =3D let len =3D String.length str in let rec loop i =3D if i =3D len then false else ( let c =3D str.[i] in if f c then true else loop (i+1) ) in loop 0 let string_is_whitespace =3D string_for_all isspace List.uniq and List.sort_uniq: let rec uniq ?(cmp =3D Pervasives.compare) =3D function [] -> [] | [x] -> [x] | x :: y :: xs when compare x y =3D 0 -> uniq (x :: xs) | x :: y :: xs -> x :: uniq (y :: xs) =20 let sort_uniq ?cmp xs =3D let xs =3D List.sort ?cmp xs in let xs =3D uniq ?cmp xs in xs List.group_by is useful when you're pulling stuff out of a database: let group_by ?(cmp =3D Pervasives.compare) ls =3D let ls' =3D List.fold_left (fun acc (day1, x1) -> match acc with [] -> [day1, [x1]] | (day2, ls2) :: acctl -> if cmp day1 day2 =3D 0 then (day1, x1 :: ls2) :: acctl else (day1, [x1]) :: acc) [] ls in let ls' =3D List.rev ls' in List.map (fun (x, xs) -> x, List.rev xs) ls' Meta-functions, like in standard ML: let notf f =3D fun v -> not (f v) That'll do for the moment, I think! Rich. --=20 Richard Jones. http://www.annexia.org/ http://www.j-london.com/ >>> http://www.team-notepad.com/ - collaboration tools for teams <<< Merjis Ltd. http://www.merjis.com/ - improving website return on investment Write Apache modules in OCaml - http://www.merjis.com/developers/mod_caml |
From: Bardur A. <ba...@sc...> - 2004-12-16 12:48:20
|
On Thu, Dec 16, 2004 at 12:03:24PM +0000, Richard Jones wrote: > On Thu, Dec 16, 2004 at 10:22:05AM +0100, Nicolas Cannasse wrote: > > Hi list, > > > > It's been long time without activity on this list, I have been quite busy > > lately but I keep on fixing bugs when reports are coming sometimes ;). Right > > now the ExtLib acheivements are pretty good : I am a happy ExtLib user and I > > don't feel anymore when I write Ocaml that I miss all theses functions from > > Stdlib. That's good, and I hope you feel same. > > > > As for the future, I am not certain which path we should follow. I tend to > > appreciate the current shape of ExtLib and I'm not wishing to include a lot > > more modules, but other people might think different. What I would like to > > be included in ExtLib would be small Xml and Date modules, but I'm not sure > > we can reach agreement on what should be put inside theses ;). Any insights > > or wishes are welcome ! > > ExtLib works great for me. The only thing I'd like to see is more of > the same. Some ideas: > > List.takewhile and List.dropwhile still seem to be missing (I posted a > patch for this on list some time ago). > > Simple config file parsing. YAML-compatibility would be awesome. Or > take a look at ConfigParser in JG's MissingLib. > > List.range: > > let rec range a b = > if a <= b then > a :: range (a+1) b > else > [] IMO, it's usually a bad idea to make both ends of a range inclusive, because you usually use a length to either derive the endpoints or expect the number of returned elements to be (b-a). ... and wouldn't range be better as an Enum constructor? > > String.truncate: > > let truncate ?(with_dots = false) ?(dots = " ...") n str = > if String.length str < n then > str > else ( > if not with_dots then > String.sub str 0 n > else ( > let n = n - String.length dots in > let str = String.sub str 0 n in > str ^ dots > ) > ) I'm not sure this is "generally useful" in any meaningful way... When you need to truncate (with dots) you usually want more control, eg. truncation of paths would usually truncate some characters in the middle. In the simple case (without dots) you can just use String.slice (which will never throw an excpetion, just like truncate). [--snip--] > From this you can have trimming functions: > > let triml ?(test = isspace) str = ... > let trimr ?(test = isspace) str = ... > let trim ?test str = ... ... or lstrip/rstrip/strip as they would be known according to the current naming conventions. :) The fact that trim takes a "test" closure instead of string with characters should IMHO be regarded as a feature and the 'old' strip should just be removed... Of course it may mean that the compiler cannot inline the "test" calls, but that is, again IMHO, a compiler issue. [--snip--] > > List.uniq and List.sort_uniq: > > let rec uniq ?(cmp = Pervasives.compare) = function > [] -> [] > | [x] -> [x] > | x :: y :: xs when compare x y = 0 -> ^--- you mean "cmp", right? > uniq (x :: xs) > | x :: y :: xs -> > x :: uniq (y :: xs) > > let sort_uniq ?cmp xs = > let xs = List.sort ?cmp xs in > let xs = uniq ?cmp xs in > xs > [--snip--] Cheers, -- Bardur Arantsson <ba...@im...> <ba...@sc...> - Slow down, sir! You're going to give yourself skin failure! Dr. Nick | The Simpsons |
From: Richard J. <ri...@an...> - 2004-12-16 13:38:14
|
On Thu, Dec 16, 2004 at 01:48:04PM +0100, Bardur Arantsson wrote: > > From this you can have trimming functions: > >=20 > > let triml ?(test =3D isspace) str =3D ... > > let trimr ?(test =3D isspace) str =3D ... > > let trim ?test str =3D ... >=20 > ... or lstrip/rstrip/strip as they would be known > according to the current naming conventions. :) >=20 > The fact that trim takes a "test" closure instead of > string with characters should IMHO be regarded as a > feature and the 'old' strip should just be removed... Of > course it may mean that the compiler cannot inline the > "test" calls, but that is, again IMHO, a compiler issue. Or have both ... Rich. --=20 Richard Jones. http://www.annexia.org/ http://www.j-london.com/ >>> http://www.team-notepad.com/ - collaboration tools for teams <<< Merjis Ltd. http://www.merjis.com/ - improving website return on investment Use Perl libs in OCaml - http://www.merjis.com/developers/perl4caml |