[Tcllib-devel] Writing tests - how?

[Lars H. <Lar...@re...>]

Apologies if this has been discussed before (at least I'm certain I've 
been bothered by this before), but a search on the SF mail archives did 
not find anything within the last year and a half or so.

Basic problem: The tcllib README states

  * the module should have both documentation ([*]) and a test suite
    (in the form of a group of *.test files in the module directory).
  * the module must have either documentation or a test suite.  It can
    not have neither.

and README.developer continues

  * If available the testsuite(s) should use 'tcltest' and the
    general format as used by the other modules in Tcllib
    (declaration of minimally needed Tcl, tcltest, supporting
    packages, etc.). The file(s) should have the extension
    '.test' for SAK to recognize them.

but what does this mean concretely? I'd interpret it as saying that 
e.g. foo.tcl should come with foo.test, where the latter looks like:


package require Tcl 8.4 ; # Declaration of minimally needed Tcl
package require tcltest 2 ; # Ditto tcltest
package require bar 1.3.2 ; # Supporting package

# Now follows the test(s). (Currently, there is only one.)
tcltest::test foo-1.1 "Testing foo" -body {
    foo
} -result "baz"


but I rather doubt this would work; for one, the foo package itself 
doesn't seem to be loaded. Is that perhaps the "etc." in 
README.developer ;-), or is there rather some automagic [source]ry 
going on? More information seems to be required. (Yes, I will find out 
below that the above probably doesn't refer to [package require] at 
all, but rather to [testsNeedTcl] and friends.)

A tcllib contributor might of course at this point choose to just 
duplicate whatever is done in existing .test files in tcllib, but 
doing-without-understanding is not a strategy I'm comfortable with. 
Besides, when I /have/ picked modules at random and looked at their 
.test files, I haven't come across anything that looked like a 
boilerplate test file.

So, I took a look around the CVS archive, in hunt for better 
documentation. There is for example a devdoc directory 
(http://tcllib.cvs.sourceforge.net/tcllib/tcllib/devdoc/) -- maybe this 
contains more information on the matter? Sadly, it doesn't. Also, most 
of the information therein seems to be rather old (even if some files 
were modified 9 months ago). What else is there?

There is also a support/devel directory, which contains all.tcl and sak 
  -- since this is the software canonically running the tests, maybe 
this is where one might find some documentation on the matter? Sadly, 
no. There are several help.txt files, but those only contain 
interactive help messages from the sak application, so the only actual 
documentation I can find therein is for the embedded pregistry package. 
Nothing to help with test files...

The mail achive search did however turn up a mention of 
devtools/testutilities -- where might this be found? As it turns out, 
devtools is a *module* in the CVS repository 
(http://tcllib.cvs.sourceforge.net/tcllib/tcllib/modules/devtools/), 
although it doesn't get installed as a module (only natural, as it has 
neither documentation nor test suite; this is beginning to feel like a 
pattern).

A lot of what is in devtools/ looks very interesting:

   * testutilities.tcl:[useLocalFile] to make sure I'm testing the 
version of my package matching the test rather than some random older 
version I happen to have installed.
   * coserv.tcl (for later, testing an IPC package I'm working on)

but then there's the matter of missing documentation, and questions it 
should have answered:

   * Should one use [useLocalFile], [useTcllibFile], [useLocal], or 
what? When and why?
   * How are these commands supposed to be made available? When can one 
rely on them?

What is currently available /may/ of course be interpreted as "well, 
just copy and modify pop3.test". I suppose this will work (many have 
probably done that) but already the first commmand

source [file join \
  	[file dirname [file dirname [file join [pwd] [info script]]]] \
  	devtools testutilities.tcl]

there is enough to make me shun away. Surely there should be a better 
way to do this?


Lars Hellström