Re: [Tcllib-devel] Writing tests - how?

[Lars H. <Lar...@re...>]

Andreas Kupries skrev:
>> Well, what about a small helper package? One could make the above
>>
>> package require tcllib::testutilities
>> tcllib::testutilities::init
>>
>> or something like that. The separate [tcllib::testutilities::init] call
>> would be needed to retrieve the [info script] data, although now that I
>> think about it, one could (if really devious) embed such a call into
>> the [package ifneeded] script as well. An explicit call does however
>> have the advantage that one can, if necessary, customise the
>> initialisation (no, I don't have a clear use-case for that right now;
>> maybe too tired).

Having slept on the matter, I now have "configuration for nonstandard 
test environment" as use-case. The initialisation command put in the 
.test file would do a default init (e.g. assuming tcllib CVS 
organisation of files, using current file as reference), but extra 
commands given before sourcing the test file could specify overrides 
for the defaults.

For example, one might do:

package require tcllib::testutilities
tcllib::testutilities::declaredir modules/math \
   /Library/Tcl/tcllib-1.6/math
source foo.test

to specify an override that "whenever the test file wants to [use] 
something in modules/math, it should use the corresponding file in 
/Library/Tcl/tcllib-1.6/math". There's no harm in doing [package 
require tcllib::testutilities] twice, unlike the corresponding explicit 
[source].


>> What is gained by [package require tcllib::testutilities] is primarily
>> that one avoids hardcoding paths (relative paths, yes, but still
>> hardcoded paths) into the test files.
> 
> The price you are paying for this is that you now cannot run the testsuite
> anymore if Tcllib is not installed because the package won't be found. Or
> using the wrong helper package if some other version of Tcllib is installed.

That's a very pessimistic view of the matter. The only case I can think
of in which one would like to run the tcllib testsuite without having
/any/ version of tcllib installed is "make test"-style activities:
testing new software before actually installing it. In this situation,
the environment is fairly well controlled, so providing an extra
package is not a problem.

Developers working on tcllib should be expected to at least have some
version installed, so unless you foresee several radical changes in the
testutilities API, problems because the installed version is not that
of CVS HEAD should be rare, and easily recognised when they eventually
occur. What could be inconvenient is the transition period when such a
package is being introduced into tcllib, but OTOH it would probably
take a couple of years before any large portion of tests were rewritten
to take advantage of it. ;-)

> Running the testsuite of Tcllib without Tcllib installed is IMHO an
> important use case during development.

What is *lost* with the

source [file join \
	[file dirname [file dirname [file join [pwd] [info script]]]] \
	devtools testutilities.tcl]

arrangement is the ability to run tests /without having all of the
tcllib repository at hand/. Wherever I have a foo.test, it is assumed
that there is a ../devtools/testutilities.tcl to source. There will be
such a file if you've checked out all of tcllib or tcllib/modules from
CVS, but at least I find it much more prudent to check out only "my
own" module (especially if the project hierarchy in which it was
originally developed doesn't match tcllib).

> To re-enable execution you have to extend the auto_path, 

Which is quite feasible to do from the environment in the "make test"
scenario, and unnecessary otherwise.

> adding again the path information you do not want to the .test files.

Of course it shouldn't be the responsibility of the .test files to
provide this information! The main point of having the [package]
mechanism is to allow code to call upon other code without knowing
exactly where it is found!

> Even so it stays
> brittle because anybody can intercept this by adding a higher version of the
> helper package anywhere on the auto_path.

If that is brittle, then it is so only because the development of the
helper package is being mismanaged! A higher version of the package
will only be picked if it has the same major version number as the one
you require. The major version number must be incremented if the
package is changed so that it is not backwards compatible. So how could
newer versions be a problem?

Lars Hellström