Re: [Ocaml-lib-devel] Future of ExtLib

[Bardur A. <oca...@sc...>]

On Sat, Dec 18, 2004 at 12:07:30AM +1100, skaller wrote:

> > To my mind, using full-blown parsers is overkill for
> > splitting UNIX paths into their constituent parts, 
> 
> But we're not restricted to Unix..

I was thinking specifically of the UNIX case...

[--snip--] 
> > Maybe I'm just stupid, but I don't see why a test harness
> > would require lots of work...?
> 
> Because it has to
> 
> (a) Collate all the tests
> (b) Run the tests -- terminating rogues 
> (c) Collate the results
> (d) Standardise a way to actually report results

... all of which my small code snippet does (well, would
do if it were actually syntactically valid...) when
combined with OUnit...?!?

   (a) Tests are combined in all_tests.
   (b) OUnit takes care of that (there is no parallel
       execution, but that's hardly a huge problem for
       testing ExtLib code).
   (c) OUnit
   (d) ./test returns 0 if all tests were succesful,
       non-zero otherwise. This enables the "make check"
       target to fail, install.ml can run ./test and check its
       return code.

I think we may be discussing different things... I'm only
talking about something which ExtLib developers can use
exclusively for testing ExtLib itself, not about making a
test harness available to ExtLib users.

> > Am I missing something?
> 
> Yeah -- designing a test system is probably harder
> than designing a library or an average application.
> This is especially the case for Ocaml I suspect,
> since it doesn't support dynamic loading and unloading. 
> That seems to mean each test must be a separate process.

In general, yes, but it doesn't mean that we can't agree
on an *internal* method for testing ExtLib itself.

> 
> In addition some tests are dangerous, especially ones
> that mess with the filesystem or network -- usually you'd run
> your tests in a suitably restricted environment
> as an low privilege user.. which makes the test
> harnes OS specific ..

Sure, but what I'm talking about is much simpler: *We*
would be the ones writing the tests, so we simply avoid
writing test cases for potentially dangerous things (ie.
tests with side effects outide of the "virtual machine").
Thus, there is no need to protect from such secondary
effects. Of course it means less coverage, but I can
certainly live with that, especially since the alternative
is *no* coverage.

> 
> I personally think 'unit' testing is a bit silly.
> It rarely finds bugs because it can't cover enough cases,
> can't handle integration, etc.. I prefer a more sloppy
> concept of just collecting whatever test code you can
> and running it, just to get some confidence you didn't
> completely mess something up whilst committing a minor
> change to CVS.

Yes, perhaps I shouldn't have used the term 'unit test',
but I was thinking along the same lines you are, ie. basic
sanity/logic tests and regression tests. Trusting unit
tests to find all bugs is silly, but they can provide a
bit more confidence that you haven't introduced really
obvious bugs...

-- 
Bardur Arantsson
<ba...@im...>
<ba...@sc...>

- "Eternal happiness for one dollar? I'd be happier with the
dollar."
                                  Montgomery Burns | The Simpsons