Re: [Ocaml-lib-devel] Future of ExtLib

[Janne H. <ja...@hy...>]

Bardur Arantsson wrote:

>
>>I personally think 'unit' testing is a bit silly.
>>It rarely finds bugs because it can't cover enough cases,
>>can't handle integration, etc.. I prefer a more sloppy
>>concept of just collecting whatever test code you can
>>and running it, just to get some confidence you didn't
>>completely mess something up whilst committing a minor
>>change to CVS.
>>    
>>
>
>Yes, perhaps I shouldn't have used the term 'unit test',
>but I was thinking along the same lines you are, ie. basic
>sanity/logic tests and regression tests. Trusting unit
>tests to find all bugs is silly, but they can provide a
>bit more confidence that you haven't introduced really
>obvious bugs...
>  
>
I find these types of tests extremely handy for developing any 
software.  In fact we use this kinds of sanity/regression tests in all 
our software at our company.  What I normally do (for C and O'Caml) is 
that I use a lot of asserts to do sanity checks in library code and then 
write a tester (sometimes called smoketester) that calls the library 
functions with heavily biased random input.  The input is biased towards 
corner cases, such as array length zero etc.  Every test case will 
naturally test that the function(s) it is testing is working in the 
expected manner.

The tester's "coverage" can be easily metered with gcov.  This way you 
can get sort of profiling of your test code.. E.g., only spend time 
writing test cases that actually reach yet unentered parts of the library.

I have personally found this technique to really effective in 
development.  Modifying large parts of code is much easier if you have a 
good tester that actually tests everything.

I guess for Extlib, you're always assumed to pass *all* tests before 
making a release or committing changes.  If this is the case, I don't 
see much need for fancy logging machinery, since the tests would pretty 
much just report PASSED.

I don't think it would be so hard to find volunteers for this kind of 
testing.  In fact I'd be willing to write a few tests for it myself.

Best regards,
Janne Hellsten