Re: [Haskelldb-users] Release

[Bjorn B. <bri...@cs...>]

On Jul 17, 2006, at 1:37 PM, Bjorn Bringert wrote:

> On Jul 17, 2006, at 1:14 PM, Matthias Radestock wrote:
>
>> Bjorn Bringert <bri...@cs...> writes:
>>
>>> I think that some sort of test suite would be very nice to have
>>> before a release, but that might just be my personal opinion.
>>
>> IMHO the test suite is *essential*. HaskellDB is quite a complex
>> library
>> and changes in one area can have far-reaching and non-obvious
>> consequences. In absence of a large user base, a reasonably
>> comprehensive test suite is crucial for tracking down bugs. I was
>> horrified when I found out that the current test suite had
>> bit-rotted. It means that all recent changes are highly suspect and
>> will
>> have most likely broken something somewhere.
>>
>> Bjorn, do you know when the test suite was last run successfully? =20
>> Then
>> we could take a look what changes were introduced since then and add
>> test cases for those. Ideally we'd also do coverage testing to
>> determine
>> what other tests we should add to get a reasonably good coverage of
>> the
>> library.
>>
>> One added complication is that the test suite should be run against a
>> variety of db backends, in order to identify db-specific code that =20=

>> has
>> crept into the generic code base, and bugs in the db-specific code
>> itself.
>
> I don't think that it would be too hard to get the current tests
> running again, but they were never that good:
>
> - There has never been a time when all of them worked on all back-
> ends, because of back-end incompatibilities.
> - The coverage is probably quite poor.
> - Many of them don't check their outputs, they mostly just test that
> HaskellDB doesn't generate bad SQL or crash in some other way.
>
> Some ideas for getting a good test suite together:
>
> - Design a test database and write DBInfo for it.
>
> - Use DynConnect for database connections. Extend DynConnect to
> support all driver (missing HDBC drivers at the moment)
>
> - Have some way for the user to provide connection parameters to all
> the database back-ends to test with.
>
> - Use HUnit, and maybe even QuickCheck.
>
> - Maybe the FlatDB [1] backend could be used as a reference. It's a
> simple database back-end whose query language is HaskellDB's internal
> PrimQuery type.
>
>
> One of the reasons that this hasn't been done yet is that the back-
> end incompatibility problem is so frustrating. But a test suite would
> be a good way to identify the things that need to be taken care of by
> any back-end specific SQL generation.
>
> I can try to get a test framework together. Are there any volunteers
> who would like to contribute test DB layout, data, unit tests or
> QuickCheck properties?
>
> /Bj=F6rn
>
> [1] http://www.cs.chalmers.se/~bringert/darcs/haskelldb-flat/

I've cleaned up the repo a bit, and added a unit test framework and =20
some unit tests. I have bumped the version number to 0.10, to =20
distinguish it from the released 0.9.

To run the current tests:

$ cd make

Edit test.config and set parameters for whatever database back-ends =20
you have available.

$ make

If you want to run tests for just one of the back-ends in test.config =20=

(for example "HSQL PostgreSQL"):

$ make tests

$ ./tests "HSQL PostgreSQL"

The current tests are very simple, but a large number of them fail on =20=

the some of the back-ends. There even seems to be one in the HDBC =20
back-end which crashes the test framework occasionally.

You are all welcome to contribute more unit tests. Very little of the =20=

API is currently covered. There should also be test for known bugs, =20
see http://sourceforge.net/tracker/?group_id=3D101095&atid=3D629040

I haven't gotten around to adding support for QuickCheck properties yet.


/Bj=F6rn=