Re: [Cdk-devel] RFC: Separate JUnit tree for performance tests

[Rich A. <rap...@oc...>]

On Tue, 6 Dec 2005 15:04:42 +0100, Egon Willighagen wrote
> Hi all,
> 
> all many of you have noticed, the tests take very long at this 
> moment. It is actually a small subset of tests that really take long,
>  e.g. the comprehensive atom type testing.
> 
> I would like to propose a new package cdk.test.slow in which we can 
> put JUnit tests that test performance or are slow otherwise. They 
> should always be tested, but it would be nice for quick hacking to 
> be able to do the fast regression tests separately. The hierarchy of 
> classes in cdk.test.slow should be identical to that of the cdk.test 
> package itself.

The concept is good. I'm wondering if it would be possible to create separate
test suites and keep the existing package heirarchy. Making separate packages
is really changing the unit test api for a performance issue. What if certain
tests in this package start to run faster? Do they get moved or stay? What if
other tests start to run slow? Moving classes between packages - even if they
are unit tests, leaves the door open for creating bugs. It also makes is more
difficult to stabilize the API.

Would it be possible to do the unit test aggregation at the level of the
TestSuite? Create two (or more) TestSuites: FastTests and SlowTests. Use Ant
to run one or both, depending on a command line argument. Then, if tests need
to be reclassified as fast or slow, those designations are invisibile to
developers. This would accomplish the same thing as moving tests into separate
packages, but would be much less disruptive.

rich