[Cppunit-devel] A new architecture ?

[Steve M. R. <ste...@vi...>]

Hello,

So I spent some time yesterday thinking about the architecture of
CppUnit, basically the same issues that I raised previously in two
messages:

    http://www.geocrawler.com/archives/3/6780/2001/7/0/6126116/
    http://www.geocrawler.com/archives/3/6780/2001/7/0/6126118/

In those messages, I tried to outline my understanding of the various
classes in the current CppUnit.  In the end, I concluded that a lot of
my difficulty in understanding CppUnit came from a couple of rather
odd bits of system architecture.  As one example, there is the class
TestCase that is used in two different ways: it can be used to
implement a single test case, or it can be used as a "fixture" to
implement several test cases.  In addition, there is the fact that a
test case can be programmed to check for an "expected" exception if
the case is constructed with TestCaller, but not if it is a direct
subclass of TestCase.  And there is the truly baroque TestResult class
that is very inflexible and heavy.

I spent some time considering what a clean design would look like, and
then some more time actually coding it.  There are two large groups of
classes needed for unit testing: classes to set up the tests, and
classes to run them and capture the output.  I thought about putting
these two classes into separate namespaces, which I will tentatively
call CppUnit::Test and CppUnit::Result.


	Test Construction Classes (CppUnit::Test)
	-----------------------------------------

I'll leave aside the assertions and the exception classes for
the moment.

The current design includes single test cases and suites of test
cases, both of which implement a common interface.  As Baptiste
pointed out these are, respectively, the leaf, composite, and
component classes of a "Composite Pattern".  I propose to have three
classes that serve the same purposes, all in CppUnit::Test
namespace:

Base  - a pure virtual class that defines the interface (i.e. the Component)
Case  - the leaf class that implements a *single* test case; subclass of Base
Suite - the composite component (also a subclass of Base), consisting 
        of a set of tests where a test could be any subclass of Base

To these basic three -- each of which are fairly simple and clean -- \
I also added the following.

CaseWithException: a single test that requires that a specified
exception is thrown.  The class is templated on the exception
type.

FunctionAdaptor: takes a pointer to a "void foo(void)" function
and turns it into a CppUnit::Test::Case object.

MethodAdaptor: takes a pointer to a "void foo(void)" class method
and turns it into a CppUnit::Test::Case object.


These three are all subclasses of CppUnit::Test::Case, so very little
extra code is required.


Using the MethodAdaptor and CaseWithException, I can easily implement
the CPPUNIT_TEST_SUITE(), CPPUNIT_TEST(), CPPUNIT_TEST_EXCEPTION(),
and CPPUNIT_TEST_SUITE_END() macros.  The beauty of this scheme is
that *any* class can contain a test suite.  In the current scheme, I
think that you can only use these macros in a subclass of
CppUnit::TestCase.  And with the FunctionAdaptor you can make test
suites out of arbitrary global functions.

So the test-definition side seems sufficiently flexible, at least
to me :-).  Have I missed something out?



	Test Result Classes (CppUnit::Result)
	-------------------------------------

To keep the code clean and flexible, I opted here to use the composite
pattern again.  I used CppUnit::Result::Base for the interface class,

class Base
{
public:
    virtual ~Base() {};

    //! Callback to indicate a test is about to start.
    virtual void startTest( CppUnit::Test::Base* test ) = 0;

    //! Callback to indicate that a failure has occurred.
    virtual void addFailure( CppUnit::Failure* failure ) = 0;

    //! Callback to indicate a test has just ended.
    virtual void endTest( CppUnit::Test::Base* test ) = 0;
};

and CppUnit::Result::Group for the composite class.  The latter is
basically identical to the test suite class in that it contains
a vector of pointers to result classes, and the callbacks are
passed along to each result object in the group.

There are a number of possible leaf classes.  The simplest ones
will just print the test progress to the screen.  For example,
here is one in the same vein as the CompilerOutputter classes
recently added.  This one is for emacs-compatible output.


class EmacsOutput : public CppUnit::Result::Base
{
public:
    void startTest( CppUnit::Test::Base* test ) {}

    void addFailure( CppUnit::Failure* failure )
    {
	if ( failure->getSourceLine().isValid() ) {
	    std::cerr << failure->getSourceLine().fileName()
		      << ":"
		      << failure->getSourceLine().lineNumber()
		      << ":";
	} else {
	    std::cerr << "Error at unknown location:";
	}

	std::cerr << failure->getMessage() 
		  << std::endl;
    };

    void endTest( CppUnit::Test::Base* test )
    {}
};

It cannot get any simpler than this!  Output for other IDEs will be
similarly trivial.  If you have a GUI running, then you can implement
a leaf leaf class that updates the progress meter.


If you want to save the failures and generate a report at the
end, then you just implement another leaf class that stores
a copy of the failures received.


	What's Next
	-----------

I believe that the current functionality is available using the simple
classes outlined above.  Those, plus the current assertion functions,
exception class, and test registry classes.  I haven't looked too
carefully at the latter, however; perhaps there is room to simplify
those also.

There are two questions to consider.  First, is it worth reworking the
foundations of CppUnit?  Certainly, the gain in clarity is persuasive
to me; how about the rest of you?  Second, if we decide to go ahead
with this, how much compability must be maintained?


I have implemented the classes mentioned above along with the
CPPUNIT_TEST_SUITE() macros, and was able to use the macro-based test
suites previously written with no modifications.  Since none of the
assertion macros had changed, my custom assertion_traits worked okay.
I did have to change my custom TestResult class, but since it was
customized precisely to put the output in emacs format, I can happily
change to the EmacsOutput class.  ;-)

My initial ambition was to reimplement completely the current Test,
TestCase, TestCaller, TestResult classes using the new classes.  I
think that is a lot work and I'm not sure whether the gain is worth
it.  It is straightforward to replace the TEST_SUITE macros with
equivalent ones using the new classes.  So any tests that use the
macros alone will continue to function.  What other techniques are in
use?  Derivation of TestCase?  Direct use of TestCaller and TestCase
subclass methods?  How many are worth supporting?


At the moment, none of the code is in CVS.  There are two reasons for
this.  First, I'd like to see how people feel about pursuing this.
Second, I made a mistake in choosing the namespace CppUnit::Test.
Using that namespace will prevent it from coexisting with the old
CppUnit::Test class.  Since I'd like, as much as possible, to
reimplement the old classes in terms of the new I'm going to change
the new classes to a different namespace.  Perhaps CppUnit::Setup ?
Suggestions welcomed.

-Steve


-- 
by Rocket to the Moon,
by Airplane to the Rocket,
by Taxi to the Airport,
by Frontdoor to the Taxi,
by throwing back the blanket and laying down the legs ...
- They Might Be Giants