[Cppunit-devel] class overview II: reporting test results

[Steve M. R. <ste...@vi...>]

This is the second part of my attempt to understand the CppUnit
framework.  I concentrate on classes for running and reporting
test results.



		class TestResult
		----------------

The fundamental run() method of the Test class hierarchy takes a
pointer to a TestResult object.  This TestResult object is responsible
for "collecting the results of executing a test case".

If I may go into Critic Mode for a moment, it seems to me that this
class is responsible for far too many things.  In addition to
collecting test results (i.e. a list of Exceptions and Failures), the
TestResult object also: keeps track of whether some agent (the user?) 
has requested that the tests stop, and keeps a set of TestListeners
and feeds them reports of errors, failures, and tests starting and
stopping.  In addition, the class is cluttered up with synchronization
code.


		class TestListener
		------------------

This class has the clean design that the TestResult ought to have
had.  (IMHO)

I think if I were to design CppUnit from scratch, I'd have Test::run
take a TestListener* parameter instead of TestResult as is currently
done.  From TestListener, I'd make 4 subclasses:

	1. to accumulate errors and failures
	2. to print E/F/. to stdout (c.f. TextTestResult)
	   - you'd also want GUI test listeners to do similar things
	3. to contain a set of listeners, and feed the startTest, addError, 
	   etc callbacks to each in turn (in the spirit of TestSuite)
	4. a TestListener decorator that synchronizes the callbacks
	   to the decorated listener

The functionality of the current TestResult and TextTestResult can be
made from listeners of these four types, with the exception of the
"should I stop?" state (i.e. TestResult's shouldStop() and stop()
methods).

I'm rather tempted to do rework CppUnit in this way.  [See below for
my suggestion on shouldStop().]  Is that too radical?  What if it can
be done keeping backwards compatibility?  How much backwards
compatibility is necessary?


		class TestRunner
		----------------

This class is currently MSVC-only code.  I propose that
CppUnit::TestRunner be reclaimed to the generic portion of the
library.

This class contains a set of tests, and has a run() method.
It should be much like the TestSuite class.  The main difference
is that running the tests will be enclosed in a try block 
that catches a "please stop" exception.  This mechanism
replaces the TestResult methods shouldStop() and stop().

Folks who want a GUI can subclass TestRunner, and use
appropriate TestListeners.

Whaddya think?

-Steve

P.S.  I haven't yet digested the registry and test factory classes.
How do they fit in here?  What can you do with a registry?  Why would
someone want more than one test suite?


-- 
by Rocket to the Moon,
by Airplane to the Rocket,
by Taxi to the Airport,
by Frontdoor to the Taxi,
by throwing back the blanket and laying down the legs ...
- They Might Be Giants