Class hierarchy testing

[Rob]

I have a class hierarchy with a single base class and several derived classes.
The situation at the moment is that I write the tests for the base class, then duplicate all of that for each base class, and extend it. This seems wrong to me - just the same as a derived class is aiming to not duplicate base class functionality, so the derived class's test should not duplicate the base class test functionality IMHO.
How can I write the base class test so that I can apply it to each classes in the hierarchy?

TIA,
--Rob

[Oliver Goodman]

How about:

TestsForAllClasses(MyClass& obj)
{
  //tests which make sense for all derived classes
}

  // The actual tests
  Derived1 obj1;
  TestsForAllClasses(obj1);
  Derived2 obj2;
  TestsForAllClasses(obj2); // etc.

  //then if Derived classes add any extra methods
  //more tests for the extra methods..
  CPPUNIT_ASSERT(obj1.func() == 1);

Note that the objects must be passed by reference
(or pointer).

Of course the line on which an assertion fails
won't necessarily tell you which derived class failed
a test.

Cheers,
Oliver

[Code Guru]

Is the base class abstract?  In otherwords, does it have any pure virtual functions?  If not, just write tests for the base class since you can instantiate it.  And write separate tests for any methods that the derived classes add or override.

If the base class, DOES have any pure virtual functions, then create a "mock" derived class that provides concrete, EMPTY implementations for these functions and use this class to test any concrete methods in the base class.  Don't worry about repeating these same tests for the "real" derived classes since they will already be tested using the mockup class.

[Rob]

Layne - where a derived class can override or extend base class functionality, I need to test it with the base class tests to prove that it fulfils the same basic guarantees and hasn't broken any of the functionality. The base class is concrete.

Oliver,
I was trying out something slightly neater using a templated test class:

//------------------------
template<class EventType>
class EventTest
   : public CppUnit::TestFixture
{ ... };
CPPUNIT_TEST_SUITE_REGISTRATION(EventTest<Event>);
// etc with other Event-derived types.
//------------------------

If all derived classes' constructors had the same argument list, this would work. However, they aren't, and it doesn't, for example:

//------------------------
void EventTest::TestWithoutData()
{
   Event e("eventType");
   // ... tests
}
//------------------------

The whole of EventTest is populated with code such as this, what I really want is to change it in such a way that the line of construction of each event is all that is replaced, but tests remain the same. Perhaps templating the TestFixture and overriding a 'GetEvent(commonParameters...)' function for each derived type. Then I could do:

//------------------------
void EventTest::TestWithoutData()
{
   Event* e = GetEvent("eventType");
   // ... tests
   delete e;
}

// I know 'virtual' doesn't belong here, it's to show it's virtual without requiring class def. as well!
virtual Event* EventTest::GetEvent(string type)
{
   return Event(type);
}

virtual Event* DerivedEventTest::GetEvent(string type)
{
   return DerivedEvent(type, "another parameter");
}
//------------------------

Any comments on that sort of approach?
Cheers, Rob.

[Oliver Goodman]

How much repetition you can avoid would seem to depend on how much commonality there is between the set of constructors of the different derived classes. If each derived class has a syntactically identical set of constructors (not usually the case) then template code would help a lot:

E.g. if each event had default, string and (int,int) constructors you could write

void CommonTests(Event& e)
{ ... }

template <class Derived>
void TestDerived()
{
    Derived d;
    CommonTests(d);
    Derived d1("testString");
    CommonTests(d1);
    Derived d2(3,5);
    CommonTests(d2);
}

void testEvent::TestAllDerived()
{
    TestDerived<MouseEvent>();
    TestDerived<KeyboardEvent>();
    // etc.
}

Doesn't sound like that's the case for you. Usually the constructors are exactly the parts of a hierarchy that vary the most. Then forget templates: they can only help where there is some syntactic uniformity.

The Factory pattern could be of some help. The basic thing it allows you to do is move the construction into the CommonTests function:

class Factory
{
public:
    virtual ~Factory() {}
    virtual Event* GetEvent() = 0;
};

void CommonTests(Factory& f)
{
    Event* e = f.GetEvent();
    ...
    delete e;
}

class MouseEventFactory : public Factory
{
    MouseEvent* GetEvent() { return new MouseEvent(3,5); }
};

class KeyboardEventFactory : public Factory
{
    KeyboardEvent* GetEvent() { return new KeyboardEvent('K'); }
};

void testEvent::TestAllDerived()
{
    CommonTests(MouseEventFactory());
    CommonTests(KeyboardEventFactory());
}

As it stands this doesn't buy you much, however you do now have an enormous amount of flexibility. You can call f.GetEvent() as many times as you like in your test functions. Your factory can provide different kinds of Event creating methods such as Event* GetEvent(string s), the test function can vary s, and the various Factories can create different events in response. Or you can store constructor arguments in the Factory objects themselves to use when GetEvent is called. (No idea whether any of this would actually be useful!)

Again, if you want/need what factories can do for you and the constructor syntax *is* uniform, you can use templates to save on typing:

template <class EventType>
class EventFactory : public Factory
{
        EventType* GetEvent() { return new EventType(); }
};

and write EventFactory<MouseEvent>() instead of MouseEventFactory() etc.

Finally, if you need to convert the name of an event type to an object of that type you can do this:

static map<string,Factory*> factory;

void RegisterFactories()
{
    factory["MouseEvent"] = new MouseEventFactory();
    // repeat for each type ...
}

// global GetEvent function..
Event* GetEvent(string type)
{
    return factory[type]->GetEvent();
}

So anyway, I don't know if that helps. None of this really has much to do with CppUnit (about which I know very little in truth!) but those are some of the ways around the fact that constructors can't be virtual and classes aren't objects.

One final off-the-wall idea:

void EventTest::runAllTests()
{
    vector<Event*> events;

    events.push_back(new KeyboardEvent('K'));
    events.push_back(new MouseEvent(2,3));
    ...

    vector<Event*>::const_iterator i,e=events.end();
    for (i=events.begin(); i!=e; ++i)
    {
        CommonTests(**i); // pass an Event&
    }

    // then delete them all
}

But of course that's back to the assumption that each test just needs one Event to play with.

Cheers,
Oliver

[Rob]

Err yeah, in the second solution I described, ignore the templating - it's unnecessary because of course, polymorphism is used by returning Event* from the GetEvent() function.
D'oh!

[Rob]

Oliver,

That's an immense post! I hadn't thought of using a factory to get around this, but of course it makes the most sense.

Thanks for all the info,
--Rob

[atlwin]

See documentation of of the macro CPPUNIT_TEST_SUITE_END_ABSTRACT(). This solution makes a virtual base class for testing a virtual base class.  You do have to make a concrete class for each concrete class you test, but you would only have to fulfill one virtual function.