cppunit-devel

[Cppunit-devel] Bug or misunderstanding?

[<Seb...@en...>]

Hi,

I'm just starting on a new project and we are going to use CppUnit as a
framework for our tests. Right now I'm working on the UI part of the
application and so I started out with creating a Fixture that contains a
UserInterface object as a member and has a static suite() function just as
described in the CookBook. The suite creates two (for now) TestCallers and
pass in test methods of the fixture. I expected one Fixture (and therefor
one UserInterface) to be created and that all the testmethods are called on
this one object (framed by calles to setUp() and tearDown() which reset the
UI object to a
known state.)

However, it turned out that for each TestCaller instantiated one Fixture
was instantiated and the testmethods were each called on a different
Fixture object. This resulted in multiple UI objects to be created and
since they create threads each calling a display driver which only expects
calls from one UI at a time ..... I think you can imagine ...

My understanding came from the following text from the CookBook:

"Ordinarily, you'll have many little test cases that you'll want to run on
the same set of objects. To do this use a fixture.      [...]  A fixture is
a known set of objects that serves as a base for a set of test cases."


Now my question is: Am I doing something wrong or is this a bug in CppUnit
or is this intended behaviour (in which case I think the docu should be
updated.)?

Also if this is intended, can you give me just a pointer on how to do this
kind of thing? Should I subclass TestSuite and keep the persistent objects
there?



Another, completely unrelated, issue is that I get a warning from the
following code:

#ifndef _test_sm70Test_h_
#define _test_sm70Test_h_

#include "cppunit/ui/text/TestRunner.h"
#include "pfl/thread.h"

namespace sm70
{
  class RunCppUint : public pfl::Thread
  {
    CppUnit::TextUi::TestRunner runner;

    void* run();

  public:

    RunCppUint();
  };
}

#endif

The warning:
"/usr/local/include/cppunit/ui/text/TextTestRunner.h", line 88: Warning:
CppUnit::TextTestRunner::run hides the virtual function
CppUnit::TestRunner::run(CppUnit::TestResult&, const
std::basic_string<char, std::char_traits<char>, std::allocator<char>>&)

It does not keep me from compiling or running the tests (they seem to run
fine apart from above's problem) but it is rather anoying.

Thanks for your help,
Seb

Re: [Cppunit-devel] Bug or misunderstanding?

[Duane M. <dua...@ma...>]

--- At Fri, 20 Dec 2002 11:45:45 +1300,
Seb...@en... wrote:

>Hi,
>
>I'm just starting on a new project and we are going to use CppUnit as a
>framework for our tests. Right now I'm working on the UI part of the
>application and so I started out with creating a Fixture that contains a
>UserInterface object as a member and has a static suite() function just as
>described in the CookBook. The suite creates two (for now) TestCallers and
>pass in test methods of the fixture. I expected one Fixture (and therefor
>one UserInterface) to be created and that all the testmethods are called on
>this one object (framed by calles to setUp() and tearDown() which reset the
>UI object to a
>known state.)
>
>However, it turned out that for each TestCaller instantiated one Fixture
>was instantiated and the testmethods were each called on a different
>Fixture object. This resulted in multiple UI objects to be created and
>since they create threads each calling a display driver which only expects
>calls from one UI at a time ..... I think you can imagine ...
>
>My understanding came from the following text from the CookBook:
>
>"Ordinarily, you'll have many little test cases that you'll want to run on
>the same set of objects. To do this use a fixture.      [...]  A fixture is
>a known set of objects that serves as a base for a set of test cases."
>
>
>Now my question is: Am I doing something wrong or is this a bug in CppUnit
>or is this intended behaviour (in which case I think the docu should be
>updated.)?
>
>Also if this is intended, can you give me just a pointer on how to do this
>kind of thing? Should I subclass TestSuite and keep the persistent objects
>there?

Others that work on CppUnit can speak up for why it is this way. I have
long considered this a bug. CppUnit likes calling new() :-). They
instiate a new Fixture() for each test case. You will have to work around
it somehow.

I worked around it by making a few additions to the CppUnit code to allow
me to register test cases that all share a single Fixture; which is how I
always thought the system should work. 

I could dig into my code to find the changes that I made if you would like.

 ...Duane

Re: [Cppunit-devel] Bug or misunderstanding?

[Baptiste L. <gai...@fr...>]

----- Original Message -----
From: "Duane Murphy" <dua...@ma...>
To: <Seb...@en...>; "CppUnit Developers"
<cpp...@li...>
Sent: Friday, December 20, 2002 12:00 AM
Subject: Re: [Cppunit-devel] Bug or misunderstanding?


> --- At Fri, 20 Dec 2002 11:45:45 +1300,
> Seb...@en... wrote:
>
> >Hi,
> >
> >I'm just starting on a new project and we are going to use CppUnit as a
> >framework for our tests. Right now I'm working on the UI part of the
> >application and so I started out with creating a Fixture that contains a
> >UserInterface object as a member and has a static suite() function just
as
> >described in the CookBook. The suite creates two (for now) TestCallers
and
> >pass in test methods of the fixture. I expected one Fixture (and therefor
> >one UserInterface) to be created and that all the testmethods are called
on
> >this one object (framed by calles to setUp() and tearDown() which reset
the
> >UI object to a
> >known state.)
> >
> >However, it turned out that for each TestCaller instantiated one Fixture
> >was instantiated and the testmethods were each called on a different
> >Fixture object. This resulted in multiple UI objects to be created and
> >since they create threads each calling a display driver which only
expects
> >calls from one UI at a time ..... I think you can imagine ...
> >
> >My understanding came from the following text from the CookBook:
> >
> >"Ordinarily, you'll have many little test cases that you'll want to run
on
> >the same set of objects. To do this use a fixture.      [...]  A fixture
is
> >a known set of objects that serves as a base for a set of test cases."
> >
> >
> >Now my question is: Am I doing something wrong or is this a bug in
CppUnit
> >or is this intended behaviour (in which case I think the docu should be
> >updated.)?
> >
> >Also if this is intended, can you give me just a pointer on how to do
this
> >kind of thing? Should I subclass TestSuite and keep the persistent
objects
> >there?
>
> Others that work on CppUnit can speak up for why it is this way. I have
> long considered this a bug. CppUnit likes calling new() :-). They
> instiate a new Fixture() for each test case. You will have to work around
> it somehow.

This is a normal behavior. It provides maximum isolation between each test
case of the fixture. As far as I know JUnit behave the same way.

> CppUnit likes calling new() :-).

What do you mean ?

Baptiste.

[...]
>  ...Duane

Re: [Cppunit-devel] Bug or misunderstanding?

[Duane M. <dua...@ma...>]

--- At Tue, 7 Jan 2003 10:38:34 +0100, Baptiste Lepilleur wrote:

>>
>> Others that work on CppUnit can speak up for why it is this way. I have
>> long considered this a bug. CppUnit likes calling new() :-). They
>> instiate a new Fixture() for each test case. You will have to work around
>> it somehow.
>
>This is a normal behavior. It provides maximum isolation between each test
>case of the fixture. As far as I know JUnit behave the same way.

There is a chance I missed something, but from my brief review of the
JUnit code and especially the JUnit articles and examples I have read, it
appeared to me that the point of the Fixture was indeed to manage the a
set of tests and not a single test. If it was meant to manage a single
test that what is the point of setUp() and tearDown()? I could just as
easily do these in a constructor or a destructor.

>> CppUnit likes calling new() :-).
>
>What do you mean ?

I find the use of new() to be a liability. It is the leading cause of
memory leaks and object lifetime errors. I have in fact had bugs with
CppUnit where the lifetime of an object was indeed shorter that I had
expected. 

I avoid pointers and unmanaged memory as much as possible. I use
references and shared pointers (from boost) for all of my memory
management. This makes the code safer, more stable, and easier to read
and understand.

I view projects like CppUnit() as "industry examples." Maybe this is too
strong of an expectation, but I would expect to see "best practices" in a
widely used framework like CppUnit. I view memory management as a best
practice.

 ...Duane

Re: [Cppunit-devel] Bug or misunderstanding?

[Baptiste L. <gai...@fr...>]

----- Original Message -----
From: "Duane Murphy" <dua...@ma...>
To: "Baptiste Lepilleur" <gai...@fr...>; "CppUnit Developers"
<cpp...@li...>
Sent: Wednesday, January 08, 2003 5:35 PM
Subject: Re: [Cppunit-devel] Bug or misunderstanding?


> --- At Tue, 7 Jan 2003 10:38:34 +0100, Baptiste Lepilleur wrote:
>
> >>
> >> Others that work on CppUnit can speak up for why it is this way. I have
> >> long considered this a bug. CppUnit likes calling new() :-). They
> >> instiate a new Fixture() for each test case. You will have to work
around
> >> it somehow.
> >
> >This is a normal behavior. It provides maximum isolation between each
test
> >case of the fixture. As far as I know JUnit behave the same way.
>
> There is a chance I missed something, but from my brief review of the
> JUnit code and especially the JUnit articles and examples I have read, it
> appeared to me that the point of the Fixture was indeed to manage the a
> set of tests and not a single test. If it was meant to manage a single
> test that what is the point of setUp() and tearDown()? I could just as
> easily do these in a constructor or a destructor.

I believe you're mistaken concerning JUnit. Give a look at
TestSuite.createTest(), and you'll see that the constructor is called for
each test (there is no choice, since the name of the method to be called is
stored in the test case name (see TestCase documentation)...).

It could indeed be the constructor and destructor, but using virtual method
is more flexible:
- setUp/tearDown can be called multiple time (that is what is done when you
click the 'run' button in the MFC/QT test runner).
- setUp and tearDown can call virtual method, which are implemented in
subclass. This is not possible in constructor/destructor.

That being said, I think it will be possible to support both mode in CppUnit
2.0: using a single fixture instance is indeed useful when initialization
cost much.

> >> CppUnit likes calling new() :-).
> >
> >What do you mean ?
>
> I find the use of new() to be a liability. It is the leading cause of
> memory leaks and object lifetime errors. I have in fact had bugs with
> CppUnit where the lifetime of an object was indeed shorter that I had
> expected.
>
> I avoid pointers and unmanaged memory as much as possible. I use
> references and shared pointers (from boost) for all of my memory
> management. This makes the code safer, more stable, and easier to read
> and understand.

I believe you meant to say "CppUnit use delete a lot" ;-). I'm also a fair
user of boost and boost::shared_ptr.

CppUnit 2.0 makes use of smart-pointer. In fact, I borrowed the 'no template
member' shared_ptr implementation and modified it to be more portable and
provide a static_cast. For instance, the TestFactoryRegistry code is much
simpler (doesn't have all that memory management code).

> I view projects like CppUnit() as "industry examples." Maybe this is too
> strong of an expectation, but I would expect to see "best practices" in a
> widely used framework like CppUnit. I view memory management as a best
> practice.

Keep in mind that I'm still fairly young to the world of C++. I started
learning 5 years ago, and I only have 3,5 years of professionnal experience.

I learned a lot during the last year, and I believe some of this will be
reflected in CppUnit 2.0. Anyway, I intend CppUnit 2.0 to go through a
review phase when I'll have a basic version done, so features/style... can
be discussed at length.

Baptiste.

>
>  ...Duane