Thread: [Cppunit-devel] include directory clean-up
Brought to you by:
blep
From: Baptiste L. <gai...@fr...> - 2001-06-14 14:39:46
|
We had some talk about the namespace a while ago, and questions about why so many stuffs were in the "extensions" directory. I think the current directory orgnization in "include" can be confusing. Here I propose a new organization below the "include" directory: cppunit/ extensions/ : only stuffs that really "extends" the framework Orthodox.h RepeatedTest.h TestDecorator.h TestSetUp.h helper/: stuffs that make using the framework easier AutoRegisterSuite.h HelperMacros.h TestFactory.h TestFactoryRegistry.h TestSuiteBuilder.h TestSuiteFactory.h TypeInfoHelper.h textui/: text based TestRunner TestRunner.h (within the CppUnit::TextUi namespace, was TextTestRunner.h) mfcui/: MFC based TestRunner TestRunner.h (within the CppUnit::MfcUi namsepace, was msvc6/TestRunner/TestRunner.h ) So, what do you think ? Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Language: English, French |
From: Steve M. R. <ste...@vi...> - 2001-06-14 15:16:42
|
On Thu, Jun 14, 2001 at 04:39:41PM +0200, Baptiste Lepilleur wrote: > We had some talk about the namespace a while ago, and questions > about why so many stuffs were in the "extensions" directory. I think > the current directory orgnization in "include" can be confusing. I agree with this sentiment. It confuses me :-) Unfortunately, I have more questions than answers... I think that drafting an outline of the library will help in organizing the directory structure (and possibly the namespace structure). Questions that I have, for example: what is the difference between an "extension" and a "helper"? What is the "core" of the framework, anyway? I'd like to start documenting the classes better. I don't understand it well enough, yet, though. If we can get some discussion going here, I will add the documentation strings. I expect this process will clarify class relationships and groupings will emerge, facilitating directory/namespace structuring. For example, we can start with the core classes such as Test, TestCase, TestCaller, TestSuite. The latter three are each a subclass of Test, which is pure virtual. Is it fair to say, then, that Test is an "interface" class? What are the intended semantics of its methods: run(), countTestCases(), toString() and getName()? We just had some questions on whether the latter two should be merged. I don't think a conclusion was reached. > textui/: text based TestRunner > TestRunner.h (within the CppUnit::TextUi namespace, was TextTestRunner.h) > mfcui/: MFC based TestRunner > TestRunner.h (within the CppUnit::MfcUi namsepace, was > msvc6/TestRunner/TestRunner.h ) Here, the directory structure is intended to mirror the namespace structure, right? I wonder what is intended to be in the CppUnit::<UI> namespace. Currently, there is only TestRunner; are more classes envisioned? If not, we'd be creating multiple <UI> namespaces with a single member each. That doesn't sound right. Why not create a TestRunner base class, with specializations for each UI? -Steve -- by Rocket to the Moon, by Airplane to the Rocket, by Taxi to the Airport, by Frontdoor to the Taxi, by throwing back the blanket and laying down the legs ... - They Might Be Giants |
From: Baptiste L. <gai...@fr...> - 2001-06-14 16:54:20
|
Quoting "Steve M. Robbins" <ste...@vi...>: > On Thu, Jun 14, 2001 at 04:39:41PM +0200, Baptiste Lepilleur wrote: > > > We had some talk about the namespace a while ago, and questions > > about why so many stuffs were in the "extensions" directory. I think > > the current directory orgnization in "include" can be confusing. > > I agree with this sentiment. It confuses me :-) > > Unfortunately, I have more questions than answers... > > I think that drafting an outline of the library will help in > organizing the directory structure (and possibly the namespace > structure). Questions that I have, for example: what is the > difference between an "extension" and a "helper"? What is the "core" > of the framework, anyway? I'll try to define those as I perceive them: core: it is the minimal(not really) set of classes for unit testing. It provides the following functionnalities: - writing (TestAssert) and running test (Test). - report test failure (TestCase,TestResult,Exception,NotEqualException,TestListener) - organize test in suite (TestSuite) I would also include: - being able to write a test in a single class method (TestCaller) - should be extensible extensions: they extends the functionnalities provided by the framework. helper(s?): they don't add functionnalities but make using existing functionnalities easier. Well, I'll have to go back home. I'll extend a bit later on this that evening. Later, Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Language: English, French |
From: Baptiste L. <bl...@cl...> - 2001-06-14 21:09:10
|
> Quoting "Steve M. Robbins" <ste...@vi...>: > > > On Thu, Jun 14, 2001 at 04:39:41PM +0200, Baptiste Lepilleur wrote: > > I think that drafting an outline of the library will help in > > organizing the directory structure (and possibly the namespace > > structure). Questions that I have, for example: what is the > > difference between an "extension" and a "helper"? What is the "core" > > of the framework, anyway? > > I'll try to define those as I perceive them: > > core: it is the minimal(not really) set of classes for unit testing. It > provides the following functionnalities: > - writing (TestAssert) and running test (Test). > - report test failure > (TestCase,TestResult,Exception,NotEqualException,TestListener) > - organize test in suite (TestSuite) > I would also include: > - being able to write a test in a single class method (TestCaller) > - should be extensible > > extensions: they extends the functionnalities provided by the framework. > helper(s?): they don't add functionnalities but make using existing > functionnalities easier. I admit the difference can be tricky. Does the TestFactoryRegistry add to the core functionnality ? I would say no: you can't do anymore than you were doing. It only make it easier: you don't have to create class for each suite, remember to add the new suite in the suite... You even lose extensibility: the suite instantiated will all be TestSuite. You lose some portability (it use a static variable and it has been reported that some compiler on embedded do not support this well). The macros case is a bit easier: they help you implement your everyday test case. The TestSuiteBuilder ? Well, it's just a factorization of the code that is commonly used to construct a suite. But it also adds a functionnality: TestCase name extraction using RTTI (note, that it was originaly located in the TestSuite constructor, which make more sense to me). TestSetUp allow you to set up an environment to run a test. RepeatedTest allow you to run a test a specified number of times. For those last two, their only purpose is to provide new functionnalities. So the edge is somewhat blurred. The hardest to define probably being the TestFactoryRegistry. Ahh, I think I found a good criteria: "If you did not have this class, could you do what it is doing without rewriting it ? If so, then it is not an extension". I hope this help, it sure sharpened some blurred shapes for me. I like that last criteria. Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Author of The Text Reformatter, a tool for fanfiction readers and writers. Language: English, French (Well, I'm French). |
From: Baptiste L. <bl...@cl...> - 2001-06-14 21:09:11
|
Well, now for the last part ;-) ----- Original Message ----- From: "Steve M. Robbins" <ste...@vi...> To: <cpp...@li...> Sent: Thursday, June 14, 2001 5:16 PM Subject: Re: [Cppunit-devel] include directory clean-up > On Thu, Jun 14, 2001 at 04:39:41PM +0200, Baptiste Lepilleur wrote: > > textui/: text based TestRunner > > TestRunner.h (within the CppUnit::TextUi namespace, was TextTestRunner.h) > > mfcui/: MFC based TestRunner > > TestRunner.h (within the CppUnit::MfcUi namsepace, was > > msvc6/TestRunner/TestRunner.h ) > > Here, the directory structure is intended to mirror the namespace > structure, right? I wonder what is intended to be in the > CppUnit::<UI> namespace. Currently, there is only TestRunner; are > more classes envisioned? If not, we'd be creating multiple <UI> > namespaces with a single member each. That doesn't sound right. Why > not create a TestRunner base class, with specializations for each UI? Well, for the TextUi I can't see anymore classes coming. We are limited by portability which means no RTTI and therefore no test hierarchy browsing (we could still add a "dump" test hierarchy option). For the MfcUi, I have restricted the "published" part to the strict minimum because the other parts are evolving a lot. But ultimately, we should reach a well factored and solid TestRunner that allow subclassing. The dialog, and the models would be published and the user could subclass them. For example, I have planned a "detail" feature, that show the detail about the selected failure in the list. For a basic "Exception" failure, you would just have larger text fields to read them properly. For the "NotEqualException", you would have two text fields to compare the "expected" and "actual" value (with a nice diff option of course ;-) ). When the user extends the framework by subclassing Exception, the user should be able to provide a new "detail" view for that exception. This is a very far goal, but one none the less. Closer goal, would be to be able to subclass the TestRunnerModel (being worked on, it is a model that contains the list of test failure reported in the list), so the user could add functionnalities, such as the time is took for each test to run, write a duplicate of the failure to a file... It's going on slowly (I'm not a MFC expert), but certainly. As a alternative for the directories, we could have: include/ cppunit/ cppunitui/ mfc/ text/ I think I like that one better, Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Author of The Text Reformatter, a tool for fanfiction readers and writers. Language: English, French (Well, I'm French). |
From: Baptiste L. <bl...@cl...> - 2001-06-20 06:25:09
|
Well, onward for the answer to the next part of the mail... ----- Original Message ----- From: "Steve M. Robbins" <ste...@vi...> To: <cpp...@li...> Sent: Thursday, June 14, 2001 5:16 PM Subject: Re: [Cppunit-devel] include directory clean-up > On Thu, Jun 14, 2001 at 04:39:41PM +0200, Baptiste Lepilleur wrote: > I'd like to start documenting the classes better. I don't understand > it well enough, yet, though. If we can get some discussion going > here, I will add the documentation strings. I expect this process > will clarify class relationships and groupings will emerge, > facilitating directory/namespace structuring. OK. I'd rather not write doc because my written english is quite strange, and I fear I would put more confusion in the mind of the user than anything else. > For example, we can start with the core classes such as Test, > TestCase, TestCaller, TestSuite. The latter three are each a subclass > of Test, which is pure virtual. Is it fair to say, then, that > Test is an "interface" class? What are the intended semantics > of its methods: run(), countTestCases(), toString() and getName()? > We just had some questions on whether the latter two should be > merged. I don't think a conclusion was reached. Well, let's get started. I'll give you my view on this classes. *** Test: provides a way to run a specified test and tracks the result of the run. void run (TestResult *result): - Run the test. - The test should report each stage of its run to the TestResult (startTest/addError/addFailure/endTest). - Should never ever throw an exception except for the case of fatal errors (can't allocate memory to add the failure to TestResult...) int countTestCases () const: - predict the number of times the TestCase::runTest() method will be called if this test is run(). Returning a value lower than 1 doesn't make sense (what would be the point of running the test if no unit test is run?). std::string toString () const: - My best guess would be describes the test. In Java, and Michael Feather's original version there is no getName() method, and the toString() has the role of the getName() method. std::string getName () const: - Identify the test. The returned name should as unique as possible to identify the test (that why the TestCase name is included by the TestSuiteBuilder). TestRunners rely on that name to run a specified test (or suite). Having a name that identify the test without ambiguity help reading the failures report (you don't have to bother with the filename). Conclusion: ditch either toString() or getName(). I find that getName() as a more intention revealing name ;-) *** TestCase: provides a convenient way to write test with a setup/test/tearoff stage. Provides track of the test run to the TestResult. void run(TestResult *result): - provides TestResult with progress of the test run. - run the actual test with the runTest() method which should be overrided. Provides runTest() initialization with a setUp() method and clean up tearOff() method which should be overrided to create/destroy fixtures (note that creating fixture in the class constructor does not make it possible to run the test more than 1 time if the fixture is modified by the test). - "translate" exception thrown by either setUp(), runTest() or tearOff() into failure or error report to the TestResult. Exception with CppUnit::Exception as a base class are "failure" (a failed condition, such as a failed assertion or an uncaught excepted exception), report to the TestResult with addFailure() other exception are "error", reported with addError(). TestResult *run(): - a convenient method to run a test in the wild ? It's not used by the framework itself. Only use I can see is if you want to run a test case without a test runner. TestResult *defaultResult(): - used by the previous run() method. Should be overrided to use a test result other than TestResult. Can't think of a use for those two... int countTestCases() const: - always returns 1 since run() call runTest() only once. Should not be overrided. std::string getName() const (or toString() depending on the survivor ;-) ): - identifiy the test. Among all the classes this is the most important implementation of this method since it is the one use by the TestResult (and TestRunner) to indicate which test as failed. Should be unique, but it not always possible (template test case on platform without RTTI support :-( ). void setUp(): - called by run() before calling runTest(). Should be override to initialize the test. If an exception is throw, then runTest() and tearDown() won't be called and an "error" is reported for that test. void runTest(): - you should override this method to do the actual test. Test failure should be reported by throwing a exception derived from CppUnit::Exception. This is the place where you use CPPUNIT_ASSERTs macros. void tearDown(): - you should override this method to clean up after runTest() is called. This method is always called if setUp() has been called successfuly (no exception were thrown). That about it for those two nasties. More about the other another day (note that test caller is a specialization of TestCase). I'll try to do a short synops when I'll be done with each of classes. Good night, Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Author of The Text Reformatter, a tool for fanfiction readers and writers. Language: English, French (Well, I'm French). |
From: Steve M. R. <ste...@vi...> - 2001-07-07 20:05:06
|
Hi, On Wed, Jun 20, 2001 at 08:42:04AM +0200, Baptiste Lepilleur wrote: [ ... while describing the core classes' functionality ... ] > std::string toString () const: > - My best guess would be describes the test. In Java, and Michael > Feather's original version there is no getName() method, and the toString() > has the role of the getName() method. > > std::string getName () const: > - Identify the test. The returned name should as unique as possible to > identify the test (that why the TestCase name is included by the > TestSuiteBuilder). TestRunners rely on that name to run a specified test (or > suite). Having a name that identify the test without ambiguity help reading > the failures report (you don't have to bother with the filename). > > Conclusion: ditch either toString() or getName(). I find that getName() as a > more intention revealing name ;-) I agree with your conclusion. And I volunteer to do it (remove toString). My plan of attack is as follows: 1. Remove all use of toString() in the library. 2. Document toString() as deprecated in all classes that define it. 3. Replace all toString() implementations with "return getName()". 4. After a respectable number of public releases have been made, remove all toString() methods. Optionally, we could replace toString() methods with toString() { std::cerr << "Replace toString() with getName()." << endl; return getName(); } ... or would that be too obnoxious? ;-) Comments? -Steve -- by Rocket to the Moon, by Airplane to the Rocket, by Taxi to the Airport, by Frontdoor to the Taxi, by throwing back the blanket and laying down the legs ... - They Might Be Giants |
From: Baptiste L. <gai...@fr...> - 2001-07-10 14:37:02
|
Quoting "Steve M. Robbins" <ste...@vi...>: > Hi, > > On Wed, Jun 20, 2001 at 08:42:04AM +0200, Baptiste Lepilleur wrote: > > Conclusion: ditch either toString() or getName(). I find that > getName() as a > > more intention revealing name ;-) > > I agree with your conclusion. And I volunteer to do it (remove > toString). > > My plan of attack is as follows: > > 1. Remove all use of toString() in the library. > 2. Document toString() as deprecated in all classes that define it. > 3. Replace all toString() implementations with "return getName()". > 4. After a respectable number of public releases have been made, > remove all toString() methods. > > Optionally, we could replace toString() methods with > > toString() { > std::cerr << "Replace toString() with getName()." << endl; > return getName(); > } (note that this is of no use for Windows user in GUI mode). > > ... or would that be too obnoxious? ;-) I wouldn't even bother with point (2), (3) and (4). The next release will break so many existing stuffs, and this one is really a minor detail (toString isn't even used by CppUnit). CppUnit status is "pre-alpha", which to me means "might be working some, but don't trust it". Seriously, I thing that releasing a list of change since 1.5.5 is enough. People using CppUnit at pre-alpha stage for serious work should be following this list (or they like to live very dangerously). So, if someone as problem with toString() methods removal, speak up now ;-) Concerning deprecation (no use for now, but might be for later), we could set up "compatibility" flag (just like we did for assert). Those flag would be up for a few releases before being removed (so you could have the compiler locating all the usages of toString() by disabling support for deprecated stuff). But I don't think it's worth bothering with deprecation support until we reach alpha or beta stage (does somebody have a clear definition of what each stage is supposed to be ?). Documentation how to do the transition from one version to another should be enough for now (we should start a document for that). Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Language: English, French |
From: Steve M. R. <ste...@vi...> - 2001-07-10 14:58:33
|
Hi, With respect to the toString/getName controversy: I realized yesterday that the "assertion_traits" class also has a toString() method. In this case one is converting an arbitrary type T to a string representation, so "toString()" fits better than "getName()". I'm starting to wonder whether it is better to eliminate getName() on the Test classes, and stick with toString() universally. Or, we could go ahead with the initial plan, and leave assertion_traits to be an exception. I'm leaning towards the first option. Thoughts? On Tue, Jul 10, 2001 at 04:36:57PM +0200, Baptiste Lepilleur wrote: > Quoting "Steve M. Robbins" <ste...@vi...>: > > My plan of attack is as follows: > > > > 1. Remove all use of toString() in the library. > > 2. Document toString() as deprecated in all classes that define it. > > 3. Replace all toString() implementations with "return getName()". > > 4. After a respectable number of public releases have been made, > > remove all toString() methods. > > > > Optionally, we could replace toString() methods with > > > > toString() { > > std::cerr << "Replace toString() with getName()." << endl; > > return getName(); > > } > (note that this is of no use for Windows user in GUI mode). Ah. What happens to the standard streams in GUI mode? Are they logged to a file somewhere, simply ignored, or does it cause a "general protection fault" :-) ? > I wouldn't even bother with point (2), (3) and (4). The next release will > break so many existing stuffs, and this one is really a minor detail (toString > isn't even used by CppUnit). True, it is a minor detail. On the other hand, if we maintain backwards compatibility without too much trouble, I'm willing to do that. > Documentation how to do the transition from one version to another should be > enough for now (we should start a document for that). I think that is a good idea. I have not used any release prior to 1.5.5, so I don't really know what is being broken or not broken. -S -- by Rocket to the Moon, by Airplane to the Rocket, by Taxi to the Airport, by Frontdoor to the Taxi, by throwing back the blanket and laying down the legs ... - They Might Be Giants |
From: Baptiste L. <gai...@fr...> - 2001-07-10 15:25:57
|
Quoting "Steve M. Robbins" <ste...@vi...>: > Hi, > > With respect to the toString/getName controversy: I realized yesterday > that the "assertion_traits" class also has a toString() method. In > this case one is converting an arbitrary type T to a string > representation, so "toString()" fits better than "getName()". > > I'm starting to wonder whether it is better to eliminate getName() > on the Test classes, and stick with toString() universally. Or, > we could go ahead with the initial plan, and leave assertion_traits > to be an exception. > > I'm leaning towards the first option. > Thoughts? toString() and getName() have very different semantic. * getName(): give me the name of that test so I can identify it. * toString(): dump this object as a string so I can diagnose the problem (why is not equal to that other one) without running the debugger. So, I would exterminate toString() from CppUnit, with the exception of assertion_traits. Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Language: English, French |
From: Steve M. R. <ste...@vi...> - 2001-07-14 06:01:08
|
On Tue, Jul 10, 2001 at 05:25:51PM +0200, Baptiste Lepilleur wrote: > Quoting "Steve M. Robbins" <ste...@vi...>: > > > Hi, > > > > With respect to the toString/getName controversy: I realized yesterday > > that the "assertion_traits" class also has a toString() method. In > > this case one is converting an arbitrary type T to a string > > representation, so "toString()" fits better than "getName()". > > > > I'm starting to wonder whether it is better to eliminate getName() > > on the Test classes, and stick with toString() universally. Or, > > we could go ahead with the initial plan, and leave assertion_traits > > to be an exception. > > > > I'm leaning towards the first option. > > Thoughts? > > toString() and getName() have very different semantic. > > * getName(): give me the name of that test so I can identify it. > > * toString(): dump this object as a string so I can diagnose the problem (why > is not equal to that other one) without running the debugger. > > So, I would exterminate toString() from CppUnit, with the exception of > assertion_traits. Mmm. Upon closer examination still, I find that all the exception classes *also* have a toString() method. I understand that one can make the semantic distinction between getName() and toString(), as you say. On the other hand, if a number of the classes use getName() (roughly, the Test hierarchy), and another good fraction use toString() (roughly, the test result classes: assertions and exceptions), folks are likely to be confused. I, for one, will be unable to keep things straight. Why not move to "toString()" across the board? One could say that the semantic of toString() is, as you say, to "dump the object as a string ... for diagnosis". The name of a simple test --- e.g. a TestCase object --- *is* the best textual description of the object. For compounds and composed tests, we are not using the naked name. TestSuite::toString() returns "suite " + name, and RepeatedTest::toString() returns name + " (repeated)". The diagnostic routines, e.g. TestFailure currently use Test::toString(), not Test::getName(), so it is the latter that we can safely discard. -Steve -- by Rocket to the Moon, by Airplane to the Rocket, by Taxi to the Airport, by Frontdoor to the Taxi, by throwing back the blanket and laying down the legs ... - They Might Be Giants |
From: Steve M. R. <ste...@vi...> - 2001-07-14 06:15:26
|
On Tue, Jul 10, 2001 at 05:25:51PM +0200, Baptiste Lepilleur wrote: > toString() and getName() have very different semantic. > > * getName(): give me the name of that test so I can identify it. > > * toString(): dump this object as a string so I can diagnose the problem (why > is not equal to that other one) without running the debugger. > > So, I would exterminate toString() from CppUnit, with the exception of > assertion_traits. OK, now I have found one place where the real name of the test is required, and not just the "diagnostic string". In TextTestRunner, there are findTestByName() and runTestByName() methods. Presumably, one doesn't want to resort to searching for "footest (suite)". Perhaps it is best to retain both getName() and toString(). -S -- by Rocket to the Moon, by Airplane to the Rocket, by Taxi to the Airport, by Frontdoor to the Taxi, by throwing back the blanket and laying down the legs ... - They Might Be Giants |
From: Baptiste L. <gai...@fr...> - 2001-07-16 14:44:53
|
Quoting "Steve M. Robbins" <ste...@vi...>: > On Tue, Jul 10, 2001 at 05:25:51PM +0200, Baptiste Lepilleur wrote: > > > toString() and getName() have very different semantic. > > > > * getName(): give me the name of that test so I can identify it. > > > > * toString(): dump this object as a string so I can diagnose the > problem (why > > is not equal to that other one) without running the debugger. > > > > So, I would exterminate toString() from CppUnit, with the exception > of > > assertion_traits. > > OK, now I have found one place where the real name of the test is > required, and not just the "diagnostic string". In TextTestRunner, > there are findTestByName() and runTestByName() methods. Presumably, > one doesn't want to resort to searching for "footest (suite)". > > Perhaps it is best to retain both getName() and toString(). As I said, getName() is used to "identify" the test. Identification is used in three ways at the current time: - reporting which test failed - look up a test in the tests provided to the TestRunner - browsing the tests provided to the TestRunner. toString() make a diagnostic string for an object. Therefore, for TestFailure, it dump the test name and the exception message. This implementation of toString () is not use for reporting test failure and is of no use for CppUnit. Baptiste. --- Baptiste Lepilleur <gai...@fr...> http://gaiacrtn.free.fr/index.html Language: English, French |