RE: [Cppunit-devel] How to use clocker in my own project ?

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Well,

I've been interested in timing of CppUnit tests for a while - so here's my
2p worth ;)

I think the reason for questioning timing of unit tests is that this sort of
thing "should be" done with batch mode tests (i.e. where your app is invoked
batch mode, once per test) as opposed to unit tests.  Assuming, that is that
you have a batch interface for your app.

I guess this is because the things you want to time are generally specific
bits of functionality, rather than say only a particular method of a class
(which is usually too fast).

Personally I don't see anything wrong with calling larger bits of
functionality as unit tests, or even stress-testing lower level things like
file copying routines and such.  Where to draw the line between unit tests
and batch mode tests is a personal choice (and a can of worms;)

For these larger unit tests, the inaccuraries of the NT (and I believe UNIX)
clock() calls are reasonable for operations that take several seconds of
CPU.

In this sense I would suggest that timing of tests is sometimes relevant to
have per-test but not particularly relevant for the whole test run.

Regards
--
Jake

-----Original Message-----
From: Philippe Lavoie [mailto:phi...@ca...]
Sent: 23 April 2002 15:08
To: Baptiste Lepilleur; cpp...@li...
Subject: RE: [Cppunit-devel] How to use clocker in my own project ?

I want to use the timing for statistics. Often developpers will do a small
change that has a big impact on performance, they added expensive
copy-constructors or other such nuisance.

I just want to keep track of the overall progress of the timings. I'm using
cppunit as part of an integrated build process based on ant. At every hours
or so, the build process finds out if there is something new. It then
compiles and runs the tests. A new result-YYYYMMDDHHMM.xml file is created. 

The build process then merges the information with the other result-XXXXX
files. A graph of the timings can be made, and other statistical information
too (number of line of codes changed, number of tests added, etc.). 

Whether this is really important, I don't know. I just know that we
shouldn't be making things slower as we progress and that catching this as
part of the build process is as a good place as any other :)

Phil

PS This comes from one of our clients where the test stress every bugfix to
see if a fix comes with a speed degradation cost. 

> -----Original Message-----
> From: Baptiste Lepilleur [mailto:gai...@fr...]
> Sent: Tuesday, April 23, 2002 9:42 AM
> To: cpp...@li...
> Subject: Re: [Cppunit-devel] How to use clocker in my own project ?
> 
> 
> Clocker was note designed for 'real life' project (just an 
> example of weird
> stuff you can do with the TestListener). As you may have 
> noticed, reported
> time are far from accurate (even using the supposed high 
> precision timer of
> Nt 100 nanosecond, you don't get anything better that a 10ms 
> resolution).
> 
> Still you raise an interesting problem. I can think of two 
> ways of doing
> what you want at the current time:
> - Clocker write a second XML file, it is later merged with 
> the first one
> (using XSL or whatever)
> - Extending XmlOutputter in Clocker to write the Xml ouput 
> with test time.
> 
> What are you using test time for ? I don't really see any use 
> to this (total
> test time yes, but individual...)
> 
> You're right about the average test time bug: the time of the 
> test case is
> counted twice: for the test and the suite... I'll fix that.
> 
> Baptiste.
> 
> ----- Original Message -----
> From: "Philippe Lavoie" <phi...@ca...>
> To: <cpp...@li...>
> Sent: Tuesday, April 23, 2002 1:25 AM
> Subject: [Cppunit-devel] How to use clocker in my own project ?
> 
> 
> 
> Hi guys,
> 
> Now that I've managed to figure out how to do a proper copy 
> from one place
> to the other :) I'm ready to incorporate cppunit results 
> inside the build
> process (I'm using ant to do the builds).
> 
> I'd prefer to have the xml output with the test duration 
> times included. I
> want to have the default XML output from DllPluginTester to 
> also have timing
> values. What is the best way to achieve this ?
> 
> I think I might be missing some basic concepts on how I can 
> mix and match
> the different listener and outputer. I'm pretty sure I should 
> look at the
> ClockPlugIn for guidance.
> 
> I did the following naive test hoping that some magic would occur
> 
> DllPlugInTesterd_dll -x test.xml -c Mine.dll clockerplugind.dll
> .
> (0.050s) All Tests
>    +--(0.050s) InputErrorTest
>    |  +--(0.050s) InputErrorTest.testNormalUsage
> 
> Total elapsed time: (0.050s) , average test case time: (0.150s) OK (1)
> 
> The command line output is indeed magic :) (although there is 
> a bug in the
> average test case time). However, the XML output is as follows.
> 
> 
> <?xml version="1.0" encoding='' standalone='yes' ?>
> <TestRun>
>   <FailedTests></FailedTests>
>   <SuccessfulTests>
>     <Test id="1">
>       <Name>InputErrorTest.testNormalUsage</Name>
>     </Test>
>   </SuccessfulTests>
>   <Statistics>
>     <Tests>1</Tests>
>     <FailuresTotal>0</FailuresTotal>
>     <Errors>0</Errors>
>     <Failures>0</Failures>
>   </Statistics>
> </TestRun>
> 
> Please let me know if you know in which direction I should go 
> to add a time
> attribute to the Test results. Thanks.
> 
> 
> Phil
> 
> 
> _______________________________________________
> Cppunit-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppunit-devel
> 
> 
> 
> _______________________________________________
> Cppunit-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppunit-devel
> 

_______________________________________________
Cppunit-devel mailing list
Cpp...@li...
https://lists.sourceforge.net/lists/listinfo/cppunit-devel