Re: [Cppqed-support] new testsuite

[Raimar S. <rai...@ui...>]

Hi András,

sure take your time, I hope you will also find it useful. For me it has 
already helped to discover some bugs which would otherwise have gone 
unnoticed.

If you don't mind I will already merge it to Development (only affects the 
super project, anyway), so that I don't have to switch branches so often. This 
will then also introduce cpypyqed to the super repository.

Best regards
Raimar

On Thursday, December 19, 2013 17:44:35 Andras Vukics wrote:
> Hi Raimar,
> Again, this looks like a great achievement. Please allow me a few days for
> delving into it.
> Just one note: instead of creating a separate project Testing, we could
> consider leaving the testsuite in the meta-repository C++QED. In my mind,
> this is similar to the overall pieces of documentation that I mentioned
> earlier, which span over the whole project core-elements-scripts (or at
> least the first two).
> Best regards,
> András
> 
> 
> 
> On Thu, Dec 19, 2013 at 12:55 PM, Raimar Sandner
> 
> <rai...@ui...>wrote:
> >  There was a commit not pushed to sourceforge in CCQEDcore, this is now
> > 
> > fixed.
> > 
> > 
> > 
> > Raimar
> > 
> > 
> > 
> > On Thursday, December 19, 2013 03:40:42 you wrote:
> > 
> > Hi András,
> > 
> > 
> > 
> > I have integrated a new testsuite into cmake.
> > 
> > 
> > 
> > Please clone the testing branch of the monolithic repository from the
> > private repository:
> > 
> > 
> > 
> > git clone -b testing --recursive gi...@ge...:
> > cppqed/complete.git
> > 
> > 
> > 
> > Then build everything with the monolithic build method (using the toplevel
> > CMakeLists.txt file). Please make sure that no outside components are
> > picked up by cmake, probably it is best to delete ~/.cmake. Afterwards, in
> > the build directory run ctest -jN where N is the number of cores. Please
> > let me know if this all works and if the tests pass.
> > 
> > 
> > 
> > A few notes to the structure now, an I will try to write a complete
> > documentation soon.
> > 
> > 
> > 
> > The testsuite uses ctest from within cmake, which means the complete
> > testsuite can be run by either 'make test' or the ctest command from
> > above.
> > The ctest command has the advantage of being able to run the tests in
> > parallel with the -j switch. CTest itself is rather 'dumb', basically one
> > only has to specify a test name and a test command including command line
> > options in the CMakeLists.txt file.
> > 
> > 
> > 
> > In our case, the command is either the executable
> > target<http://bit.ly/1gHiTC0>named 'tester', which we have built with the
> > boost unittest framework <http://bit.ly/1gHjFyW>, or a python
> > script<http://bit.ly/1gHj2FA>which I wrote as test driver. The former is
> > for testing code snippets, your boost tests can be imported that way. The
> > latter is basically for running scripts. At the moment the testdriver.py
> > is capable of:
> > 
> > 
> > 
> > * running a C++QED script in all modes (single, ensemble, master)
> > 
> > * continuing a script
> > 
> > * comparing the outcome of a C++QED script to saved and version controlled
> > output in Testing/expected
> > 
> > * comparing the outcome of a C++QED script to the outcome of another
> > script
> > 
> > * check that the compilation of a cmake target fails in an expected
> > way<http://bit.ly/1gHlPi4>
> > 
> > 
> > 
> > and the functionality can be easily extended. The behavior of the
> > testdriver is controlled by command line options and configuration files
> > testdriver.conf in the subdirectories.
> > 
> > 
> > 
> > Note that neither 'make test' nor 'ctest' will trigger a re-configuration
> > or re-compilation in the case of changed CMakeLists.txt files or changed
> > source files. To overcome this problem I have defined the target 'check'.
> > This should depend on everything essential and therefore trigger
> > re-configuration and compilation. Every subdirectory of the testsuite (at
> > the moment boost, compilefail,continue and run) has its own 'check_subdir'
> > target, and additionally every single test has its own target. This is
> > very
> > convenient when developing or debugging a feature, one does not have to
> > run
> > the whole testsuite all the time but can fine-tune which tests to run by
> > adding them to the buildset in kdevelop. Additionally, all the check*
> > targets are a lot more verbose, because they use ctest -v internally.
> > 
> > 
> > 
> > The next steps:
> > 
> > 
> > 
> > * documentation of testdriver.py, the testdriver.conf syntax and how to
> > add new tests to cmake
> > 
> > * making Testing a standalone project, at the moment the testsuite is only
> > available in monolithic builds
> > 
> > * import all the other boost unittests which exist
> > 
> > * import the other CompositeError files into the compilefail submodule
> > (and I think the CompositeError2 which I used so far gives additional
> > compilation errors which should not exist, but I'm not sure. Please check
> > the output of make CompositeError2.)
> > 
> > * write more tests to cover a broader range of features and design
> > promises
> > 
> > * integrate the physics testsuite
> > 
> > * anything else?
> > 
> > 
> > 
> > Best regards
> > 
> > Raimar
> > 
> > 
> > 
> > 
> > 
> > --------------------------------------------------------------------------
> > ---- Rapidly troubleshoot problems before they affect your business. Most
> > IT organizations don't have a clear picture of how application
> > performance affects their revenue. With AppDynamics, you get 100%
> > visibility into your Java,.NET, & PHP application. Start your 15-day FREE
> > TRIAL of AppDynamics Pro!
> > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktr
> > k
> > _______________________________________________
> > Cppqed-support mailing list
> > Cpp...@li...
> > https://lists.sourceforge.net/lists/listinfo/cppqed-support