Re: [Cppqed-support] new testsuite

[Andras V. <and...@ui...>]

Hi Raimar,
Again, this looks like a great achievement. Please allow me a few days for
delving into it.
Just one note: instead of creating a separate project Testing, we could
consider leaving the testsuite in the meta-repository C++QED. In my mind,
this is similar to the overall pieces of documentation that I mentioned
earlier, which span over the whole project core-elements-scripts (or at
least the first two).
Best regards,
András



On Thu, Dec 19, 2013 at 12:55 PM, Raimar Sandner
<rai...@ui...>wrote:

>  There was a commit not pushed to sourceforge in CCQEDcore, this is now
> fixed.
>
>
>
> Raimar
>
>
>
> On Thursday, December 19, 2013 03:40:42 you wrote:
>
> Hi András,
>
>
>
> I have integrated a new testsuite into cmake.
>
>
>
> Please clone the testing branch of the monolithic repository from the
> private repository:
>
>
>
> git clone -b testing --recursive gi...@ge...:
> cppqed/complete.git
>
>
>
> Then build everything with the monolithic build method (using the toplevel
> CMakeLists.txt file). Please make sure that no outside components are
> picked up by cmake, probably it is best to delete ~/.cmake. Afterwards, in
> the build directory run ctest -jN where N is the number of cores. Please
> let me know if this all works and if the tests pass.
>
>
>
> A few notes to the structure now, an I will try to write a complete
> documentation soon.
>
>
>
> The testsuite uses ctest from within cmake, which means the complete
> testsuite can be run by either 'make test' or the ctest command from above.
> The ctest command has the advantage of being able to run the tests in
> parallel with the -j switch. CTest itself is rather 'dumb', basically one
> only has to specify a test name and a test command including command line
> options in the CMakeLists.txt file.
>
>
>
> In our case, the command is either the executable target<http://bit.ly/1gHiTC0>named 'tester', which we have built with the boost
> unittest framework <http://bit.ly/1gHjFyW>, or a python script<http://bit.ly/1gHj2FA>which I wrote as test driver. The former is for testing code snippets, your
> boost tests can be imported that way. The latter is basically for running
> scripts. At the moment the testdriver.py is capable of:
>
>
>
> * running a C++QED script in all modes (single, ensemble, master)
>
> * continuing a script
>
> * comparing the outcome of a C++QED script to saved and version controlled
> output in Testing/expected
>
> * comparing the outcome of a C++QED script to the outcome of another script
>
> * check that the compilation of a cmake target fails in an expected way<http://bit.ly/1gHlPi4>
>
>
>
> and the functionality can be easily extended. The behavior of the
> testdriver is controlled by command line options and configuration files
> testdriver.conf in the subdirectories.
>
>
>
> Note that neither 'make test' nor 'ctest' will trigger a re-configuration
> or re-compilation in the case of changed CMakeLists.txt files or changed
> source files. To overcome this problem I have defined the target 'check'.
> This should depend on everything essential and therefore trigger
> re-configuration and compilation. Every subdirectory of the testsuite (at
> the moment boost, compilefail,continue and run) has its own 'check_subdir'
> target, and additionally every single test has its own target. This is very
> convenient when developing or debugging a feature, one does not have to run
> the whole testsuite all the time but can fine-tune which tests to run by
> adding them to the buildset in kdevelop. Additionally, all the check*
> targets are a lot more verbose, because they use ctest -v internally.
>
>
>
> The next steps:
>
>
>
> * documentation of testdriver.py, the testdriver.conf syntax and how to
> add new tests to cmake
>
> * making Testing a standalone project, at the moment the testsuite is only
> available in monolithic builds
>
> * import all the other boost unittests which exist
>
> * import the other CompositeError files into the compilefail submodule
> (and I think the CompositeError2 which I used so far gives additional
> compilation errors which should not exist, but I'm not sure. Please check
> the output of make CompositeError2.)
>
> * write more tests to cover a broader range of features and design promises
>
> * integrate the physics testsuite
>
> * anything else?
>
>
>
> Best regards
>
> Raimar
>
>
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Cppqed-support mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppqed-support
>
>