cppqed-support

[Cppqed-support] new repository layout

[Raimar S. <rai...@ui...>]

Hi András,

over the weekend I finished the split into core, elements and scripts and adapted the cmake build process accordingly. It still needs some polishing, but in general it works quite well I think.

First of all you will need cmake 2.8.9. I'm sorry about this dependency, I know your Ubuntu distribution doesn't have it yet. But there are some very nice features, which if missing would need ugly workarounds. You can install the most recent version of cmake with:

wget http://www.cmake.org/files/v2.8/cmake-2.8.12-Linux-i386.sh
chmod +x cmake-2.8.12-Linux-i386.sh
./cmake-2.8.12-Linux-i386.sh

In the new layout we have four repositories:

* C++QED: very lightweight repository bringing along the other repositories as git submodules. A simple CMakeLists.txt enables the user to compile everything at once.
* C++QEDcore: branch split_elements
* C++QEDelements: branch master (as this is a new repository I didn't have to avoid master)
* C++QEDscripts: branch split_elements

The latter three repositories can be compiled as standalone cmake projects, given they find their dependencies.

There are three modes of compilation, it would be great if you could test all of them.

1. Monolithic build from within C++QED
**************************************
$ git clone --recursive ssh://vu...@gi.../p/cppqed/cppqed C++QED
Then open the top level CMakeLists.txt as usual in kdevelop and compile.

2. Modular build without installation
*************************************
$ git clone -b split_elements ssh://vu...@gi.../p/cppqed/cppqed_core CPPQEDcore
$ git clone ssh://vu...@gi.../p/cppqed/cppqed_elements CPPQEDelements
$ git clone -b split_elements ssh://vu...@gi.../p/cppqed/cppqed_scripts CPPQEDscripts

* build CPPQEDcore as usual
* when configuring CPPQEDelements, call cmake with -DCPPQED_DIR=/path/to/CPPQEDcore/build, then compile CPPQEDelements
* when configuring CPPQEDscripts, call cmake with -DCPPQED_DIR=/path/to/CPPQEDcore/build and 
-DCPPQelements_DIR=/path/to/CPPQEDelements/build, then compile

It is also possible to skip the -D, let the configuration in kdevelop fail for the first time and then set the path in the project configuration (Project->Open configuration, set CPPQED_DIR and CPPQEDelements_DIR there).

3. Modular build with installation
**********************************
* Clone the three repositories as in 2.
* Call cmake with -DCMAKE_INSTALL_PREFIX=/some/prefix (to install into prefix) and -DCMAKE_PREFIX_PATH=/some/prefix (to find stuff in prefix)
* Build and install CPPQEDcore, then CPPQEDelements, then CPPQEDscripts (the scripts are not installed as of now)


Method 1 is very similar to the way it used to be, but one has to handle git submodules, which can be confusing at first (only if one wants to use this as a working directory for development). It also has the advantage that cmake knows how to build every target in the project and rebuilds all dependencies automatically.

Method 2 might prove useful for development, especially when maintaining additional elements repositories. One does not have to remember to install all the time. Dependencies cannot be built automatically, i.e. if core changes it is not sufficient to rebuild the scripts, as the scripts project does not know how to build the core. One has to rebuild core and then scripts. But if the scripts project sees that the core has changed, it will relink automatically.

Method 3 will come in most convenient when binary packages exist of the libraries. In this scenario, the libraries are found automatically by cmake.

Some technical details:

* I implemented the right include path dependencies between components, both in core and in elements.
* I removed the details subdirectories. Include files former in details are now prefixed with "details_", this way the include guards stay as the were. When installed, include files will go into a flat directory CPPQED-2.9/{core,elements}. This just resembles the way cmake works better, otherwise a lot of boilerplate code has to be used to keep the directory structure of include files.
* Given the first two points, there is now a problem with structure/Free.h, where a file from ../quantumoperator is included. When installed, this path is not correct anymore. To work around this problem until it is fixed, I have introduced a compiler macro indicating whether to include ../quantumoperator/TridiagonalFwd.h or just TridiagonalFwd.h.
* If one repository is built in DEBUG mode it cannot be used together with another repository in RELEASE mode. At the moment this will be detected at link time. With method 3 it is also possible to install both DEBUG and RELEASE in parallel. Dependent components will pick up the right library automatically.
* Libraries and executables should run without the need of LD_LIBRARY_PATH environment variable. All non-standard library paths are built into the binaries, the resulting executables should be relocatable as long as the libraries stay where they are. It would also be possible to make the libraries relocatable, but only if they are in standard paths or LD_LIBRARY_PATH is used.
* I have not touched the development branch yet
* I have not tested what happens if clang and g++ are mixed. This should actually work, right? If not one would have to test for consistent compilers.

Todo: provide a simple way for users to create their own elements- and script-projects.

Please keep in touch, this definitely needs a lot of testing and probably still contains bugs. We should also talk about a good workflow regarding the git submodules. Some reading on submodules:
http://git-scm.com/book/en/Git-Tools-Submodules

Best regards
Raimar

Re: [Cppqed-support] new repository layout

[Raimar S. <rai...@ui...>]

Hi András,

I have created two new repositories, CustomElementsExample and 
CustomScriptsExample. They contain very simple and almost self-explanatory 
CMakeLists.txt which show how to create custom projects for elements and 
scripts.

Additionally I added an experimental feature to the build process, the cmake 
user registry. This is a quite cool feature of cmake, let's see if it proves 
useful. Wen building CPPQEDcore, CPPQEDelements or any of the custom elements 
project, the library locations are recorded in ~/.cmake, and projects 
depending on those libraries can find them without user intervention. The only 
problem I see is that this might interfere with installed versions of the 
library...

This means you can now (after updating all the submodules) just go ahead and 
build everything, either monolithic or modular, and don't have to care about 
CMAKE_PREFIX_PATH or <something>_DIR variables. Everything should be found 
automatically. This also holds for the custom projects (elements and scripts).

Best regards
Raimar

Re: [Cppqed-support] new repository layout, Development branch ready

[Raimar S. <rai...@ui...>]

Dear András,

I have merged the new repository layout into Development. This was certainly one of the more interesting merges I have had so far :)

The branches are currently named:

C++QED -> Development
C++QEDcore -> Development_split_elements
CPPQEDelements -> Development
C++QEDscripts -> Development_split_elements

I think after some testing we can merge the Development_split_elements branches back into Development (this will be a fast-forward), thus concluding the split. Then new doxygen-branches can be created as well. For the OpenMP branch it might be an option to rebase and push --force (will be a simpler history graph), or otherwise merge. This should be straight forward, as the changes in OpenMP touch only the core.

I had to disable some scripts, could you check if this was to be expected (I think so) or if the failures result from the merge? To re-enable for testing, remove the scripts from the EXCLUDE_SCRIPTS variable in CMakeLists.txt.

Best regards
Raimar

P.S.: Some technical details...

Are you currently working with the C++QED repository and submodules or with the standalone repositories? If you have standalone repositories, just checkout the corresponding branches, you can ignore the rest.

If you are using C++QED, checkout Development in C++QED. This will not touch the submodules, therefore you will have a dirty working directory because the recorded commits of the submodules don't match. You have to switch branches in the submodules manually, or to automate this, call inside C++QED:

$ git submodule foreach 'BRANCH=$(git config --file $toplevel/.gitmodules --get submodule.${name}.branch); git checkout ${BRANCH:-master}

This command is so useful that I have created an alias in ~/.gitconfig:

[alias]
        attach = "submodule foreach 'BRANCH=$(git config --file $toplevel/.gitmodules --get submodule.${name}.branch); git checkout ${BRANCH:-master}'"

With this alias in place, the work flow for cloning and switching branches would be:

git clone --recurse <URL_of_C++QED> C++QED
cd C++QED
git attach

Change to Development branch:

git checkout Development
git attach

Change back to master:

git checkout master
git attach