#389 parallel build fails

[Zbigniew Jędrzejewski-Szmek]

I'm compiling libsbml on Fedora 21:

{tar -xvf ~/libSBML-5.11.0-core-plus-packages-src.tar.gz && cd libSBML-5.11.0-Source && ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --libdir=/usr/lib64 --disable-static --with-check --with-swig --with-doxygen --with-python --with-python-interpreter=/usr/bin/python2 --with-perl --enable-layout --enable-comp --enable-fbc --enable-qual && make -j20; }|tee build.log

...
make[3]: Leaving directory '/var/tmp/libSBML-5.11.0-Source/src/bindings/python'
make[2]: Leaving directory '/var/tmp/libSBML-5.11.0-Source/src/bindings'
make[1]: *** [bindings-recurse] Error 2
../config/makefile-common-actions.mk:373: recipe for target 'bindings-recurse' failed
make[1]: Leaving directory '/var/tmp/libSBML-5.11.0-Source/src'
make: *** [src-recurse] Error 2
config/makefile-common-actions.mk:373: recipe for target 'src-recurse' failed
make: Leaving directory '/var/tmp/libSBML-5.11.0-Source'

[Michael Hucka]

Thanks for the report.

Parallel builds actually do work when configuring using cmake and using the cmake-generated makefiles.

Parallel builds indeed do not work when using gnu autonconf/configure and using the hand-written makefiles. Unfortunately we lack the resources to fix that, so it will probably never be fixed. However, your report is a reminder that we need to document this fact, so I added an item to our tracker to do this.

Perhaps you can use cmake instead?

[Zbigniew Jędrzejewski-Szmek]

Sure, cmake would work for me too. Since I'm just building a package, it's pretty much the same.

[Lucian Smith]

It's possible that using cmake might fix your other valgrind test problem, too, though we do of course appreciate the report, and will work to fix it.

[Zbigniew Jędrzejewski-Szmek]

Hm, building with cmake works nicely, but I have no idea how to compile or run the tests:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$ mkdir build && cd build && cmake -DENABLE_{LAYOUT,QUAL,COMP,FBC}=ON -DWITH_{PYTHON,PERL}=ON -DCMAKE_INSTALL_PREFIX:PATH=/usr .. && make ...this part works ok... $ make check gmake[4]: *** No rule to make target 'test'. Stop. CMakeFiles/check.dir/build.make:49: recipe for target 'CMakeFiles/check' failed make[3]: *** [CMakeFiles/check] Error 2 CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/check.dir/all' failed make[2]: *** [CMakeFiles/check.dir/all] Error 2 CMakeFiles/Makefile2:68: recipe for target 'CMakeFiles/check.dir/rule' failed make[1]: *** [CMakeFiles/check.dir/rule] Error 2 Makefile:171: recipe for target 'check' failed make: *** [check] Error 2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Lucian Smith]

The CMake option for turning on the tests is WITH_CHECK. At that point, 'make check' should work, assuming CMake can find libcheck. (If it can't, use 'LIBCHECK_INCLUDE_DIR' and 'LIBCHECK_LIBRARY'.)

[Zbigniew Jędrzejewski-Szmek]

Thank you for the hint.

Indeed, with cmake the tests behave better, although they still fail.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make check
Running tests...
Test project /var/tmp/libSBML-5.11.0-Source/build
Start 1: test_sbml_package_layout_run
1/22 Test #1: test_sbml_package_layout_run ...... Passed 0.54 sec
Start 2: test_layout_validator_run
2/22 Test #2: test_layout_validator_run ......... Passed 0.64 sec
Start 3: test_comp_extensions_run
3/22 Test #3: test_comp_extensions_run .......... Passed 0.39 sec
Start 4: test_comp_sbase_run
4/22 Test #4: test_comp_sbase_run ............... Passed 0.10 sec
Start 5: test_comp_util_run
5/22 Test #5: test_comp_util_run ................ Passed 3.60 sec
Start 6: test_comp_validator_run
6/22 Test #6: test_comp_validator_run ........... Passed 0.52 sec
Start 7: test_sbml_package_fbc_run
7/22 Test #7: test_sbml_package_fbc_run ......... Passed 0.05 sec
Start 8: test_fbc_validator_run
8/22 Test #8: test_fbc_validator_run ............ Passed 0.12 sec
Start 9: test_extension_package_qual_run
9/22 Test #9: test_extension_package_qual_run ... Passed 0.05 sec
Start 10: test_sbml_package_qual_run
10/22 Test #10: test_sbml_package_qual_run ........ Passed 0.09 sec
Start 11: test_qual_validator_run
11/22 Test #11: test_qual_validator_run ........... Passed 0.17 sec
Start 12: test_sbml_annotation_run
12/22 Test #12: test_sbml_annotation_run .......... Passed 0.38 sec
Start 13: test_sbml_conversion_run
13/22 Test #13: test_sbml_conversion_run .......... Passed 0.45 sec
Start 14: test_sbml_extension_run
14/22 Test #14: test_sbml_extension_run ........... Passed 0.05 sec
Start 15: test_sbml_math_run
15/22 Test #15: test_sbml_math_run ................ Passed 1.30 sec
Start 16: test_sbml_sbml_run
16/22 Test #16: test_sbml_sbml_run ................ Passed 2.39 sec
Start 17: test_sbml_units_run
17/22 Test #17: test_sbml_units_run ............... Passed 0.77 sec
Start 18: test_sbml_util_run
18/22 Test #18: test_sbml_util_run ................ Passed 0.06 sec
Start 19: test_sbml_validator_run
19/22 Test #19: test_sbml_validator_run ........... Passed 2.37 sec
Start 20: test_sbml_xml_run
20/22 Test #20: test_sbml_xml_run ................. Passed 0.15 sec
Start 21: test_perl_binding
21/22 Test #21: test_perl_binding ................. Passed 6.63 sec
Start 22: test_python_binding
22/22 Test #22: test_python_binding ...............***Failed 0.29 sec

95% tests passed, 1 tests failed out of 22

Total Test time (real) = 21.11 sec

The following tests FAILED:
22 - test_python_binding (Failed)
Errors while running CTest
Makefile:137: recipe for target 'test' failed
gmake[4]: *** [test] Error 8
CMakeFiles/check.dir/build.make:49: recipe for target 'CMakeFiles/check' failed
make[3]: *** [CMakeFiles/check] Error 2
CMakeFiles/Makefile2:928: recipe for target 'CMakeFiles/check.dir/all' failed
make[2]: *** [CMakeFiles/check.dir/all] Error 2
CMakeFiles/Makefile2:936: recipe for target 'CMakeFiles/check.dir/rule' failed
make[1]: *** [CMakeFiles/check.dir/rule] Error 2
Makefile:545: recipe for target 'check' failed
make: *** [check] Error 2
LANG=C make check 16,11s user 5,50s system 101% cpu 21,199 total
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I'll try to debug this tomorrow.

[Frank Bergmann]

if you run ctest directly, you could isolate the issue more quickly. Try running

ctest -R python -V

this will only run the python binding test in verbose mode. I have a hunch that this might have to do with cmake having picked up a different python library, than the interpreter that you are using.

Could you please attach your CMakeCache.txt file from the build directory, along with the output of the ctest command above?

thank you
Frank

[Zbigniew Jędrzejewski-Szmek]

I forgot to update the status here. Your haunch was 100% correct: cmake picked up Python 3 compilation flags even when using python2 binary. Using something like

cd build2
%cmake -DENABLE_{LAYOUT,QUAL,COMP,FBC,RENDER}=ON \
       -DWITH_PYTHON=ON \
       -DWITH_CHECK=ON \
       -DPYTHON_INCLUDE_DIR:PATH=$(pkg-config --cflags python2|sed s/-I//) \
       -DPYTHON_LIBRARY:FILEPATH=$(pkg-config --libs python2) \
       -DPYTHON_EXECUTABLE:FILEPATH=%{__python2} \
       -DCMAKE_INSTALL_LIBDIR:PATH=%{_libdir} \
       ..
cd ../build3
%cmake -DENABLE_{LAYOUT,QUAL,COMP,FBC,RENDER}=ON \
       -DWITH_PYTHON=ON \
       -DWITH_CHECK=ON \
       -DPYTHON_INCLUDE_DIR:PATH=$(pkg-config --cflags python3|sed s/-I//) \
       -DPYTHON_LIBRARY:FILEPATH=$(pkg-config --libs python3) \
       -DPYTHON_EXECUTABLE:FILEPATH=%{__python3} \
       -DCMAKE_INSTALL_LIBDIR:PATH=%{_libdir} \
       ..
cd ..
make -C build2 %{?_smp_mflags} check
make -C build3 %{?_smp_mflags} check

the tests pass for both versions.

Please close!