#8 make forcegen fails after configure in Ubuntu

[Cristian Serpell]

Hi, I am trying to install swIMP in my Ubuntu, to link Java code to Osi.

I downloaded version 0.9.5, and then ran ./configure. It found these solvers:

checking CbcSolver.hpp usability... no
checking CbcSolver.hpp presence... no
checking for CbcSolver.hpp... no
checking ClpModel.hpp usability... no
checking ClpModel.hpp presence... no
checking for ClpModel.hpp... no
checking VolVolume.hpp usability... no
checking VolVolume.hpp presence... no
checking for VolVolume.hpp... no
checking symphony.h usability... no
checking symphony.h presence... no
checking for symphony.h... no
checking glpk.h usability... yes
checking glpk.h presence... yes
checking for glpk.h... yes
checking mosek.h usability... no
checking mosek.h presence... no
checking for mosek.h... no

I tried to make forcegen, and it failed because I didn't have swig installed (note that configure didn't fail). I installed it, and now it is failing with this error message:

coin_wrap.cxx: In function 'std::string CoinPackedMatrix_toString(CoinPackedMatrix)':
coin_wrap.cxx:976:20: error: 'unlink' was not declared in this scope
coin_wrap.cxx: In function 'std::string OsiSolverInterface_getLPString(OsiSolverInterface)':
coin_wrap.cxx:1071:20: error: 'unlink' was not declared in this scope

I found that unlink is included in unistd.h. I opened coin_wrap.cxx, added "#include <unistd.h>" after some other includes in the file, and then I can compile it manually. Anyway, the make forcegen continues failing. Probably that file is auto generated.

So, how could I fix so it doesn't happen anymore?

Any help is appreciated, thanks.

A side note, the build process gives a lot of warnings, both from multiple definitions on the code, and also from SWIG, like this one:

/usr/bin/swig -c++ -java \ \ -I.. -Icoin -Icoin/static \ -I./../coin/include -I./coin \ -package net.sourceforge.swimp.coin \ -outdir ./../java/net/sourceforge/swimp/coin \ -o coin_wrap.cxx \ coin.i
SWIG:1: Warning 125: Use of the include path to find the input file is deprecated and will not work with ccache. Please include the path when specifying the input file.

[Cristian Serpell]

I have been trying to make it run the whole day...

I modified several lines of configure file. For example, libtool generated file has an ECHO variable that is not working, and making an exact copy as "echo" works, as it is used that way below in the file. Then, the generated coin_wrap.cxx can be fixed including unistd.h, so I can do it manually. But I didn't manage to do it in the configure file.

Any help? Thanks.

[combean]

This can probably be fixed by adding

#include <unistd.h>

to iface/coin/CoinPackedMatrix.i and OsiSolverInterface.i.

But based on your most recent post I guess that you have already found that out yourself.

[Cristian Serpell]

Well, I made several modifications into configure, Makefile.am, coin.i and project.properties files, and now I get the whole thing compiling.

I can run correctly make forcegen, make, but then make checks fails with Cbc and Clp. Thus, I added them to project.properties test.excluded variable, so they are not tested. After doing so, it continues failing, but now with a different error, for Glpk. Could you please give me some help?

Thanks, this is the test output:

Testsuite: net.sourceforge.swimp.test.osiglpk.TestOsiGlpkSolverInterface
Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0,012 sec

Testcase: testRandomAssignment took 0,008 sec
Caused an ERROR
/home/cserpell/Projects/RoLT/downloads/cvs_swimp/swimp-code/swimp/iface/.libs/libswIMPOsiGlpk.so.0.0.0: /home/cserpell/Projects/RoLT/downloads/Osi/Osi-0.106.2/lib/libOsiGlpk.so.1: undefined symbol: _ZN18OsiSolverInterface22activateRowCutDebuggerEPKdb
java.lang.UnsatisfiedLinkError: /home/cserpell/Projects/RoLT/downloads/cvs_swimp/swimp-code/swimp/iface/.libs/libswIMPOsiGlpk.so.0.0.0: /home/cserpell/Projects/RoLT/downloads/Osi/Osi-0.106.2/lib/libOsiGlpk.so.1: undefined symbol: _ZN18OsiSolverInterface22activateRowCutDebuggerEPKdb
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1953)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1878)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1868)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1087)
at net.sourceforge.swimp.osiglpk.osiglpkJNI.<clinit>(osiglpkJNI.java:14)
at net.sourceforge.swimp.osiglpk.OsiGlpkSolverInterface.<init>(OsiGlpkSolverInterface.java:41)
at net.sourceforge.swimp.test.osiglpk.TestOsiGlpkSolverInterface.setUp(TestOsiGlpkSolverInterface.java:47)

[Cristian Serpell]

Thanks! Yes, but I modified coin_base.i. Wow, I started all over, being more careful with libraries. I started linking against my own Osi compiled .so files, instead of the ones that come with the distribution. I reached the same point of tests, but look at the error I managed to create:

test:
[junit] Running net.sourceforge.swimp.test.coin.TestCoinPackedMatrix
[junit] Ordering 0 rhs - 1
[junit] Ordering 1 rhs - 0
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 0,082 sec
[junit] Running net.sourceforge.swimp.test.coin.TestCoinPackedVector
[junit] Tests run: 31, Failures: 0, Errors: 0, Time elapsed: 0,017 sec
Error in `/usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java': malloc(): memory corruption: 0x00002adb94404500
[junit] Running net.sourceforge.swimp.test.coin.TestCoinShallowPackedVector
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0,013 sec
[junit] Running net.sourceforge.swimp.test.osiclp.TestOsiClpSolverInterface

and then it hangs.

[combean]

That looks bad, indeed. Perhaps a 64 bit compatibility problem because I haven't tested the code on 64 bit (it's quite some time that I have worked last with it). Well, it does make sense to update my environment here to 64 bit and to give it a try but I cannot promise that I will manage to do that in the next few days. Perhaps you can isolate the problem by tracking it down to an individual test?

[combean]

This looks like a problem during the loading of libOsiGlpk.so, which is taken from your local installation under /home/cserpell/Projects/RoLT/downloads/Osi/Osi-0.106.2. The undefined symbol is referenced from Osi itself and not from the swIMP frontend. Are your sure that this installation works stand-alone (I mean, when you write a little C++ program to test it)? Have you run the test suite that comes with Osi? What does ldd /home/cserpell/Projects/RoLT/downloads/Osi/Osi-0.106.2/lib/libOsiGlpk.so.1 say?

Have you tried the binary distribuion of swIMP that comes with a Osi version with which it has been tested? Unfortunately, this version is a bit older and compiled for 32 bits but I may get you started. I hope that I find the time to update swIMP to a more recent binary distribution of Osi soon...

Please keep me posted on your progress. Thanks.

[Cristian Serpell]

I ran the tests that come with their distribution, but that may be the issue. Don't worry regarding the help, I know you do your best. Hopefully we can find the issue so everyone else gets the benefit.

This is the output from ldd:

ldd /home/cserpell/Projects/RoLT/downloads/Osi/Osi-0.106.2/lib/libOsiGlpk.so.1
linux-vdso.so.1 => (0x00007fffd21ed000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f967e38c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f967dfc3000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f967ddad000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f967daa8000)
/lib64/ld-linux-x86-64.so.2 (0x00007f967e8d0000)

[Cristian Serpell]

Almost there. I re built Osi from scratch, removed all binaries that come with swIMP, and other I had installed in my machine, and now I can do everything until make check, that fails like this:

test:
[junit] Running net.sourceforge.swimp.test.coin.TestCoinPackedMatrix
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 0,086 sec
[junit] Running net.sourceforge.swimp.test.coin.TestCoinPackedVector
[junit] Ordering 0 rhs - 1
[junit] Ordering 1 rhs - 0
[junit] Tests run: 31, Failures: 0, Errors: 0, Time elapsed: 0,017 sec
[junit] Running net.sourceforge.swimp.test.coin.TestCoinShallowPackedVector
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0,009 sec
[junit] Running net.sourceforge.swimp.test.osiglpk.TestOsiGlpkSolverInterface
[junit] glp_add_cols: ncs = 809720848; too many columns
[junit] Error detected in file glpapi01.c at line 357
[junit] Running net.sourceforge.swimp.test.probe.Batch-With-Multiple-Tests
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec

BUILD FAILED
/home/cserpell/Projects/RoLT/downloads/svn_swimp/swimp-code/swimp/build.xml:166: Tests failed (crashed)

The number after glp_add_cols: ncs = is always different. Then it crashes, so no output in testrep.

[Cristian Serpell]

Hi Thomas, I finally made it!! at least it is passing all tests. Now I will try to use it from my Java program. Probably it is going to be another big challenge.

I compiled this bash-like instructions on how I did it, starting from available code in sourceforge SVN. I hope this is helpful.

===========================

Step 1: Preparing the files

===========================

Useful shortcut

LOCAL_OSI_DIR="/home/cserpell/Projects/RoLT/downloads/Osi/Osi-0.106.2"

Get the code

svn checkout svn://svn.code.sf.net/p/swimp/code/trunk swimp-code
cd swimp-code/swimp

Remove some warnings

echo "AC_CONFIG_MACRO_DIR([m4])" >> configure.in # optional
echo "" >> Makefile.am # optional
echo "ACLOCAL_AMFLAGS = -I m4" >> Makefile.am # optional

Change wrong ifdefs to ifs. Also remove wrong includes

sed 's/^ifdef /if /g' iface/Makefile.am > tmpfile
sed 's/INCLUDES = $(LOCALINCLUDE) $(JAVA_INCLUDE)/INCLUDES = $(LOCALINCLUDE) # $(JAVA_INCLUDE)/g' > tmpfile2
mv tmpfile2 iface/Makefile.am

Comment lines of lib_LTLIBRARIES from solvers you don't have in Osi in iface/Makefile.am. For example:

libswIMPOsiSym.la

Add missing libraries

sed 's/LIBS=-lCoinUtils/LIBS=-lCoinUtils -lglpk -lbz2 -llapack/g' iface/perftest/Makefile.am > tmpfile
mv tmpfile iface/perftest/Makefile.am

Add missing include

sed 's/#include "config.h"/#include <unistd.h>\n#include "config.h"/g' iface/coin/coin_base.i > tmpfile
mv tmpfile iface/coin/coin_base.i

Just use my own Osi installation

rm coin/.so coin/.so.0
ln -s ${LOCAL_OSI_DIR}/lib/*.so coin/.
rm -r coin/include
ln -s ${LOCAL_OSI_DIR}/include/coin include

=============================================

Step 2: Remove unavailable solvers from tests

=============================================

Repeat this for each solver you don't have or don't compile.

sed 's/^.TestOsiClp.$//g' java/net/sourceforge/swimp/test/suites/AllCoinBinaryDistTests.java > tmpfile
mv tmpfile java/net/sourceforge/swimp/test/suites/AllCoinBinaryDistTests.java

Add this line in java/net/sourceforge/swimp/test/suites/AllCoinBinaryDistTests.java

for solvers you compiled that were not there originally.

suite.addTestSuite(TestOsiGlpkSolverInterface.class);

Repeat this for each solver you don't have, or change assertion to false

sed 's/^."clp".$//g' java/net/sourceforge/swimp/test/probe/TestCoinProbe.java > tmpfile
mv tmpfile java/net/sourceforge/swimp/test/probe/TestCoinProbe.java

Add this line in java/net/sourceforge/swimp/test/probe/TestCoinProbe.java

assertTrue(CoinProbe.probeOsiSolver("glpk"));

========================

Step 3: Compile and test

========================

. swimpEnv.sh -c

./prepareGnuBuild.sh

./configure --disable-binbundle

make forcegen

make

make check

[combean]

Thanks for the detailed explanation. To let everybody benefit from that easily I plan to finally update my machine to 64 bit as soon as OpenSuse 13.1 is out in a month from now and then get swIMP running against the latest binary distribution from Osi. When doing so, I will go to your report and apply your changes (or equivalent actions).