Re: [moose-generic] MOOSE_installation_problem

[Subhasis R. <ray...@gm...>]

Hi,
You seem to have a non-standard location for Python library. You should
update this:

PYTHON_LDFLAGS := -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker
-export-dynamic* -L{location of libpython2.7.so}*

Best,
 Subha


On Sat, Mar 9, 2013 at 5:36 AM, Vadim A <ast...@nc...> wrote:

> I add those paths in Makefile
>
> PYTHON_INCLUDES := -I/home/diag/opt/python2.7/include/python2.7
> -I/home/diag/opt/python2.7/include/python2.7
> -I/home/diag/opt/python2.7/lib/python2.7/site-packages/numpy/core/include/
>
> PYTHON_LDFLAGS := -lpthread -ldl -lutil -lm *-lpython2.7* -Xlinker
> -export-dynamic
>
>
> Python.h and all numpy header are there
>
> and build went through further
>
> but now complain that "/usr/bin/ld: cannot find -lpython2.7"
>
>
> Is there another place where we should set some flags ?
>
>
> g++ -shared -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic
>  -O3 -pthread -fpermissive -fno-strict-aliasing -fPIC -Wall -Wno-long-long
> -pedantic -DNDEBUG -DUSE_GENESIS_PARSER -I
> /home/diag/opt/openmpi/1.4.3/gnu/include  -DSVN_REVISION=\"4401M\" -DLINUX
> -DUSE_GSL -I /home/diag/opt/gsl/1.15/gnu/include/    -DPYMOOSE
> -I/home/diag/opt/python2.7/include/python2.7
> -I/home/diag/opt/python2.7/include/python2.7
> -I/home/diag/opt/python2.7/lib/python2.7/site-packages/numpy/core/include/
> -o python/moose/_moose.so msg/_msg.o shell/_shell.o
> biophysics/_biophysics.o hsolve/_hsolve.o randnum/_randnum.o
> scheduling/_scheduling.o builtins/_builtins.o device/_device.o
> kinetics/_kinetics.o ksolve/_ksolve.o regressionTests/_rt.o
> utility/_utility.o geom/_geom.o mesh/_mesh.o manager/_manager.o
> signeur/_signeur.o pymoose/_pymoose.o basecode/_basecode_pymoose.o
> -L/usr/lib -L/usr/local/lib -lpthread -L/lib64 -L/usr/lib64 -L/usr/lib
> -L/home/diag/opt/gsl/1.15/gnu/lib/ -lgsl -lgslcblas -lm
>
> /usr/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching
> for -lpthread
> /usr/bin/ld: skipping incompatible /usr/lib/libpthread.a when searching
> for -lpthread
> /usr/bin/ld: skipping incompatible /usr/lib/libdl.so when searching for
> -ldl
> /usr/bin/ld: skipping incompatible /usr/lib/libdl.a when searching for -ldl
> /usr/bin/ld: skipping incompatible /usr/lib/libutil.so when searching for
> -lutil
> /usr/bin/ld: skipping incompatible /usr/lib/libutil.a when searching for
> -lutil
> /usr/bin/ld: cannot find -lpython2.7
>
>
>
> Thanks a lot in advance for any advice.
>
> V.
>
> On Mar 6, 2013, at 9:56 PM, Subhasis Ray wrote:
>
> Dear Vadim,
>   You can change the following with the corresponding values for your
> system:
>
> ~ Line # 339 and 340 ...
>
> PYTHON_INCLUDES := $(shell python-config --includes)
> PYTHON_LDFLAGS := $(shell python-config --ldflags)
>
> And as I suspected, your system does not seems have python-config
> installed (standard locations are /usr/bin or /usr/local/bin) because that
> is the command we are using in the Makefile to get the flags. It is
> supposed to come with Python development library, which is also a
> prerequisite for building moose. So please make sure you have that
> installed (does locate Python.h show something like:
> /usr/include/python2.7/Python.h? ).
>
> Best,
>  Subha
>
>
> On Thu, Mar 7, 2013 at 6:16 AM, Vadim A <ast...@nc...> wrote:
>
>> Also this example is not clear where you've mentioned
>>
>>
>> "...and add the following to LIBS (output of python-config --ldflags)
>> updated according to local system paths:
>>
>> -L/usr/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7
>> -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
>>
>> "
>>
>>
>> I do not see  "python2.7/config" in our system
>>
>>
>> Vadim.
>>
>> On Mar 6, 2013, at 4:31 PM, Vadim A wrote:
>>
>> Hi Subha.
>>
>> Just wan to clarify one think. We decided to concentrate on simples
>> installation for now and do the following steps:
>>
>> make clean
>> make BUILD=release USE_HDF5=0
>>
>>
>> And it still give me the error
>>
>> g++ -O3 -pthread -fpermissive -fno-strict-aliasing -fPIC -Wall
>> -Wno-long-long -pedantic -DNDEBUG -DUSE_GENESIS_PARSER -I
>> /home/diag/opt/openmpi/1.4.3/gnu/include  -DSVN_REVISION=\"4401M\" -DLINUX
>> -DUSE_GSL -I /home/diag/opt/gsl/1.15/gnu/include/    -DPYMOOSE
>> -I/home/diag/opt/python2.7/include/python2.7
>> -I/home/diag/opt/python2.7/include/python2.7 moosemodule.cpp -c
>>
>> moosemodule.cpp:80:31: error: numpy/arrayobject.h: No such file or
>> directory
>>
>>
>>
>> But we loaded that module and it is located in our system here :
>> /home/diag/opt/python2.7/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h
>>
>>
>> You've suggested to add it to CXXFLAGS flags. But where in the Makefile
>> should we do that ?
>>
>>
>> Best regards,
>>
>> Vadim.
>>
>>
>>
>> On Mar 5, 2013, at 7:43 PM, Subhasis Ray wrote:
>>
>> Dear Vadim,
>>  Did you run "make clean" before rebuilding each time? If you specify
>> USE_HDF5=0, then the entire content of HDF5 related files get excluded from
>> the compilation process. But from the build log for the first case it seems
>> the object files were compiled with HDF5 support but the linking is being
>> done without.
>>  In the second case, is it possible that your version of HDF5 library is
>> linked to a different MPI library than what you set in MOOSE Makefile?
>> Otherwise, it is also possible that HDF5 lib is dynamically linked to MPI
>> and when you are building MOOSE without MPI support the MPI-lib location is
>> not passed to compiler. In that case you may want to use mpicxx as the
>> compiler at line 250:
>>
>> # Here we automagically change compilers to deal with MPI.
>> ifdef USE_MPI
>>     CXX = mpicxx
>> #    CXX = /usr/local/mvapich2/bin/mpicxx
>> #    PARALLEL_DIR = parallel
>> #    PARALLEL_LIB = parallel/parallel.o
>> else
>>     CXX = g++ # <-- replace with mpicxx
>> #    CXX = CC    # Choose between Solaris CC and g++ on a Solaris machine
>> endif
>>
>>
>>  For the third case, we use the command:
>>
>> python-config --includes
>>
>> in the Makefile in order to get the location of header files for numpy
>> and python. Looks like that does not work on your system. So you need to
>> explicitly add
>> -I/usr/include/python2.7 -I/usr/include/python2.7
>>
>> and
>>
>> -I{location of numpy/arrayobject.h}
>>
>> to CXXFLAGS
>>
>> and add the following to LIBS (output of python-config --ldflags) updated
>> according to local system paths:
>>
>> -L/usr/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7
>> -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
>>
>>
>> hth,
>>   Subha
>>
>>
>> On Wed, Mar 6, 2013 at 1:43 AM, Vadim A <ast...@nc...> wrote:
>>
>>> Dear Subha.
>>>
>>> I was trying three different ways to install MOOSE: without HDF5, with
>>> HDF5 and with MPI
>>>
>>> Unfortunatelly, I did fail in all. Here the list of problems which we
>>> have. We would very appreciate you comments or suggestions what we might
>>> missing here.
>>>
>>>
>>>
>>> (1) First I did
>>>
>>> make BUILD=release USE_HDF5=0
>>>
>>>
>>>
>>> g++ -O3 -pthread -fpermissive -fno-strict-aliasing -fPIC -Wall
>>> -Wno-long-long -pedantic -DNDEBUG -DUSE_GENESIS_PARSER -I
>>> /home/diag/opt/openmpi/1.4.3/gnu/include  -DSVN_REVISION=\"4401M\" -DLINUX
>>> -DUSE_GSL -I /home/diag/opt/gsl/1.15/gnu/include/    basecode/_basecode.o
>>> msg/_msg.o shell/_shell.o biophysics/_biophysics.o hsolve/_hsolve.o
>>> randnum/_randnum.o scheduling/_scheduling.o builtins/_builtins.o
>>> device/_device.o kinetics/_kinetics.o ksolve/_ksolve.o
>>> regressionTests/_rt.o utility/_utility.o geom/_geom.o mesh/_mesh.o
>>> manager/_manager.o signeur/_signeur.o     -L/usr/lib -L/usr/local/lib
>>> -lpthread -L/lib64 -L/usr/lib64 -L/usr/lib
>>> -L/home/diag/opt/gsl/1.15/gnu/lib/ -lgsl -lgslcblas -lm -o moose
>>>
>>>
>>>
>>> And getting a lot of error like this:
>>>
>>>
>>> builtins/_builtins.o: In function `HDF5WriterBase::setMode(unsigned
>>> int)':
>>> (.text+0x11f40): undefined reference to `H5check_version'
>>> ……………….
>>>
>>> builtins/_builtins.o:(.data.rel.ro._ZTVN3MPI8DatatypeE[vtable for
>>> MPI::Datatype]+0x78): undefined reference to `MPI::Datatype::Free()'
>>> collect2: ld returned 1 exit status
>>>
>>>
>>>
>>>
>>>
>>>
>>> (2) Then
>>>
>>> We have installed  HDF5 1.8.9 and I was trying to do build release:
>>>
>>>
>>> make BUILD=release
>>>
>>>
>>> It seems that all libs get compiled and it can find HDF5 1.8.9
>>>
>>>
>>> All Libs compiled
>>> g++ -O3 -pthread -fpermissive -fno-strict-aliasing -fPIC -Wall
>>> -Wno-long-long -pedantic -DNDEBUG -DUSE_GENESIS_PARSER -I
>>> /home/diag/opt/openmpi/1.4.3/gnu/include  -DSVN_REVISION=\"4401M\" -DLINUX
>>> -DUSE_GSL -I /home/diag/opt/gsl/1.15/gnu/include/    -DUSE_HDF5
>>>  -DH5_NO_DEPRECATED_SYMBOLS -*I
>>>  /home/diag/opt/hdf5/1.8.9/gnu/openmpi/include* basecode/_basecode.o
>>> msg/_msg.o shell/_shell.o biophysics/_biophysics.o hsolve/_hsolve.o
>>> randnum/_randnum.o scheduling/_scheduling.o builtins/_builtins.o
>>> device/_device.o kinetics/_kinetics.o ksolve/_ksolve.o
>>> regressionTests/_rt.o utility/_utility.o geom/_geom.o mesh/_mesh.o
>>> manager/_manager.o signeur/_signeur.o     -L/usr/lib -L/usr/local/lib
>>> -lpthread -L/lib64 -L/usr/lib64 -L/usr/lib
>>> -L/home/diag/opt/gsl/1.15/gnu/lib/ -lgsl -lgslcblas -lm -lhdf5
>>> -L/home/diag/opt/hdf5/1.8.9/gnu/openmpi/lib    -o moose
>>>
>>>
>>>
>>> but it complain about a lot of "undefined references like this:
>>>
>>>
>>> builtins/_builtins.o:(.data.rel.ro._ZTVN3MPI8DatatypeE[vtable for
>>> MPI::Datatype]+0x78): undefined reference to `MPI::Datatype::Free()'
>>> collect2: ld returned 1 exit status
>>> make: *** [moose] Error 1
>>>
>>> Looks like we still missing something. Should we have exactly same HDF5
>>> library as you are using -> *libhdf5-serial   ?*
>>>
>>>
>>>
>>>
>>> (3)
>>>
>>> Also I did try to compile with MPI:
>>>
>>>
>>> make BUILD=release USE_MPI=1
>>>
>>> All Libs compiled
>>> mpicxx -O3 -pthread -fpermissive -fno-strict-aliasing -fPIC -Wall
>>> -Wno-long-long -pedantic -DNDEBUG -DUSE_GENESIS_PARSER -I
>>> /home/diag/opt/openmpi/1.4.3/gnu/include  -DSVN_REVISION=\"4401M\" -DLINUX
>>> -DUSE_MPI -DMPICH_IGNORE_CXX_SEEK -DUSE_GSL -I
>>> /home/diag/opt/gsl/1.15/gnu/include/    -DUSE_HDF5
>>>  -DH5_NO_DEPRECATED_SYMBOLS -I
>>>  /home/diag/opt/hdf5/1.8.9/gnu/openmpi/include basecode/_basecode.o
>>> msg/_msg.o shell/_shell.o biophysics/_biophysics.o hsolve/_hsolve.o
>>> randnum/_randnum.o scheduling/_scheduling.o builtins/_builtins.o
>>> device/_device.o kinetics/_kinetics.o ksolve/_ksolve.o
>>> regressionTests/_rt.o utility/_utility.o geom/_geom.o mesh/_mesh.o
>>> manager/_manager.o signeur/_signeur.o     -L/usr/lib -L/usr/local/lib
>>> -lpthread -L/lib64 -L/usr/lib64 -L/usr/lib
>>> -L/home/diag/opt/gsl/1.15/gnu/lib/ -lgsl -lgslcblas -lm -lhdf5
>>> -L/home/diag/opt/hdf5/1.8.9/gnu/openmpi/lib    -o moose
>>>
>>>
>>>
>>> But getting error:
>>>
>>>
>>> moosemodule.cpp:80:31: error: numpy/arrayobject.h: No such file or
>>> directory
>>>
>>>
>>>
>>> But We have "numpy 1.6" loaded.  Should we set some other flag or should
>>> we have different version of "numpy" ?
>>>
>>>
>>>
>>> Thanks a lot in advance for any advice.
>>>
>>> Vadim.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mar 2, 2013, at 10:58 PM, Subhasis Ray wrote:
>>>
>>> Hi,
>>> In the serial version, HDF5 1.8.9 should work fine. It is used in
>>> HDF5DataWriter class to incrementally dump data into an HDF5 file. It was
>>> implemented to reduce memory usage for long running simulations generating
>>> gigabytes of data.
>>> Best,
>>>   Subha
>>>
>>>
>>> On Sun, Mar 3, 2013 at 1:22 AM, Majumdar, Amitava <maj...@sd...>wrote:
>>>
>>>>  If HDF5 version is the only issue, we can make HDF51.8.0 or higher
>>>> available (actually we have HDF5 1.8.9 available
>>>> on another parallel computer at SDSC).
>>>>
>>>> -Amitvai
>>>>  ------------------------------
>>>> *From:* Subhasis Ray [ray...@gm...]
>>>> *Sent:* Saturday, March 02, 2013 1:35 AM
>>>> *To:* Vadim A
>>>>
>>>> *Cc:* Niraj Dudani; Yoshimoto, Kenneth; Majumdar, Amitava;
>>>> moo...@li...
>>>> *Subject:* Re: [moose-generic] MOOSE_installation_problem
>>>>
>>>>    Dear Vadim,
>>>>    I would recommend that you avoid the MPI build at this point. The
>>>> errors again are due to a minor issue with HDF5 library that can vary from
>>>> package to package. These functions were introduced in HDF5 1.8.0. Either
>>>> you need the same or a newer version installed. Otherwise, you can skip
>>>> HDF5 support (which is somewhat experimental anyways) altogether by passing
>>>> USE_HDF5=0 to "make" command.
>>>>    Hope this helps,
>>>>      Subha
>>>>
>>>>
>>>> On Sat, Mar 2, 2013 at 5:35 AM, Vadim A <ast...@nc...>wrote:
>>>>
>>>>> Dear Subha.
>>>>>
>>>>>  I set CXXFLAG to include mpi.h in "ifeq ($(BUILD),release)"
>>>>>
>>>>>
>>>>>  like you've suggested  and it seems that it can find all libraries
>>>>> now but unfortunately
>>>>>
>>>>>
>>>>>  make BUILD=release USE_MPI=1
>>>>>
>>>>>
>>>>>  produce another error
>>>>>
>>>>>
>>>>>  mpicxx -O3 -pthread -fpermissive -fno-strict-aliasing -fPIC -Wall
>>>>> -Wno-long-long -pedantic -DNDEBUG -DUSE_GENESIS_PARSER -I
>>>>> /home/diag/opt/openmpi/1.4.3/gnu/include  -DSVN_REVISION=\"4401M\" -DLINUX
>>>>> -DUSE_MPI -DMPICH_IGNORE_CXX_SEEK -DUSE_GSL -I
>>>>> /home/diag/opt/gsl/1.15/gnu/include/    -DUSE_HDF5
>>>>>  -DH5_NO_DEPRECATED_SYMBOLS -I  /home/diag/opt/hdf5/1.6/gnu/include  -I.
>>>>> -I../basecode -I../msg HDF5DataWriter.cpp -c
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  ../basecode/Conv.h:1009: warning: converting to 'unsigned int' from
>>>>> 'const double'
>>>>> HDF5DataWriter.cpp: In member function 'hid_t
>>>>> HDF5DataWriter::get_dataset(std::string)':
>>>>> HDF5DataWriter.cpp:222: error: 'H5Gcreate2' was not declared in this
>>>>> scope
>>>>> HDF5DataWriter.cpp:225: error: 'H5Gopen2' was not declared in this
>>>>> scope
>>>>> HDF5DataWriter.cpp:258: error: 'H5Dopen2' was not declared in this
>>>>> scope
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  HDF5  and openmpi  libraries included. Do I still missing something ?
>>>>>
>>>>>  Thanks a lot in advance for any comments.
>>>>>
>>>>>  Vadim.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  On Feb 28, 2013, at 7:37 PM, Subhasis Ray wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Do not enable MPI unless you want to try the parallel version. To
>>>>> enable MPI, you have to pass the argument "USE_MPI=1" to make command.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>  # Here we automagically change compilers to deal with MPI.
>>>>>> ifdef USE_MPI
>>>>>>         CXX = mpicxx
>>>>>> #       CXX = /usr/local/mvapich2/bin/mpicxx
>>>>>>         PARALLEL_DIR = /home/diag/opt/openmpi/1.4.3/gnu/  #parallel
>>>>>>          PARALLEL_LIB = /home/diag/opt/openmpi/1.4.3/gnu/lib/
>>>>>> #parallel/parallel.o
>>>>>> else
>>>>>>         CXX = g++
>>>>>> #       CXX = CC        # Choose between Solaris CC and g++ on a
>>>>>> Solaris machine
>>>>>> endif
>>>>>>
>>>>>>
>>>>>>
>>>>>  In the above, the PARALLEL_DIR and PARALLEL_LIB are MOOSE source
>>>>> directory and object file for the parallel version. They are not to be
>>>>> modified.
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  We are using "openmpi" which does not have "parallel.o"
>>>>>>
>>>>>
>>>>>  As mentioned above, parallel.o will be generated by the build
>>>>> process. It looks like your version of hdf5 includes mpi.h and the compiler
>>>>> cannot find it. So you have to add -I{location of mpi.h} to the CXXFLAGS
>>>>> variable somewhere.Depending on your build type, you can append this to the
>>>>> CXXFLAG assignment under :
>>>>>
>>>>> ifeq ($(BUILD),debug)
>>>>>
>>>>>  and/or under
>>>>>
>>>>> ifeq ($(BUILD),release)
>>>>>
>>>>>
>>>>>  Best,
>>>>>    Subha
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>