Re: [scipion-users] [EXT] Re: Problem compiling xmipp in Scipion3 on Centos 7 cluster

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Dear Wolfgang,

cc 8.6 certainly requires CUDA11
<https://en.wikipedia.org/wiki/CUDA#GPUs_supported>, since this is for
RTX30xx cards. There's no cc8.6 suggestions by default (see here
<https://github.com/I2PC/xmipp/wiki/Xmipp-configuration-(version-20.07)#cuda-configuration>),
so I believe you have added this argument.

I have xmipp compiled on Scientific Linux 7.9 with gcc10.2.0, cuda 11.4 (
cc7.5 max, since I have Tesla T4 cards) and hdf5 1.12.1 installed via conda.

Best regards,
Grigory

--------------------------------------------------------------------------------
Grigory Sharov, Ph.D.

MRC Laboratory of Molecular Biology,
Francis Crick Avenue,
Cambridge Biomedical Campus,
Cambridge CB2 0QH, UK.
tel. +44 (0) 1223 267228 <+44%201223%20267228>
e-mail: gs...@mr...

On Wed, Mar 16, 2022 at 12:24 PM Lugmayr, Wolfgang <w.l...@uk...> wrote:

> Dear Grigory,
>
> I have the same problem and tried a complete fresh install of scipion3.
> In your
> https://scipion-em.github.io/docs/docs/scipion-modes/how-to-install.html
> you say CUDA 10 but it seems you need CUDA 11
>
> Checking CUDA configuration ...
>  /beegfs/cssb/software/spack/opt/spack/linux-centos7-x86_64/gcc-10.3.0/cuda-10.2.89-4fkqx2xzcctfvrzwzuqiylwdrxhdywro/bin/nvcc
> -c -w --x cu -D_FORCE_INLINES -Xcompiler -fPIC -ccbin
> /beegfs/cssb/software/spack/opt/spack/linux-centos7-x86_64/gcc-10.3.0/gcc-8.5.0-vo5ghaznhl5vu4k77vimk2bqs5e53kil/bin/g++
> -std=c++14 --expt-extended-lambda -gencode=arch=compute_60,code=compute_60
> -gencode=arch=compute_61,code=compute_61
> -gencode=arch=compute_75,code=compute_75
> -gencode=arch=compute_86,code=compute_86 -I../
> -I/beegfs/cssb/software/em/scipion/3.0/miniconda/envs/scipion3/include
> xmipp_cuda_test.cpp -o xmipp_cuda_test.o
> nvcc fatal   : Unsupported gpu architecture 'compute_86'
>  Check the NVCC, NVCC_CXXFLAGS and INCDIRFLAGS
>  Cannot compile with NVCC, continuing without CUDA
>
> And then later i have problems with the hdf5 libraries. The ondes from the
> distribution do not seem to have the cxx bindings and hdf5 1.12 is too new.
> So I assume you need hdf5 1.8?
>
> cheers,
> Wolfgang
>
>
> ------------------------------
> *From: *"Grigory Sharov" <sha...@gm...>
> *To: *"Mailing list for Scipion users" <
> sci...@li...>
> *Sent: *Wednesday, 16 March, 2022 11:09:42
> *Subject: *Re: [scipion-users] [EXT] Re: Problem compiling xmipp in
> Scipion3 on Centos 7 cluster
>
> Also, what do you have in:
>
>> ls /usr/lib64/libhdf*
>> ls /usr/include/hdf*
>> ls /share/apps/anaconda3/envs/scipion3/lib/libhdf*
>
> ls /share/apps/anaconda3/envs/scipion3/include/*
>
>
>
> Best regards,
> Grigory
>
>
> --------------------------------------------------------------------------------
> Grigory Sharov, Ph.D.
>
> MRC Laboratory of Molecular Biology,
> Francis Crick Avenue,
> Cambridge Biomedical Campus,
> Cambridge CB2 0QH, UK.
> tel. +44 (0) 1223 267228
> e-mail: gs...@mr...
>
>
> On Wed, Mar 16, 2022 at 9:50 AM Grigory Sharov <sha...@gm...>
> wrote:
>
>> Hi Thu,
>> xmipp script needs a second parameter, but anyway I suggest going step by
>> step:
>>
>> 1) conda activate scipion3
>> 2) cd /share/apps/scipion3/software/em/xmippSrc-v3.22.01.0/
>> 3) ./xmipp cleanBin
>> 4) edit xmipp.conf and set Verified=False
>> 5) ./xmipp check_config -> check what it returns
>> 6) ./xmipp compile | tee -a compile.log -> attach here compile.log
>>
>> Best regards,
>> Grigory
>>
>>
>> --------------------------------------------------------------------------------
>> Grigory Sharov, Ph.D.
>>
>> MRC Laboratory of Molecular Biology,
>> Francis Crick Avenue,
>> Cambridge Biomedical Campus,
>> Cambridge CB2 0QH, UK.
>> tel. +44 (0) 1223 267228
>> e-mail: gs...@mr...
>>
>>
>> On Tue, Mar 15, 2022 at 11:16 AM Thu Nguyen <t.n...@un...>
>> wrote:
>>
>>> Running:
>>>
>>>  /share/apps/scipion3/scipion3 run
>>> /share/apps/scipion3/software/em/xmippSrc-v3.22.01.0/xmipp
>>> Scipion v3.0.10 - Eugenius
>>>
>>>  >>> WARNING: Image library not found!
>>>   > Please install Xmipp to get full functionality.
>>> (Configuration->Plugins->scipion-em-xmipp -> expand, in Scipion plugin
>>> manager window)
>>>
>>>
>>> ** Running command:
>>>  /share/apps/scipion3/software/em/xmippSrc-v3.22.01.0/xmipp
>>>  'xmipp.conf' detected.
>>> Checking configuration ------------------------------
>>>  'xmipp.conf' is already checked. Set VERIFIED=False to re-checked
>>> Getting Dependencies -------------------------------------
>>>  Checkouting cuFFTAdvisor:master ...
>>>  Repository contains uncommitted changes.
>>>  Use 'compileAndInstall' mode to keep developing.
>>>  Cannot checkout branch 'master'. Remaining on the branch 'None'.
>>>  Cannot get dependencies
>>> Traceback (most recent call last):
>>>   File "/share/apps/anaconda3/envs/scipion3/bin/emprogram", line 8, in
>>> <module>
>>>     sys.exit(main())
>>>   File
>>> "/share/apps/anaconda3/envs/scipion3/lib/python3.8/site-packages/pwem/cmd/program.py",
>>> line 43, in main
>>>     runProgram(program, params)
>>>   File
>>> "/share/apps/anaconda3/envs/scipion3/lib/python3.8/site-packages/pwem/utils.py",
>>> line 73, in runProgram
>>>     pwutils.runJob(None, program, params, env=env)
>>>   File
>>> "/share/apps/anaconda3/envs/scipion3/lib/python3.8/site-packages/pyworkflow/utils/process.py",
>>> line 52, in runJob
>>>     return runCommand(command, env, cwd)
>>>   File
>>> "/share/apps/anaconda3/envs/scipion3/lib/python3.8/site-packages/pyworkflow/utils/process.py",
>>> line 67, in runCommand
>>>     check_call(command, shell=True, stdout=sys.stdout, stderr=sys.stderr,
>>>   File
>>> "/share/apps/anaconda3/envs/scipion3/lib/python3.8/subprocess.py", line
>>> 364, in check_call
>>>     raise CalledProcessError(retcode, cmd)
>>> subprocess.CalledProcessError: Command '
>>> /share/apps/scipion3/software/em/xmippSrc-v3.22.01.0/xmipp ' returned
>>> non-zero exit status 1.
>>>
>>> ---------------------------------
>>>
>>> And here is my xmipp.conf:
>>>
>>> [BUILD]
>>> BUILD_TESTS=False
>>> CC=gcc
>>> CCFLAGS=-std=c99
>>> CONFIG_VERSION=release-22.01 (97a0c0dc)
>>> CUDA=True
>>> CXX=/share/apps/gcc/8.5.0/bin/g++
>>> CXXFLAGS=-mtune=native -march=native -flto -std=c++17 -O3
>>> CXX_CUDA=/share/apps/gcc/8.5.0/bin
>>> DEBUG=False
>>> INCDIRFLAGS=-I../ -I/share/apps/anaconda3/envs/scipion3/include
>>> -I/usr/include
>>> JAR=/usr/java/latest/bin/jar
>>> JAVAC=/usr/java/latest/bin/javac
>>> JAVA_BINDIR=/usr/java/latest/bin
>>> JAVA_HOME=/usr/java/latest
>>> JNI_CPPPATH=/usr/java/latest/include:/usr/java/latest/include/linux
>>> LIBDIRFLAGS=-L/share/apps/anaconda3/envs/scipion3/lib -L/usr/lib64
>>> LINKERFORPROGRAMS=/share/apps/gcc/8.5.0/bin/g++
>>> LINKFLAGS=-flto
>>> MATLAB=False
>>> MATLAB_DIR=
>>> MPI_CC=mpicc
>>> MPI_CXX=mpicxx
>>> MPI_CXXFLAGS=
>>> MPI_LINKERFORPROGRAMS=mpicxx
>>> MPI_LINKFLAGS=
>>> MPI_RUN=mpirun
>>> NVCC=/opt/cuda_10.2.89/bin/nvcc
>>> NVCC_CXXFLAGS=--x cu -D_FORCE_INLINES -Xcompiler -fPIC -ccbin
>>> /share/apps/gcc/8.5.0/bin -std=c++11 --expt-extended-lambda
>>> -gencode=arch=compute_35,code=compute_35
>>> -gencode=arch=compute_50,code=compute_50
>>> -gencode=arch=compute_60,code=compute_60
>>> -gencode=arch=compute_61,code=compute_61
>>> NVCC_LINKFLAGS=-L/opt/cuda_10.2.89/targets/x86_64-linux/lib
>>> -L/opt/cuda_10.2.89/targets/x86_64-linux/lib/stubs
>>> OPENCV=False
>>> OPENCV3=False
>>> OPENCVSUPPORTSCUDA=False
>>> PYTHONINCFLAGS=-I/share/apps/anaconda3/envs/scipion3/include/python3.8
>>> -I/share/apps/anaconda3/envs/scipion3/lib/python3.8/site-packages/numpy/core/include
>>> PYTHON_LIB=python3.8
>>> STARPU=False
>>> STARPU_HOME=
>>> STARPU_INCLUDE=/include/starpu/1.3
>>> STARPU_LIB=/lib
>>> STARPU_LIBRARY=libstarpu-1.3
>>> USE_DL=False
>>> VERIFIED=True
>>>
>>> Thank you.
>>>
>>>
>>>
>>> ___________________________________________________
>>>
>>> Thu D. Nguyen
>>>
>>> Bio21 Institute
>>>
>>> Thu University of Melbourne
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* Alberto Garcia Mena <alb...@cn...>
>>> *Sent:* Tuesday, 15 March 2022 9:36 PM
>>> *To:* Thu Nguyen <t.n...@un...>; Mailing list for Scipion
>>> users <sci...@li...>
>>> *Subject:* Re: [scipion-users] [EXT] Re: Problem compiling xmipp in
>>> Scipion3 on Centos 7 cluster
>>>
>>> * External email: Please exercise caution *
>>> ------------------------------
>>>
>>> Hi again,
>>>
>>>
>>> Could you share the xmipp.config file? You can try to install xmipp with
>>> scipion scipion3 by running: *./scipion3 run pathToXmipp/xmipp *to run
>>> it with the scipion enviroment.
>>>
>>>
>>>
>>> On 15/3/22 11:15, Thu Nguyen wrote:
>>>
>>> I am sorry to report it made no differences.
>>>
>>> It's not gcc, as I switched to gcc 8.5.0, the error is still there.
>>> Here is where it came out in my latest attempt:
>>>
>>> g++ -o bindings/python/python_metadata.os -c -mtune=native -march=native
>>> -flto -std=c++17 -O3 -I../ -I/share/apps/anaconda3/envs/scipion3/include
>>> -I/usr/include -fPIC
>>> -I/share/apps/anaconda3/envs/scipion3/include/python3.8
>>> -I/share/apps/anaconda3/envs/scipion3/lib/python3.8/site-packages/numpy/core/include
>>> -Iexternal -Ilibraries
>>> -I/share/apps/scipion3/software/em/xmippSrc-v3.22.01.0/src/xmippCore
>>> bindings/python/python_metadata.cpp
>>>
>>>
>>> g++ -o bin/xmipp_transform_mirror -flto
>>> applications/programs/transform_mirror/transform_mirror_main.o
>>> -L/share/apps/scipion3/software/em/xmippSrc-v3.22.01.0/src/xmippCore/lib
>>> -Llib -Llib -L/share/apps/anaconda3/envs/scipion3/lib -L/usr/lib64
>>> -L/share/apps/gcc/8.5.0/lib64 -L/share/apps/gcc/8.5.0/lib
>>> -L/opt/cuda_10.2.89/lib64 -L/opt/gridengine/lib/lx-amd64 -L/opt/openmpi/lib
>>> -lXmipp -lhdf5 -lhdf5_cpp -lfftw3 -lfftw3_threads -ljpeg -ltiff -lfftw3f
>>> -lfftw3f_threads -lpython3.8 -lpthread -lsvm -lXmippCore
>>> lib/libXmipp.so: undefined reference to `non-virtual thunk to
>>> H5::H5File::throwException(std::__cxx11::basic_string<char,
>>> std::char_traits<char>, std::allocator<char> > const&,
>>> std::__cxx11::basic_string<char, std::char_traits<char>,
>>> std::allocator<char> > const&) const'
>>> collect2: error: ld returned 1 exit status
>>> scons: *** [bin/xmipp_pdb_analysis] Error 1
>>>
>>>
>>>
>>> ___________________________________________________
>>>
>>> Thu D. Nguyen
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* Alberto Garcia Mena <alb...@cn...>
>>> <alb...@cn...>
>>> *Sent:* Tuesday, 15 March 2022 5:50 PM
>>> *To:* Thu Nguyen <t.n...@un...> <t.n...@un...>;
>>> Mailing list for Scipion users <sci...@li...>
>>> <sci...@li...>
>>> *Subject:* Re: [scipion-users] [EXT] Re: Problem compiling xmipp in
>>> Scipion3 on Centos 7 cluster
>>>
>>> * External email: Please exercise caution *
>>> ------------------------------
>>>
>>> Hi again Thu,
>>>
>>> Xmipp looks for libhdf5* and hdf5.h files. For Centos7 libhdf5* are in*
>>> /usr/lib64/ *and hdf5.h is in* /usr/include/.* Please verify that you
>>> have the files in those paths and add them to INCDIRFLAGS and LIBDIRFLAGS
>>> in xmipp.config as
>>>
>>> LIBDIRFLAGS= ... -L/usr/lib64
>>> INCDIRFLAGS= ... -I/usr/include
>>>
>>> After that run *./xmipp compileAndInstall*
>>>
>>> Thank you for your detailed report. (It seems that for Centos7 Xmipp is
>>> not able to find hdf5 by default)
>>>
>>>
>>> On 14/3/22 23:59, Thu Nguyen wrote:
>>>
>>> Thanks to all that responded.
>>>
>>>
>>>
>>> Unfortunately I am not faring any better.  I have tried different
>>> versions of hdf5-devel (1.8.12, 1.10.0,1.10.4,1.10.6), correctly edit
>>> xmipp.conf for INCDIRFLAGS and LIBDIRFLAGS each time (as shown in my latest
>>> error post), and even tried the 20.07 release as some suggested.
>>>
>>>
>>>
>>> The error seemed to turn up at different stages each time.
>>>
>>>
>>>
>>> One thing I just notice, in all my hdf5-devel installs, I never seem to
>>> get the “serial” folder as shown in some discussions.  What does this
>>> folder have?
>>>
>>>
>>>
>>>
>>>
>>>
>>> *----------------------------------------------------------------------------------------------------------------*
>>>
>>> *Thu D. Nguyen*
>>>
>>>
>>>
>>> Bio21 Institute
>>>
>>> The University of Melbourne, Victoria 3010 Australia
>>>
>>>
>>>
>>> *From:* Alberto Garcia Mena <alb...@cn...>
>>> <alb...@cn...>
>>> *Sent:* Monday, 14 March 2022 6:49 PM
>>> *To:* Thu Nguyen <t.n...@un...> <t.n...@un...>
>>> *Subject:* Re: [scipion-users] [EXT] Re: Problem compiling xmipp in
>>> Scipion3 on Centos 7 cluster
>>>
>>>
>>>
>>> *External email: *Please exercise caution
>>>
>>>
>>> ------------------------------
>>>
>>> Hello thu Nguyen,
>>>
>>> To avoid issues related to hdf5 we recommend uninstalling all hdf5
>>> version and install only the hdf5-devel version using* sudo yum install
>>> hdf5-devel.* We have a installation guide for Centos
>>> <https://github.com/I2PC/xmipp/wiki/Installing-Xmipp-on-CentOS-7-9.2009>
>>> .
>>>
>>> After the installation of hdf5-devel, run ./xmipp config which should
>>> find the hdf5 path (usr/include/hdf5/serial) and should add this path on
>>> the INDIRFLAGS in the xmipp.config file, after checking this you could run
>>> ./xmipp and install it.
>>>
>>>
>>>
>>> --
>>> - Alberto GM -
>>>
>>> --
>>> - Alberto GM -
>>> _______________________________________________
>>> scipion-users mailing list
>>> sci...@li...
>>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>>>
>>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
>

Re: [scipion-users] [EXT] Re: Problem compiling xmipp in Scipion3 on Centos 7 cluster

Image processing framework to integrate EM software packages.

Re: [scipion-users] [EXT] Re: Problem compiling xmipp in Scipion3 on Centos 7 cluster