Menu
▾
▴
Re: [scipion-users] Scipion 3.0 cluster config
|
From: Pablo C. <pc...@cn...> - 2021-01-19 11:37:23
|
Hi Yangyang,
I assume now that xmipp seems to be installed it fails differently?
If so, what is the error you find in the output log?
On 19/1/21 10:11, Yangyang Yi wrote:
> Thanks for your reply. Happy new year. I see. I tried to install
> scipion-em-xmipp again, it has reported a successful install, but when
> I submit job via slurm, I failed to run. Also, I tried to run a small
> job on headnode (only system since our headnode do not have GPU
> cards), and it works. Maybe something related to my config files,
> especially my host.conf? Here’s my config files:
> host.conf
> [localhost]
> PARALLEL_COMMAND = mpirun -np %_(JOB_NODES)d %_(COMMAND)s
> NAME = SCIPION_SLURM
> MANDATORY = False
> SUBMIT_COMMAND = sbatch %_(JOB_SCRIPT)s
> CANCEL_COMMAND = scancel %_(JOB_ID)s
> CHECK_COMMAND = squeue -h -j %_(JOB_ID)s
> SUBMIT_TEMPLATE = #!/bin/bash
> ### Inherit all current environment variables
> #SBATCH --export=ALL
> ### Job name
> #SBATCH -J %_(JOB_NAME)s
> ### Queue name
> #SBATCH -p %_(JOB_QUEUE)s
> ### Standard output and standard error messages
> #SBATCH -o %_(JOB_LOGS)s.out
> #SBATCH -e %_(JOB_LOGS)s.err
> ### Specify the number of nodes and thread (ppn) for your job.
> #SBATCH --ntasks=%_(JOB_NODES)d
> #SBATCH --cpus-per-task=%_(JOB_THREADS)d
> #SBATCH --mem=%_(JOB_MEMORY)s
> #SBATCH --gres=gpu:%_(GPU_COUNT)s
> ### Tell PBS the anticipated run-time for your job, where
> walltime=HH:MM:SS
> #SBATCH --time=%_(JOB_TIME)s:00:00
> # Use as working dir the path where qsub was launched
> WORKDIR=$SLURM_SUBMIT_DIR
> #################################
> ### Set environment variable to know running mode is non interactive
> export XMIPP_IN_QUEUE=1
> ### Switch to the working directory;
> cd $WORKDIR
> # Make a copy of PBS_NODEFILE
> cp $SLURM_JOB_NODELIST > %_(JOB_NODEFILE)s
> # Calculate the number of processors allocated to this run.
> NPROCS=`wc -l < $SLURM_JOB_NODELIST`
> # Calculate the number of nodes allocated.
> ###NNODES=`uniq $SLURM_JOB_NODELIST | wc -l`
> ### Display the job context
> echo Running on host `hostname`
> echo Time is `date`
> echo Working directory is `pwd`
> ###echo Using ${NPROCS} processors across ${NNODES} nodes
> echo NODE LIST - config:
> echo $SLURM_JOB_NODELIST
> echo CUDA_VISIBLE_DEVICES: $CUDA_VISIBLE_DEVICES
> module load cuda-10.1
> module load impi-2019.4
> module load relion-3.1.0
> module load java-1.8.0
> #################################
> # echo '%_(JOB_COMMAND)s' >> /tmp/slurm-jobs.log
> %_(JOB_COMMAND)s
> ###find "$SLURM_SUBMIT_DIR" -type f -user $USER -perm 644 -exec
> chmod 664 {} +
> ; #Next variable is used to provide a regex to check if a job is
> finished on a queue system
> QUEUES = {"gpu": [["JOB_MEMORY", "8192", "Memory (MB)", "Select amount
> of memory (in megabytes) for this job"],
> ["JOB_TIME", "48", "Time (hours)", "Select the time expected
> (in hours) for this job"],
> ["GPU_COUNT", "8", "Number of GPUs", "Select the number of
> GPUs if protocol has been set up to use them"],
> ["QUEUE_FOR_JOBS", "N", "Use queue for jobs", "Send individual
> jobs to queue"]]}
> scipion.conf:
> [PYWORKFLOW]
> CONDA_ACTIVATION_CMD = eval "$(/share/apps/software/anconda/bin/conda
> shell.bash hook)"
> SCIPION_DOMAIN = pwem
> SCIPION_FONT_NAME = Helvetica
> SCIPION_LOG = ~/ScipionUserData/logs/scipion.log
> SCIPION_LOGO = scipion_logo.gif
> SCIPION_LOGS = ~/ScipionUserData/logs
> SCIPION_NOTES_FILE = notes.txt
> SCIPION_NOTIFY = True
> SCIPION_PLUGIN_REPO_URL = http://scipion.i2pc.es/getplugins/
> SCIPION_SOFTWARE = ${SCIPION_HOME}/software
> SCIPION_SUPPORT_EMAIL = sc...@cn... <mailto:sc...@cn...>
> SCIPION_TESTS = ${SCIPION_HOME}/data/tests
> SCIPION_TESTS_CMD = scipion3 tests
> SCIPION_TESTS_OUTPUT = ~/ScipionUserData/Tests
> SCIPION_TMP = ~/ScipionUserData/tmp
> SCIPION_URL = http://scipion.cnb.csic.es/downloads/scipion
> SCIPION_URL_SOFTWARE =
> http://scipion.cnb.csic.es/downloads/scipion/software
> SCIPION_URL_TESTDATA =
> http://scipion.cnb.csic.es/downloads/scipion/data/tests
> SCIPION_USER_DATA = ~/ScipionUserData
> WIZARD_MASK_COLOR = [0.125, 0.909, 0.972]
> SCIPION_NOTES_ARGS =
> SCIPION_NOTES_PROGRAM =
>
> [PLUGINS]
> EM_ROOT = software/em
> MAXIT_HOME = %(EM_ROOT)s/maxit-10.1
> XMIPP_HOME = %(EM_ROOT)s/xmipp
> CUDA_BIN = /share/apps/software/cuda-10.1/bin
> CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> CHIMERA_HOME = %(EM_ROOT)s/chimerax-1.1
> GCTF_HOME = %(EM_ROOT)s/Gctf_v1.18
> GAUTOMATCH = Gautomatch_v0.56_sm30-75_cu10.1
> MOTIONCOR2_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> RELION_HOME = software/em/relion-3.1.0
> RESMAP = ResMap-1.95-cuda-Centos7x64
> GCTF_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> MOTIONCOR2_BIN = MotionCor2_1.4.0_Cuda101
> CRYOSPARC_HOME =
> RESMAP_GPU_LIB = ResMap_krnl-cuda-V8.0.61-sm60_gpu.so
> CRYO_PROJECTS_DIR = scipion_projects
> GAUTOMATCH_HOME = software/em/gautomatch-0.56
> RELION_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> RELION_CUDA_BIN = /share/apps/software/cuda-10.1/bin
> CTFFIND4_HOME = software/em/ctffind4-4.1.14
> RESMAP_HOME = software/em/resmap-1.95
> GAUTOMATCH_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> GCTF = Gctf_v1.18_sm30-75_cu10.1
> CISTEM_HOME = software/em/cistem-1.0.0-beta
> MOTIONCOR2_HOME = software/em/motioncor2-1.4.0
>
>> 2020年12月30日 下午4:50，Grigory Sharov <sha...@gm...
>> <mailto:sha...@gm...>> 写道：
>>
>> Hi Yangyang, the line
>> 00016: GHOST in place, read call ignored
>> means that you don't have xmipp installed which is still required for
>> all basic operations
>>
>> Grigory .
>>
>>
>> On Wed, Dec 30, 2020, 08:44 Yangyang Yi <yy...@si...
>> <mailto:yy...@si...>> wrote:
>>
>> Hi,
>> Happy Holiday!
>> Thanks for the help in last email about Scipion-2.0 cluster
>> config. I have solved the problem, since I used “Use queue for
>> jobs:Y” instead of "N", and some plugins only download partly. I
>> edited those parameter and reinstalled some of the plugin, now it
>> works.
>> However, our users now prefer scipion-3.0 and I found new
>> errors with similar settings. Now all the motion correction job
>> works and result files in extra directory (micrographs and
>> shifts) and generated, however, I failed to run CTF after motion
>> correction. I have imported some motion correction micrographs
>> but still failed to run CTF (both Gctf and CTFFIND). The error is
>> similar as below:
>>
>> CTFFIND:
>> 00002: Hostname: gpu04
>> 00003: PID: 138990
>> 00004: pyworkflow: 3.0.8
>> 00005: plugin: cistem
>> 00006: plugin v: 3.0.9
>> 00007: currentDir:
>> /share/home/biotest/stu01/ScipionUserData/projects/test
>> 00008: workingDir: Runs/001767_CistemProtCTFFind
>> 00009: runMode: Restart
>> 00010: MPI: 1
>> 00011: threads: 1
>> 00012: Starting at step: 1
>> 00013: Running steps
>> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30
>> 15:35:57.887435
>> 00015: Estimating CTF of micrograph: 1
>> 00016: GHOST in place, read call ignored!.
>> 00017: Traceback (most recent call last):
>> 00018: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 189, in run
>> 00019: self._run()
>> 00020: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 240, in _run
>> 00021: resultFiles = self._runFunc()
>> 00022: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 236, in _runFunc
>> 00023: return self._func(*self._args)
>> 00024: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_micrographs.py",
>> line 247, in estimateCtfStep
>> 00025: self._estimateCTF(mic, *args)
>> 00026: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py",
>> line 108, in _estimateCTF
>> 00027: self._doCtfEstimation(mic)
>> 00028: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py",
>> line 78, in _doCtfEstimation
>> 00029: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
>> 00030: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py",
>> line 170, in convert
>> 00031: self._img.write(outputLoc)
>> 00032: AttributeError: 'Image' object has no attribute 'write'
>> 00033: Protocol failed: 'Image' object has no attribute 'write'
>> 00034: FAILED: estimateCtfStep, step 1, time 2020-12-30
>> 15:35:57.909908
>>
>> 00002: Hostname: gpu08
>> 00003: PID: 101015
>> 00004: pyworkflow: 3.0.8
>> 00005: plugin: gctf
>> 00006: plugin v: 3.0.11
>> 00007: currentDir:
>> /share/home/biotest/stu01/ScipionUserData/projects/test
>> 00008: workingDir: Runs/001508_ProtGctf
>> 00009: runMode: Restart
>> 00010: MPI: 1
>> 00011: threads: 1
>> 00012: Starting at step: 1
>> 00013: Running steps
>> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30
>> 16:20:37.950836
>> 00015: Estimating CTF of micrograph: 1
>> 00016: GHOST in place, read call ignored!.
>> 00017: ERROR: Gctf has failed on
>> Runs/001508_ProtGctf/tmp/mic_0001/*.mrc
>> 00018: Traceback (most recent call last):
>> 00019: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/gctf/protocols/protocol_gctf.py",
>> line 82, in _estimateCtfList
>> 00020: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
>> 00021: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py",
>> line 170, in convert
>> 00022: self._img.write(outputLoc)
>> 00023: AttributeError: 'Image' object has no attribute 'write'
>> 00024: FINISHED: estimateCtfStep, step 1, time 2020-12-30
>> 16:20:38.060255
>>
>> Is there any suggestions? Thanks!
>>
>>
>>
>> _______________________________________________
>> scipion-users mailing list
>> sci...@li...
>> <mailto:sci...@li...>
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>>
>> _______________________________________________
>> scipion-users mailing list
>> sci...@li...
>> <mailto:sci...@li...>
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
--
Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team*
|
