Menu
▾
▴
scipion-users — Mailing list for Scipion users
You can subscribe to this list here.
| 2016 |
Jan
(2) |
Feb
(13) |
Mar
(9) |
Apr
(4) |
May
(5) |
Jun
(2) |
Jul
(8) |
Aug
(3) |
Sep
(25) |
Oct
(7) |
Nov
(49) |
Dec
(15) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2017 |
Jan
(24) |
Feb
(36) |
Mar
(53) |
Apr
(44) |
May
(37) |
Jun
(34) |
Jul
(12) |
Aug
(15) |
Sep
(14) |
Oct
(9) |
Nov
(9) |
Dec
(7) |
| 2018 |
Jan
(16) |
Feb
(9) |
Mar
(27) |
Apr
(39) |
May
(8) |
Jun
(24) |
Jul
(22) |
Aug
(11) |
Sep
(1) |
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
(4) |
Feb
(5) |
Mar
|
Apr
(1) |
May
(21) |
Jun
(13) |
Jul
(31) |
Aug
(22) |
Sep
(9) |
Oct
(19) |
Nov
(24) |
Dec
(12) |
| 2020 |
Jan
(30) |
Feb
(12) |
Mar
(16) |
Apr
(4) |
May
(37) |
Jun
(17) |
Jul
(19) |
Aug
(15) |
Sep
(26) |
Oct
(84) |
Nov
(64) |
Dec
(55) |
| 2021 |
Jan
(18) |
Feb
(58) |
Mar
(26) |
Apr
(88) |
May
(51) |
Jun
(36) |
Jul
(31) |
Aug
(37) |
Sep
(79) |
Oct
(15) |
Nov
(29) |
Dec
(8) |
| 2022 |
Jan
(5) |
Feb
(8) |
Mar
(29) |
Apr
(21) |
May
(11) |
Jun
(11) |
Jul
(18) |
Aug
(16) |
Sep
(6) |
Oct
(10) |
Nov
(23) |
Dec
(1) |
| 2023 |
Jan
(18) |
Feb
|
Mar
(4) |
Apr
|
May
(3) |
Jun
(10) |
Jul
(1) |
Aug
|
Sep
|
Oct
(1) |
Nov
(3) |
Dec
(5) |
| 2024 |
Jan
(2) |
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2025 |
Jan
(1) |
Feb
|
Mar
|
Apr
(5) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Huiskonen, J. T <juh...@he...> - 2021-01-24 11:48:35
|
Found the answer: I added JAVA_MAX_MEMORY = 15 in my scipion.conf And now it works. From: "Huiskonen, Juha T" <juh...@he...> Date: Sunday 24. January 2021 at 11.23 To: "sci...@li..." <sci...@li...> Subject: Out of memory with "Show classification in Scipion" Hi all, I have classified 3.8M particles to 4 classes. Next I would like to create a new SetOfParticles with particles from 2 classes. Normally I do this by clicking “Analyse Results”, "Show classification in Scipion" and saving particles from manually selected classes. In the current case, I run out of memory (the maximum for interactive jobs is 15 Gb on the HPC cluster I am using) when I click “Show classification in Scipion”. Is there any other way making a subset of particles after 3D classification than to visualize the classes? Best wishes, Juha |
|
From: Huiskonen, J. T <juh...@he...> - 2021-01-24 09:39:04
|
Hi all, I have classified 3.8M particles to 4 classes. Next I would like to create a new SetOfParticles with particles from 2 classes. Normally I do this by clicking “Analyse Results”, "Show classification in Scipion" and saving particles from manually selected classes. In the current case, I run out of memory (the maximum for interactive jobs is 15 Gb on the HPC cluster I am using) when I click “Show classification in Scipion”. Is there any other way making a subset of particles after 3D classification than to visualize the classes? Best wishes, Juha |
|
From: Pablo C. <pc...@cn...> - 2021-01-22 15:13:51
|
I believe it should be: GPUs: 0 1 0 1 MPI: 3 I also think there is a bug when using this case we missed when migrating to python 3: https://github.com/scipion-em/scipion-pyworkflow/blob/devel/pyworkflow/protocol/executor.py#L175 chunk = nGpu / nThreads should be chunk = int(nGpu / nThreads) I have to test this and release a fix. On 22/1/21 14:34, Hoover, David (NIH/CIT) [E] via scipion-users wrote: > Hi all, > > I am trying to launch a MotionCorr job using Scipion across multiple > nodes with multiple GPUs per node. > > If I leave GPU IDs blank, it attempts to run the tasks, but fails > because the gpu ids are missing (command has -Gpu %(GPU)s). > > If I set GPU IDs to "0", it finishes some of the tasks (where the > command includes -Gpu 0.0), but it fails on other tasks with > non-integer values for the gpu (e.g. -gpu 2.5). > > If I set GPU IDs to "0 1" (there are 2 gpus per node), the entire job > fails with this error: > > Traceback (most recent call last): > File > "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/apps/pw_protocol_mpirun.py", > line 54, in <module> > runProtocolMainMPI(projectPath, dbPath, protId, comm) > File > "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 2229, in runProtocolMainMPI > executor = MPIStepExecutor(hostConfig, protocol.numberOfMpi.get() > - 1, > File > "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", > line 342, in __init__ > ThreadStepExecutor.__init__(self, hostConfig, nMPI, **kwargs) > File > "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", > line 177, in __init__ > self.gpuDict[node] = list(self.gpuList[i*chunk:(i+1)*chunk]) > TypeError: slice indices must be integers or None or have an __index__ > method > > How exactly should one designate the GPU IDs for such a situation? > For example, 2 nodes, each with 2 GPU? > > David -- Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team* |
|
From: Hoover, D. (NIH/C. [E] <hoo...@hp...> - 2021-01-22 13:51:00
|
Hi all,
I am trying to launch a MotionCorr job using Scipion across multiple nodes with multiple GPUs per node.
If I leave GPU IDs blank, it attempts to run the tasks, but fails because the gpu ids are missing (command has -Gpu %(GPU)s).
If I set GPU IDs to "0", it finishes some of the tasks (where the command includes -Gpu 0.0), but it fails on other tasks with non-integer values for the gpu (e.g. -gpu 2.5).
If I set GPU IDs to "0 1" (there are 2 gpus per node), the entire job fails with this error:
Traceback (most recent call last):
File "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/apps/pw_protocol_mpirun.py", line 54, in <module>
runProtocolMainMPI(projectPath, dbPath, protId, comm)
File "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 2229, in runProtocolMainMPI
executor = MPIStepExecutor(hostConfig, protocol.numberOfMpi.get() - 1,
File "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 342, in __init__
ThreadStepExecutor.__init__(self, hostConfig, nMPI, **kwargs)
File "/usr/local/apps/scipion/3.0.6/anaconda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 177, in __init__
self.gpuDict[node] = list(self.gpuList[i*chunk:(i+1)*chunk])
TypeError: slice indices must be integers or None or have an __index__ method
How exactly should one designate the GPU IDs for such a situation? For example, 2 nodes, each with 2 GPU?
David
--
David Hoover, Ph.D.
Computational Biologist
High Performance Computing Services,
Center for Information Technology,
National Institutes of Health
12 South Dr., Rm 2N207
Bethesda, MD 20892, USA
TEL: (+1) 301-435-2986
Email: hoo...@hp...
|
|
From: Grigory S. <sha...@gm...> - 2021-01-19 11:37:26
|
Hi Yangyang,
happy new year!
Let's try a step-by-step approach.
First, you might check that xmipp was installed successfully by importing
any 3D volume and trying to display results. If that fails then xmipp
compilation was not successful.
Second, does the same job you are trying to run, runs locally (without
submission to slurm cluster) with less mpis/threads? This will sort out
MPI-related issues.
Third, what is the error you are getting with slurm (I didn't notice
anything wrong with the config)? You can attach run.stdout to the email.
Best regards,
Grigory
--------------------------------------------------------------------------------
Grigory Sharov, Ph.D.
MRC Laboratory of Molecular Biology,
Francis Crick Avenue,
Cambridge Biomedical Campus,
Cambridge CB2 0QH, UK.
tel. +44 (0) 1223 267228 <+44%201223%20267228>
e-mail: gs...@mr...
On Tue, Jan 19, 2021 at 9:12 AM Yangyang Yi <yy...@si...> wrote:
> Thanks for your reply. Happy new year. I see. I tried to install
> scipion-em-xmipp again, it has reported a successful install, but when I
> submit job via slurm, I failed to run. Also, I tried to run a small job on
> headnode (only system since our headnode do not have GPU cards), and it
> works. Maybe something related to my config files, especially my host.conf?
> Here’s my config files:
> host.conf
> [localhost]
> PARALLEL_COMMAND = mpirun -np %_(JOB_NODES)d %_(COMMAND)s
> NAME = SCIPION_SLURM
> MANDATORY = False
> SUBMIT_COMMAND = sbatch %_(JOB_SCRIPT)s
> CANCEL_COMMAND = scancel %_(JOB_ID)s
> CHECK_COMMAND = squeue -h -j %_(JOB_ID)s
> SUBMIT_TEMPLATE = #!/bin/bash
> ### Inherit all current environment variables
> #SBATCH --export=ALL
> ### Job name
> #SBATCH -J %_(JOB_NAME)s
> ### Queue name
> #SBATCH -p %_(JOB_QUEUE)s
> ### Standard output and standard error messages
> #SBATCH -o %_(JOB_LOGS)s.out
> #SBATCH -e %_(JOB_LOGS)s.err
> ### Specify the number of nodes and thread (ppn) for your job.
> #SBATCH --ntasks=%_(JOB_NODES)d
> #SBATCH --cpus-per-task=%_(JOB_THREADS)d
> #SBATCH --mem=%_(JOB_MEMORY)s
> #SBATCH --gres=gpu:%_(GPU_COUNT)s
> ### Tell PBS the anticipated run-time for your job, where
> walltime=HH:MM:SS
> #SBATCH --time=%_(JOB_TIME)s:00:00
> # Use as working dir the path where qsub was launched
> WORKDIR=$SLURM_SUBMIT_DIR
> #################################
> ### Set environment variable to know running mode is non interactive
> export XMIPP_IN_QUEUE=1
> ### Switch to the working directory;
> cd $WORKDIR
> # Make a copy of PBS_NODEFILE
> cp $SLURM_JOB_NODELIST > %_(JOB_NODEFILE)s
> # Calculate the number of processors allocated to this run.
> NPROCS=`wc -l < $SLURM_JOB_NODELIST`
> # Calculate the number of nodes allocated.
> ###NNODES=`uniq $SLURM_JOB_NODELIST | wc -l`
> ### Display the job context
> echo Running on host `hostname`
> echo Time is `date`
> echo Working directory is `pwd`
> ###echo Using ${NPROCS} processors across ${NNODES} nodes
> echo NODE LIST - config:
> echo $SLURM_JOB_NODELIST
> echo CUDA_VISIBLE_DEVICES: $CUDA_VISIBLE_DEVICES
> module load cuda-10.1
> module load impi-2019.4
> module load relion-3.1.0
> module load java-1.8.0
> #################################
> # echo '%_(JOB_COMMAND)s' >> /tmp/slurm-jobs.log
> %_(JOB_COMMAND)s
> ###find "$SLURM_SUBMIT_DIR" -type f -user $USER -perm 644 -exec chmod
> 664 {} +
> ; #Next variable is used to provide a regex to check if a job is finished
> on a queue system
> QUEUES = {"gpu": [["JOB_MEMORY", "8192", "Memory (MB)", "Select amount of
> memory (in megabytes) for this job"],
> ["JOB_TIME", "48", "Time (hours)", "Select the time expected (in
> hours) for this job"],
> ["GPU_COUNT", "8", "Number of GPUs", "Select the number of GPUs if
> protocol has been set up to use them"],
> ["QUEUE_FOR_JOBS", "N", "Use queue for jobs", "Send individual
> jobs to queue"]]}
> scipion.conf:
> [PYWORKFLOW]
> CONDA_ACTIVATION_CMD = eval "$(/share/apps/software/anconda/bin/conda
> shell.bash hook)"
> SCIPION_DOMAIN = pwem
> SCIPION_FONT_NAME = Helvetica
> SCIPION_LOG = ~/ScipionUserData/logs/scipion.log
> SCIPION_LOGO = scipion_logo.gif
> SCIPION_LOGS = ~/ScipionUserData/logs
> SCIPION_NOTES_FILE = notes.txt
> SCIPION_NOTIFY = True
> SCIPION_PLUGIN_REPO_URL = http://scipion.i2pc.es/getplugins/
> SCIPION_SOFTWARE = ${SCIPION_HOME}/software
> SCIPION_SUPPORT_EMAIL = sc...@cn...
> SCIPION_TESTS = ${SCIPION_HOME}/data/tests
> SCIPION_TESTS_CMD = scipion3 tests
> SCIPION_TESTS_OUTPUT = ~/ScipionUserData/Tests
> SCIPION_TMP = ~/ScipionUserData/tmp
> SCIPION_URL = http://scipion.cnb.csic.es/downloads/scipion
> SCIPION_URL_SOFTWARE =
> http://scipion.cnb.csic.es/downloads/scipion/software
> SCIPION_URL_TESTDATA =
> http://scipion.cnb.csic.es/downloads/scipion/data/tests
> SCIPION_USER_DATA = ~/ScipionUserData
> WIZARD_MASK_COLOR = [0.125, 0.909, 0.972]
> SCIPION_NOTES_ARGS =
> SCIPION_NOTES_PROGRAM =
>
> [PLUGINS]
> EM_ROOT = software/em
> MAXIT_HOME = %(EM_ROOT)s/maxit-10.1
> XMIPP_HOME = %(EM_ROOT)s/xmipp
> CUDA_BIN = /share/apps/software/cuda-10.1/bin
> CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> CHIMERA_HOME = %(EM_ROOT)s/chimerax-1.1
> GCTF_HOME = %(EM_ROOT)s/Gctf_v1.18
> GAUTOMATCH = Gautomatch_v0.56_sm30-75_cu10.1
> MOTIONCOR2_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> RELION_HOME = software/em/relion-3.1.0
> RESMAP = ResMap-1.95-cuda-Centos7x64
> GCTF_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> MOTIONCOR2_BIN = MotionCor2_1.4.0_Cuda101
> CRYOSPARC_HOME =
> RESMAP_GPU_LIB = ResMap_krnl-cuda-V8.0.61-sm60_gpu.so
> CRYO_PROJECTS_DIR = scipion_projects
> GAUTOMATCH_HOME = software/em/gautomatch-0.56
> RELION_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> RELION_CUDA_BIN = /share/apps/software/cuda-10.1/bin
> CTFFIND4_HOME = software/em/ctffind4-4.1.14
> RESMAP_HOME = software/em/resmap-1.95
> GAUTOMATCH_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> GCTF = Gctf_v1.18_sm30-75_cu10.1
> CISTEM_HOME = software/em/cistem-1.0.0-beta
> MOTIONCOR2_HOME = software/em/motioncor2-1.4.0
>
> 2020年12月30日 下午4:50，Grigory Sharov <sha...@gm...> 写道：
>
> Hi Yangyang, the line
> 00016: GHOST in place, read call ignored
> means that you don't have xmipp installed which is still required for all
> basic operations
>
> Grigory .
>
>
> On Wed, Dec 30, 2020, 08:44 Yangyang Yi <yy...@si...> wrote:
>
>> Hi,
>> Happy Holiday!
>> Thanks for the help in last email about Scipion-2.0 cluster config.
>> I have solved the problem, since I used “Use queue for jobs:Y” instead of
>> "N", and some plugins only download partly. I edited those parameter and
>> reinstalled some of the plugin, now it works.
>> However, our users now prefer scipion-3.0 and I found new errors
>> with similar settings. Now all the motion correction job works and result
>> files in extra directory (micrographs and shifts) and generated, however, I
>> failed to run CTF after motion correction. I have imported some motion
>> correction micrographs but still failed to run CTF (both Gctf and CTFFIND).
>> The error is similar as below:
>>
>> CTFFIND:
>> 00002: Hostname: gpu04
>> 00003: PID: 138990
>> 00004: pyworkflow: 3.0.8
>> 00005: plugin: cistem
>> 00006: plugin v: 3.0.9
>> 00007: currentDir:
>> /share/home/biotest/stu01/ScipionUserData/projects/test
>> 00008: workingDir: Runs/001767_CistemProtCTFFind
>> 00009: runMode: Restart
>> 00010: MPI: 1
>> 00011: threads: 1
>> 00012: Starting at step: 1
>> 00013: Running steps
>> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.887435
>> 00015: Estimating CTF of micrograph: 1
>> 00016: GHOST in place, read call ignored!.
>> 00017: Traceback (most recent call last):
>> 00018: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 189, in run
>> 00019: self._run()
>> 00020: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 240, in _run
>> 00021: resultFiles = self._runFunc()
>> 00022: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 236, in _runFunc
>> 00023: return self._func(*self._args)
>> 00024: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_micrographs.py",
>> line 247, in estimateCtfStep
>> 00025: self._estimateCTF(mic, *args)
>> 00026: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py",
>> line 108, in _estimateCTF
>> 00027: self._doCtfEstimation(mic)
>> 00028: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py",
>> line 78, in _doCtfEstimation
>> 00029: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
>> 00030: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py",
>> line 170, in convert
>> 00031: self._img.write(outputLoc)
>> 00032: AttributeError: 'Image' object has no attribute 'write'
>> 00033: Protocol failed: 'Image' object has no attribute 'write'
>> 00034: FAILED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.909908
>>
>> 00002: Hostname: gpu08
>> 00003: PID: 101015
>> 00004: pyworkflow: 3.0.8
>> 00005: plugin: gctf
>> 00006: plugin v: 3.0.11
>> 00007: currentDir:
>> /share/home/biotest/stu01/ScipionUserData/projects/test
>> 00008: workingDir: Runs/001508_ProtGctf
>> 00009: runMode: Restart
>> 00010: MPI: 1
>> 00011: threads: 1
>> 00012: Starting at step: 1
>> 00013: Running steps
>> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 16:20:37.950836
>> 00015: Estimating CTF of micrograph: 1
>> 00016: GHOST in place, read call ignored!.
>> 00017: ERROR: Gctf has failed on Runs/001508_ProtGctf/tmp/mic_0001/*.mrc
>> 00018: Traceback (most recent call last):
>> 00019: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/gctf/protocols/protocol_gctf.py",
>> line 82, in _estimateCtfList
>> 00020: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
>> 00021: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py",
>> line 170, in convert
>> 00022: self._img.write(outputLoc)
>> 00023: AttributeError: 'Image' object has no attribute 'write'
>> 00024: FINISHED: estimateCtfStep, step 1, time 2020-12-30
>> 16:20:38.060255
>>
>> Is there any suggestions? Thanks!
>>
>>
>>
>> _______________________________________________
>> scipion-users mailing list
>> sci...@li...
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
>
|
|
From: Pablo C. <pc...@cn...> - 2021-01-19 11:37:23
|
Hi Yangyang,
I assume now that xmipp seems to be installed it fails differently?
If so, what is the error you find in the output log?
On 19/1/21 10:11, Yangyang Yi wrote:
> Thanks for your reply. Happy new year. I see. I tried to install
> scipion-em-xmipp again, it has reported a successful install, but when
> I submit job via slurm, I failed to run. Also, I tried to run a small
> job on headnode (only system since our headnode do not have GPU
> cards), and it works. Maybe something related to my config files,
> especially my host.conf? Here’s my config files:
> host.conf
> [localhost]
> PARALLEL_COMMAND = mpirun -np %_(JOB_NODES)d %_(COMMAND)s
> NAME = SCIPION_SLURM
> MANDATORY = False
> SUBMIT_COMMAND = sbatch %_(JOB_SCRIPT)s
> CANCEL_COMMAND = scancel %_(JOB_ID)s
> CHECK_COMMAND = squeue -h -j %_(JOB_ID)s
> SUBMIT_TEMPLATE = #!/bin/bash
> ### Inherit all current environment variables
> #SBATCH --export=ALL
> ### Job name
> #SBATCH -J %_(JOB_NAME)s
> ### Queue name
> #SBATCH -p %_(JOB_QUEUE)s
> ### Standard output and standard error messages
> #SBATCH -o %_(JOB_LOGS)s.out
> #SBATCH -e %_(JOB_LOGS)s.err
> ### Specify the number of nodes and thread (ppn) for your job.
> #SBATCH --ntasks=%_(JOB_NODES)d
> #SBATCH --cpus-per-task=%_(JOB_THREADS)d
> #SBATCH --mem=%_(JOB_MEMORY)s
> #SBATCH --gres=gpu:%_(GPU_COUNT)s
> ### Tell PBS the anticipated run-time for your job, where
> walltime=HH:MM:SS
> #SBATCH --time=%_(JOB_TIME)s:00:00
> # Use as working dir the path where qsub was launched
> WORKDIR=$SLURM_SUBMIT_DIR
> #################################
> ### Set environment variable to know running mode is non interactive
> export XMIPP_IN_QUEUE=1
> ### Switch to the working directory;
> cd $WORKDIR
> # Make a copy of PBS_NODEFILE
> cp $SLURM_JOB_NODELIST > %_(JOB_NODEFILE)s
> # Calculate the number of processors allocated to this run.
> NPROCS=`wc -l < $SLURM_JOB_NODELIST`
> # Calculate the number of nodes allocated.
> ###NNODES=`uniq $SLURM_JOB_NODELIST | wc -l`
> ### Display the job context
> echo Running on host `hostname`
> echo Time is `date`
> echo Working directory is `pwd`
> ###echo Using ${NPROCS} processors across ${NNODES} nodes
> echo NODE LIST - config:
> echo $SLURM_JOB_NODELIST
> echo CUDA_VISIBLE_DEVICES: $CUDA_VISIBLE_DEVICES
> module load cuda-10.1
> module load impi-2019.4
> module load relion-3.1.0
> module load java-1.8.0
> #################################
> # echo '%_(JOB_COMMAND)s' >> /tmp/slurm-jobs.log
> %_(JOB_COMMAND)s
> ###find "$SLURM_SUBMIT_DIR" -type f -user $USER -perm 644 -exec
> chmod 664 {} +
> ; #Next variable is used to provide a regex to check if a job is
> finished on a queue system
> QUEUES = {"gpu": [["JOB_MEMORY", "8192", "Memory (MB)", "Select amount
> of memory (in megabytes) for this job"],
> ["JOB_TIME", "48", "Time (hours)", "Select the time expected
> (in hours) for this job"],
> ["GPU_COUNT", "8", "Number of GPUs", "Select the number of
> GPUs if protocol has been set up to use them"],
> ["QUEUE_FOR_JOBS", "N", "Use queue for jobs", "Send individual
> jobs to queue"]]}
> scipion.conf:
> [PYWORKFLOW]
> CONDA_ACTIVATION_CMD = eval "$(/share/apps/software/anconda/bin/conda
> shell.bash hook)"
> SCIPION_DOMAIN = pwem
> SCIPION_FONT_NAME = Helvetica
> SCIPION_LOG = ~/ScipionUserData/logs/scipion.log
> SCIPION_LOGO = scipion_logo.gif
> SCIPION_LOGS = ~/ScipionUserData/logs
> SCIPION_NOTES_FILE = notes.txt
> SCIPION_NOTIFY = True
> SCIPION_PLUGIN_REPO_URL = http://scipion.i2pc.es/getplugins/
> SCIPION_SOFTWARE = ${SCIPION_HOME}/software
> SCIPION_SUPPORT_EMAIL = sc...@cn... <mailto:sc...@cn...>
> SCIPION_TESTS = ${SCIPION_HOME}/data/tests
> SCIPION_TESTS_CMD = scipion3 tests
> SCIPION_TESTS_OUTPUT = ~/ScipionUserData/Tests
> SCIPION_TMP = ~/ScipionUserData/tmp
> SCIPION_URL = http://scipion.cnb.csic.es/downloads/scipion
> SCIPION_URL_SOFTWARE =
> http://scipion.cnb.csic.es/downloads/scipion/software
> SCIPION_URL_TESTDATA =
> http://scipion.cnb.csic.es/downloads/scipion/data/tests
> SCIPION_USER_DATA = ~/ScipionUserData
> WIZARD_MASK_COLOR = [0.125, 0.909, 0.972]
> SCIPION_NOTES_ARGS =
> SCIPION_NOTES_PROGRAM =
>
> [PLUGINS]
> EM_ROOT = software/em
> MAXIT_HOME = %(EM_ROOT)s/maxit-10.1
> XMIPP_HOME = %(EM_ROOT)s/xmipp
> CUDA_BIN = /share/apps/software/cuda-10.1/bin
> CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> CHIMERA_HOME = %(EM_ROOT)s/chimerax-1.1
> GCTF_HOME = %(EM_ROOT)s/Gctf_v1.18
> GAUTOMATCH = Gautomatch_v0.56_sm30-75_cu10.1
> MOTIONCOR2_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> RELION_HOME = software/em/relion-3.1.0
> RESMAP = ResMap-1.95-cuda-Centos7x64
> GCTF_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> MOTIONCOR2_BIN = MotionCor2_1.4.0_Cuda101
> CRYOSPARC_HOME =
> RESMAP_GPU_LIB = ResMap_krnl-cuda-V8.0.61-sm60_gpu.so
> CRYO_PROJECTS_DIR = scipion_projects
> GAUTOMATCH_HOME = software/em/gautomatch-0.56
> RELION_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> RELION_CUDA_BIN = /share/apps/software/cuda-10.1/bin
> CTFFIND4_HOME = software/em/ctffind4-4.1.14
> RESMAP_HOME = software/em/resmap-1.95
> GAUTOMATCH_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
> GCTF = Gctf_v1.18_sm30-75_cu10.1
> CISTEM_HOME = software/em/cistem-1.0.0-beta
> MOTIONCOR2_HOME = software/em/motioncor2-1.4.0
>
>> 2020年12月30日 下午4:50，Grigory Sharov <sha...@gm...
>> <mailto:sha...@gm...>> 写道：
>>
>> Hi Yangyang, the line
>> 00016: GHOST in place, read call ignored
>> means that you don't have xmipp installed which is still required for
>> all basic operations
>>
>> Grigory .
>>
>>
>> On Wed, Dec 30, 2020, 08:44 Yangyang Yi <yy...@si...
>> <mailto:yy...@si...>> wrote:
>>
>> Hi,
>> Happy Holiday!
>> Thanks for the help in last email about Scipion-2.0 cluster
>> config. I have solved the problem, since I used “Use queue for
>> jobs:Y” instead of "N", and some plugins only download partly. I
>> edited those parameter and reinstalled some of the plugin, now it
>> works.
>> However, our users now prefer scipion-3.0 and I found new
>> errors with similar settings. Now all the motion correction job
>> works and result files in extra directory (micrographs and
>> shifts) and generated, however, I failed to run CTF after motion
>> correction. I have imported some motion correction micrographs
>> but still failed to run CTF (both Gctf and CTFFIND). The error is
>> similar as below:
>>
>> CTFFIND:
>> 00002: Hostname: gpu04
>> 00003: PID: 138990
>> 00004: pyworkflow: 3.0.8
>> 00005: plugin: cistem
>> 00006: plugin v: 3.0.9
>> 00007: currentDir:
>> /share/home/biotest/stu01/ScipionUserData/projects/test
>> 00008: workingDir: Runs/001767_CistemProtCTFFind
>> 00009: runMode: Restart
>> 00010: MPI: 1
>> 00011: threads: 1
>> 00012: Starting at step: 1
>> 00013: Running steps
>> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30
>> 15:35:57.887435
>> 00015: Estimating CTF of micrograph: 1
>> 00016: GHOST in place, read call ignored!.
>> 00017: Traceback (most recent call last):
>> 00018: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 189, in run
>> 00019: self._run()
>> 00020: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 240, in _run
>> 00021: resultFiles = self._runFunc()
>> 00022: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py",
>> line 236, in _runFunc
>> 00023: return self._func(*self._args)
>> 00024: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_micrographs.py",
>> line 247, in estimateCtfStep
>> 00025: self._estimateCTF(mic, *args)
>> 00026: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py",
>> line 108, in _estimateCTF
>> 00027: self._doCtfEstimation(mic)
>> 00028: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py",
>> line 78, in _doCtfEstimation
>> 00029: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
>> 00030: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py",
>> line 170, in convert
>> 00031: self._img.write(outputLoc)
>> 00032: AttributeError: 'Image' object has no attribute 'write'
>> 00033: Protocol failed: 'Image' object has no attribute 'write'
>> 00034: FAILED: estimateCtfStep, step 1, time 2020-12-30
>> 15:35:57.909908
>>
>> 00002: Hostname: gpu08
>> 00003: PID: 101015
>> 00004: pyworkflow: 3.0.8
>> 00005: plugin: gctf
>> 00006: plugin v: 3.0.11
>> 00007: currentDir:
>> /share/home/biotest/stu01/ScipionUserData/projects/test
>> 00008: workingDir: Runs/001508_ProtGctf
>> 00009: runMode: Restart
>> 00010: MPI: 1
>> 00011: threads: 1
>> 00012: Starting at step: 1
>> 00013: Running steps
>> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30
>> 16:20:37.950836
>> 00015: Estimating CTF of micrograph: 1
>> 00016: GHOST in place, read call ignored!.
>> 00017: ERROR: Gctf has failed on
>> Runs/001508_ProtGctf/tmp/mic_0001/*.mrc
>> 00018: Traceback (most recent call last):
>> 00019: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/gctf/protocols/protocol_gctf.py",
>> line 82, in _estimateCtfList
>> 00020: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
>> 00021: File
>> "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py",
>> line 170, in convert
>> 00022: self._img.write(outputLoc)
>> 00023: AttributeError: 'Image' object has no attribute 'write'
>> 00024: FINISHED: estimateCtfStep, step 1, time 2020-12-30
>> 16:20:38.060255
>>
>> Is there any suggestions? Thanks!
>>
>>
>>
>> _______________________________________________
>> scipion-users mailing list
>> sci...@li...
>> <mailto:sci...@li...>
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>>
>> _______________________________________________
>> scipion-users mailing list
>> sci...@li...
>> <mailto:sci...@li...>
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
--
Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team*
|
|
From: Yangyang Yi <yy...@si...> - 2021-01-19 09:12:45
|
Thanks for your reply. Happy new year. I see. I tried to install scipion-em-xmipp again, it has reported a successful install, but when I submit job via slurm, I failed to run. Also, I tried to run a small job on headnode (only system since our headnode do not have GPU cards), and it works. Maybe something related to my config files, especially my host.conf? Here’s my config files:
host.conf
[localhost]
PARALLEL_COMMAND = mpirun -np %_(JOB_NODES)d %_(COMMAND)s
NAME = SCIPION_SLURM
MANDATORY = False
SUBMIT_COMMAND = sbatch %_(JOB_SCRIPT)s
CANCEL_COMMAND = scancel %_(JOB_ID)s
CHECK_COMMAND = squeue -h -j %_(JOB_ID)s
SUBMIT_TEMPLATE = #!/bin/bash
### Inherit all current environment variables
#SBATCH --export=ALL
### Job name
#SBATCH -J %_(JOB_NAME)s
### Queue name
#SBATCH -p %_(JOB_QUEUE)s
### Standard output and standard error messages
#SBATCH -o %_(JOB_LOGS)s.out
#SBATCH -e %_(JOB_LOGS)s.err
### Specify the number of nodes and thread (ppn) for your job.
#SBATCH --ntasks=%_(JOB_NODES)d
#SBATCH --cpus-per-task=%_(JOB_THREADS)d
#SBATCH --mem=%_(JOB_MEMORY)s
#SBATCH --gres=gpu:%_(GPU_COUNT)s
### Tell PBS the anticipated run-time for your job, where walltime=HH:MM:SS
#SBATCH --time=%_(JOB_TIME)s:00:00
# Use as working dir the path where qsub was launched
WORKDIR=$SLURM_SUBMIT_DIR
#################################
### Set environment variable to know running mode is non interactive
export XMIPP_IN_QUEUE=1
### Switch to the working directory;
cd $WORKDIR
# Make a copy of PBS_NODEFILE
cp $SLURM_JOB_NODELIST > %_(JOB_NODEFILE)s
# Calculate the number of processors allocated to this run.
NPROCS=`wc -l < $SLURM_JOB_NODELIST`
# Calculate the number of nodes allocated.
###NNODES=`uniq $SLURM_JOB_NODELIST | wc -l`
### Display the job context
echo Running on host `hostname`
echo Time is `date`
echo Working directory is `pwd`
###echo Using ${NPROCS} processors across ${NNODES} nodes
echo NODE LIST - config:
echo $SLURM_JOB_NODELIST
echo CUDA_VISIBLE_DEVICES: $CUDA_VISIBLE_DEVICES
module load cuda-10.1
module load impi-2019.4
module load relion-3.1.0
module load java-1.8.0
#################################
# echo '%_(JOB_COMMAND)s' >> /tmp/slurm-jobs.log
%_(JOB_COMMAND)s
###find "$SLURM_SUBMIT_DIR" -type f -user $USER -perm 644 -exec chmod 664 {} +
; #Next variable is used to provide a regex to check if a job is finished on a queue system
QUEUES = {"gpu": [["JOB_MEMORY", "8192", "Memory (MB)", "Select amount of memory (in megabytes) for this job"],
["JOB_TIME", "48", "Time (hours)", "Select the time expected (in hours) for this job"],
["GPU_COUNT", "8", "Number of GPUs", "Select the number of GPUs if protocol has been set up to use them"],
["QUEUE_FOR_JOBS", "N", "Use queue for jobs", "Send individual jobs to queue"]]}
scipion.conf:
[PYWORKFLOW]
CONDA_ACTIVATION_CMD = eval "$(/share/apps/software/anconda/bin/conda shell.bash hook)"
SCIPION_DOMAIN = pwem
SCIPION_FONT_NAME = Helvetica
SCIPION_LOG = ~/ScipionUserData/logs/scipion.log
SCIPION_LOGO = scipion_logo.gif
SCIPION_LOGS = ~/ScipionUserData/logs
SCIPION_NOTES_FILE = notes.txt
SCIPION_NOTIFY = True
SCIPION_PLUGIN_REPO_URL = http://scipion.i2pc.es/getplugins/
SCIPION_SOFTWARE = ${SCIPION_HOME}/software
SCIPION_SUPPORT_EMAIL = sc...@cn...
SCIPION_TESTS = ${SCIPION_HOME}/data/tests
SCIPION_TESTS_CMD = scipion3 tests
SCIPION_TESTS_OUTPUT = ~/ScipionUserData/Tests
SCIPION_TMP = ~/ScipionUserData/tmp
SCIPION_URL = http://scipion.cnb.csic.es/downloads/scipion
SCIPION_URL_SOFTWARE = http://scipion.cnb.csic.es/downloads/scipion/software
SCIPION_URL_TESTDATA = http://scipion.cnb.csic.es/downloads/scipion/data/tests
SCIPION_USER_DATA = ~/ScipionUserData
WIZARD_MASK_COLOR = [0.125, 0.909, 0.972]
SCIPION_NOTES_ARGS =
SCIPION_NOTES_PROGRAM =
[PLUGINS]
EM_ROOT = software/em
MAXIT_HOME = %(EM_ROOT)s/maxit-10.1
XMIPP_HOME = %(EM_ROOT)s/xmipp
CUDA_BIN = /share/apps/software/cuda-10.1/bin
CUDA_LIB = /share/apps/software/cuda-10.1/lib64
CHIMERA_HOME = %(EM_ROOT)s/chimerax-1.1
GCTF_HOME = %(EM_ROOT)s/Gctf_v1.18
GAUTOMATCH = Gautomatch_v0.56_sm30-75_cu10.1
MOTIONCOR2_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
RELION_HOME = software/em/relion-3.1.0
RESMAP = ResMap-1.95-cuda-Centos7x64
GCTF_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
MOTIONCOR2_BIN = MotionCor2_1.4.0_Cuda101
CRYOSPARC_HOME =
RESMAP_GPU_LIB = ResMap_krnl-cuda-V8.0.61-sm60_gpu.so
CRYO_PROJECTS_DIR = scipion_projects
GAUTOMATCH_HOME = software/em/gautomatch-0.56
RELION_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
RELION_CUDA_BIN = /share/apps/software/cuda-10.1/bin
CTFFIND4_HOME = software/em/ctffind4-4.1.14
RESMAP_HOME = software/em/resmap-1.95
GAUTOMATCH_CUDA_LIB = /share/apps/software/cuda-10.1/lib64
GCTF = Gctf_v1.18_sm30-75_cu10.1
CISTEM_HOME = software/em/cistem-1.0.0-beta
MOTIONCOR2_HOME = software/em/motioncor2-1.4.0
> 2020年12月30日 下午4:50，Grigory Sharov <sha...@gm...> 写道：
>
> Hi Yangyang, the line
> 00016: GHOST in place, read call ignored
> means that you don't have xmipp installed which is still required for all basic operations
>
> Grigory .
>
>
> On Wed, Dec 30, 2020, 08:44 Yangyang Yi <yy...@si... <mailto:yy...@si...>> wrote:
> Hi,
> Happy Holiday!
> Thanks for the help in last email about Scipion-2.0 cluster config. I have solved the problem, since I used “Use queue for jobs:Y” instead of "N", and some plugins only download partly. I edited those parameter and reinstalled some of the plugin, now it works.
> However, our users now prefer scipion-3.0 and I found new errors with similar settings. Now all the motion correction job works and result files in extra directory (micrographs and shifts) and generated, however, I failed to run CTF after motion correction. I have imported some motion correction micrographs but still failed to run CTF (both Gctf and CTFFIND). The error is similar as below:
>
> CTFFIND:
> 00002: Hostname: gpu04
> 00003: PID: 138990
> 00004: pyworkflow: 3.0.8
> 00005: plugin: cistem
> 00006: plugin v: 3.0.9
> 00007: currentDir: /share/home/biotest/stu01/ScipionUserData/projects/test
> 00008: workingDir: Runs/001767_CistemProtCTFFind
> 00009: runMode: Restart
> 00010: MPI: 1
> 00011: threads: 1
> 00012: Starting at step: 1
> 00013: Running steps
> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.887435
> 00015: Estimating CTF of micrograph: 1
> 00016: GHOST in place, read call ignored!.
> 00017: Traceback (most recent call last):
> 00018: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 189, in run
> 00019: self._run()
> 00020: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 240, in _run
> 00021: resultFiles = self._runFunc()
> 00022: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 236, in _runFunc
> 00023: return self._func(*self._args)
> 00024: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_micrographs.py", line 247, in estimateCtfStep
> 00025: self._estimateCTF(mic, *args)
> 00026: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py", line 108, in _estimateCTF
> 00027: self._doCtfEstimation(mic)
> 00028: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py", line 78, in _doCtfEstimation
> 00029: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
> 00030: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py", line 170, in convert
> 00031: self._img.write(outputLoc)
> 00032: AttributeError: 'Image' object has no attribute 'write'
> 00033: Protocol failed: 'Image' object has no attribute 'write'
> 00034: FAILED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.909908
>
> 00002: Hostname: gpu08
> 00003: PID: 101015
> 00004: pyworkflow: 3.0.8
> 00005: plugin: gctf
> 00006: plugin v: 3.0.11
> 00007: currentDir: /share/home/biotest/stu01/ScipionUserData/projects/test
> 00008: workingDir: Runs/001508_ProtGctf
> 00009: runMode: Restart
> 00010: MPI: 1
> 00011: threads: 1
> 00012: Starting at step: 1
> 00013: Running steps
> 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 16:20:37.950836
> 00015: Estimating CTF of micrograph: 1
> 00016: GHOST in place, read call ignored!.
> 00017: ERROR: Gctf has failed on Runs/001508_ProtGctf/tmp/mic_0001/*.mrc
> 00018: Traceback (most recent call last):
> 00019: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/gctf/protocols/protocol_gctf.py", line 82, in _estimateCtfList
> 00020: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
> 00021: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py", line 170, in convert
> 00022: self._img.write(outputLoc)
> 00023: AttributeError: 'Image' object has no attribute 'write'
> 00024: FINISHED: estimateCtfStep, step 1, time 2020-12-30 16:20:38.060255
>
> Is there any suggestions? Thanks!
>
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li... <mailto:sci...@li...>
> https://lists.sourceforge.net/lists/listinfo/scipion-users <https://lists.sourceforge.net/lists/listinfo/scipion-users>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
|
|
From: Pablo C. <pc...@cn...> - 2021-01-18 09:53:49
|
We are trying to organize better our support and have trigger a massive send of emails. Apologies. -- Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team* |
|
From: Pablo C. <pc...@cn...> - 2021-01-14 12:10:45
|
Hi Sudhir. Seems like a hard thing to do. I'm not saying is impossible but you'll face many challenges. Probably the simplest way would be to use a docker-nvidia image but then not sure how it performs and you'll need docker and the image Is that an option. docker images of scipion3 are currently being tested. On 14/1/21 12:55, Sudhir Baghel wrote: > Hi, > > We want to install Scipion on a server where internet is not > available. Kindly guide us. > > Thank you, > With regards, > > Sudhir Kumar > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users -- Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team* |
|
From: Sudhir B. <bag...@gm...> - 2021-01-14 11:56:36
|
Hi, We want to install Scipion on a server where internet is not available. Kindly guide us. Thank you, With regards, Sudhir Kumar |
|
From: Grigory S. <sha...@gm...> - 2021-01-05 15:11:11
|
Hi Dmitry, The opening project slowdown is fixed in the latest plugin release. In my experience this saved 50% of time to open/load a project with many cryosparc protocols inside. Let us know if you still experience the performance issues. Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Sun, Dec 27, 2020 at 2:41 PM <pc...@cn...> wrote: > We recently saw cryosparc not responding to our request. We weren't > defining a timeout so the consequence is that, a project never opens and > freezes. > > A restart of cryosparc fixed it. Additionally we are working, if not > released yet (Yunior should know), on a tolerance fix to avoid the freeze > even if cs does not respond. > > El 27 dic. 2020 11:56, Grigory Sharov <sha...@gm...> escribió: > > Hi Dmitry, > > So you have a project with few cryosparc protocols and nothing running > there. > Do you mean that the project window takes a lot of time to open? What's > the total number of protocols? Could you maybe share the whole project > workflow (export it to json) privately? > > We did start working on profiling scipion performance so this will be a > useful case > > Thanks, > Grigory > > > On Tue, Dec 22, 2020, 09:20 Dmitry Semchonok <Sem...@gm...> wrote: > > Dear colleagues, > > We have a slowing down of any of our projects that contains any of > cryoSPARC protocol inside the SCIPION even when no active process is > running. > (I do not mean the conversion step). > > > At the same time, running the same task separately on the cryoSPARC server > only does not give any delay in response and finished much faster. > > > Do you perhaps know the reason and the possible fix? > > > Thank you! > > Sincerely, > Dmitry > > > > > > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Grigory S. <sha...@gm...> - 2021-01-01 17:41:36
|
Hi Dmitry, Killed with signal 9 in most cases means you have run out of memory. You are running training with 421029 particles, which is a bad idea. Please see page 46 of the relion 3.1 tutorial about how many particles to use. Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Fri, Jan 1, 2021 at 5:04 PM Dmitry Semchonok <sem...@gm...> wrote: > Dear colleagues & Grigory, > > > > Happy new year! > > > > > > > > > > Once again the issue with relion Bayesian polishing. > > > > I have 421029 particles that I would like to apply for Bayesian polishing > (training) protocol. > > > > > > But every time this step collapses with the following error + > Signals.SIGKILL: 9 (please see below). > > > > I suspect that the problem can be in too large number of particles. > > What is your opinion? > > If that is an issue how still is it possible to run this Bayesian > polishing (training) protocol? > > > > > > > Thank you! > > > > Sincerely, > > Dmitry > > > > > ============================================================================================================= > > > > 02234: 1985: > GridSquare_11360233_Data_FoilHole_13166836_Data_11396369_11396371_20201215_204224_Fractions.mrc > > 02235: 1404: > GridSquare_11359542_Data_FoilHole_11378215_Data_11396369_11396371_20201214_055446_Fractions.mrc > > 02236: 361: > GridSquare_11359008_Data_FoilHole_11380461_Data_11396369_11396371_20201214_181814_Fractions.mrc > > 02237: > > 02238: + 421029 particles found in 2199 micrographs > > 02239: + preparing alignment data... > > 02240: Traceback (most recent call last): > > 02241: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 189, in run > > 02242: self._run() > > 02243: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 240, in _run > > 02244: resultFiles = self._runFunc() > > 02245: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 236, in _runFunc > > 02246: return self._func(*self._args) > > 02247: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", > line 309, in trainOrPolishStep > > 02248: self.runJob(prog, args) > > 02249: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 1357, in runJob > > 02250: self._stepsExecutor.runJob(self._log, program, arguments, > **kwargs) > > 02251: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", > line 65, in runJob > > 02252: process.runJob(log, programName, params, > > 02253: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 51, in runJob > > 02254: return runCommand(command, env, cwd) > > 02255: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 66, in runCommand > > 02256: check_call(command, shell=True, stdout=sys.stdout, > stderr=sys.stderr, > > 02257: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line > 364, in check_call > > 02258: raise CalledProcessError(retcode, cmd) > > 02259: subprocess.CalledProcessError: Command 'relion_motion_refine --i > Runs/006095_ProtRelionBayesianPolishing/input_particles.star --o > Runs/006095_ProtRelionBayesianPolishing/extra --f > Runs/006046_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/006095_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 20 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 ' died with <Signals.SIGKILL: 9>. > > 02260: Protocol failed: Command 'relion_motion_refine --i > Runs/006095_ProtRelionBayesianPolishing/input_particles.star --o > Runs/006095_ProtRelionBayesianPolishing/extra --f > Runs/006046_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/006095_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 20 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 ' died with <Signals.SIGKILL: 9>. > > 02261: FAILED: trainOrPolishStep, step 2, time 2021-01-01 13:40:51.032080 > > 02262: *** Last status is failed > |
|
From: Dmitry S. <sem...@gm...> - 2021-01-01 17:04:57
|
<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style></head><body lang=RU link=blue vlink="#954F72" style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Dear colleagues & Grigory,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Happy new year!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Once again the issue with relion Bayesian polishing.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I have 421029 particles that I would like to apply for Bayesian polishing (training) protocol.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>But every time this step collapses with the following error + Signals.SIGKILL: 9 (please see below).<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I suspect that the problem can be in too large number of particles.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>What is your opinion?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>If that is an issue how still is it possible to run this Bayesian polishing (training) protocol?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><br>Thank you!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Sincerely,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Dmitry<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>=============================================================================================================<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>02234: 1985: GridSquare_11360233_Data_FoilHole_13166836_Data_11396369_11396371_20201215_204224_Fractions.mrc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02235: 1404: GridSquare_11359542_Data_FoilHole_11378215_Data_11396369_11396371_20201214_055446_Fractions.mrc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02236: 361: GridSquare_11359008_Data_FoilHole_11380461_Data_11396369_11396371_20201214_181814_Fractions.mrc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02237: <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02238: + 421029 particles found in 2199 micrographs<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02239: + preparing alignment data... <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02240: Traceback (most recent call last):<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02241: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 189, in run<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02242: self._run()<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02243: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 240, in _run<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02244: resultFiles = self._runFunc()<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02245: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 236, in _runFunc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02246: return self._func(*self._args)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02247: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", line 309, in trainOrPolishStep<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02248: self.runJob(prog, args)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02249: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 1357, in runJob<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02250: self._stepsExecutor.runJob(self._log, program, arguments, **kwargs)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02251: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 65, in runJob<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02252: process.runJob(log, programName, params,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02253: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 51, in runJob<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02254: return runCommand(command, env, cwd)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02255: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 66, in runCommand<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02256: check_call(command, shell=True, stdout=sys.stdout, stderr=sys.stderr,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02257: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line 364, in check_call<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02258: raise CalledProcessError(retcode, cmd)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02259: subprocess.CalledProcessError: Command 'relion_motion_refine --i Runs/006095_ProtRelionBayesianPolishing/input_particles.star --o Runs/006095_ProtRelionBayesianPolishing/extra --f Runs/006046_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/006095_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 20 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' died with <Signals.SIGKILL: 9>.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02260: Protocol failed: Command 'relion_motion_refine --i Runs/006095_ProtRelionBayesianPolishing/input_particles.star --o Runs/006095_ProtRelionBayesianPolishing/extra --f Runs/006046_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/006095_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 20 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' died with <Signals.SIGKILL: 9>.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02261: FAILED: trainOrPolishStep, step 2, time 2021-01-01 13:40:51.032080<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>02262: *** Last status is failed<o:p></o:p></span></p></div></body></html> |
|
From: Grigory S. <sha...@gm...> - 2020-12-30 17:05:07
|
CC for the records: The problem was in xmipp center particles protocol that did not preserve micnames. I've reported this to the xmipp dev team which will be on it. Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Wed, Dec 30, 2020 at 4:23 PM Dmitry A. Semchonok <sem...@gm...> wrote: > Thank you very much, Grigory, for your detailed investigation and > recommendations! > > > Sincerely, > Dmitry > > On December 30, 2020 6:16:12 PM Grigory Sharov <sha...@gm...> > wrote: > >> Hi, just confirmed, polishing fails with the same error as you reported. >> I'll inform the xmipp team to fix this. Thanks a lot for reporting this! >> >> In Relion motioncor you set the number of patches to 1x1 which means you >> only did global movie alignment. It's okay, but probably better results can >> be obtained with a number of patches 5x5 (see relion tutorial). >> >> Best regards, >> Grigory >> >> >> -------------------------------------------------------------------------------- >> Grigory Sharov, Ph.D. >> >> MRC Laboratory of Molecular Biology, >> Francis Crick Avenue, >> Cambridge Biomedical Campus, >> Cambridge CB2 0QH, UK. >> tel. +44 (0) 1223 267228 <+44%201223%20267228> >> e-mail: gs...@mr... >> >> >> On Wed, Dec 30, 2020 at 4:11 PM Dmitry A. Semchonok <sem...@gm...> >> wrote: >> >>> Dear Grigory, >>> >>> Thank you! >>> >>> So now it is clear that the xmipp centering doesnot preserve the >>> required info for Bayesian polishing - micName. >>> Maybe it can be corrected by Scipion/xmipp team sometime? >>> >>> I have a question - what do you mean by >>> >>> 1) you run only global mc without local patch-based alignment >>>> >>> >>> Usually I run relion - motioncor protocol for the motion alignment, and >>> continue further with the image processing. >>> >>> Am I doing something wrong? >>> >>> What could you recommend to run inorder to correct for the motion? >>> >>> Thank you! >>> >>> Sincerely, >>> Dmitry >>> >>> On December 30, 2020 5:47:22 PM Grigory Sharov <sha...@gm...> >>> wrote: >>> >>>> Hi again, >>>> >>>> I think I found the culprit which is xmipp center particles protocol. >>>> It does not preserve coordinate micName. I will run polishing to confirm. >>>> >>>> PS. Some things I've noticed: >>>> 1) you run only global mc without local patch-based alignment >>>> 2) for all cryosparc protocols numbers of mpi/threads are irrelevant >>>> since they are not available in cryosparc for modification. So you could >>>> use always 1 mpi, 1 thread >>>> >>>> >>>> Best regards, >>>> Grigory >>>> >>>> >>>> -------------------------------------------------------------------------------- >>>> Grigory Sharov, Ph.D. >>>> >>>> MRC Laboratory of Molecular Biology, >>>> Francis Crick Avenue, >>>> Cambridge Biomedical Campus, >>>> Cambridge CB2 0QH, UK. >>>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>>> e-mail: gs...@mr... >>>> >>>> >>>> On Wed, Dec 30, 2020 at 2:20 PM Grigory Sharov < >>>> sha...@gm...> wrote: >>>> >>>>> Thanks! That's quite a workflow. I did a simple test (below) already >>>>> and it worked. So must be some other combination of protocols. I'll try to >>>>> follow your project with my test data. >>>>> [image: image.png] >>>>> Best regards, >>>>> Grigory >>>>> >>>>> >>>>> -------------------------------------------------------------------------------- >>>>> Grigory Sharov, Ph.D. >>>>> >>>>> MRC Laboratory of Molecular Biology, >>>>> Francis Crick Avenue, >>>>> Cambridge Biomedical Campus, >>>>> Cambridge CB2 0QH, UK. >>>>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>>>> e-mail: gs...@mr... >>>>> >>>>> >>>>> On Wed, Dec 30, 2020 at 1:59 PM Dmitry Semchonok <sem...@gm...> >>>>> wrote: >>>>> >>>>>> Dear Grigory, >>>>>> >>>>>> >>>>>> >>>>>> Please find the file in the attachment. >>>>>> >>>>>> >>>>>> >>>>>> I also progressed further, trying to re-extract the good particles >>>>>> and run the Bayesian polishing in relion protocols above only (not yet >>>>>> there). >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Dmitry >>>>>> >>>>>> >>>>>> >>>>>> *От: *Grigory Sharov <sha...@gm...> >>>>>> *Отправлено: *30 декабря 2020 г. в 14:51 >>>>>> *Кому: *Dmitry A. Semchonok <sem...@gm...> >>>>>> *Тема: *Re: [scipion-users] Bayesian polishing issue >>>>>> >>>>>> >>>>>> >>>>>> Sure, you click Ctrl+A to select all protocols, then press Export and >>>>>> choose a json file to save. The file will be very small so you can attach >>>>>> it to the email. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Grigory >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -------------------------------------------------------------------------------- >>>>>> >>>>>> Grigory Sharov, Ph.D. >>>>>> >>>>>> MRC Laboratory of Molecular Biology, >>>>>> Francis Crick Avenue, >>>>>> Cambridge Biomedical Campus, >>>>>> Cambridge CB2 0QH, UK. >>>>>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>>>>> >>>>>> e-mail: gs...@mr... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Dec 30, 2020 at 12:50 PM Dmitry A. Semchonok < >>>>>> sem...@gm...> wrote: >>>>>> >>>>>> Dear Grigory, >>>>>> >>>>>> >>>>>> >>>>>> I already thought about that, but I forgot how to do that. >>>>>> >>>>>> >>>>>> >>>>>> Could you please remind me how can I copy my workflow? >>>>>> >>>>>> >>>>>> >>>>>> Thank you, >>>>>> >>>>>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Dmitry >>>>>> >>>>>> >>>>>> >>>>>> On December 30, 2020 2:10:53 PM Grigory Sharov < >>>>>> sha...@gm...> wrote: >>>>>> >>>>>> Hi Dmitry, >>>>>> >>>>>> >>>>>> >>>>>> Could you possibly send me your workflow for the project? Since there >>>>>> are so many cryosparc protocols and I'm not familiar with the code it will >>>>>> take time to figure it out. >>>>>> >>>>>> >>>>>> >>>>>> Grigory >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 29, 2020, 14:54 Dmitry Semchonok <sem...@gm...> >>>>>> wrote: >>>>>> >>>>>> I run the motion correction with relion – motioncor >>>>>> >>>>>> Is there a way to do the cryosparc- motion correction inside the >>>>>> SCIPON3 (I was not aware) ? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Dmitry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *От: *Grigory Sharov <sha...@gm...> >>>>>> *Отправлено: *29 декабря 2020 г. в 16:45 >>>>>> *Кому: *Mailing list for Scipion users >>>>>> <sci...@li...> >>>>>> *Тема: *Re: [scipion-users] Bayesian polishing issue >>>>>> >>>>>> >>>>>> >>>>>> Oh, I see. You are probably one of the first users trying to combine >>>>>> cryosparc and relion protocols in scipion. I will investigate whether >>>>>> cryosparc plugin does not keep the original micrographName and get back to >>>>>> you. >>>>>> >>>>>> >>>>>> >>>>>> Did you run motion correction also in cryosparc or in relion? >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Grigory >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -------------------------------------------------------------------------------- >>>>>> >>>>>> Grigory Sharov, Ph.D. >>>>>> >>>>>> MRC Laboratory of Molecular Biology, >>>>>> Francis Crick Avenue, >>>>>> Cambridge Biomedical Campus, >>>>>> Cambridge CB2 0QH, UK. >>>>>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>>>>> >>>>>> e-mail: gs...@mr... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 29, 2020 at 2:36 PM Dmitry Semchonok <sem...@gm...> >>>>>> wrote: >>>>>> >>>>>> Dear Grigory, >>>>>> >>>>>> >>>>>> >>>>>> Thank you. >>>>>> >>>>>> >>>>>> >>>>>> That is quite interesting result. >>>>>> >>>>>> Could you please check the new Bayesian run (I copied the files and >>>>>> will send you with Wetransfer again) >>>>>> >>>>>> >>>>>> >>>>>> If the problem remains the same it means that the cryosparc 3D refine >>>>>> output can not be followed up with relion Bayesian polishing. >>>>>> >>>>>> >>>>>> >>>>>> Please see the project tree. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> What is your opinion? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Dmitry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Отправлено из приложения "Почта >>>>>> <https://go.microsoft.com/fwlink/?LinkId=550986>" для Windows 10 >>>>>> >>>>>> >>>>>> >>>>>> *От: *Grigory Sharov <sha...@gm...> >>>>>> *Отправлено: *29 декабря 2020 г. в 16:22 >>>>>> *Кому: *Mailing list for Scipion users >>>>>> <sci...@li...> >>>>>> *Тема: *Re: [scipion-users] Bayesian polishing issue >>>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> Thank you for sending over the files. rlnMicrographName in the >>>>>> input_particles.star and input_corrected_micrographs.star are completely >>>>>> different. I do not know how you have generated your particles, but you >>>>>> cannot run polishing with them. >>>>>> >>>>>> >>>>>> >>>>>> rlnMicrographName >>>>>> >>>>>> Best regards, >>>>>> Grigory >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -------------------------------------------------------------------------------- >>>>>> >>>>>> Grigory Sharov, Ph.D. >>>>>> >>>>>> MRC Laboratory of Molecular Biology, >>>>>> Francis Crick Avenue, >>>>>> Cambridge Biomedical Campus, >>>>>> Cambridge CB2 0QH, UK. >>>>>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>>>>> >>>>>> e-mail: gs...@mr... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 29, 2020 at 11:32 AM Grigory Sharov < >>>>>> sha...@gm...> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> You forgot to attach Runs/004138_ProtRelionBayesianPolishing/input_particles.star >>>>>> file >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 29, 2020, 11:26 Dmitry Semchonok <sem...@gm...> >>>>>> wrote: >>>>>> >>>>>> Dear Grigory, >>>>>> >>>>>> >>>>>> >>>>>> Please find the file attached >>>>>> >>>>>> >>>>>> >>>>>> Thank you! >>>>>> >>>>>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Dmitry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov < >>>>>> sha...@gm...> wrote: >>>>>> >>>>>> Hi Dmitry, >>>>>> >>>>>> >>>>>> >>>>>> this usually happens if you have moved your raw movie files from >>>>>> where Scipion expects them to be. >>>>>> >>>>>> >>>>>> >>>>>> Could you please send me >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star and >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>>>> files? >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Grigory >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -------------------------------------------------------------------------------- >>>>>> >>>>>> Grigory Sharov, Ph.D. >>>>>> >>>>>> MRC Laboratory of Molecular Biology, >>>>>> Francis Crick Avenue, >>>>>> Cambridge Biomedical Campus, >>>>>> Cambridge CB2 0QH, UK. >>>>>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>>>>> >>>>>> e-mail: gs...@mr... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok < >>>>>> sem...@gm...> wrote: >>>>>> >>>>>> Dear colleagues, >>>>>> >>>>>> >>>>>> >>>>>> *Bayesian polishing issue –* >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 00008: workingDir: Runs/004138_ProtRelionBayesianPolishing >>>>>> >>>>>> 00009: runMode: Restart >>>>>> >>>>>> 00010: MPI: 1 >>>>>> >>>>>> 00011: threads: 14 >>>>>> >>>>>> 00012: Starting at step: 1 >>>>>> >>>>>> 00013: Running steps >>>>>> >>>>>> 00014: STARTED: convertInputStep, step 1, time 2020-12-28 >>>>>> 23:03:24.304754 >>>>>> >>>>>> 00015: Converting set from >>>>>> 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into >>>>>> 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star' >>>>>> >>>>>> 00016: convertBinaryFiles: creating soft links. >>>>>> >>>>>> 00017: Root: >>>>>> Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> >>>>>> Runs/000427_ProtRelionExtractParticles/extra >>>>>> >>>>>> 00018: FINISHED: convertInputStep, step 1, time 2020-12-28 >>>>>> 23:04:33.416996 >>>>>> >>>>>> 00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 >>>>>> 23:04:33.422574 >>>>>> >>>>>> 00020: relion_motion_refine --i >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>>>>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>>>>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>>>>> 0.59200 --corr_mic >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>>>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>>>>> --align_frac 0.500 --params3 --j 14 >>>>>> >>>>>> 00021: + Reading >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star.. >>>>>> >>>>>> 00022: in: >>>>>> /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, >>>>>> line 261 >>>>>> >>>>>> 00023: ERROR: >>>>>> >>>>>> 00024: There is no movie metadata STAR file for any micrographs! >>>>>> >>>>>> 00025: === Backtrace === >>>>>> >>>>>> 00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) >>>>>> [0x453571] >>>>>> >>>>>> 00027: >>>>>> relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) >>>>>> [0x531275] >>>>>> >>>>>> 00028: >>>>>> relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) >>>>>> [0x533a88] >>>>>> >>>>>> 00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) >>>>>> [0x47d1a6] >>>>>> >>>>>> 00030: relion_motion_refine(main+0x31) [0x428841] >>>>>> >>>>>> 00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555] >>>>>> >>>>>> 00032: relion_motion_refine() [0x42955f] >>>>>> >>>>>> 00033: ================== >>>>>> >>>>>> 00034: ERROR: >>>>>> >>>>>> 00035: There is no movie metadata STAR file for any micrographs! >>>>>> >>>>>> 00036: Traceback (most recent call last): >>>>>> >>>>>> 00037: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>>>> line 189, in run >>>>>> >>>>>> 00038: self._run() >>>>>> >>>>>> 00039: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>>>> line 240, in _run >>>>>> >>>>>> 00040: resultFiles = self._runFunc() >>>>>> >>>>>> 00041: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>>>> line 236, in _runFunc >>>>>> >>>>>> 00042: return self._func(*self._args) >>>>>> >>>>>> 00043: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", >>>>>> line 309, in trainOrPolishStep >>>>>> >>>>>> 00044: self.runJob(prog, args) >>>>>> >>>>>> 00045: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>>>> line 1357, in runJob >>>>>> >>>>>> 00046: self._stepsExecutor.runJob(self._log, program, >>>>>> arguments, **kwargs) >>>>>> >>>>>> 00047: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", >>>>>> line 65, in runJob >>>>>> >>>>>> 00048: process.runJob(log, programName, params, >>>>>> >>>>>> 00049: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >>>>>> line 51, in runJob >>>>>> >>>>>> 00050: return runCommand(command, env, cwd) >>>>>> >>>>>> 00051: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >>>>>> line 66, in runCommand >>>>>> >>>>>> 00052: check_call(command, shell=True, stdout=sys.stdout, >>>>>> stderr=sys.stderr, >>>>>> >>>>>> 00053: File >>>>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line >>>>>> 364, in check_call >>>>>> >>>>>> 00054: raise CalledProcessError(retcode, cmd) >>>>>> >>>>>> 00055: subprocess.CalledProcessError: Command 'relion_motion_refine >>>>>> --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>>>>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>>>>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>>>>> 0.59200 --corr_mic >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>>>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>>>>> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >>>>>> >>>>>> 00056: Protocol failed: Command 'relion_motion_refine --i >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>>>>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>>>>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>>>>> 0.59200 --corr_mic >>>>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>>>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>>>>> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >>>>>> >>>>>> 00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 >>>>>> 23:04:45.953315 >>>>>> >>>>>> 00058: *** Last status is failed >>>>>> >>>>>> 00059: ------------------- PROTOCOL FAILED (DONE 2/2) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Any ideas? >>>>>> >>>>>> Thank you! >>>>>> >>>>>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Dmitry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> scipion-users mailing list >>>>>> sci...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>>>>> >>>>>> _______________________________________________ >>>>>> scipion-users mailing list >>>>>> sci...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>>>>> >>>>>> _______________________________________________ >>>>>> scipion-users mailing list >>>>>> sci...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> scipion-users mailing list >>>>>> sci...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>> > > > |
|
From: Grigory S. <sha...@gm...> - 2020-12-30 08:50:27
|
Hi Yangyang, the line 00016: GHOST in place, read call ignored means that you don't have xmipp installed which is still required for all basic operations Grigory . On Wed, Dec 30, 2020, 08:44 Yangyang Yi <yy...@si...> wrote: > Hi, > Happy Holiday! > Thanks for the help in last email about Scipion-2.0 cluster config. I > have solved the problem, since I used “Use queue for jobs:Y” instead of > "N", and some plugins only download partly. I edited those parameter and > reinstalled some of the plugin, now it works. > However, our users now prefer scipion-3.0 and I found new errors with > similar settings. Now all the motion correction job works and result files > in extra directory (micrographs and shifts) and generated, however, I > failed to run CTF after motion correction. I have imported some motion > correction micrographs but still failed to run CTF (both Gctf and CTFFIND). > The error is similar as below: > > CTFFIND: > 00002: Hostname: gpu04 > 00003: PID: 138990 > 00004: pyworkflow: 3.0.8 > 00005: plugin: cistem > 00006: plugin v: 3.0.9 > 00007: currentDir: > /share/home/biotest/stu01/ScipionUserData/projects/test > 00008: workingDir: Runs/001767_CistemProtCTFFind > 00009: runMode: Restart > 00010: MPI: 1 > 00011: threads: 1 > 00012: Starting at step: 1 > 00013: Running steps > 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.887435 > 00015: Estimating CTF of micrograph: 1 > 00016: GHOST in place, read call ignored!. > 00017: Traceback (most recent call last): > 00018: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 189, in run > 00019: self._run() > 00020: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 240, in _run > 00021: resultFiles = self._runFunc() > 00022: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 236, in _runFunc > 00023: return self._func(*self._args) > 00024: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_micrographs.py", > line 247, in estimateCtfStep > 00025: self._estimateCTF(mic, *args) > 00026: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py", > line 108, in _estimateCTF > 00027: self._doCtfEstimation(mic) > 00028: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py", > line 78, in _doCtfEstimation > 00029: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT) > 00030: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py", > line 170, in convert > 00031: self._img.write(outputLoc) > 00032: AttributeError: 'Image' object has no attribute 'write' > 00033: Protocol failed: 'Image' object has no attribute 'write' > 00034: FAILED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.909908 > > 00002: Hostname: gpu08 > 00003: PID: 101015 > 00004: pyworkflow: 3.0.8 > 00005: plugin: gctf > 00006: plugin v: 3.0.11 > 00007: currentDir: > /share/home/biotest/stu01/ScipionUserData/projects/test > 00008: workingDir: Runs/001508_ProtGctf > 00009: runMode: Restart > 00010: MPI: 1 > 00011: threads: 1 > 00012: Starting at step: 1 > 00013: Running steps > 00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 16:20:37.950836 > 00015: Estimating CTF of micrograph: 1 > 00016: GHOST in place, read call ignored!. > 00017: ERROR: Gctf has failed on Runs/001508_ProtGctf/tmp/mic_0001/*.mrc > 00018: Traceback (most recent call last): > 00019: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/gctf/protocols/protocol_gctf.py", > line 82, in _estimateCtfList > 00020: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT) > 00021: File > "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py", > line 170, in convert > 00022: self._img.write(outputLoc) > 00023: AttributeError: 'Image' object has no attribute 'write' > 00024: FINISHED: estimateCtfStep, step 1, time 2020-12-30 16:20:38.060255 > > Is there any suggestions? Thanks! > > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Yangyang Yi <yy...@si...> - 2020-12-30 08:44:02
|
Hi,
Happy Holiday!
Thanks for the help in last email about Scipion-2.0 cluster config. I have solved the problem, since I used “Use queue for jobs:Y” instead of "N", and some plugins only download partly. I edited those parameter and reinstalled some of the plugin, now it works.
However, our users now prefer scipion-3.0 and I found new errors with similar settings. Now all the motion correction job works and result files in extra directory (micrographs and shifts) and generated, however, I failed to run CTF after motion correction. I have imported some motion correction micrographs but still failed to run CTF (both Gctf and CTFFIND). The error is similar as below:
CTFFIND:
00002: Hostname: gpu04
00003: PID: 138990
00004: pyworkflow: 3.0.8
00005: plugin: cistem
00006: plugin v: 3.0.9
00007: currentDir: /share/home/biotest/stu01/ScipionUserData/projects/test
00008: workingDir: Runs/001767_CistemProtCTFFind
00009: runMode: Restart
00010: MPI: 1
00011: threads: 1
00012: Starting at step: 1
00013: Running steps
00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.887435
00015: Estimating CTF of micrograph: 1
00016: GHOST in place, read call ignored!.
00017: Traceback (most recent call last):
00018: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 189, in run
00019: self._run()
00020: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 240, in _run
00021: resultFiles = self._runFunc()
00022: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 236, in _runFunc
00023: return self._func(*self._args)
00024: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_micrographs.py", line 247, in estimateCtfStep
00025: self._estimateCTF(mic, *args)
00026: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py", line 108, in _estimateCTF
00027: self._doCtfEstimation(mic)
00028: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/cistem/protocols/protocol_ctffind.py", line 78, in _doCtfEstimation
00029: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
00030: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py", line 170, in convert
00031: self._img.write(outputLoc)
00032: AttributeError: 'Image' object has no attribute 'write'
00033: Protocol failed: 'Image' object has no attribute 'write'
00034: FAILED: estimateCtfStep, step 1, time 2020-12-30 15:35:57.909908
00002: Hostname: gpu08
00003: PID: 101015
00004: pyworkflow: 3.0.8
00005: plugin: gctf
00006: plugin v: 3.0.11
00007: currentDir: /share/home/biotest/stu01/ScipionUserData/projects/test
00008: workingDir: Runs/001508_ProtGctf
00009: runMode: Restart
00010: MPI: 1
00011: threads: 1
00012: Starting at step: 1
00013: Running steps
00014: STARTED: estimateCtfStep, step 1, time 2020-12-30 16:20:37.950836
00015: Estimating CTF of micrograph: 1
00016: GHOST in place, read call ignored!.
00017: ERROR: Gctf has failed on Runs/001508_ProtGctf/tmp/mic_0001/*.mrc
00018: Traceback (most recent call last):
00019: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/gctf/protocols/protocol_gctf.py", line 82, in _estimateCtfList
00020: ih.convert(micFn, micFnMrc, emlib.DT_FLOAT)
00021: File "/share/apps/software/anconda/envs/scipion3/lib/python3.8/site-packages/pwem/emlib/image/image_handler.py", line 170, in convert
00022: self._img.write(outputLoc)
00023: AttributeError: 'Image' object has no attribute 'write'
00024: FINISHED: estimateCtfStep, step 1, time 2020-12-30 16:20:38.060255
Is there any suggestions? Thanks!
|
|
From: Dmitry S. <sem...@gm...> - 2020-12-29 14:55:04
|
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} </style><![endif]--><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style></head><body lang=RU link=blue vlink="#954F72" style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span lang=EN-US>I run the motion correction with relion – motioncor<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Is there a way to do the cryosparc- motion correction inside the SCIPON3 (I was not aware) ?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><img width=1005 height=680 style='width:10.4687in;height:7.0833in' id="Рисунок_x0020_5" src="cid:image001.png@01D6DE03.1F46A540"> <span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US>Sincerely,</span></p><p class=MsoNormal><span lang=EN-US>Dmitry</span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><div style='mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='border:none;padding:0cm'><b>От: </b><a href="mailto:sha...@gm...">Grigory Sharov</a><br><b>Отправлено: </b>29 декабря 2020 г. в 16:45<br><b>Кому: </b><a href="mailto:sci...@li...">Mailing list for Scipion users</a><br><b>Тема: </b>Re: [scipion-users] Bayesian polishing issue</p></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>Oh, I see. You are probably one of the first users trying to combine cryosparc and relion protocols in scipion. I will investigate whether cryosparc plugin does not keep the original micrographName and get back to you.</p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Did you run motion correction also in cryosparc or in relion?</p></div><div><p class=MsoNormal><br clear=all></p><div><div><div><div><div><div><div><div><div><div><div><div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>Best regards,<br>Grigory</span><span style='color:black'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>--------------------------------------------------------------------------------</span><span style='color:black'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;color:black'>Grigory Sharov, Ph.D.<br><br>MRC Laboratory of Molecular Biology,<br>Francis Crick Avenue,<br>Cambridge Biomedical Campus,<br>Cambridge CB2 0QH, UK.<br>tel. </span><span style='color:black'><a href="tel:+44%201223%20267228" target="_blank"><span style='font-size:9.5pt;color:#1155CC'>+44 (0) 1223 267228</span></a><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;color:black'>e-mail: </span><span style='color:black'><a href="mailto:gs...@mr..." target="_blank"><span style='font-size:9.5pt;color:#1155CC'>gs...@mr...</span></a><o:p></o:p></span></p></div></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Tue, Dec 29, 2020 at 2:36 PM Dmitry Semchonok <<a href="mailto:sem...@gm...">sem...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Dear Grigory,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Thank you.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>That is quite interesting result.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Could you please check the new Bayesian run (I copied the files and will send you with Wetransfer again)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>If the problem remains the same it means that the cryosparc 3D refine output can not be followed up with relion Bayesian polishing.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Please see the project tree.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><img border=0 width=998 height=663 style='width:10.3958in;height:6.9062in' id="gmail-m_6268838057735507349Рисунок_x0020_4" src="cid:176aef393adf07cdd741"></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>What is your opinion?</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Sincerely,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Dmitry</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Отправлено из приложения "<a href="https://go.microsoft.com/fwlink/?LinkId=550986" target="_blank">Почта</a>" для Windows 10</p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b>От: </b><a href="mailto:sha...@gm..." target="_blank">Grigory Sharov</a><br><b>Отправлено: </b>29 декабря 2020 г. в 16:22<br><b>Кому: </b><a href="mailto:sci...@li..." target="_blank">Mailing list for Scipion users</a><br><b>Тема: </b>Re: [scipion-users] Bayesian polishing issue</p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi,</p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thank you for sending over the files. rlnMicrographName in the input_particles.star and input_corrected_micrographs.star are completely different. I do not know how you have generated your particles, but you cannot run polishing with them.</p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>rlnMicrographName<br clear=all></p><div><div><div><div><div><div><div><div><div><div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>Best regards,<br>Grigory</span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>--------------------------------------------------------------------------------</span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.5pt;color:black'>Grigory Sharov, Ph.D.<br><br>MRC Laboratory of Molecular Biology,<br>Francis Crick Avenue,<br>Cambridge Biomedical Campus,<br>Cambridge CB2 0QH, UK.<br>tel. </span><span style='color:black'><a href="tel:+44%201223%20267228" target="_blank"><span style='font-size:9.5pt;color:#1155CC'>+44 (0) 1223 267228</span></a></span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.5pt;color:black'>e-mail: </span><span style='color:black'><a href="mailto:gs...@mr..." target="_blank"><span style='font-size:9.5pt;color:#1155CC'>gs...@mr...</span></a></span></p></div></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On Tue, Dec 29, 2020 at 11:32 AM Grigory Sharov <<a href="mailto:sha...@gm..." target="_blank">sha...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi,</p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>You forgot to attach <span style='font-size:9.5pt;font-family:"Arial",sans-serif'>Runs/004138_ProtRelionBayesianPolishing/input_particles.star file </span></p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On Tue, Dec 29, 2020, 11:26 Dmitry Semchonok <<a href="mailto:sem...@gm..." target="_blank">sem...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Dear Grigory,</p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Please find the file attached</p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thank you!</p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Sincerely,</p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Dmitry</p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov <<a href="mailto:sha...@gm..." target="_blank">sha...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi Dmitry,</p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>this usually happens if you have moved your raw movie files from where Scipion expects them to be. </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Could you please send me Runs/004138_ProtRelionBayesianPolishing/input_particles.star and Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star files?</p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br clear=all></p><div><div><div><div><div><div><div><div><div><div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>Best regards,<br>Grigory</span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>--------------------------------------------------------------------------------</span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.5pt;color:black'>Grigory Sharov, Ph.D.<br><br>MRC Laboratory of Molecular Biology,<br>Francis Crick Avenue,<br>Cambridge Biomedical Campus,<br>Cambridge CB2 0QH, UK.<br>tel. </span><span style='color:black'><a href="tel:+44%201223%20267228" target="_blank"><span style='font-size:9.5pt;color:#1155CC'>+44 (0) 1223 267228</span></a></span></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.5pt;color:black'>e-mail: </span><span style='color:black'><a href="mailto:gs...@mr..." target="_blank"><span style='font-size:9.5pt;color:#1155CC'>gs...@mr...</span></a></span></p></div></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <<a href="mailto:sem...@gm..." target="_blank">sem...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Dear colleagues,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><u><span lang=EN-US>Bayesian polishing issue –</span></u></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00008: workingDir: Runs/004138_ProtRelionBayesianPolishing</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00009: runMode: Restart</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00010: MPI: 1</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00011: threads: 14</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00012: Starting at step: 1</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00013: Running steps </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00014: STARTED: convertInputStep, step 1, time 2020-12-28 23:03:24.304754</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00015: Converting set from 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star'</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00016: convertBinaryFiles: creating soft links.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00017: Root: Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> Runs/000427_ProtRelionExtractParticles/extra</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00018: FINISHED: convertInputStep, step 1, time 2020-12-28 23:04:33.416996</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 23:04:33.422574</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00020: relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00021: + Reading Runs/004138_ProtRelionBayesianPolishing/input_particles.star..</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00022: in: /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, line 261</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00023: ERROR: </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00024: There is no movie metadata STAR file for any micrographs!</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00025: === Backtrace ===</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) [0x453571]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00027: relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) [0x531275]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00028: relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) [0x533a88]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00030: relion_motion_refine(main+0x31) [0x428841]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00032: relion_motion_refine() [0x42955f]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00033: ==================</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00034: ERROR: </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00035: There is no movie metadata STAR file for any micrographs!</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00036: Traceback (most recent call last):</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00037: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 189, in run</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00038: self._run()</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00039: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 240, in _run</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00040: resultFiles = self._runFunc()</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00041: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 236, in _runFunc</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00042: return self._func(*self._args)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00043: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", line 309, in trainOrPolishStep</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00044: self.runJob(prog, args)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00045: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 1357, in runJob</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00046: self._stepsExecutor.runJob(self._log, program, arguments, **kwargs)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00047: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 65, in runJob</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00048: process.runJob(log, programName, params,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00049: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 51, in runJob</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00050: return runCommand(command, env, cwd)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00051: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 66, in runCommand</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00052: check_call(command, shell=True, stdout=sys.stdout, stderr=sys.stderr,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00053: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line 364, in check_call</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00054: raise CalledProcessError(retcode, cmd)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00055: subprocess.CalledProcessError: Command 'relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00056: Protocol failed: Command 'relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 23:04:45.953315</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00058: *** Last status is failed </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00059: ------------------- PROTOCOL FAILED (DONE 2/2)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><img border=0 width=226 height=1 style='width:2.3541in;height:.0104in' id="gmail-m_6268838057735507349Horizontal_x0020_Line_x0020_1" src="cid:176aef393ad13dbd25e2"></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Any ideas?</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Thank you!</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Sincerely,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Dmitry</span></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p></blockquote></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p></blockquote></div></blockquote></div></blockquote></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:9.6pt'>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div></div></blockquote></div><p class=MsoNormal style='margin-left:4.8pt'>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p><p class=MsoNormal><o:p> </o:p></p></div></body></html> |
|
From: Grigory S. <sha...@gm...> - 2020-12-29 14:44:57
|
Oh, I see. You are probably one of the first users trying to combine cryosparc and relion protocols in scipion. I will investigate whether cryosparc plugin does not keep the original micrographName and get back to you. Did you run motion correction also in cryosparc or in relion? Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Tue, Dec 29, 2020 at 2:36 PM Dmitry Semchonok <sem...@gm...> wrote: > Dear Grigory, > > > > Thank you. > > > > That is quite interesting result. > > Could you please check the new Bayesian run (I copied the files and will > send you with Wetransfer again) > > > > If the problem remains the same it means that the cryosparc 3D refine > output can not be followed up with relion Bayesian polishing. > > > > Please see the project tree. > > > > > > > > What is your opinion? > > > > > > > > > > Sincerely, > > Dmitry > > > > > > > > > > > > > > Отправлено из приложения "Почта > <https://go.microsoft.com/fwlink/?LinkId=550986>" для Windows 10 > > > > *От: *Grigory Sharov <sha...@gm...> > *Отправлено: *29 декабря 2020 г. в 16:22 > *Кому: *Mailing list for Scipion users > <sci...@li...> > *Тема: *Re: [scipion-users] Bayesian polishing issue > > > > Hi, > > > > Thank you for sending over the files. rlnMicrographName in the > input_particles.star and input_corrected_micrographs.star are completely > different. I do not know how you have generated your particles, but you > cannot run polishing with them. > > > > rlnMicrographName > > Best regards, > Grigory > > > > > -------------------------------------------------------------------------------- > > Grigory Sharov, Ph.D. > > MRC Laboratory of Molecular Biology, > Francis Crick Avenue, > Cambridge Biomedical Campus, > Cambridge CB2 0QH, UK. > tel. +44 (0) 1223 267228 <+44%201223%20267228> > > e-mail: gs...@mr... > > > > > > On Tue, Dec 29, 2020 at 11:32 AM Grigory Sharov <sha...@gm...> > wrote: > > Hi, > > > > You forgot to attach Runs/004138_ProtRelionBayesianPolishing/input_particles.star > file > > > > On Tue, Dec 29, 2020, 11:26 Dmitry Semchonok <sem...@gm...> wrote: > > Dear Grigory, > > > > Please find the file attached > > > > Thank you! > > > > Sincerely, > > Dmitry > > > > > > On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov <sha...@gm...> > wrote: > > Hi Dmitry, > > > > this usually happens if you have moved your raw movie files from where > Scipion expects them to be. > > > > Could you please send me > Runs/004138_ProtRelionBayesianPolishing/input_particles.star and > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > files? > > > Best regards, > Grigory > > > > > -------------------------------------------------------------------------------- > > Grigory Sharov, Ph.D. > > MRC Laboratory of Molecular Biology, > Francis Crick Avenue, > Cambridge Biomedical Campus, > Cambridge CB2 0QH, UK. > tel. +44 (0) 1223 267228 <+44%201223%20267228> > > e-mail: gs...@mr... > > > > > > On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <sem...@gm...> > wrote: > > Dear colleagues, > > > > *Bayesian polishing issue –* > > > > > > 00008: workingDir: Runs/004138_ProtRelionBayesianPolishing > > 00009: runMode: Restart > > 00010: MPI: 1 > > 00011: threads: 14 > > 00012: Starting at step: 1 > > 00013: Running steps > > 00014: STARTED: convertInputStep, step 1, time 2020-12-28 23:03:24.304754 > > 00015: Converting set from > 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into > 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star' > > 00016: convertBinaryFiles: creating soft links. > > 00017: Root: > Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> > Runs/000427_ProtRelionExtractParticles/extra > > 00018: FINISHED: convertInputStep, step 1, time 2020-12-28 > 23:04:33.416996 > > 00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 > 23:04:33.422574 > > 00020: relion_motion_refine --i > Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o > Runs/004138_ProtRelionBayesianPolishing/extra --f > Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 > > 00021: + Reading > Runs/004138_ProtRelionBayesianPolishing/input_particles.star.. > > 00022: in: > /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, > line 261 > > 00023: ERROR: > > 00024: There is no movie metadata STAR file for any micrographs! > > 00025: === Backtrace === > > 00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) [0x453571] > > 00027: > relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) > [0x531275] > > 00028: > relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) > [0x533a88] > > 00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6] > > 00030: relion_motion_refine(main+0x31) [0x428841] > > 00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555] > > 00032: relion_motion_refine() [0x42955f] > > 00033: ================== > > 00034: ERROR: > > 00035: There is no movie metadata STAR file for any micrographs! > > 00036: Traceback (most recent call last): > > 00037: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 189, in run > > 00038: self._run() > > 00039: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 240, in _run > > 00040: resultFiles = self._runFunc() > > 00041: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 236, in _runFunc > > 00042: return self._func(*self._args) > > 00043: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", > line 309, in trainOrPolishStep > > 00044: self.runJob(prog, args) > > 00045: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 1357, in runJob > > 00046: self._stepsExecutor.runJob(self._log, program, arguments, > **kwargs) > > 00047: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", > line 65, in runJob > > 00048: process.runJob(log, programName, params, > > 00049: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 51, in runJob > > 00050: return runCommand(command, env, cwd) > > 00051: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 66, in runCommand > > 00052: check_call(command, shell=True, stdout=sys.stdout, > stderr=sys.stderr, > > 00053: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line > 364, in check_call > > 00054: raise CalledProcessError(retcode, cmd) > > 00055: subprocess.CalledProcessError: Command 'relion_motion_refine --i > Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o > Runs/004138_ProtRelionBayesianPolishing/extra --f > Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. > > 00056: Protocol failed: Command 'relion_motion_refine --i > Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o > Runs/004138_ProtRelionBayesianPolishing/extra --f > Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. > > 00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 23:04:45.953315 > > 00058: *** Last status is failed > > 00059: ------------------- PROTOCOL FAILED (DONE 2/2) > > > > > > > > Any ideas? > > Thank you! > > > > Sincerely, > > Dmitry > > > > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Dmitry S. <sem...@gm...> - 2020-12-29 14:35:56
|
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} </style><![endif]--><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style></head><body lang=RU link=blue vlink="#954F72" style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Dear Grigory,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thank you.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>That is quite interesting result.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Could you please check the new Bayesian run (I copied the files and will send you with Wetransfer again)<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US>If the problem remains the same it means that the cryosparc 3D refine output can not be followed up with relion Bayesian polishing.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Please see the project tree.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><img width=998 height=663 style='width:10.3958in;height:6.9062in' id="Рисунок_x0020_4" src="cid:image001.png@01D6DE00.93310D40"><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>What is your opinion?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Sincerely,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Dmitry<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Отправлено из приложения "<a href="https://go.microsoft.com/fwlink/?LinkId=550986">Почта</a>" для Windows 10</p><p class=MsoNormal><o:p> </o:p></p><div style='mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='border:none;padding:0cm'><b>От: </b><a href="mailto:sha...@gm...">Grigory Sharov</a><br><b>Отправлено: </b>29 декабря 2020 г. в 16:22<br><b>Кому: </b><a href="mailto:sci...@li...">Mailing list for Scipion users</a><br><b>Тема: </b>Re: [scipion-users] Bayesian polishing issue</p></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>Hi,</p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thank you for sending over the files. rlnMicrographName in the input_particles.star and input_corrected_micrographs.star are completely different. I do not know how you have generated your particles, but you cannot run polishing with them.</p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>rlnMicrographName<br clear=all></p><div><div><div><div><div><div><div><div><div><div><div><div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>Best regards,<br>Grigory</span><span style='color:black'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>--------------------------------------------------------------------------------</span><span style='color:black'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;color:black'>Grigory Sharov, Ph.D.<br><br>MRC Laboratory of Molecular Biology,<br>Francis Crick Avenue,<br>Cambridge Biomedical Campus,<br>Cambridge CB2 0QH, UK.<br>tel. </span><span style='color:black'><a href="tel:+44%201223%20267228" target="_blank"><span style='font-size:9.5pt;color:#1155CC'>+44 (0) 1223 267228</span></a><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;color:black'>e-mail: </span><span style='color:black'><a href="mailto:gs...@mr..." target="_blank"><span style='font-size:9.5pt;color:#1155CC'>gs...@mr...</span></a><o:p></o:p></span></p></div></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Tue, Dec 29, 2020 at 11:32 AM Grigory Sharov <<a href="mailto:sha...@gm...">sha...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><p class=MsoNormal>Hi,</p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>You forgot to attach <span style='font-size:9.5pt;font-family:"Arial",sans-serif'>Runs/004138_ProtRelionBayesianPolishing/input_particles.star file </span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Tue, Dec 29, 2020, 11:26 Dmitry Semchonok <<a href="mailto:sem...@gm..." target="_blank">sem...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><p class=MsoNormal>Dear Grigory,</p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Please find the file attached</p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thank you!</p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Sincerely,</p></div><div><p class=MsoNormal>Dmitry</p></div><div><p class=MsoNormal><o:p> </o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov <<a href="mailto:sha...@gm..." target="_blank">sha...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><p class=MsoNormal>Hi Dmitry,</p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>this usually happens if you have moved your raw movie files from where Scipion expects them to be. </p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Could you please send me Runs/004138_ProtRelionBayesianPolishing/input_particles.star and Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star files?</p></div><div><p class=MsoNormal><br clear=all></p><div><div><div><div><div><div><div><div><div><div><div><div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>Best regards,<br>Grigory</span><span style='color:black'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial",sans-serif;color:black'>--------------------------------------------------------------------------------</span><span style='color:black'><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;color:black'>Grigory Sharov, Ph.D.<br><br>MRC Laboratory of Molecular Biology,<br>Francis Crick Avenue,<br>Cambridge Biomedical Campus,<br>Cambridge CB2 0QH, UK.<br>tel. </span><span style='color:black'><a href="tel:+44%201223%20267228" target="_blank"><span style='font-size:9.5pt;color:#1155CC'>+44 (0) 1223 267228</span></a><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;color:black'>e-mail: </span><span style='color:black'><a href="mailto:gs...@mr..." target="_blank"><span style='font-size:9.5pt;color:#1155CC'>gs...@mr...</span></a><o:p></o:p></span></p></div></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <<a href="mailto:sem...@gm..." target="_blank">sem...@gm...</a>> wrote:</p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Dear colleagues,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><u><span lang=EN-US>Bayesian polishing issue –</span></u></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00008: workingDir: Runs/004138_ProtRelionBayesianPolishing</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00009: runMode: Restart</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00010: MPI: 1</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00011: threads: 14</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00012: Starting at step: 1</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00013: Running steps </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00014: STARTED: convertInputStep, step 1, time 2020-12-28 23:03:24.304754</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00015: Converting set from 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star'</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00016: convertBinaryFiles: creating soft links.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00017: Root: Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> Runs/000427_ProtRelionExtractParticles/extra</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00018: FINISHED: convertInputStep, step 1, time 2020-12-28 23:04:33.416996</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 23:04:33.422574</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00020: relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00021: + Reading Runs/004138_ProtRelionBayesianPolishing/input_particles.star..</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00022: in: /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, line 261</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00023: ERROR: </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00024: There is no movie metadata STAR file for any micrographs!</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00025: === Backtrace ===</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) [0x453571]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00027: relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) [0x531275]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00028: relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) [0x533a88]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00030: relion_motion_refine(main+0x31) [0x428841]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00032: relion_motion_refine() [0x42955f]</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00033: ==================</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00034: ERROR: </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00035: There is no movie metadata STAR file for any micrographs!</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00036: Traceback (most recent call last):</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00037: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 189, in run</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00038: self._run()</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00039: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 240, in _run</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00040: resultFiles = self._runFunc()</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00041: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 236, in _runFunc</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00042: return self._func(*self._args)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00043: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", line 309, in trainOrPolishStep</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00044: self.runJob(prog, args)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00045: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 1357, in runJob</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00046: self._stepsExecutor.runJob(self._log, program, arguments, **kwargs)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00047: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 65, in runJob</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00048: process.runJob(log, programName, params,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00049: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 51, in runJob</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00050: return runCommand(command, env, cwd)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00051: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 66, in runCommand</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00052: check_call(command, shell=True, stdout=sys.stdout, stderr=sys.stderr,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00053: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line 364, in check_call</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00054: raise CalledProcessError(retcode, cmd)</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00055: subprocess.CalledProcessError: Command 'relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00056: Protocol failed: Command 'relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1.</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 23:04:45.953315</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00058: *** Last status is failed </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>00059: ------------------- PROTOCOL FAILED (DONE 2/2)</span></p><p class=MsoNormal><img border=0 width=226 height=1 style='width:2.3541in;height:.0104in' id="Horizontal_x0020_Line_x0020_1" src="cid:image003.png@01D6DE00.A342F950"></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Any ideas?</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Thank you!</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US> </span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Sincerely,</span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US>Dmitry</span></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> </p></div></div><p class=MsoNormal>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p></blockquote></div><p class=MsoNormal>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p></blockquote></div></blockquote></div></blockquote></div><p class=MsoNormal style='margin-left:9.6pt'>_______________________________________________<br>scipion-users mailing list<br><a href="mailto:sci...@li..." target="_blank">sci...@li...</a><br><a href="https://lists.sourceforge.net/lists/listinfo/scipion-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/scipion-users</a></p><p class=MsoNormal><o:p> </o:p></p></div></body></html> |
|
From: Grigory S. <sha...@gm...> - 2020-12-29 14:22:18
|
Hi, Thank you for sending over the files. rlnMicrographName in the input_particles.star and input_corrected_micrographs.star are completely different. I do not know how you have generated your particles, but you cannot run polishing with them. rlnMicrographName Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Tue, Dec 29, 2020 at 11:32 AM Grigory Sharov <sha...@gm...> wrote: > Hi, > > You forgot to attach Runs/004138_ProtRelionBayesianPolishing/input_particles.star > file > > On Tue, Dec 29, 2020, 11:26 Dmitry Semchonok <sem...@gm...> wrote: > >> Dear Grigory, >> >> Please find the file attached >> >> Thank you! >> >> Sincerely, >> Dmitry >> >> >> On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov <sha...@gm...> >> wrote: >> >>> Hi Dmitry, >>> >>> this usually happens if you have moved your raw movie files from where >>> Scipion expects them to be. >>> >>> Could you please send me >>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star and >>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>> files? >>> >>> Best regards, >>> Grigory >>> >>> >>> -------------------------------------------------------------------------------- >>> Grigory Sharov, Ph.D. >>> >>> MRC Laboratory of Molecular Biology, >>> Francis Crick Avenue, >>> Cambridge Biomedical Campus, >>> Cambridge CB2 0QH, UK. >>> tel. +44 (0) 1223 267228 <+44%201223%20267228> >>> e-mail: gs...@mr... >>> >>> >>> On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <sem...@gm...> >>> wrote: >>> >>>> Dear colleagues, >>>> >>>> >>>> >>>> *Bayesian polishing issue –* >>>> >>>> >>>> >>>> >>>> >>>> 00008: workingDir: Runs/004138_ProtRelionBayesianPolishing >>>> >>>> 00009: runMode: Restart >>>> >>>> 00010: MPI: 1 >>>> >>>> 00011: threads: 14 >>>> >>>> 00012: Starting at step: 1 >>>> >>>> 00013: Running steps >>>> >>>> 00014: STARTED: convertInputStep, step 1, time 2020-12-28 >>>> 23:03:24.304754 >>>> >>>> 00015: Converting set from >>>> 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into >>>> 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star' >>>> >>>> 00016: convertBinaryFiles: creating soft links. >>>> >>>> 00017: Root: >>>> Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> >>>> Runs/000427_ProtRelionExtractParticles/extra >>>> >>>> 00018: FINISHED: convertInputStep, step 1, time 2020-12-28 >>>> 23:04:33.416996 >>>> >>>> 00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 >>>> 23:04:33.422574 >>>> >>>> 00020: relion_motion_refine --i >>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>>> 0.59200 --corr_mic >>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>>> --align_frac 0.500 --params3 --j 14 >>>> >>>> 00021: + Reading >>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star.. >>>> >>>> 00022: in: >>>> /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, >>>> line 261 >>>> >>>> 00023: ERROR: >>>> >>>> 00024: There is no movie metadata STAR file for any micrographs! >>>> >>>> 00025: === Backtrace === >>>> >>>> 00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) >>>> [0x453571] >>>> >>>> 00027: >>>> relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) >>>> [0x531275] >>>> >>>> 00028: >>>> relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) >>>> [0x533a88] >>>> >>>> 00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) >>>> [0x47d1a6] >>>> >>>> 00030: relion_motion_refine(main+0x31) [0x428841] >>>> >>>> 00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555] >>>> >>>> 00032: relion_motion_refine() [0x42955f] >>>> >>>> 00033: ================== >>>> >>>> 00034: ERROR: >>>> >>>> 00035: There is no movie metadata STAR file for any micrographs! >>>> >>>> 00036: Traceback (most recent call last): >>>> >>>> 00037: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>> line 189, in run >>>> >>>> 00038: self._run() >>>> >>>> 00039: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>> line 240, in _run >>>> >>>> 00040: resultFiles = self._runFunc() >>>> >>>> 00041: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>> line 236, in _runFunc >>>> >>>> 00042: return self._func(*self._args) >>>> >>>> 00043: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", >>>> line 309, in trainOrPolishStep >>>> >>>> 00044: self.runJob(prog, args) >>>> >>>> 00045: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>>> line 1357, in runJob >>>> >>>> 00046: self._stepsExecutor.runJob(self._log, program, arguments, >>>> **kwargs) >>>> >>>> 00047: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", >>>> line 65, in runJob >>>> >>>> 00048: process.runJob(log, programName, params, >>>> >>>> 00049: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >>>> line 51, in runJob >>>> >>>> 00050: return runCommand(command, env, cwd) >>>> >>>> 00051: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >>>> line 66, in runCommand >>>> >>>> 00052: check_call(command, shell=True, stdout=sys.stdout, >>>> stderr=sys.stderr, >>>> >>>> 00053: File >>>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line >>>> 364, in check_call >>>> >>>> 00054: raise CalledProcessError(retcode, cmd) >>>> >>>> 00055: subprocess.CalledProcessError: Command 'relion_motion_refine >>>> --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>>> 0.59200 --corr_mic >>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>>> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >>>> >>>> 00056: Protocol failed: Command 'relion_motion_refine --i >>>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>>> 0.59200 --corr_mic >>>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>>> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >>>> >>>> 00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 >>>> 23:04:45.953315 >>>> >>>> 00058: *** Last status is failed >>>> >>>> 00059: ------------------- PROTOCOL FAILED (DONE 2/2) >>>> ------------------------------ >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Any ideas? >>>> >>>> Thank you! >>>> >>>> >>>> >>>> Sincerely, >>>> >>>> Dmitry >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> scipion-users mailing list >>>> sci...@li... >>>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>>> >>> _______________________________________________ >>> scipion-users mailing list >>> sci...@li... >>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>> >> _______________________________________________ >> scipion-users mailing list >> sci...@li... >> https://lists.sourceforge.net/lists/listinfo/scipion-users >> > |
|
From: Grigory S. <sha...@gm...> - 2020-12-29 11:32:47
|
Hi, You forgot to attach Runs/004138_ProtRelionBayesianPolishing/input_particles.star file On Tue, Dec 29, 2020, 11:26 Dmitry Semchonok <sem...@gm...> wrote: > Dear Grigory, > > Please find the file attached > > Thank you! > > Sincerely, > Dmitry > > > On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov <sha...@gm...> > wrote: > >> Hi Dmitry, >> >> this usually happens if you have moved your raw movie files from where >> Scipion expects them to be. >> >> Could you please send me >> Runs/004138_ProtRelionBayesianPolishing/input_particles.star and >> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >> files? >> >> Best regards, >> Grigory >> >> >> -------------------------------------------------------------------------------- >> Grigory Sharov, Ph.D. >> >> MRC Laboratory of Molecular Biology, >> Francis Crick Avenue, >> Cambridge Biomedical Campus, >> Cambridge CB2 0QH, UK. >> tel. +44 (0) 1223 267228 <+44%201223%20267228> >> e-mail: gs...@mr... >> >> >> On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <sem...@gm...> >> wrote: >> >>> Dear colleagues, >>> >>> >>> >>> *Bayesian polishing issue –* >>> >>> >>> >>> >>> >>> 00008: workingDir: Runs/004138_ProtRelionBayesianPolishing >>> >>> 00009: runMode: Restart >>> >>> 00010: MPI: 1 >>> >>> 00011: threads: 14 >>> >>> 00012: Starting at step: 1 >>> >>> 00013: Running steps >>> >>> 00014: STARTED: convertInputStep, step 1, time 2020-12-28 >>> 23:03:24.304754 >>> >>> 00015: Converting set from >>> 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into >>> 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star' >>> >>> 00016: convertBinaryFiles: creating soft links. >>> >>> 00017: Root: >>> Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> >>> Runs/000427_ProtRelionExtractParticles/extra >>> >>> 00018: FINISHED: convertInputStep, step 1, time 2020-12-28 >>> 23:04:33.416996 >>> >>> 00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 >>> 23:04:33.422574 >>> >>> 00020: relion_motion_refine --i >>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>> 0.59200 --corr_mic >>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>> --align_frac 0.500 --params3 --j 14 >>> >>> 00021: + Reading >>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star.. >>> >>> 00022: in: >>> /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, >>> line 261 >>> >>> 00023: ERROR: >>> >>> 00024: There is no movie metadata STAR file for any micrographs! >>> >>> 00025: === Backtrace === >>> >>> 00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) >>> [0x453571] >>> >>> 00027: >>> relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) >>> [0x531275] >>> >>> 00028: >>> relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) >>> [0x533a88] >>> >>> 00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6] >>> >>> 00030: relion_motion_refine(main+0x31) [0x428841] >>> >>> 00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555] >>> >>> 00032: relion_motion_refine() [0x42955f] >>> >>> 00033: ================== >>> >>> 00034: ERROR: >>> >>> 00035: There is no movie metadata STAR file for any micrographs! >>> >>> 00036: Traceback (most recent call last): >>> >>> 00037: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>> line 189, in run >>> >>> 00038: self._run() >>> >>> 00039: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>> line 240, in _run >>> >>> 00040: resultFiles = self._runFunc() >>> >>> 00041: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>> line 236, in _runFunc >>> >>> 00042: return self._func(*self._args) >>> >>> 00043: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", >>> line 309, in trainOrPolishStep >>> >>> 00044: self.runJob(prog, args) >>> >>> 00045: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >>> line 1357, in runJob >>> >>> 00046: self._stepsExecutor.runJob(self._log, program, arguments, >>> **kwargs) >>> >>> 00047: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", >>> line 65, in runJob >>> >>> 00048: process.runJob(log, programName, params, >>> >>> 00049: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >>> line 51, in runJob >>> >>> 00050: return runCommand(command, env, cwd) >>> >>> 00051: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >>> line 66, in runCommand >>> >>> 00052: check_call(command, shell=True, stdout=sys.stdout, >>> stderr=sys.stderr, >>> >>> 00053: File >>> "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line >>> 364, in check_call >>> >>> 00054: raise CalledProcessError(retcode, cmd) >>> >>> 00055: subprocess.CalledProcessError: Command 'relion_motion_refine >>> --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>> 0.59200 --corr_mic >>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >>> >>> 00056: Protocol failed: Command 'relion_motion_refine --i >>> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >>> Runs/004138_ProtRelionBayesianPolishing/extra --f >>> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >>> 0.59200 --corr_mic >>> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >>> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >>> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >>> >>> 00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 >>> 23:04:45.953315 >>> >>> 00058: *** Last status is failed >>> >>> 00059: ------------------- PROTOCOL FAILED (DONE 2/2) >>> ------------------------------ >>> >>> >>> >>> >>> >>> >>> >>> Any ideas? >>> >>> Thank you! >>> >>> >>> >>> Sincerely, >>> >>> Dmitry >>> >>> >>> >>> >>> _______________________________________________ >>> scipion-users mailing list >>> sci...@li... >>> https://lists.sourceforge.net/lists/listinfo/scipion-users >>> >> _______________________________________________ >> scipion-users mailing list >> sci...@li... >> https://lists.sourceforge.net/lists/listinfo/scipion-users >> > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Grigory S. <sha...@gm...> - 2020-12-29 11:31:04
|
Hi, I need more information. What command did you use to install relion and what is the full output? Grigory On Tue, Dec 29, 2020, 11:26 Eng. Adil <ala...@gm...> wrote: > Hi, > > Thanks in advance, recently I was using scipion and it does work > perfectly. I got some issues while I tried to install RELION it keeps > saying FAILED, any suggestions please ? > > I did follow the installation using ubuntu with conda > > Thanks, > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Dmitry S. <sem...@gm...> - 2020-12-29 09:54:35
|
Dear Grigory, Please find the file attached Thank you! Sincerely, Dmitry On Tue, Dec 29, 2020 at 12:28 AM Grigory Sharov <sha...@gm...> wrote: > Hi Dmitry, > > this usually happens if you have moved your raw movie files from where > Scipion expects them to be. > > Could you please send me > Runs/004138_ProtRelionBayesianPolishing/input_particles.star and > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > files? > > Best regards, > Grigory > > > -------------------------------------------------------------------------------- > Grigory Sharov, Ph.D. > > MRC Laboratory of Molecular Biology, > Francis Crick Avenue, > Cambridge Biomedical Campus, > Cambridge CB2 0QH, UK. > tel. +44 (0) 1223 267228 <+44%201223%20267228> > e-mail: gs...@mr... > > > On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <sem...@gm...> > wrote: > >> Dear colleagues, >> >> >> >> *Bayesian polishing issue –* >> >> >> >> >> >> 00008: workingDir: Runs/004138_ProtRelionBayesianPolishing >> >> 00009: runMode: Restart >> >> 00010: MPI: 1 >> >> 00011: threads: 14 >> >> 00012: Starting at step: 1 >> >> 00013: Running steps >> >> 00014: STARTED: convertInputStep, step 1, time 2020-12-28 >> 23:03:24.304754 >> >> 00015: Converting set from >> 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into >> 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star' >> >> 00016: convertBinaryFiles: creating soft links. >> >> 00017: Root: >> Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> >> Runs/000427_ProtRelionExtractParticles/extra >> >> 00018: FINISHED: convertInputStep, step 1, time 2020-12-28 >> 23:04:33.416996 >> >> 00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 >> 23:04:33.422574 >> >> 00020: relion_motion_refine --i >> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >> Runs/004138_ProtRelionBayesianPolishing/extra --f >> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >> 0.59200 --corr_mic >> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >> --align_frac 0.500 --params3 --j 14 >> >> 00021: + Reading >> Runs/004138_ProtRelionBayesianPolishing/input_particles.star.. >> >> 00022: in: >> /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, >> line 261 >> >> 00023: ERROR: >> >> 00024: There is no movie metadata STAR file for any micrographs! >> >> 00025: === Backtrace === >> >> 00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) [0x453571] >> >> 00027: >> relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) >> [0x531275] >> >> 00028: >> relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) >> [0x533a88] >> >> 00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6] >> >> 00030: relion_motion_refine(main+0x31) [0x428841] >> >> 00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555] >> >> 00032: relion_motion_refine() [0x42955f] >> >> 00033: ================== >> >> 00034: ERROR: >> >> 00035: There is no movie metadata STAR file for any micrographs! >> >> 00036: Traceback (most recent call last): >> >> 00037: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >> line 189, in run >> >> 00038: self._run() >> >> 00039: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >> line 240, in _run >> >> 00040: resultFiles = self._runFunc() >> >> 00041: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >> line 236, in _runFunc >> >> 00042: return self._func(*self._args) >> >> 00043: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", >> line 309, in trainOrPolishStep >> >> 00044: self.runJob(prog, args) >> >> 00045: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", >> line 1357, in runJob >> >> 00046: self._stepsExecutor.runJob(self._log, program, arguments, >> **kwargs) >> >> 00047: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", >> line 65, in runJob >> >> 00048: process.runJob(log, programName, params, >> >> 00049: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >> line 51, in runJob >> >> 00050: return runCommand(command, env, cwd) >> >> 00051: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", >> line 66, in runCommand >> >> 00052: check_call(command, shell=True, stdout=sys.stdout, >> stderr=sys.stderr, >> >> 00053: File >> "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line >> 364, in check_call >> >> 00054: raise CalledProcessError(retcode, cmd) >> >> 00055: subprocess.CalledProcessError: Command 'relion_motion_refine --i >> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >> Runs/004138_ProtRelionBayesianPolishing/extra --f >> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >> 0.59200 --corr_mic >> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >> >> 00056: Protocol failed: Command 'relion_motion_refine --i >> Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o >> Runs/004138_ProtRelionBayesianPolishing/extra --f >> Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref >> 0.59200 --corr_mic >> Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star >> --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 >> --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. >> >> 00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 >> 23:04:45.953315 >> >> 00058: *** Last status is failed >> >> 00059: ------------------- PROTOCOL FAILED (DONE 2/2) >> ------------------------------ >> >> >> >> >> >> >> >> Any ideas? >> >> Thank you! >> >> >> >> Sincerely, >> >> Dmitry >> >> >> >> >> _______________________________________________ >> scipion-users mailing list >> sci...@li... >> https://lists.sourceforge.net/lists/listinfo/scipion-users >> > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Grigory S. <sha...@gm...> - 2020-12-28 22:28:04
|
Hi Dmitry, this usually happens if you have moved your raw movie files from where Scipion expects them to be. Could you please send me Runs/004138_ProtRelionBayesianPolishing/input_particles.star and Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star files? Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Mon, Dec 28, 2020 at 10:16 PM Dmitry Semchonok <sem...@gm...> wrote: > Dear colleagues, > > > > *Bayesian polishing issue –* > > > > > > 00008: workingDir: Runs/004138_ProtRelionBayesianPolishing > > 00009: runMode: Restart > > 00010: MPI: 1 > > 00011: threads: 14 > > 00012: Starting at step: 1 > > 00013: Running steps > > 00014: STARTED: convertInputStep, step 1, time 2020-12-28 23:03:24.304754 > > 00015: Converting set from > 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into > 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star' > > 00016: convertBinaryFiles: creating soft links. > > 00017: Root: > Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> > Runs/000427_ProtRelionExtractParticles/extra > > 00018: FINISHED: convertInputStep, step 1, time 2020-12-28 > 23:04:33.416996 > > 00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 > 23:04:33.422574 > > 00020: relion_motion_refine --i > Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o > Runs/004138_ProtRelionBayesianPolishing/extra --f > Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 > > 00021: + Reading > Runs/004138_ProtRelionBayesianPolishing/input_particles.star.. > > 00022: in: > /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, > line 261 > > 00023: ERROR: > > 00024: There is no movie metadata STAR file for any micrographs! > > 00025: === Backtrace === > > 00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) [0x453571] > > 00027: > relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) > [0x531275] > > 00028: > relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) > [0x533a88] > > 00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6] > > 00030: relion_motion_refine(main+0x31) [0x428841] > > 00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555] > > 00032: relion_motion_refine() [0x42955f] > > 00033: ================== > > 00034: ERROR: > > 00035: There is no movie metadata STAR file for any micrographs! > > 00036: Traceback (most recent call last): > > 00037: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 189, in run > > 00038: self._run() > > 00039: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 240, in _run > > 00040: resultFiles = self._runFunc() > > 00041: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 236, in _runFunc > > 00042: return self._func(*self._args) > > 00043: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", > line 309, in trainOrPolishStep > > 00044: self.runJob(prog, args) > > 00045: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 1357, in runJob > > 00046: self._stepsExecutor.runJob(self._log, program, arguments, > **kwargs) > > 00047: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", > line 65, in runJob > > 00048: process.runJob(log, programName, params, > > 00049: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 51, in runJob > > 00050: return runCommand(command, env, cwd) > > 00051: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 66, in runCommand > > 00052: check_call(command, shell=True, stdout=sys.stdout, > stderr=sys.stderr, > > 00053: File > "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line > 364, in check_call > > 00054: raise CalledProcessError(retcode, cmd) > > 00055: subprocess.CalledProcessError: Command 'relion_motion_refine --i > Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o > Runs/004138_ProtRelionBayesianPolishing/extra --f > Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. > > 00056: Protocol failed: Command 'relion_motion_refine --i > Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o > Runs/004138_ProtRelionBayesianPolishing/extra --f > Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref > 0.59200 --corr_mic > Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star > --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 > --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1. > > 00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 23:04:45.953315 > > 00058: *** Last status is failed > > 00059: ------------------- PROTOCOL FAILED (DONE 2/2) > ------------------------------ > > > > > > > > Any ideas? > > Thank you! > > > > Sincerely, > > Dmitry > > > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
|
From: Dmitry S. <sem...@gm...> - 2020-12-28 22:16:34
|
<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style></head><body lang=RU link=blue vlink="#954F72"><div class=WordSection1><div><p class=MsoNormal><span lang=EN-US>Dear colleagues,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><u><span lang=EN-US>Bayesian polishing issue –<o:p></o:p></span></u></p><p class=MsoNormal><u><span lang=EN-US><o:p><span style='text-decoration:none'> </span></o:p></span></u></p><p class=MsoNormal><u><span lang=EN-US><o:p><span style='text-decoration:none'> </span></o:p></span></u></p><p class=MsoNormal><span lang=EN-US>00008: workingDir: Runs/004138_ProtRelionBayesianPolishing<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00009: runMode: Restart<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00010: MPI: 1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00011: threads: 14<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00012: Starting at step: 1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00013: Running steps <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00014: STARTED: convertInputStep, step 1, time 2020-12-28 23:03:24.304754<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00015: Converting set from 'Runs/003937_ProtRelionRefine3D/particles.sqlite' into 'Runs/004138_ProtRelionBayesianPolishing/input_particles.star'<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00016: convertBinaryFiles: creating soft links.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00017: Root: Runs/004138_ProtRelionBayesianPolishing/input/particles/input -> Runs/000427_ProtRelionExtractParticles/extra<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00018: FINISHED: convertInputStep, step 1, time 2020-12-28 23:04:33.416996<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00019: STARTED: trainOrPolishStep, step 2, time 2020-12-28 23:04:33.422574<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00020: relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00021: + Reading Runs/004138_ProtRelionBayesianPolishing/input_particles.star..<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00022: in: /home/user/Data/Software/Scipion3/software/em/relion-3.1.0/src/jaz/micrograph_handler.cpp, line 261<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00023: ERROR: <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00024: There is no movie metadata STAR file for any micrographs!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00025: === Backtrace ===<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00026: relion_motion_refine(_ZN11RelionErrorC1ERKSsS1_l+0x41) [0x453571]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00027: relion_motion_refine(_ZN17MicrographHandler11loadInitialERKSt6vectorI13MetaDataTableSaIS1_EEbRiRdRSs+0xb95) [0x531275]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00028: relion_motion_refine(_ZN17MicrographHandler4initERKSt6vectorI13MetaDataTableSaIS1_EEbiRiRdRSs+0x78) [0x533a88]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00029: relion_motion_refine(_ZN13MotionRefiner4initEv+0x546) [0x47d1a6]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00030: relion_motion_refine(main+0x31) [0x428841]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00031: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff26cfb7555]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00032: relion_motion_refine() [0x42955f]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00033: ==================<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00034: ERROR: <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00035: There is no movie metadata STAR file for any micrographs!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00036: Traceback (most recent call last):<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00037: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 189, in run<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00038: self._run()<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00039: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 240, in _run<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00040: resultFiles = self._runFunc()<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00041: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 236, in _runFunc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00042: return self._func(*self._args)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00043: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/relion/protocols/protocol_bayesian_polishing.py", line 309, in trainOrPolishStep<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00044: self.runJob(prog, args)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00045: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 1357, in runJob<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00046: self._stepsExecutor.runJob(self._log, program, arguments, **kwargs)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00047: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 65, in runJob<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00048: process.runJob(log, programName, params,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00049: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 51, in runJob<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00050: return runCommand(command, env, cwd)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00051: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 66, in runCommand<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00052: check_call(command, shell=True, stdout=sys.stdout, stderr=sys.stderr,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00053: File "/home/user/.conda/envs/.scipion3env/lib/python3.8/subprocess.py", line 364, in check_call<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00054: raise CalledProcessError(retcode, cmd)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00055: subprocess.CalledProcessError: Command 'relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00056: Protocol failed: Command 'relion_motion_refine --i Runs/004138_ProtRelionBayesianPolishing/input_particles.star --o Runs/004138_ProtRelionBayesianPolishing/extra --f Runs/004089_ProtRelionPostprocess/extra/postprocess.star --angpix_ref 0.59200 --corr_mic Runs/004138_ProtRelionBayesianPolishing/input_corrected_micrographs.star --first_frame 3 --last_frame 0 --min_p 421029 --eval_frac 0.500 --align_frac 0.500 --params3 --j 14 ' returned non-zero exit status 1.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00057: FAILED: trainOrPolishStep, step 2, time 2020-12-28 23:04:45.953315<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00058: *** Last status is failed <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>00059: ------------------- PROTOCOL FAILED (DONE 2/2)<hr align=left size=1 width="33%"><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Any ideas?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Thank you!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Sincerely,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Dmitry<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p></div></body></html> |
1 message has been excluded from this view by a project administrator.
