Menu
▾
▴
[scipion-users] Relion auto-picking job fails, looking for help
|
From: Anastasina, M. <mar...@he...> - 2022-01-17 14:19:38
|
Dear colleagues, Relion auto-picking jobs within Scipion 3.0.8 fail with the error message pasted below. I am picking particles using 2D averages as references. These averages come from a different dataset than the micrographs, and have been re-scaled to match the pixel size using crop/resize protocol in Scipion. I don’t understand what exactly is going wrong and would be very appreciative if any of you could suggest the reason for failure. Best regards, Maria 00001: RUNNING PROTOCOL ----------------- 00002: Hostname: r15g05.bullx 00003: PID: 135115 00004: pyworkflow: 3.0.16 00005: plugin: relion 00006: plugin v: 3.1.1 00007: currentDir: /scratch/project_2004278/ScipionUserData/projects/immTBEV 00008: workingDir: Runs/025798_ProtRelion2Autopick 00009: runMode: Continue 00010: MPI: 4 00011: threads: 1 00012: Starting at step: 1 00013: Running steps 00014: STARTED: convertInputStep, step 1, time 2022-01-17 15:15:42.193627 00015: FINISHED: convertInputStep, step 1, time 2022-01-17 15:15:42.565530 00016: STARTED: pickMicrographStep, step 2, time 2022-01-17 15:15:42.591236 00017: Picking micrograph: Runs/000799_ProtMotionCorr/extra/GridSquare_29251179_Data_FoilHole_30265827_Data_29254225_29254227_20210330_060030_fractions_aligned_mic_DW.mrc 00018: Some paths do not exist in: /usr/local/cuda/lib64 00019: srun `which relion_autopick_mpi` --pickname autopick --odir "./" --particle_diameter 666 --angpix 1.10410 --shrink 0.000 --gpu "" --ref /scratch/project_2004278/ScipionUserData/projects/immTBEV/Runs/025798_ProtRelion2Autopick/reference_2d.stk --angpix_ref 1.10000 --invert --ang 5 --lowpass 20 --i input_micrographs.star --threshold 0.250 --min_distance -1.000 --max_stddev_noise 1.100 --min_avg_noise -999.000 00020: RELION version: 3.1.1 00021: Precision: BASE=double, CUDA-ACC=single 00022: 00023: === RELION MPI setup === 00024: + Number of MPI processes = 5 00025: + Master (0) runs on host = r15g05.bullx 00026: ================= 00027: + Slave 1 runs on host = r15g05.bullx 00028: + Slave 2 runs on host = r15g05.bullx 00029: + Slave 3 runs on host = r15g05.bullx 00030: + Slave 4 runs on host = r15g05.bullx 00031: + Run autopicking on the following micrographs: 00032: * mic_000007.mrc 00033: WARNING!!! Pixel size in reference image header= 0 but you have provided --angpix_ref 1.1 00034: + WARNING: Requested rescale of micrographs is 636 pixels. The largest prime factor in FFTs is 53 00035: + WARNING: Will change rescaling of micrographs to 638 pixels, because the prime factor then becomes 29 00036: + WARNING: add --skip_optimise_scale to your autopick command to prevent rescaling 00037: Initialising FFTs for the references and masks ... 00038: 000/??? sec ~~(,_,"> [oo] 00039: 2/ 2 sec ............................................................~~(,_,"> 00040: + Using GPU device: 0 on MPI node: 0 00041: + Using GPU device: 1 on MPI node: 1 00042: + Using GPU device: 2 on MPI node: 2 00043: + Using GPU device: 3 on MPI node: 3 00044: + Using GPU device: 0 on MPI node: 4 00045: Autopicking ... 00046: 000/??? sec ~~(,_,"> [oo] 00047: 6/ 6 sec ............................................................~~(,_,"> 00048: Generating logfile.pdf ... 00049: 000/??? sec ~~(,_,"> [oo] 00050: 0/ 0 sec ............................................................~~(,_,"> 00051: Total number of particles from 1 micrographs is 0 00052: i.e. on average there were 0 particles per micrograph 00053: terminate called after throwing an instance of 'std::bad_alloc' 00054: what(): std::bad_alloc 00055: [r15g05:135359] *** Process received signal *** 00056: [r15g05:135359] Signal: Aborted (6) 00057: [r15g05:135359] Signal code: (-6) 00058: [r15g05:135359] [ 0] /lib64/libpthread.so.0(+0xf630)[0x7f0aad9e5630] 00059: [r15g05:135359] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x7f0aa8d15387] 00060: [r15g05:135359] [ 2] /lib64/libc.so.6(abort+0x148)[0x7f0aa8d16a78] 00061: [r15g05:135359] [ 3] /appl/spack/install-tree/gcc-4.8.5/gcc-8.3.0-qzmzn5/lib64/libstdc++.so.6(+0x94983)[0x7f0aa9443983] 00062: [r15g05:135359] [ 4] /appl/spack/install-tree/gcc-4.8.5/gcc-8.3.0-qzmzn5/lib64/libstdc++.so.6(+0x9aaf6)[0x7f0aa9449af6] 00063: [r15g05:135359] [ 5] /appl/spack/install-tree/gcc-4.8.5/gcc-8.3.0-qzmzn5/lib64/libstdc++.so.6(+0x9ab31)[0x7f0aa9449b31] 00064: [r15g05:135359] [ 6] /appl/spack/install-tree/gcc-4.8.5/gcc-8.3.0-qzmzn5/lib64/libstdc++.so.6(+0x9ad63)[0x7f0aa9449d63] 00065: [r15g05:135359] [ 7] /appl/spack/install-tree/gcc-4.8.5/gcc-8.3.0-qzmzn5/lib64/libstdc++.so.6(+0x9b22c)[0x7f0aa944a22c] 00066: [r15g05:135359] [ 8] /appl/soft/math/relion/relion-3.1.1-gpubuild/lib/librelion_lib.so(_ZNSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS5_EE17_M_realloc_insertIJS5_EEEvN9__gnu_cxx17__normal_iteratorIPS5_S7_EEDpOT_+0x57)[0x7f0ab6c64b87] 00067: [r15g05:135359] [ 9] /appl/soft/math/relion/relion-3.1.1-gpubuild/lib/librelion_lib.so(_ZN7CPlot2D23ComputeLabelTickSpacingEddPdS0_S0_iNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x315)[0x7f0ab6c626b5] 00068: [r15g05:135359] [10] /appl/soft/math/relion/relion-3.1.1-gpubuild/lib/librelion_lib.so(_ZN7CPlot2D20PrecomputeDimensionsEv+0x489)[0x7f0ab6c62eb9] 00069: [r15g05:135359] [11] /appl/soft/math/relion/relion-3.1.1-gpubuild/lib/librelion_lib.so(_ZN7CPlot2D20OutputPostScriptPlotENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x51)[0x7f0ab6c63441] 00070: [r15g05:135359] [12] /appl/soft/math/relion/relion-3.1.1-gpubuild/lib/librelion_lib.so(_ZN10AutoPicker18generatePDFLogfileEv+0x762)[0x7f0ab6c849a2] 00071: [r15g05:135359] [13] /appl/soft/math/scipion/3.0.7/software/em/relion-3.1.0/bin/relion_autopick_mpi(main+0x11c)[0x4024ec] 00072: [r15g05:135359] [14] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f0aa8d01555] 00073: [r15g05:135359] [15] /appl/soft/math/scipion/3.0.7/software/em/relion-3.1.0/bin/relion_autopick_mpi[0x4025dc] 00074: [r15g05:135359] *** End of error message *** 00075: srun: error: r15g05: task 0: Aborted 00076: srun: launch/slurm: _step_signal: Terminating StepId=10234377.0 00077: slurmstepd: error: *** STEP 10234377.0 ON r15g05 CANCELLED AT 2022-01-17T15:25:37 *** 00078: srun: error: r15g05: tasks 1-4: Terminated 00079: srun: Force Terminated StepId=10234377.0 00080: Traceback (most recent call last): 00081: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 197, in run 00082: self._run() 00083: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 248, in _run 00084: resultFiles = self._runFunc() 00085: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 244, in _runFunc 00086: return self._func(*self._args) 00087: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pwem/protocols/protocol_particles_picking.py", line 241, in pickMicrographStep 00088: self._pickMicrograph(mic, *args) 00089: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/relion/protocols/protocol_autopick.py", line 44, in _pickMicrograph 00090: self._pickMicrographList([mic], *args) 00091: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/relion/protocols/protocol_autopick.py", line 54, in _pickMicrographList 00092: self._pickMicrographsFromStar(micStar, micsDir, *args) 00093: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/relion/protocols/protocol_autopick_ref.py", line 401, in _pickMicrographsFromStar 00094: self.runJob(program, params, cwd=cwd) 00095: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", line 1389, in runJob 00096: self._stepsExecutor.runJob(self._log, program, arguments, **kwargs) 00097: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", line 65, in runJob 00098: process.runJob(log, programName, params, 00099: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 52, in runJob 00100: return runCommand(command, env, cwd) 00101: File "/appl/soft/math/scipion/3.0.7/.scipion3/lib/python3.8/site-packages/pyworkflow/utils/process.py", line 67, in runCommand 00102: check_call(command, shell=True, stdout=sys.stdout, stderr=sys.stderr, 00103: File "/appl/soft/math/scipion/3.8.7-gnu8.3.0/lib/python3.8/subprocess.py", line 364, in check_call 00104: raise CalledProcessError(retcode, cmd) 00105: subprocess.CalledProcessError: Command ' srun `which relion_autopick_mpi` --pickname autopick --odir "./" --particle_diameter 666 --angpix 1.10410 --shrink 0.000 --gpu "" --ref /scratch/project_2004278/ScipionUserData/projects/immTBEV/Runs/025798_ProtRelion2Autopick/reference_2d.stk --angpix_ref 1.10000 --invert --ang 5 --lowpass 20 --i input_micrographs.star --threshold 0.250 --min_distance -1.000 --max_stddev_noise 1.100 --min_avg_noise -999.000' returned non-zero exit status 143. 00106: Protocol failed: Command ' srun `which relion_autopick_mpi` --pickname autopick --odir "./" --particle_diameter 666 --angpix 1.10410 --shrink 0.000 --gpu "" --ref /scratch/project_2004278/ScipionUserData/projects/immTBEV/Runs/025798_ProtRelion2Autopick/reference_2d.stk --angpix_ref 1.10000 --invert --ang 5 --lowpass 20 --i input_micrographs.star --threshold 0.250 --min_distance -1.000 --max_stddev_noise 1.100 --min_avg_noise -999.000' returned non-zero exit status 143. 00107: FAILED: pickMicrographStep, step 2, time 2022-01-17 15:25:37.465641 00108: *** Last status is failed 00109: ------------------- PROTOCOL FAILED (DONE 2/2002) |
