Menu
▾
▴
Re: [scipion-users] Export particles from cryosparc
|
From: Grigory S. <sha...@gm...> - 2021-05-18 13:03:12
|
Hi Dmitry, as the error states your star file points to a non-existing image in the mrcs stack. You need to check first if you import from cryosparc with mrcs worked correctly (open / display particles) then trace all the steps you did before 2D classification. Best regards, Grigory -------------------------------------------------------------------------------- Grigory Sharov, Ph.D. MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge CB2 0QH, UK. tel. +44 (0) 1223 267228 <+44%201223%20267228> e-mail: gs...@mr... On Tue, May 18, 2021 at 1:25 PM Dmitry Semchonok <Sem...@gm...> wrote: > Dear Pablo, > > > Thank you. I heard about this option. For that I guess the > https://pypi.org/project/cs2star/ needs to be installed. > > > > > In Cryosparc itself there is an option to export files. And then what we > get is the mrc files with different number of particles in each. > > It appeared to be possible to rename mrc —> to —>mrcs. Then SCIPION can > import those particles. > > Currently the problem is that not relion nor cryosparc can run these > particles. > > > > > Relion stops with error: > > > 00001: RUNNING PROTOCOL ----------------- > 00002: Hostname: cryoem01 > 00003: PID: 46455 > 00004: pyworkflow: 3.0.13 > 00005: plugin: relion > 00006: plugin v: 3.1.2 > 00007: currentDir: /data1/ScipionUserData/projects/Caro__helix > 00008: workingDir: Runs/001546_ProtRelionClassify2D > 00009: runMode: Continue > 00010: MPI: 3 > 00011: threads: 3 > 00012: Starting at step: 1 > 00013: Running steps > 00014: STARTED: convertInputStep, step 1, time 2021-05-18 12:33:26.198123 > 00015: Converting set from 'Runs/001492_ProtUserSubSet/particles.sqlite' > into 'Runs/001546_ProtRelionClassify2D/input_particles.star' > 00016: convertBinaryFiles: creating soft links. > 00017: Root: Runs/001546_ProtRelionClassify2D/extra/input -> > Runs/001057_ProtImportParticles/extra > 00018: FINISHED: convertInputStep, step 1, time 2021-05-18 > 12:33:27.117588 > 00019: STARTED: runRelionStep, step 2, time 2021-05-18 12:33:27.145974 > 00020: mpirun -np 3 `which relion_refine_mpi` --i > Runs/001546_ProtRelionClassify2D/input_particles.star --particle_diameter > 690 --zero_mask --K 64 --norm --scale --o > Runs/001546_ProtRelionClassify2D/extra/relion --oversampling 1 > --flatten_solvent --tau2_fudge 2.0 --iter 25 --offset_range 5.0 > --offset_step 2.0 --psi_step 10.0 --dont_combine_weights_via_disc > --scratch_dir /data1/new_scratch/ --pool 3 --gpu --j 3 > 00021: RELION version: 3.1.2 > 00022: Precision: BASE=double, CUDA-ACC=single > 00023: > 00024: === RELION MPI setup === > 00025: + Number of MPI processes = 3 > 00026: + Number of threads per MPI process = 3 > 00027: + Total number of threads therefore = 9 > 00028: + Leader (0) runs on host = cryoem01 > 00029: + Follower 1 runs on host = cryoem01 > 00030: + Follower 2 runs on host = cryoem01 > 00031: ================= > 00032: uniqueHost cryoem01 has 2 ranks. > 00033: GPU-ids not specified for this rank, threads will automatically > be mapped to available devices. > 00034: Thread 0 on follower 1 mapped to device 0 > 00035: Thread 1 on follower 1 mapped to device 0 > 00036: Thread 2 on follower 1 mapped to device 0 > 00037: GPU-ids not specified for this rank, threads will automatically > be mapped to available devices. > 00038: Thread 0 on follower 2 mapped to device 1 > 00039: Thread 1 on follower 2 mapped to device 1 > 00040: Thread 2 on follower 2 mapped to device 1 > 00041: Running CPU instructions in double precision. > 00042: + WARNING: Changing psi sampling rate (before oversampling) to > 5.625 degrees, for more efficient GPU calculations > 00043: + On host cryoem01: free scratch space = 448.252 Gb. > 00044: Copying particles to scratch directory: > /data1/new_scratch/relion_volatile/ > 00045: 000/??? sec ~~(,_,"> > [oo] > 00046: 1/ 60 sec ~~(,_,">in: > /opt/Scipion3/software/em/relion-3.1.2/src/rwMRC.h, line 192 > 00047: ERROR: > 00048: readMRC: Image number 11 exceeds stack size 1 of image 000011@Runs > /001546_ProtRelionClassify2D/extra/input/1024562735536827037_FoilHole_1618719_Data_1621438_1621440_20200703_085118_Fractions_patch_aligned_doseweighted_particles.mrcs > 00049: === Backtrace === > 00050: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(_ZN11RelionErrorC1ERKSsS1_l+0x41) > [0x4786a1] > 00051: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(_ZN5ImageIdE7readMRCElbRK8FileName+0x99f) > [0x4b210f] > 00052: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(_ZN5ImageIdE5_readERK8FileNameR13fImageHandlerblbb+0x17b) > [0x4b407b] > 00053: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(_ZN10Experiment22copyParticlesToScratchEibbd+0xda7) > [0x5b8f87] > 00054: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(_ZN14MlOptimiserMpi18initialiseWorkLoadEv+0x210) > [0x498540] > 00055: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(_ZN14MlOptimiserMpi10initialiseEv+0x9aa) > [0x49ab2a] > 00056: > /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi(main+0x55) > [0x4322a5] > 00057: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f657f51a555] > 00058: /opt/Scipion3/software/em/relion-3.1.2/bin/relion_refine_mpi() > [0x435fbf] > 00059: ================== > 00060: ERROR: > 00061: readMRC: Image number 11 exceeds stack size 1 of image 000011@Runs > /001546_ProtRelionClassify2D/extra/input/1024562735536827037_FoilHole_1618719_Data_1621438_1621440_20200703_085118_Fractions_patch_aligned_doseweighted_particles.mrcs > 00062: application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1 > 00063: Traceback (most recent call last): > 00064: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 197, in run > 00065: self._run() > 00066: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 248, in _run > 00067: resultFiles = self._runFunc() > 00068: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 244, in _runFunc > 00069: return self._func(*self._args) > 00070: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/relion/protocols/protocol_base.py", > line 811, in runRelionStep > 00071: self.runJob(self._getProgram(), params) > 00072: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/protocol.py", > line 1388, in runJob > 00073: self._stepsExecutor.runJob(self._log, program, arguments, > **kwargs) > 00074: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/protocol/executor.py", > line 65, in runJob > 00075: process.runJob(log, programName, params, > 00076: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 52, in runJob > 00077: return runCommand(command, env, cwd) > 00078: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/site-packages/pyworkflow/utils/process.py", > line 67, in runCommand > 00079: check_call(command, shell=True, stdout=sys.stdout, > stderr=sys.stderr, > 00080: File > "/usr/local/miniconda/envs/scipion3/lib/python3.8/subprocess.py", line 364, > in check_call > 00081: raise CalledProcessError(retcode, cmd) > 00082: subprocess.CalledProcessError: Command ' mpirun -np 3 `which > relion_refine_mpi` --i > Runs/001546_ProtRelionClassify2D/input_particles.star --particle_diameter > 690 --zero_mask --K 64 --norm --scale --o > Runs/001546_ProtRelionClassify2D/extra/relion --oversampling 1 > --flatten_solvent --tau2_fudge 2.0 --iter 25 --offset_range 5.0 > --offset_step 2.0 --psi_step 10.0 --dont_combine_weights_via_disc > --scratch_dir /data1/new_scratch/ --pool 3 --gpu --j 3' returned > non-zero exit status 1. > 00083: Protocol failed: Command ' mpirun -np 3 `which > relion_refine_mpi` --i > Runs/001546_ProtRelionClassify2D/input_particles.star --particle_diameter > 690 --zero_mask --K 64 --norm --scale --o > Runs/001546_ProtRelionClassify2D/extra/relion --oversampling 1 > --flatten_solvent --tau2_fudge 2.0 --iter 25 --offset_range 5.0 > --offset_step 2.0 --psi_step 10.0 --dont_combine_weights_via_disc > --scratch_dir /data1/new_scratch/ --pool 3 --gpu --j 3' returned > non-zero exit status 1. > 00084: FAILED: runRelionStep, step 2, time 2021-05-18 12:33:29.230213 > 00085: *** Last status is failed > 00086: ------------------- PROTOCOL FAILED (DONE 2/3) > > > Cryosparc (in SCIPION) requires CTF to run. > > > > Thais is where I am now. > > Perhaps there is a solution? > > > Sincerely, > Dmitry > > > > On 18. May 2021, at 14:16, Pablo Conesa <pc...@cn...> wrote: > > Dear Dmitry, the import of CS metadata files (*.cs) is not supported in > Scipion. Does CS has an option to export to star files. It rings a bell. > On 18/5/21 9:53, Dmitry Semchonok wrote: > > Dear Grigory, > > > The files are in mrc format. > > > Please, let me try to explain more plan: > > I have a project in cryosparc. There I have cryosparc selected 2D > classes. I want to export the particles of those classes into SCIPION. > > So I I pressed Export (fig 1) and the program(cryosparc) created the > folder with mrc + other files (fig 2;3). I looked into J48 and found many > *.mrc files of the particles. But it is not 1 mrc = 1 particle. It seems to > be a mrc stuck - so I have several files inside 1 *.mrc (fig 4) (you can > also notice that they all have different sizes) > > So I need to export them somehow in SCIPION > > For that, I used the SCIPION export - images protocol where for the files > to add I put *.mrc. But the protocol seems to be added only 1 mrc as 1 > picture and instead of having 46392 particles I have ~600 particles. > > (Also the geometry seems not preserved). > > > So my question how to export the particles from cryosparc into SCIPION > correctly? > > > Thank you! > > > > https://disk.yandex.com/d/Fv3Q1lpwEzSisg > > Sincerely, > > Dmitry > > > > On 17. May 2021, at 18:12, Grigory Sharov <sha...@gm...> > wrote: > > Hi Dmitry, > > mrc stacks should have "mrcs" extension. Is this the problem you are > getting? > > Best regards, > Grigory > > > -------------------------------------------------------------------------------- > Grigory Sharov, Ph.D. > > MRC Laboratory of Molecular Biology, > Francis Crick Avenue, > Cambridge Biomedical Campus, > Cambridge CB2 0QH, UK. > tel. +44 (0) 1223 267228 <+44%201223%20267228> > e-mail: gs...@mr... > > > On Mon, May 17, 2021 at 4:49 PM Dmitry Semchonok <Sem...@gm...> > wrote: > >> Dear colleagues, >> >> I would like to export the particles from cryosparc to SCIPION. >> >> How to do that? >> >> >> >> >> What I tried: >> >> >> 1. In cryosparc I pressed Export – to export the particles I am >> interested in. >> >> 2. In the folder Export – I found many mrc stacks with particles in >> each. >> >> 3. I tried to export them to SCIPION using Export particles but >> instead of reading each stack and combine them in the 1 dataset I received >> 1 particle / per each mrc stack. >> >> >> Any ideas? >> >> >> Thank you >> >> Sincerely, >> >> Dmitry >> >> _______________________________________________ >> scipion-users mailing list >> sci...@li... >> https://lists.sourceforge.net/lists/listinfo/scipion-users >> > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > > > > > _______________________________________________ > scipion-users mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/scipion-users > > -- > Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es/> team* > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > > > _______________________________________________ > scipion-users mailing list > sci...@li... > https://lists.sourceforge.net/lists/listinfo/scipion-users > |
