[scipion-users] Scipion v1.1 difficulty continuing relion2d & relating joined particle set to micrographs via .xmd

[Matthews-Palmer, T. R. S. <t.m...@im...>]

Dear Scipion Users,

When running a large relion 2D classification, which fails unavoidably (this class of error: https://github.com/3dem/relion/issues/155) at a late iteration but has not reached the specified 25 iterations, scipion has not processed the relion output in its sqlite database (I’m guessing) and so when analysing the completed iterations, information is missing e.g. size=0 for all classes. Therefore subsets of particles cannot be selected to remove bad classes, re-extract with recentering and keep processing.
This is effectively a roadblock to continue processing inside scipion.
Is there a way to get the scipion DB / GUI to recover the information from any given iteration?
(Tangentially, although running a relion command with —continue flag worked, in the scipion GUI the continue option failed, perhaps because ‘consider previous alignment’ was set to No? Scipion responded by starting at iteration 1, threatening to delete the successful iterations so far - which took ~2weeks! From there on scipion could not be persuaded to continue from iteration 18, and I do not know what would happen if I let it continue what looks like a restart.)

I can take the optimiser.star file to relion and make a subset selection, fine.
However, now I would like to re-extract particles with recentering, and un-bin the particles later. These both seem to be difficult now. Especially because I joined two sets of particles before binning, and the MicrographIDs were not adjusted to be unique values in the unioned set.
If I import the subset particle.star back in to scipion, it generates outputMicrographs & outputParticles - the Micrograph info is wrong & can’t find the micrographs, it thinks there are 5615 micrographs, but this is wrong since there are now 1114 non-unique micIDs.
Does anyone know how the relationship between particles in the joined set and their micrographs can be re-established?

for xmipp particle extraction, the image.xmd files (/star files) lose the connection to the micrographs because the _micrograph field is /tmp/[…]noDust.xmp rather than the .mrc output from motioncorr. Other than that there is  _micrographId ; does anyone know where _micrographId is related to the corresponding micrograph?

Any help is greatly appreciated.
All the best,
Teige
[cid:1FA...@th...]