[scipion-users] Streaming/On-the-Fly processing

[David T. <dav...@bi...>]

Hello, Sorry in advance for the long email.

After watching the recent SBGrid video on Streaming in Scipion (https://www.youtube.com/watch?v=5fLrZ2BYJ28), I tried testing streaming processing with v1.2 (2018-04-02) Caligula.

The 'scipion demo' worked fine with this version (as described https://github.com/I2PC/scipion/wiki/Streaming-Processing).

I next transferred 20 apoferritin movies one at a time to a processing directory over a 100 minute time period (5 min between each movie). Several movies transferred before I started streaming importing in scipion.

The Protocols SPA Protocol: scipion - import movies worked fine and contained a Streaming tab selected yes with 600 sec time out and 30 sec File timeout. Run mode continue was also selected. Started 15:47:41... Finished: 17:18:19...

Next, I executed the motioncorr Protocol (Protocols SPA, motioncor2). Run mode Continue, no Streaming tab available, never the less all 20 movies got motion corrected in a streaming manner. Started 15:51:40 ... Finished: 17:18:21...

Likewise, the gctf Protocol (Protocols SPA) has no Streaming tab available but appeared to stream just fine with one potential issue that _psdFile column (Analyze Results) shows half image with calc'd FFT from fit CTF, but half where observed FFT should be appears black. Started 16:02:44... Finished 17:18:22...

The motioncor2 and gctf outputs are consistent with results I've previously obtained via command line.

***QUESTION: Why is the observed FFT black, while the fit CTF in the analyze results table shows up just fine?***

I also included a relion - auto-picking protocol, again no Streaming tab was present in the Protocols SPA protocol. This appeared to get off to a decent start, but in the end only the first 11 aligned micrographs got picked (Analyze Results: State = Manual), no particles were picked for the last 9 mics (State = Available). Started 16:17:21...  Finished 17:18:25... run.stderr is empty. I've attached run.stdout herein, all the autopicks including and beyond 'Picking micrograph: Runs/000079_ProtMotionCorr/extra/May08_05.55.31_aligned_mic_DW.mrc ' show 0 Particles and State = Available in Analyze Results table. All but one of these micrographs autopick reasonably well using relion via command line with identical input parameters.

***QUESTION: Any ideas why the final 9 micrographs did not yield any autopicked particles and what does Available mean on the Analyze Results State column?***

Thank you! Dave Timm

=======================================
David E. Timm
Electron Microscopy Core Laboratory
Department of Biochemistry
Crocker Science Center, room 038
University of Utah
Salt Lake City, Utah 84112 USA
Phone  801.587.0353
FAX  801.587.3077
dav...@bi...
=======================================