Re: [scipion-users] movie alignment protocols error

[Pablo C. <pc...@cn...>]

This seems chaotic!

Dmitry, can you try the same workflow outside a cluster? Just to discard 
cluster configuration.

Could you also send what are the values for MOTIONCOR AND MOTIONCOR2 
variables?:

Maybe a:  scipion printenv will help.


Cheers.



On 11/04/17 13:40, Dmitry Semchonok wrote:
>
> Dear colleagues,
>
>
> The issue is  the following - none of the movie alignment protocols is 
> currently working fine at our cluster.
>
>
> Please see the errors below:
>
> *The xmipp3 – movie alignment (xmipp3-optical alignment) (run on GPU)*
>
> 2017-04-11 10:56:49.125723
>
> 00080:Traceback (most recent call last):
>
> 00081:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 184, in run
>
> 00082:self._run()
>
> 00083:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 1034, in _run
>
> 00084:self._runSteps(startIndex)
>
> 00085:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 919, in _runSteps
>
> 00086:self._stepsCheck)
>
> 00087:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/executor.py", 
> line 119, in runSteps
>
> 00088:stepsCheckCallback()
>
> 00089:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_movies.py",line 
> 150, in _stepsCheck
>
> 00090:self._checkNewOutput()
>
> 00091:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_align_movies.py", 
> line 222, in _checkNewOutput
>
> 00092:'outputMicrographs')
>
> 00093:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_align_movies.py", 
> line 193, in _updateOutputMicSet
>
> 00094:micSet = self._loadOutputSet(SetOfMicrographs, sqliteFn)
>
> 00095:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_align_movies.py", 
> line 122, in _loadOutputSet
>
> 00096:outputSet.loadAllProperties()
>
> 00097:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/object.py",line 
> 1196, in loadAllProperties
>
> 00098:for key in self._getMapper().getPropertyKeys():
>
> 00099:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/mapper/sqlite.py", 
> line 920, in getPropertyKeys
>
> 00100:return self.db.getPropertyKeys()
>
> 00101:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/mapper/sqlite.py", 
> line 1017, in getPropertyKeys
>
> 00102:self.executeCommand(self.SELECT_PROPERTY_KEYS)
>
> 00103:OperationalError: no such table: Properties
>
> 00104:------------------- PROTOCOL FAILED (DONE 2/5141)
>
> *Motioncorr – motioncorr alignment (GPU)*
>
> STARTED: processMovieStep, step 1
>
> 00012:2017-04-11 10:44:05.586468
>
> 00013:Processing movie: 
> Runs/021173_ProtMotionCorr/tmp/movie_000001/FoilHole_26594250_Data_25564800_25564801_20160826_2007-21923.mrc
>
> 00014:Traceback (most recent call last):
>
> 00015:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 184, in run
>
> 00016:self._run()
>
> 00017:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 230, in _run
>
> 00018:resultFiles = self._runFunc()
>
> 00019:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 226, in _runFunc
>
> 00020:return self._func(*self._args)
>
> 00021:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_movies.py", 
> line 267, in processMovieStep
>
> 00022:self._processMovie(movie)
>
> 00023:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/packages/motioncorr/protocol_motioncorr.py", 
> line 304, in _processMovie
>
> 00024:if self.doMagCor and self._supportsMagCorrection():
>
> 00025:AttributeError: 'ProtMotionCorr' object has no attribute 'doMagCor'
>
> 00026:Protocol failed: 'ProtMotionCorr' object has no attribute 'doMagCor'
>
> 00027:FAILED: processMovieStep, step 1
>
> 00028:2017-04-11 10:44:05.613162
>
> 00029:------------------- PROTOCOL FAILED (DONE 1/5141)
>
> *Grigoriefflab-unblur*
>
> 00738:Total execution time : 1 minute and 7 seconds
>
> 00739:2017-04-11 13:20:05: UnBlur finished cleanly.
>
> 00740:
>
> 00741:Erasing.....movieFolder: Runs/022622_ProtUnblur/tmp/movie_000005
>
> 00742:Traceback (most recent call last):
>
> 00743:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 184, in run
>
> 00744:self._run()
>
> 00745:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 1034, in _run
>
> 00746:self._runSteps(startIndex)
>
> 00747:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/protocol.py", 
> line 919, in _runSteps
>
> 00748:self._stepsCheck)
>
> 00749:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/protocol/executor.py",line 
> 220, in runSteps
>
> 00750:stepsCheckCallback()
>
> 00751:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_movies.py", 
> line 150, in _stepsCheck
>
> 00752:self._checkNewOutput()
>
> 00753:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_align_movies.py", 
> line 180, in _checkNewOutput
>
> 00754:newMovie = self._createOutputMovie(movie)
>
> 00755:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/protocol/protocol_align_movies.py",line 
> 352, in _createOutputMovie
>
> 00756:xshifts, yshifts = self._getMovieShifts(movie)
>
> 00757:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/packages/grigoriefflab/protocol_unblur.py", 
> line 316, in _getMovieShifts
>
> 00758:xShifts, yShifts = readShiftsMovieAlignment(shiftFn)
>
> 00759:File 
> "/software/software/Scipion/1.1_2017-02-06-foss-2016a/pyworkflow/em/packages/grigoriefflab/convert.py", 
> line 247, in readShiftsMovieAlignment
>
> 00760:f = open(shiftFn)
>
> 00761:IOError: [Errno 2] No such file or directory: 
> 'Runs/022622_ProtUnblur/extra/FoilHole_26594250_Data_25564906_25564907_20160826_2009-21925_shifts.txt'
>
> 00762:------------------- PROTOCOL FAILED (DONE 1/5141)
>
> *Xmipp3 – correlation alignment –(Threads)*
>
> Was running till step 1075 then failed – I resubmitted it and again it 
> crashed after a few images.
>
> 00423:2017-04-11 12:54:52,577 ERROR:Protocol failed: (-10.0, 0.0, 
> u'\u221210', 'k', 'top', 'center', 7057509111269643423, None, None, 
> 100, 140391271392104)
>
> 00424:2017-04-11 12:54:52,639 INFO:FAILED: processMovieStep, step 1075
>
> 00425:2017-04-11 12:54:52,640 INFO:2017-04-11 12:54:34.176712
>
> 00426:2017-04-11 12:54:52,779 INFO:------------------- PROTOCOL FAILED 
> (DONE 1077/5141)
>
> 00427:2017-04-11 12:55:08,985 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001097
>
> 00428:2017-04-11 12:56:23,125 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001098
>
> 00429:2017-04-11 12:56:36,576 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001099
>
> 00430:2017-04-11 12:57:14,722 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001102
>
> 00431:2017-04-11 12:57:18,171 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001101
>
> 00432:2017-04-11 12:57:18,744 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001103
>
> 00433:2017-04-11 12:57:21,819 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001100
>
> 00434:2017-04-11 12:57:22,569 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001105
>
> 00435:2017-04-11 12:57:30,870 INFO:Erasing.....movieFolder: 
> Runs/022537_XmippProtMovieCorr/tmp/movie_001104
>
> 00436:2017-04-11 12:57:31,472 INFO:------------------- PROTOCOL FAILED 
> (DONE 1077/5141)
>
> Any ideas why?
>
> Thank you.
>
> Sincerely,
>
> Dmitry
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users