Re: [scipion-users] Antw: Re: Re: Antw: RE: deep consensus pickling error

[Pablo C. <pc...@cn...>]

You have eman2 acting as a conda manager, that is a problem.

At least the environment is not listed there.

Does the other plugins work? e.g. cryolo?

Can you send the output of:

scipion3 printenv

and

scipion3 run printenv

?

On 21/4/21 17:06, Dmitry Semchonok wrote:
> Dear colleagues,
> Just to add our current error from the xmipp deep learning installation.
>
> Interestingly we almost fun to an end of this step but conda issue 
> appeared.
>
> Any suggestions?
>
>
>
> currentDir: /data1/ScipionUserData/projects/Image_processing_cn_10688
> 00890:   workingDir: Runs/007643_XmippProtScreenDeepLearning
> 00891:   runMode: Continue
> 00892: MPI: 1
> 00893: threads: 4
> 00894:    Starting at step: 3
> 00895:    Running steps
> 00896:   STARTED: predict, step 3, time 2021-04-20 11:08:26.633086
> 00897:
> 00898:   $ /usr/local/EMAN_2.21/bin/conda info --env
> 00899:   # conda environments:
> 00900:   #
> 00901:   3DFSC              /home/panos/.conda/envs/3DFSC
> 00902: cryoassess-0.1.0 /home/panos/.conda/envs/cryoassess-0.1.0
> 00903:   cryolo             /home/panos/.conda/envs/cryolo
> 00904:   cryolo             /usr/local/EMAN_2.21/envs/cryolo
> 00905:   cryolo-1.5.4             /usr/local/EMAN_2.21/envs/cryolo-1.5.4
> 00906:   fsc              /usr/local/EMAN_2.21/envs/fsc
> 00907:   topaz              /usr/local/EMAN_2.21/envs/topaz
> 00908:   root          *  /usr/local/EMAN_2.21
> 00909:
> 00910:
> 00911:  >>> 'xmipp_DLTK_v0.3' conda environment not found... (check 
> list above)
> 00912:   Traceback (most recent call last):
> 00913:     File 
> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py", 
> line 189, in run
> 00914: self._run()
> 00915:     File 
> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py", 
> line 240, in _run
> 00916: resultFiles = self._runFunc()
> 00917:     File 
> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py", 
> line 236, in _runFunc
> 00918:       return self._func(*self._args)
> 00919:     File 
> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/xmipp3/protocols/protocol_screen_deeplearning.py", 
> line 319, in predict
> 00920: env=self.getCondaEnv())
> 00921:     File 
> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/xmipp3/base.py", 
> line 204, in getCondaEnv
> 00922:       return CondaEnvManager.getCondaEnv(env, envName)
> 00923:     File "/opt/Scipion3/software/bindings/xmipp_base.py", line 
> 245, in getCondaEnv
> 00924:       envBin = os.path.join(environDir, "bin")
> 00925:     File "/usr/lib64/python3.6/posixpath.py", line 80, in join
> 00926:       a = os.fspath(a)
> 00927:   TypeError: expected str, bytes or os.PathLike object, not 
> NoneType
> 00928:   Protocol failed: expected str, bytes or os.PathLike object, 
> not NoneType
> 00929:   FAILED: predict, step 3, time 2021-04-20 11:08:27.020467
> 00930:   *** Last status is failed
> 00931: ------------------- PROTOCOL FAILED (DONE 3/4)
>
>
> Thank you!
>
>
> Sincerely,
> Dmitry
>
>
>
>> On 20. Apr 2021, at 11:25, Christian Tüting 
>> <chr...@bi... 
>> <mailto:chr...@bi...>> wrote:
>>
>> Hi,
>>
>> thank you for your response. I tried this twice. The first time, the
>> "_BACKUP2" folder was popping up again under .conda/envs, so I followed
>> your steps again, deleting the folder under software and removed the
>> conda enviroment, and now, no enviroment is generated at all. Logically,
>> running a job gives this error: ">>> 'xmipp_DLTK_v0.3' conda environment
>> not found... (check list above)".
>>
>> Another long-term job is currently running, after this is done, we will
>> restart everything to have a clean system. But do you have any
>> additional tips?
>>
>> Thanks in advance
>> Christian
>>
>>
>>
>>>>> David Maluenda <dma...@ub... <mailto:dma...@ub...>> 
>>>>> 20.04.21 10.02 Uhr >>>
>> Hi Christian,
>>
>> If it already works, I would leave it as it is.
>>
>> To make a clear installation, you don't need to reinstall the whole
>> xmipp. You can just reinstall the deepLearningToolkit by
>>
>> rm -rf /opt/Scipion3/software/em/deepLearningToolkit
>> conda env remove --name xmipp_DLTK_v0.3_BACKUP2
>> scipion3 installb deepLearningToolkit
>>
>>
>> Cheers,
>>
>> <http://ucm.es/ <http://ucm.es/>>
>> _____
>> Dr. David Maluenda Niubó
>> dma...@ub... <mailto:dma...@ub...><mailto:dma...@ub... 
>> <mailto:dma...@ub...>> - (+34) 619 029 310
>> Wavefront Engineering
>> Group<https://webgrec.ub.edu/cgi-bin/3DADREC/crfitgrup.cgi?PAR=GRENFONA 
>> <https://webgrec.ub.edu/cgi-bin/3DADREC/crfitgrup.cgi?PAR=GRENFONA>>
>> Applied Physics - UB<https://www.ub.edu/portal/web/dp-fisica-aplicada 
>> <https://www.ub.edu/portal/web/dp-fisica-aplicada>>
>>
>> This email message and any documents attached to it may contain
>> confidential or legally protected material and are intended solely for
>> the use of the individual or organization to whom they are addressed. We
>> remind you that if you are not the intended recipient of this email
>> message or the person responsible for processing it, then you are not
>> authorized to read, save, modify, send, copy or disclose any of its
>> contents. If you have received this email message by mistake, we kindly
>> ask you to inform the sender of this and to eliminate both the message
>> and any attachments it carries from your account. Thank you for your
>> collaboration.
>>
>>
>>
>> On Tue, Apr 20, 2021 at 9:53 AM Christian Tüting
>> <chr...@bi... 
>> <mailto:chr...@bi...><mailto:chr...@bi... 
>> <mailto:chr...@bi...>>>
>> wrote:
>> Hi Pablo,
>>
>> I will take an eye on this. I have a question regarding dependencies:
>> tensorflow 1.15 requires keras-application>=1.0.8 and
>> keras-preprocessing>=1.0.5, but keras 2.2 requieres older versions
>> (1.0.2 and 1.0.1, respectivly).
>>
>> Should we try to uninstall and a clean reinstall of xmipp, to resolve
>> all dependency issues?
>>
>> Thank you for your support.
>>
>> Best
>> Christian
>>
>>
>>>>> Pablo Conesa <pc...@cn... 
>>>>> <mailto:pc...@cn...><mailto:pc...@cn... 
>>>>> <mailto:pc...@cn...>>>
>> 20.04.21 9.16 Uhr >>>
>> Thanks Christian.
>>
>> In theory, xmipp defines dependencies strictly
>> (https://github.com/I2PC/xmipp/blob/devel/src/xmipp/bindings/python/xmipp_conda_envs.py 
>> <https://github.com/I2PC/xmipp/blob/devel/src/xmipp/bindings/python/xmipp_conda_envs.py>)
>>
>> I do not know how it ended up with higher versions, but something to
>> keep an eye on.
>>
>> On 20/4/21 7:00, Christian Tüting wrote:
>>> Hi all,
>>>
>>> first the conda path looks actually very strange, and I fixed this
>>> direclty. We had some other issues, which were tensorflow-related, but
>>> this is fixed.
>>>
>>>
>>> Just to comment on that, there were first an issue with the tensorflow
>>> module. Installing the latest release (2.x) gives an error on "module
>>> 'tensorflow' has no attribute 'Session'". After fixing that by
>>> downgrading, keras was giving an error, that the module does not have
>> no
>>> attribute "tf", which was also fixed by downgrading.
>>>
>>>
>>>
>>> TL;DR Eventually, downgrading tensorflow to 1.15.0 and downgrading
>> keras
>>> to 2.2.4 (from 2.2.5) solved the issue
>>>
>>>
>>> Best
>>> Christian
>>>
>>>
>>>>>> "Dmitry A. Semchonok"
>> <sem...@gm... 
>> <mailto:sem...@gm...><mailto:sem...@gm... 
>> <mailto:sem...@gm...>>> 19.04.21 16.24 Uhr >>>
>>> Dear Carlos Oscar, Pablo, David, Ruben,
>>>
>>>
>>> Thank you very much for your fast response.
>>>
>>>
>>> Currently, deep learning is running.
>>> A colleague of mine, Christian, kindly looked at the problem.
>>>
>>>
>>> Let me, please post the errors we had and where we are now.
>>> May be Christian can comment more on this issue.
>>>
>>>
>>> Please see the errors here.
>>> https://yadi.sk/d/KlTeKv1O9UCTQg <https://yadi.sk/d/KlTeKv1O9UCTQg>
>>>
>>>
>>> Since> Sent: Monday, April 19, 2021 12:57 PM
>>> To: Mailing list for Scipion users; Pablo Conesa; Rubén Sánchez; David
>>> Maluenda
>>> Subject: Re: [scipion-users] deep consensus pickling error
>>>
>>>
>>> Dear Dmitry,
>>> let us see if David or Rubén knows where this environment is coming
>>> from.
>>> Kind regards, Carlos Oscar
>>> El 19/04/2021 a las 9:37, Pablo Conesa escribió:
>>>
>>> Hi Dmitry... you have no environment for xmipp_DLTK_v0.3.
>>> You have a "weird" xmipp_DLTK_v0.3_BACKUP2
>>> /home/panos/.conda/envs/xmipp_DLTK_v0.3_BACKUP2
>>>
>>> On 17/4/21 8:56, Dmitry A. Semchonok wrote:
>>>
>>> Dear colleagues,
>>>
>>> During deep consensus pickling I received the following error
>>>
>>> Any ideas?
>>>
>>> Thank you
>>>
>>> Sincerely,
>>> Dmitry
>>>
>>>
>>>
>>> [Parallel(n_jobs=12)]: Done 176 tasks      | elapsed:  1.8min
>>> [Parallel(n_jobs=12)]: Done 426 tasks      | elapsed:  4.3min
>>> [Parallel(n_jobs=12)]: Done 776 tasks      | elapsed:  7.5min
>>> [Parallel(n_jobs=12)]: Done 1226 tasks      | elapsed: 11.5min
>>> [Parallel(n_jobs=12)]: Done 1776 tasks      | elapsed: 16.7min
>>> [Parallel(n_jobs=12)]: Done 2426 tasks      | elapsed: 23.7min
>>> [Parallel(n_jobs=12)]: Done 3176 tasks      | elapsed: 32.1min
>>> [Parallel(n_jobs=12)]: Done 4026 tasks      | elapsed: 41.4min
>>> [Parallel(n_jobs=12)]: Done 4976 tasks      | elapsed: 51.8min
>>> [Parallel(n_jobs=12)]: Done 5285 out of 5285 | elapsed: 55.1min
>> finished
>>> [35mFINISHED[0m: extractParticles, step 12, time 2021-04-17
>>> 04:05:07.093613
>>> [35mSTARTED[0m: joinSetOfParticlesStep, step 13, time 2021-04-17
>>> 04:05:07.574152
>>> [35mFINISHED[0m: joinSetOfParticlesStep, step 13, time 2021-04-17
>>> 04:26:59.879548
>>> [35mSTARTED[0m: trainCNN, step 14, time 2021-04-17 04:26:59.920444
>>> None None None None
>>>
>>> $ /usr/local/EMAN_2.21/bin/conda info --env
>>> # conda environments:
>>> #
>>> 3DFSC                    /home/panos/.conda/envs/3DFSC
>>> cryoassess-0.1.0         /home/panos/.conda/envs/cryoassess-0.1.0
>>> cryolo                   /home/panos/.conda/envs/cryolo> cryolo
>>           /usr/local/EMAN_2.21/envs/cryolo
>>> cryolo-1.5.4             /usr/local/EMAN_2.21/envs/cryolo-1.5.4
>>> fsc                      /usr/local/EMAN_2.21/envs/fsc
>>> topaz                    /usr/local/EMAN_2.21/envs/topaz
>>> root                  *  /usr/local/EMAN_2.21
>>>
>>>
>>>>>> 'xmipp_DLTK_v0.3' conda environment not found... (check list above)
>>> Traceback (most recent call last):
>>>   File
>>>
>> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py",
>>> line 189, in run
>>>     self._run()
>>>   File
>>>
>> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py",
>>> line 240, in _run
>>>     resultFiles = self._runFunc()
>>>   File
>>>
>> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py",
>>> line 236, in _runFunc
>>>     return self._func(*self._args)
>>>   File
>>> "/opt/Scipion    env=self.getCondaEnv())
>>>   File
>>>
>> "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/xmipp3/base.py",
>>> line 204, in getCondaEnv
>>>     return CondaEnvManager.getCondaEnv(env, envName)
>>>   File "/opt/Scipion3/software/bindings/xmipp_base.py", line 245, in
>>> getCondaEnv
>>>     envBin = os.path.join(environDir, "bin")
>>>   File "/usr/lib64/python3.6/posixpath.py", line 80, in join
>>>     a = os.fspath(a)
>>> TypeError: expected str, bytes or os.PathLike object, not NoneType
>>> [31mProtocol failed: expected str, bytes or os.PathLike object, not
>>> NoneType[0m
>>> [35mFAILED[0m: trainCNN, step 14, time 2021-04-17 04:27:00.607417
>>> *** Last status is failed
>>> Sent from Mail for Windows 10
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________scipion-users mailing
>>>
>> lis...@li...https 
>> <mailto:lis...@li...https>://lists.sourceforge.net/lists/listinfo/scipion-users--<http://lists.sourceforge.net/lists/listinfo/scipion-users-->
>>>
>>> Pablo Conesa - Madrid Scipion team
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________scipion-users mailing
>>>
>> lis...@li...https 
>> <mailto:lis...@li...https>:/>
>>>
>>>
>>>
>> --
>> Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es 
>> <http://scipion.i2pc.es>> team*
>>
>>
>>
>> Aquest missatge, i els fitxers adjunts que hi pugui haver, pot contenir
>> informació confidencial o protegida legalment i s’adreça exclusivament a
>> la persona o entitat destinatària. Si no consteu com a destinatari final
>> o no teniu l’encàrrec de rebre’l, no esteu autoritzat a llegir-lo,
>> retenir-lo, modificar-lo, distribuir-lo, copiar-lo ni a revelar-ne el
>> contingut. Si l’heu rebut per error, informeu-ne el remitent i elimineu
>> del sistema tant el missatge com els fitxers adjunts que hi pugui haver.
>>
>> Este mensaje, y los ficheros adjuntos que pueda incluir, puede contener
>> información confidencial o legalmente protegida y está exclusivamente
>> dirigido a la persona o entidad destinataria. Si usted no consta como
>> destinatario final ni es la persona encargada de recibirlo, no está
>> autorizado a leerlo, retenerlo, modificarlo, distribuirlo o copiarlo, ni
>> a revelar su contenido. Si lo ha recibido por error, informe de ello al
>> remitente y elimine del sistema tanto el mensaje como los ficheros
>> adjuntos que pueda contener.
>>
>> This email message and any attachments it carries may contain
>> confidential or legally protected material and are intended solely for
>> the individual or organization to whom they are addressed. If you are
>> not the intended recipient of this message or the person responsible for
>> processing it, then you are not authorized to read, save, modify, send,
>> copy or disclose any part of it. If you have received the message by
>> mistake, please inform the sender of this and eliminate the message and
>> any attachments it carries from your account.
>>
>>
>> Aquest missatge, i els fitxers adjunts que hi pugui haver, pot contenir
>> informació confidencial o protegida legalment i s’adreça exclusivament a
>> la persona o entitat destinatària. Si no consteu com a destinatari final
>> o no teniu l’encàrrec de rebre’l, no esteu autoritzat a llegir-lo,
>> retenir-lo, modificar-lo, distribuir-lo, copiar-lo ni a revelar-ne el
>> contingut. Si l’heu rebut per error, informeu-ne el remitent i elimineu
>> del sistema tant el missatge com els fitxers adjunts que hi pugui haver.
>>
>> Este mensaje, y los ficheros adjuntos que pueda incluir, puede contener
>> información confidencial o legalmente protegida y está exclusivamente
>> dirigido a la persona o entidad destinataria. Si usted no consta como
>> destinatario final ni es la persona encargada de recibirlo, no está
>> autorizado a leerlo, retenerlo, modificarlo, distribuirlo o copiarlo, ni
>> a revelar su contenido. Si lo ha recibido por error, informe de ello al
>> remitente y elimine del sistema tanto el mensaje como los ficheros
>> adjuntos que pueda contener.
>>
>> This email message and any attachments it carries may contain
>> confidential or legally protected material and are intended solely for
>> the individual or organization to whom they are addressed. If you are
>> not the intended recipient of this message or the person responsible for
>> processing it, then you are not authorized to read, save, modify, send,
>> copy or disclose any part of it. If you have received the message by
>> mistake, please inform the sender of this and eliminate the message and
>> any attachments it carries from your account.
>>
>
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
-- 
Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team*