Menu
▾
▴
[scipion-users] Antw: Re: Re: Antw: RE: deep consensus pickling error
|
From: Christian T. <chr...@bi...> - 2021-04-20 09:26:18
|
Hi, thank you for your response. I tried this twice. The first time, the "_BACKUP2" folder was popping up again under .conda/envs, so I followed your steps again, deleting the folder under software and removed the conda enviroment, and now, no enviroment is generated at all. Logically, running a job gives this error: ">>> 'xmipp_DLTK_v0.3' conda environment not found... (check list above)". Another long-term job is currently running, after this is done, we will restart everything to have a clean system. But do you have any additional tips? Thanks in advance Christian >>> David Maluenda <dma...@ub...> 20.04.21 10.02 Uhr >>> Hi Christian, If it already works, I would leave it as it is. To make a clear installation, you don't need to reinstall the whole xmipp. You can just reinstall the deepLearningToolkit by rm -rf /opt/Scipion3/software/em/deepLearningToolkit conda env remove --name xmipp_DLTK_v0.3_BACKUP2 scipion3 installb deepLearningToolkit Cheers, <http://ucm.es/> _____ Dr. David Maluenda Niubó dma...@ub...<mailto:dma...@ub...> - (+34) 619 029 310 Wavefront Engineering Group<https://webgrec.ub.edu/cgi-bin/3DADREC/crfitgrup.cgi?PAR=GRENFONA> Applied Physics - UB<https://www.ub.edu/portal/web/dp-fisica-aplicada> This email message and any documents attached to it may contain confidential or legally protected material and are intended solely for the use of the individual or organization to whom they are addressed. We remind you that if you are not the intended recipient of this email message or the person responsible for processing it, then you are not authorized to read, save, modify, send, copy or disclose any of its contents. If you have received this email message by mistake, we kindly ask you to inform the sender of this and to eliminate both the message and any attachments it carries from your account. Thank you for your collaboration. On Tue, Apr 20, 2021 at 9:53 AM Christian Tüting <chr...@bi...<mailto:chr...@bi...>> wrote: Hi Pablo, I will take an eye on this. I have a question regarding dependencies: tensorflow 1.15 requires keras-application>=1.0.8 and keras-preprocessing>=1.0.5, but keras 2.2 requieres older versions (1.0.2 and 1.0.1, respectivly). Should we try to uninstall and a clean reinstall of xmipp, to resolve all dependency issues? Thank you for your support. Best Christian >>> Pablo Conesa <pc...@cn...<mailto:pc...@cn...>> 20.04.21 9.16 Uhr >>> Thanks Christian. In theory, xmipp defines dependencies strictly (https://github.com/I2PC/xmipp/blob/devel/src/xmipp/bindings/python/xmipp_conda_envs.py) I do not know how it ended up with higher versions, but something to keep an eye on. On 20/4/21 7:00, Christian Tüting wrote: > Hi all, > > first the conda path looks actually very strange, and I fixed this > direclty. We had some other issues, which were tensorflow-related, but > this is fixed. > > > Just to comment on that, there were first an issue with the tensorflow > module. Installing the latest release (2.x) gives an error on "module > 'tensorflow' has no attribute 'Session'". After fixing that by > downgrading, keras was giving an error, that the module does not have no > attribute "tf", which was also fixed by downgrading. > > > > TL;DR Eventually, downgrading tensorflow to 1.15.0 and downgrading keras > to 2.2.4 (from 2.2.5) solved the issue > > > Best > Christian > > >>>> "Dmitry A. Semchonok" <sem...@gm...<mailto:sem...@gm...>> 19.04.21 16.24 Uhr >>> > Dear Carlos Oscar, Pablo, David, Ruben, > > > Thank you very much for your fast response. > > > Currently, deep learning is running. > A colleague of mine, Christian, kindly looked at the problem. > > > Let me, please post the errors we had and where we are now. > May be Christian can comment more on this issue. > > > Please see the errors here. > https://yadi.sk/d/KlTeKv1O9UCTQg > > > Since> Sent: Monday, April 19, 2021 12:57 PM > To: Mailing list for Scipion users; Pablo Conesa; Rubén Sánchez; David > Maluenda > Subject: Re: [scipion-users] deep consensus pickling error > > > Dear Dmitry, > let us see if David or Rubén knows where this environment is coming > from. > Kind regards, Carlos Oscar > El 19/04/2021 a las 9:37, Pablo Conesa escribió: > > Hi Dmitry... you have no environment for xmipp_DLTK_v0.3. > You have a "weird" xmipp_DLTK_v0.3_BACKUP2 > /home/panos/.conda/envs/xmipp_DLTK_v0.3_BACKUP2 > > On 17/4/21 8:56, Dmitry A. Semchonok wrote: > > Dear colleagues, > > During deep consensus pickling I received the following error > > Any ideas? > > Thank you > > Sincerely, > Dmitry > > > > [Parallel(n_jobs=12)]: Done 176 tasks | elapsed: 1.8min > [Parallel(n_jobs=12)]: Done 426 tasks | elapsed: 4.3min > [Parallel(n_jobs=12)]: Done 776 tasks | elapsed: 7.5min > [Parallel(n_jobs=12)]: Done 1226 tasks | elapsed: 11.5min > [Parallel(n_jobs=12)]: Done 1776 tasks | elapsed: 16.7min > [Parallel(n_jobs=12)]: Done 2426 tasks | elapsed: 23.7min > [Parallel(n_jobs=12)]: Done 3176 tasks | elapsed: 32.1min > [Parallel(n_jobs=12)]: Done 4026 tasks | elapsed: 41.4min > [Parallel(n_jobs=12)]: Done 4976 tasks | elapsed: 51.8min > [Parallel(n_jobs=12)]: Done 5285 out of 5285 | elapsed: 55.1min finished > [35mFINISHED[0m: extractParticles, step 12, time 2021-04-17 > 04:05:07.093613 > [35mSTARTED[0m: joinSetOfParticlesStep, step 13, time 2021-04-17 > 04:05:07.574152 > [35mFINISHED[0m: joinSetOfParticlesStep, step 13, time 2021-04-17 > 04:26:59.879548 > [35mSTARTED[0m: trainCNN, step 14, time 2021-04-17 04:26:59.920444 > None None None None > > $ /usr/local/EMAN_2.21/bin/conda info --env > # conda environments: > # > 3DFSC /home/panos/.conda/envs/3DFSC > cryoassess-0.1.0 /home/panos/.conda/envs/cryoassess-0.1.0 > cryolo /home/panos/.conda/envs/cryolo> cryolo /usr/local/EMAN_2.21/envs/cryolo > cryolo-1.5.4 /usr/local/EMAN_2.21/envs/cryolo-1.5.4 > fsc /usr/local/EMAN_2.21/envs/fsc > topaz /usr/local/EMAN_2.21/envs/topaz > root * /usr/local/EMAN_2.21 > > >>>> 'xmipp_DLTK_v0.3' conda environment not found... (check list above) > Traceback (most recent call last): > File > "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py", > line 189, in run > self._run() > File > "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py", > line 240, in _run > resultFiles = self._runFunc() > File > "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/pyworkflow/protocol/protocol.py", > line 236, in _runFunc > return self._func(*self._args) > File > "/opt/Scipion env=self.getCondaEnv()) > File > "/opt/Scipion3/.scipion3env/lib/python3.6/site-packages/xmipp3/base.py", > line 204, in getCondaEnv > return CondaEnvManager.getCondaEnv(env, envName) > File "/opt/Scipion3/software/bindings/xmipp_base.py", line 245, in > getCondaEnv > envBin = os.path.join(environDir, "bin") > File "/usr/lib64/python3.6/posixpath.py", line 80, in join > a = os.fspath(a) > TypeError: expected str, bytes or os.PathLike object, not NoneType > [31mProtocol failed: expected str, bytes or os.PathLike object, not > NoneType[0m > [35mFAILED[0m: trainCNN, step 14, time 2021-04-17 04:27:00.607417 > *** Last status is failed > Sent from Mail for Windows 10 > > > > > > _______________________________________________scipion-users mailing > lis...@li...https://lists.sourceforge.net/lists/listinfo/scipion-users--<http://lists.sourceforge.net/lists/listinfo/scipion-users--> > > Pablo Conesa - Madrid Scipion team > > > > > > _______________________________________________scipion-users mailing > lis...@li...https:/> > > > -- Pablo Conesa - *Madrid Scipion <http://scipion.i2pc.es> team* Aquest missatge, i els fitxers adjunts que hi pugui haver, pot contenir informació confidencial o protegida legalment i s’adreça exclusivament a la persona o entitat destinatària. Si no consteu com a destinatari final o no teniu l’encàrrec de rebre’l, no esteu autoritzat a llegir-lo, retenir-lo, modificar-lo, distribuir-lo, copiar-lo ni a revelar-ne el contingut. Si l’heu rebut per error, informeu-ne el remitent i elimineu del sistema tant el missatge com els fitxers adjunts que hi pugui haver. Este mensaje, y los ficheros adjuntos que pueda incluir, puede contener información confidencial o legalmente protegida y está exclusivamente dirigido a la persona o entidad destinataria. Si usted no consta como destinatario final ni es la persona encargada de recibirlo, no está autorizado a leerlo, retenerlo, modificarlo, distribuirlo o copiarlo, ni a revelar su contenido. Si lo ha recibido por error, informe de ello al remitente y elimine del sistema tanto el mensaje como los ficheros adjuntos que pueda contener. This email message and any attachments it carries may contain confidential or legally protected material and are intended solely for the individual or organization to whom they are addressed. If you are not the intended recipient of this message or the person responsible for processing it, then you are not authorized to read, save, modify, send, copy or disclose any part of it. If you have received the message by mistake, please inform the sender of this and eliminate the message and any attachments it carries from your account. Aquest missatge, i els fitxers adjunts que hi pugui haver, pot contenir informació confidencial o protegida legalment i s’adreça exclusivament a la persona o entitat destinatària. Si no consteu com a destinatari final o no teniu l’encàrrec de rebre’l, no esteu autoritzat a llegir-lo, retenir-lo, modificar-lo, distribuir-lo, copiar-lo ni a revelar-ne el contingut. Si l’heu rebut per error, informeu-ne el remitent i elimineu del sistema tant el missatge com els fitxers adjunts que hi pugui haver. Este mensaje, y los ficheros adjuntos que pueda incluir, puede contener información confidencial o legalmente protegida y está exclusivamente dirigido a la persona o entidad destinataria. Si usted no consta como destinatario final ni es la persona encargada de recibirlo, no está autorizado a leerlo, retenerlo, modificarlo, distribuirlo o copiarlo, ni a revelar su contenido. Si lo ha recibido por error, informe de ello al remitente y elimine del sistema tanto el mensaje como los ficheros adjuntos que pueda contener. This email message and any attachments it carries may contain confidential or legally protected material and are intended solely for the individual or organization to whom they are addressed. If you are not the intended recipient of this message or the person responsible for processing it, then you are not authorized to read, save, modify, send, copy or disclose any part of it. If you have received the message by mistake, please inform the sender of this and eliminate the message and any attachments it carries from your account. |
