Re: [scipion-users] Out of memory error on GPU when running 2D classification with Relion

[Carlos O. S. <co...@cn...>]

Dear Veli,

I would say this is something internal of Relion and Scipion cannot do 
much about it. Adding noise to an input signal is not the best you can 
do in image processing and you may not have the desired result either. 
Alternatively, what you can do is to use other 2D classifiers. In 
Scipion you have the ones of CryoSparc, Xmipp, and Eman. The Xmipp one 
is based on CPU and takes time, but its results are pretty accurate.

Kind regards, Carlos Oscar

On 3/25/20 11:19 PM, Veli-Pekka Kestilä wrote:
> Hi,
>
> I have problem when running Relion 2D classification with setting of:
>
> When running with option: Mask particles with zeros? No, fill with 
> random noise
>
> Run ends with out of memory error on GPU:
> 00112:   KERNEL_ERROR: out of memory in 
> /data/opt/sci/scipion-2.0/software/em/relion-3.0/src/acc/utilities_impl.h 
> at line 253 (error-code 2)
> 00113:   in: 
> /data/opt/sci/scipion-2.0/software/em/relion-3.0/src/acc/cuda/cuda_settings.h, 
> line 81
> 00114:   in: 
> /data/opt/sci/scipion-2.0/software/em/relion-3.0/src/acc/cuda/cuda_settings.h, 
> line 81
> 00115:   slave 2 encountered error: === Backtrace  ===
> 00116: 
> /data/opt/sci/scipion-2.0/software/em/relion-3.0/bin/relion_refine_mpi(_ZN11RelionErrorC1ERKSsS1_l+0x41) 
> [0x447ce1]
> 00117: 
> /data/opt/sci/scipion-2.0/software/em/relion-3.0/bin/relion_refine_mpi(_Z36globalThreadExpectationSomeParticlesR14ThreadArgument+0xe8) 
> [0x5cbb48]
> 00118: 
> /data/opt/sci/scipion-2.0/software/em/relion-3.0/bin/relion_refine_mpi(_Z11_threadMainPv+0x3f) 
> [0x48fbcf]
> 00119:   /lib64/libpthread.so.0(+0x7e65) [0x7f6e6b492e65]
> 00120:   /lib64/libc.so.6(clone+0x6d) [0x7f6e6b1bb88d]
>
> Run works when selecting: Yes, fill with zeroes (but the result isn't 
> what is hoped for).
>
> Is there known fix for this situation? There was similar bug in Relion 
> bug tracker, but it was closed without explanation.
>
> Specifications are:
> System: CentOS Linux release 7.7.1908 (Core)
> GPU: 2 x Tesla P100
> Memory: 128GB
> CPU: Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz
>
> Scipion is v2.0 (2019-04-23)
> Relion 3.0 is compiled against cuda-8.0
>
> Greetings,
> Veli-Pekka Kestilä
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users