Menu
▾
▴
Re: [scipion-users] Regarding running protocol on cluster
|
From: Pablo C. <pc...@cn...> - 2016-12-27 13:12:13
|
Thank you all! Great job.
Pablo
Scipion Team.
On 26/12/16 03:31, ashutosh srivastava wrote:
> Dear Grigory and Daniel
>
> Thank you so much for the help. It's working perfectly now.
>
> Best Regards
> Ashutosh
>
> On Sat, Dec 24, 2016 at 6:59 PM, Daniel Luque Buzo <dl...@is...
> <mailto:dl...@is...>> wrote:
>
> Hello Ashutosh,
>
> Find attached he hosts.conf file of out SGE cluster. The file is
> based scipion wiki example that Grigory pointed.
>
> The module lines are specific of our cluster (if you do not use
> module to manage environments/versions in you cluster should not
> be necessary).
>
> When we started to configure our cluster, we had the same problem
> for using more than one node but the problem was related with the
> parallel environment configuration of the cluster (named orte in
> our case).
>
> Hope it helps!
>
> Best Regards
> --
> Daniel Luque Buzo
> Unidad de Microscopía Electrónica y Confocal
> Centro Nacional de Microbiología - ISCIII
> Carretera de Majadahonda - Pozuelo, Km. 2.200
> 28220 - Majadahonda (Madrid)
> Tel: +34 91 822 39 71 <tel:+34%20918%2022%2039%2071>
> e-mail: dl...@is... <mailto:dl...@is...>
>
>
> [localhost]
> PARALLEL_COMMAND = mpirun -np %_(JOB_NODES)d -bynode %_(COMMAND)s
> MANDATORY = 2
> NAME = OGE
> CANCEL_COMMAND = qdel %_(JOB_ID)s
> CHECK_COMMAND = qstat -j %_(JOB_ID)s
> SUBMIT_COMMAND = qsub %_(JOB_SCRIPT)s
> SUBMIT_PREFIX = scipion
> SUBMIT_TEMPLATE = #!/bin/bash
> ###====================================================###
> #$ -V
> #$ -S /bin/bash
> #$ -cwd ### Use the current working directory
> #$ -N scipion%_(JOB_NAME)s ### Job name
> #$ -q %_(JOB_QUEUE)s ### Queue name
> #$ -pe %_(JOB_PE)s %_(JOB_SLOTS)s
> #$ -j y ### Merge stdin and stdout
> ###=======================================================#
> module load Python/python-2.7.11
> module load OpenMPI/openmpi-1.10.0
> ##module load scipion-1.0.0
> %_(JOB_COMMAND)s
>
> QUEUES = { "all.q": [
> ["JOB_SLOTS","20","Select the proper number
> of cores"],
> ["JOB_PE","orte","Parallel Enviroment
> (PE)","Select the proper Parallel Enviroment (orte)"]
> ]
> }
>
>
>
>> El 24 dic 2016, a las 10:01, Grigory Sharov
>> <sha...@gm... <mailto:sha...@gm...>>
>> escribió:
>>
>> Hello Ashutosh,
>>
>> in scipion wiki there is an example for SGE batch system. You
>> might check it out and give it a try:
>>
>> https://github.com/I2PC/scipion/wiki/Host-Configuration
>> <https://github.com/I2PC/scipion/wiki/Host-Configuration>
>>
>> Best regards,
>> Grigory
>>
>> --------------------------------------------------------------------------------
>> Grigory Sharov, Ph.D.
>> Institute of Genetics and Molecular and Cellular Biology
>> Integrated Structural Biology Department (CBI)
>> 1, rue Laurent Fries
>> 67404 Illkirch, France
>> tel. 03 69 48 51 00
>> e-mail: sh...@ig... <mailto:sh...@ig...>
>>
>> On Sat, Dec 24, 2016 at 2:25 AM, ashutosh srivastava
>> <ash...@gm... <mailto:ash...@gm...>> wrote:
>>
>> Dear Scipion Users
>>
>> I am analysing ~65000 particles in SPA protocol. I am trying
>> to do this on a cluster which uses SGE as job scheduler.
>> Since the hosts.conf file is provided for PBS/Torque, I tried
>> converting it into SGE type script but it failed in using
>> more than one nodes. So I went ahead and used a bash
>> jobscript to run the ML2D in the following way
>>
>> #!/bin/bash
>> #$ -N classification_2d
>> #$ -pe openmpi 96
>> #$ -o pass
>> #$ -e error
>> #$ -cwd
>> #$ -q BLAB
>> export MPIDIR=/opt/tools/mpi/openmpi-1.10.2-gcc4.4.7/bin
>> $MPIDIR/mpirun -machinefile $TMPDIR/machines -np $NSLOTS
>> /opt/tools/scipion/scipion xmipp_mpi_ml_align2d -i
>> input/particles65818_export.st
>> <http://particles65818_export.st/>k --nref 200 --fast --oroot
>> output/ml2d --mirror
>>
>> This runs fine, generating following files
>> ml2dclasses.stk
>> ml2dclasses.xmd
>> ml2dextra (folder containing iteration data)
>> ml2d_images_average.xmp
>> ml2dimages.xmd
>>
>> However I am having trouble now to write a similar script for
>> generating initial volume using RANSAC.
>> Is there a way to use the protocol_ransac.py in a script
>> similar to one mentioned above? If not then could some one
>> guide me regarding this?
>> Apart from this I would also like to know as to how I can
>> import the classes generated as mentioned above to scipion
>> gui to visualize them.
>>
>> Thank you
>>
>> Best Regards
>> Ashutosh
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Developer Access Program for Intel Xeon Phi Processors
>> Access to Intel Xeon Phi processor-based developer platforms.
>> With one year of Intel Parallel Studio XE.
>> Training and support from Colfax.
>> Order your platform today.http://sdm.link/intel
>> _______________________________________________
>> scipion-users mailing list
>> sci...@li...
>> <mailto:sci...@li...>
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>> <https://lists.sourceforge.net/lists/listinfo/scipion-users>
>>
>>
>> ------------------------------------------------------------------------------
>> Developer Access Program for Intel Xeon Phi Processors
>> Access to Intel Xeon Phi processor-based developer platforms.
>> With one year of Intel Parallel Studio XE.
>> Training and support from Colfax.
>> Order your platform
>> today.http://sdm.link/intel_______________________________________________
>> <http://sdm.link/intel_______________________________________________>
>> scipion-users mailing list
>> sci...@li...
>> <mailto:sci...@li...>
>> https://lists.sourceforge.net/lists/listinfo/scipion-users
>> <https://lists.sourceforge.net/lists/listinfo/scipion-users>
>
> ************************* AVISO LEGAL *************************
> Este mensaje electrónico está dirigido exclusivamente a sus
> destinatarios, pudiendo contener documentos anexos de carácter
> privado y confidencial. Si por error ha recibido este mensaje y no
> se encuentra entre los destinatarios, por favor no use, informe,
> distribuya, imprima o copie su contenido por ningún medio. Le
> rogamos lo comunique al remitente y borre completamente el mensaje
> y sus anexos. El Instituto de Salud Carlos III no asume ningún
> tipo de responsabilidad legal por el contenido de este mensaje
> cuando no responda a las funciones atribuidas al remitente del
> mismo por la normativa vigente.
>
>
>
>
> ------------------------------------------------------------------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
>
>
> _______________________________________________
> scipion-users mailing list
> sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scipion-users
|
