denovoassembler-devel Mailing List for Ray: scalable assembly (Page 2)
Ray -- Parallel genome assemblies for parallel DNA sequencing
Brought to you by:
sebhtml
Menu
▾
▴
denovoassembler-devel — Development mailing list
You can subscribe to this list here.
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(42) |
Aug
(4) |
Sep
|
Oct
|
Nov
|
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(2) |
Nov
(8) |
Dec
(4) |
| 2013 |
Jan
(6) |
Feb
(21) |
Mar
|
Apr
|
May
(3) |
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(3) |
Nov
|
Dec
|
| 2014 |
Jan
|
Feb
(8) |
Mar
(10) |
Apr
|
May
(1) |
Jun
(5) |
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Sébastien B. <seb...@ul...> - 2013-10-28 18:23:22
|
Hi Jacques, Good news ! It worked (but did not finish as expected). Compute job: HiSeq-2500-NA12878-demo-2x150-8 - I used 8 ranks instead of 16 per node and it worked. Reads were loaded, the graph was built and the extensions generated in under 4 hours. This was with 2504 ranks. The time was capped to 12 hours because the job was not large enough. 1.1 billion input sequences, paired The graph had 5.6 billion vertices. The merging did not complete however. There is a ticket open on Ray's github page for improving the merging code. https://github.com/sebhtml/ray/issues/82 Compute job: HiSeq-2500-NA12878-demo-2x150-9 - I will use 8 ranks per node, and 626 nodes instead of 313. If you are wondering where I am pulling these numbers (313), they are from Titan's scheduling policy: https://www.olcf.ornl.gov/kb_articles/titan-scheduling-policy/ So that's 5008 ranks. This will run at most 12 hours (maximum). Accounting: 626*30*12 = 225360 (626 nodes, 30 core-hours are charged for every node-hour, duration: 12 hours) titan> cat HiSeq-2500-NA12878-demo-2x150-9.sh #PBS -N HiSeq-2500-NA12878-demo-2x150-9 #PBS -l walltime=00:12:00:00 #PBS -l nodes=626 #PBS -A LSC005 #PBS -l gres=widow1 cd $PBS_O_WORKDIR # 626 * 8 = 5008 aprun -n 5008 \ ./software/lsc005/Ray/616d2a26cc1e39f59325a0e632af46262edaa12c-1/Ray \ -k 31 \ -detect-sequence-files HiSeq-2500-NA12878-demo-2x150 \ -o HiSeq-2500-NA12878-demo-2x150-9 \ titan> qsub HiSeq-2500-NA12878-demo-2x150-9.sh 1769459 see https://github.com/sebhtml/ray/issues/197 |
|
From: Sébastien B. <seb...@ul...> - 2013-10-21 21:18:27
|
On 21/10/13 05:06 PM, Fernanda Foertter via RT wrote: > > Thanks Sebastien, > > My gut tells me you're running out of memory per core. Hugepage is busting and > the max size is 2GB. Ray does not use Hugepages. As I understand, they won't be used with standard malloc or C++'s new. You need to mmap with MAP_HUGETLB to use them, which I don't do in Ray. So AFAIK, hugepages are not at involved here, at least not in Ray (MPICH2 seems to use plenty of them though). Yes. I think also that this is the case. Is there a symmetric hard limit on the memory per core/rank (32 GiB / 16 = 2 GiB) ? But, I mean, Cached is at 22 GiB, and it is possibly 100% reclaimable via VM pressure. That's 1.3 GiB per core. > MPIU_nem_gni_get_hugepages(): large page stats: free 0 nr 211 nr_overcommit > 16154 resv 0 surplus 211 > Sure. But that seems to be related to the meminfo Cached entry which should get purged since free pagecache, dentries and inodes are not required at this point by Ray since all the data is on the heap. Linux usually purges these with vfs_cache_pressure. > The network is just the one to complain about it, but not necessarily the > cause. > > Have you tried lowering the number of MPI processes to 8/node? I have 2 jobs in the queue: - one with Ray -debug ... - one with -N 2504 instead of -N 5008 (using 313 nodes). They will probably start in 1 week or 2 weeks since our allocation for LSC005 is super small (250 k core-hours 2013, and 750 k for next year up to the summer. > > FF > > On Mon Oct 21 16:47:01 2013, seb...@ul... wrote: >> On 21/10/13 04:38 PM, Fernanda Foertter via RT wrote: >>> >>> Hi Sebastien, >> >> Hello FF, >> >>> >>> Could you send me the list of modules you have loaded for the job to >> run? ( >>> cmd: module list ) >> >> To compile Ray (C++ / MPI), I used >> >> module purge >> module load PrgEnv-intel/4.1.40 >> module load cray-mpich2/5.6.3 >> module load git/1.8.2.1 >> >> My build script is https://github.com/sebhtml/Ray-on- >> Cray/blob/master/titan.ccs.ornl.gov/Build-on-Titan.sh >> >> >> >> I am not loading any module in my job script. >> It seems that aprun is already in my PATH. Anyway, I don't know if >> aprun is forked from >> bash (which implies loading all the default modules). >> >> And my executable is statically linked: >> >> titan> file >> ./software/lsc005/Ray/616d2a26cc1e39f59325a0e632af46262edaa12c-1/Ray >> ./software/lsc005/Ray/616d2a26cc1e39f59325a0e632af46262edaa12c-1/Ray: >> ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), for >> GNU/Linux 2.6.4, statically linked, not stripped >> >> >> >> >> I personally found that the sheer amount of modules that are loaded by >> default >> is very high. Hence the module purge that I did. >> >> >> Default modules loaded: >> >> titan> module list >> Currently Loaded Modulefiles: >> 1) modules/3.2.6.6 13) csa/3.0.0- >> 1_2.0401.37452.4.50.gem 25) xe-sysroot/4.1.40 >> 2) craype-network-gemini 14) dvs/1.8.6_0.9.0- >> 1.0401.1401.1.120 26) atp/1.6.1 >> 3) xt-asyncpe/5.17 15) rca/1.0.0- >> 2.0401.38656.2.2.gem 27) PrgEnv-pgi/4.1.40 >> 4) pgi/12.10.0 16) audit/1.0.0- >> 1.0401.37969.2.32.gem 28) cray-mpich2/5.6.3 >> 5) xt-libsci/12.0.00 17) ccm/2.2.0- >> 1.0401.37254.2.142 29) xtpe-interlagos >> 6) udreg/2.3.2-1.0401.5929.3.3.gem 18) configuration/1.0- >> 1.0401.35391.1.2.gem 30) eswrap/1.0.15 >> 7) ugni/4.0-1.0401.5928.9.5.gem 19) hosts/1.0- >> 1.0401.35364.1.115.gem 31) torque/4.2.5-snap.201308291703 >> 8) pmi/4.0.1-1.0000.9421.73.3.gem 20) lbcd/2.1- >> 1.0401.35360.1.2.gem 32) moab/7.1.3 >> 9) dmapp/3.2.1-1.0401.5983.4.5.gem 21) nodehealth/5.0- >> 1.0401.38460.12.18.gem 33) lustredu/1.2 >> 10) gni-headers/2.1-1.0401.5675.4.4.gem 22) pdsh/2.26- >> 1.0401.37449.1.1.gem 34) DefApps >> 11) xpmem/0.1-2.0401.36790.4.3.gem 23) shared-root/1.0- >> 1.0401.37253.3.50.gem 35) altd/1.0 >> 12) job/1.5.5-0.1_2.0401.35380.1.10.gem 24) switch/1.0- >> 1.0401.36779.2.72.gem >> >> >> And these are loaded elsewhere, not in by bashrc: >> >> titan> echo $SHELL >> /bin/bash >> titan> cat ~/.bashrc >> >> PS1="titan> " >> >> >> >> Thank you for your time and concern. >> >>> >>> FF >>> >>> On Mon Oct 21 15:06:51 2013, seb...@ul... wrote: >>>> On 21/10/13 02:48 PM, Fernanda Foertter via RT wrote: >>>>> >>>>> Ah, thank you for clarifying. I'm not even sure what would happen >> if >>>> those two >>>>> are called. Probably a singularity in the universe...or something. >>>> >>>> Haha. >>>> >>>> I suppose it would work, but each nested mpiexec would probably be >>>> spawning >>>> within its confined compute node. >>>> >>>> >>>> >>>>> >>>>> I have some folks looking into this. When you get the debug run >>>> complete, >>>>> please let me know what happens. >>>> >>>> I will. >>>> >>>> The -debug option basically collects all RayPlatform events for >> every >>>> period of >>>> 1 second (1000 ms). It also reports the value of various memory >>>> indicators >>>> (VmData, VmStack, VmRSS, and so on) >>>> >>>>> >>>>> FF >>>>> >>>> >>> >> > > > -- > |
|
From: Sébastien B. <seb...@ul...> - 2013-10-21 15:21:02
|
Hello, I worked on this issue, but have not find a fix yet. I am running test jobs with 313 nodes and 5008 MPI ranks. Carlos P. Sosa (from Cray) suggested that I add -N 16 to my command, which is now: titan> cat HiSeq-2500-NA12878-demo-2x150-6.sh #PBS -N HiSeq-2500-NA12878-demo-2x150-6 #PBS -l walltime=12:00:00 #PBS -l nodes=313 #PBS -A LSC005 #PBS -l gres=widow1 cd $PBS_O_WORKDIR # 313 * 8 * 2 = 5008 #-debug \ aprun -n 5008 -N 16 \ ./software/lsc005/Ray/616d2a26cc1e39f59325a0e632af46262edaa12c-1/Ray \ -k 31 \ -detect-sequence-files HiSeq-2500-NA12878-demo-2x150 \ -o HiSeq-2500-NA12878-demo-2x150-6 \ The job still crashes. The virtual memory usage of one single rank looks like this before crashing: Rank 3755: assembler memory usage: 106072 KiB Rank 3755: assembler memory usage: 171804 KiB Rank 3755: Rank= 3755 Size= 5008 ProcessIdentifier= 17973 Rank 3755 is testing the network [0/1000] Rank 3755 is testing the network [1000/1000] Rank 3755: mode round trip latency when requesting a reply for a message of 4000 bytes is 27 microseconds (10^-6 seconds) Rank 3755: average round trip latency when requesting a reply for a message of 4000 bytes is 263 microseconds (10^-6 seconds) Rank 3755 is loading sequence reads Rank 3755 : partition is [878283235;878517131], 233897 sequence reads Rank 3755 is fetching file HiSeq-2500-NA12878-demo-2x150/sorted_S1_L002_R1_002.fastq.gz with lazy loading (please wait...) Rank 3755 has 0 sequence reads Rank 3755: assembler memory usage: 3410716 KiB Rank 3755 has 100000 sequence reads Rank 3755: assembler memory usage: 3410716 KiB Rank 3755 has 200000 sequence reads Rank 3755: assembler memory usage: 3410716 KiB Rank 3755 has 233897 sequence reads (completed) Rank 3755 created its Bloom filter Rank 3755 is counting k-mers in sequence reads [1/233897] Rank 3755 has 100000 vertices Rank 3755: assembler memory usage: 3423004 KiB Rank 3755 has 200000 vertices Rank 3755: assembler memory usage: 3423004 KiB Rank 3755 has 300000 vertices Rank 3755: assembler memory usage: 3423004 KiB Rank 3755 has 400000 vertices Rank 3755: assembler memory usage: 3423004 KiB So the first report of virtual memory usage is ~ 100 MiB. But then when file I/O commences, it goes up to 3.4 GiB (this must include Cached and other things shared between processes I suppose). In the error log, MPICH2 complains that he can't allocate 12 MiB: MPICH2 ERROR [Rank 3755] [job id 3742358] [Wed Oct 16 08:11:23 2013] [c1-5c1s5n1] [nid18933] - MPIU_nem_gni_get_hugepages(): Unable to mmap 12582912 bytes for file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.2.17973.kvs_3742358, err Cannot allocate memory Error log: /ccs/home/sebhtml/lsc005/projects/human-1-hour/HiSeq-2500-NA12878-demo-2x150-6.e1754627 Each node has 33 GiB of memory. When the crash strikes, 22 GiB are Cached (due to file I/O), and 12 GiB is Inactive. There is probably some sort of overlap between Cached and Inactive. See below the /proc/meminfo: [Rank 3758] Cat of /proc/meminfo [Rank 3755]: MemTotal: 33084652 kB [Rank 3755]: MemFree: 3984520 kB [Rank 3755]: Buffers: 0 kB [Rank 3755]: Cached: 22332700 kB [Rank 3755]: SwapCached: 0 kB [Rank 3755]: Active: 12556068 kB [Rank 3755]: Inactive: 12527116 kB [Rank 3758]: MemTotal: 33084652 kB [Rank 3755]: Active(anon): 2637848 kB [Rank 3758]: MemFree: 3984892 kB [Rank 3755]: Inactive(anon): 168920 kB [Rank 3758]: Buffers: 0 kB [Rank 3755]: Active(file): 9918220 kB [Rank 3758]: Cached: 22332700 kB [Rank 3755]: Inactive(file): 12358196 kB Also, DirectMap1G is 17825792 kB, which may mean that calls to brk() will join 1G segments to the virtual memory of my processes. My last piece of evidence is what is reported by slabinfo (the slab allocator is *supposed* to use slabs to allocate memory, hence should reuse memory as much as possible). Here is what slabinfo had to say: [Rank 3758]: slabinfo - version: 2.1 [Rank 3758]: # name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail> [Rank 3758]: ll_async_page 5569508 6526674 336 11 1 : tunables 54 27 8 : slabdata 593334 593334 432 So there are 5569508 active_objs in the ll_async_page slab cache. From the evidence I gathered, here is my hypothesis: This is a problem with the Lustre client on every compute nodes. I am using Spider's widow 1. I will rerun with the -debug option. Sébastien On 26/09/13 10:49 AM, Fernanda Foertter via RT wrote: > > Hi Sébastien, > > I can't find any reason why MPICH would use that much memory. Although, you > appear to list virtual memory, not actual memory. > One way to avoid running out of memory on compute nodes is to reduce the number > of cores (MPI tasks) per node. That way, there is more memory per core. > > Also, have you tried profiling your application? You can find information of > available profiling and debugging tools here: > https://www.olcf.ornl.gov/support/software/?softwaretype=kb_software_debugging_and_profiling > > Happy Computing! > F² > > > -------------------------------------------------------- > Fernanda Foertter > National Center for Computational Sciences > Oak Ridge National Laboratory > O: 865-576-9391 F: 865-241-2850 > foe...@or... > > > > On Mon Sep 23 16:54:33 2013, seb...@ul... wrote: >> Dear OLCF support: >> >> I launched a job (Job # 1732882) with 313 nodes (5008 MPI ranks) >> on titan. >> >> >> The code is Ray ( http://denovoassembler.sourceforge.net/ ) and >> does not use the GPU. I am the author of Ray. >> >> >> The job failed with a MPICH2 error. >> >> >> Standard error: >> /tmp/proj/lsc005/projects/human-1-hour/HiSeq-2500-NA12878-demo- >> 2x150-3.e1732882 >> >> Standard output: >> /tmp/proj/lsc005/projects/human-1-hour/HiSeq-2500-NA12878-demo- >> 2x150-3.o1732882 >> >> Launch script: >> /tmp/proj/lsc005/projects/human-1-hour/HiSeq-2500-NA12878-demo- >> 2x150-3.sh >> >> Output directory: >> /tmp/proj/lsc005/projects/human-1-hour/HiSeq-2500-NA12878-demo- >> 2x150-3 >> >> >> >> >> The errors I got: >> >> MPICH2 ERROR [Rank 1227] [job id 3577704] [Mon Sep 16 20:34:24 2013] >> [c19-4c0s2n1] [nid12091] - MPIU_nem_gni_get_hugepages(): Unable to >> mmap 12582912 bytes for file /var/lib/hugetlbfs/global/pagesize- >> 2097152/hugepagefile.MPICH.2.27853.kvs_3577704, err Cannot allocate >> memory >> MPICH2 ERROR [Rank 1227] [job id 3577704] [Mon Sep 16 20:34:24 2013] >> [c19-4c0s2n1] [nid12091] - MPIU_nem_gni_get_hugepages(): large page >> stats: free 0 nr 158 nr_overcommit 16154 resv 0 surplus 158 >> MPICH2 ERROR [Rank 1230] [job id 3577704] [Mon Sep 16 20:34:24 2013] >> [c19-4c0s2n1] [nid12091] - MPIU_nem_gni_get_hugepages(): Unable to >> mmap 12582912 bytes for file /var/lib/hugetlbfs/global/pagesize- >> 2097152/hugepagefile.MPICH.2.27856.kvs_3577704, err Cannot allocate >> memory >> MPICH2 ERROR [Rank 1230] [job id 3577704] [Mon Sep 16 20:34:24 2013] >> [c19-4c0s2n1] [nid12091] - MPIU_nem_gni_get_hugepages(): large page >> stats: free 0 nr 165 nr_overcommit 16154 resv 0 surplus 165 >> MPICH2 ERROR [Rank 4378] [job id 3577704] [Mon Sep 16 20:34:24 2013] >> [c0-2c1s6n0] [nid00114] - MPIU_nem_gni_get_hugepages(): Unable to >> mmap 12582912 bytes for file /var/lib/hugetlbfs/global/pagesize- >> 2097152/hugepagefile.MPICH.2.24160.kvs_3577704, err Cannot allocate >> memory >> MPICH2 ERROR [Rank 4378] [job id 3577704] [Mon Sep 16 20:34:24 2013] >> [c0-2c1s6n0] [nid00114] - MPIU_nem_gni_get_hugepages(): large page >> stats: free 0 nr 173 nr_overcommit 16154 resv 0 surplus 173 >> >> >> >> >> The reported memory usage (I use the VmData entry in >> /proc/self/status) was > 3 GiB per MPI rank at the beginning, before >> my application does anything. >> >> > > > -- > |
|
From: Sébastien B. <seb...@ul...> - 2013-09-25 14:21:06
|
Dear developers, There is a new debugging option: * New -debug option which activates RayPlatform profiler Example: [boiseb01@ls30 ray]$ mpiexec -n 16 ./Ray -debug -test-network-only -o test-debug-9 This will generate a report like this every 100 milliseconds: [/dev/actor/rank/1] [RayPlatform] epoch ends at 99 ms ! (tick # 38201), length is 100 ms, VmData is 91248 KiB Rank 1: RAY_SLAVE_MODE_TEST_NETWORK Time= 0.10 s Speed= 38201 Sent= 16 (processMessages: 7, processData: 9) Received= 17 Balance= -1 Rank 1 granularity of processData calls 0 28601 1 9576 2 1 10 2 11 2 14 5 15 1 16 1 18 4 19 3 20 2 21 2 37 1 Rank 1 END of granularity of processData calls Rank 1 received in receiveMessages: Rank 1 RAY_MPI_TAG_SET_WORD_SIZE 1 Rank 1 RAY_MPI_TAG_TEST_NETWORK_MESSAGE 7 Rank 1 RAY_MPI_TAG_TEST_NETWORK 1 Rank 1 RAY_MPI_TAG_TEST_NETWORK_MESSAGE_REPLY 8 Rank 1 sent in processMessages: Rank 1 RAY_MPI_TAG_TEST_NETWORK_MESSAGE_REPLY 7 Rank 1 sent in processData: Rank 1 RAY_MPI_TAG_TEST_NETWORK_MESSAGE 9 |
|
From: Sébastien B. <seb...@ul...> - 2013-05-29 14:21:31
|
On 27/05/13 03:03 PM, Marpaud@ULaval wrote:
> Bonjour,
>
> c'était une question théorique, l'hypothèse étant que l'utilisateur final
> peux mettre en entrée du programme un fichier de n'importe quelle taille !
A OK.
Quand ça arrive, Ray va afficher "Out of memory".
>
> Le Mon, 27 May 2013 14:57:13 -0400, Sébastien Boisvert
> <seb...@ul...> a écrit:
>
>> Combien de séquences as-tu ?
>>
>>
>> On 27/05/13 01:58 PM, Carl Poirier wrote:
>>> Bonjour Sébastien,
>>>
>>> J'ai une petite question concernant les ressources.
>>>
>>> Si on roule un gros problème d'assemblage sur un trop petit nombre de
>>> noeuds du super-calculateur (pas assez de RAM pour charger tous les
>>> reads, le graph et autres), nous croyons que le programme ne fait que
>>> planter. Sommes-nous justes?
>>>
>>> Merci,
>>>
>>> Carl
>>> ________________________________________
>>> De : Carl Poirier
>>> Date d'envoi : 24 mai 2013 14:12
>>> À : Sébastien Boisvert
>>> Cc : Cyril Marpaud; Jacques Corbeil; François Laviolette;
>>> den...@li...
>>> Objet : RE : RAY
>>>
>>> Bonjour Sébastien,
>>>
>>> Merci pour les clarifications. C'est très apprécié.
>>>
>>> Nous devrions commencer quelques expérimentations dès la semaine
>>> prochaine. Nous vous tiendrons au courant des développements.
>>>
>>> Carl Poirier
>>> ________________________________________
>>> De : Sébastien Boisvert [seb...@ul...]
>>> Date d'envoi : 23 mai 2013 11:30
>>> À : Marpaud@Gmail
>>> Cc : Carl Poirier; Cyril Marpaud; Jacques Corbeil; François Laviolette;
>>> den...@li...
>>> Objet : Re: RAY
>>>
>>> (J'ai mis en C.C. Jacques Corbeil et François Laviolette pour les
>>> laisser savoir
>>> que je mets du temps sur ce projet.)
>>>
>>> On 21/05/13 04:27 PM, Marpaud@Gmail wrote:
>>>> Rebonjour,
>>>>
>>>> j'ai joint à ce mail un .txt produit par Carl Poirier (l'étudiant en
>>>> maîtrise qui m'a rejoint sur le projet) et moi-même. Puis-je vous
>>>> demander d'y jeter un coup d'œil pour :
>>>>
>>>> * confirmer que notre analyse est correcte
>>>> * clarifier certains points
>>>>
>>>> si vous en avez le temps ? Les points à clarifier sont signalés dans
>>>> le fichier par la présence de la chaine "=> ?".
>>>>
>>>> Encore une fois, merci !
>>>>
>>>> --
>>>> Cyril MARPAUD
>>>> Étudiant ingénieur à l'ENSICAEN
>>>> 3ème année électronique et physique appliquée
>>>> Signal, Automatique et Télécoms pour l'Embarqué
>>>> Ligne fixe : 418 656-7777 poste 1-7839
>>>> @ : cma...@gm...
>>>
>>>
>>> Reviewer: Sébastien Boisvert
>>> File: descriptionAlgorithme.txt
>>>
>>> Patch applies cleanly:
>>> [ ] Yes
>>> [ ] No
>>> [x] Not applicable
>>>
>>> The code works:
>>> [ ] Yes
>>> [ ] No
>>> [x] Not applicable
>>>
>>> Major concerns:
>>>
>>> Code Review:
>>>
>>>> * : les noms de plugins précédés d'une étoile sont ceux dont les
>>>> master modes sont situés dans le fichier MachineHelper.cpp
>>>
>>> Dans le meilleur des mondes, ces adapteurs devraient être migrés dans
>>> d'autres plugins.
>>>
>>>>
>>>> Partitioner
>>>> Il s'agit de compter le nombre d'entrées dans les fichiers fournis au
>>>> séquenceur.
>>>
>>> fournis à l'assembleur, pas au séquenceur.
>>>
>>>> Chaque esclave a un certain nombre de fichiers à compter.
>>>
>>> Exactement ! Ça comprend aussi le rang 0.
>>>
>>>>
>>>> Les fichiers peuvent être de 3 formats différents:
>>>> -SFF: Voir ray-master/code/SequencesLoader/SffLoader.cpp
>>>> -FASTA: Voir ray-master/code/SequencesLoader/FastaLoader.cpp
>>>> -FASTQ: Voir ray-master/code/SequencesLoader/FastqLoader.cpp
>>>>
>>>
>>> Il y a beaucoup plus de formats:
>>>
>>> .fasta
>>> .fa
>>> .fasta.gz (needs HAVE_LIBZ=y at compilation)
>>> .fa.gz (needs HAVE_LIBZ=y at compilation)
>>> .fasta.bz2 (needs HAVE_LIBBZ2=y at compilation)
>>> .fa.bz2 (needs HAVE_LIBBZ2=y at compilation)
>>> .fastq
>>> .fq
>>> .fastq.gz (needs HAVE_LIBZ=y at compilation)
>>> .fq.gz (needs HAVE_LIBZ=y at compilation)
>>> .fastq.bz2 (needs HAVE_LIBBZ2=y at compilation)
>>> .fq.bz2 (needs HAVE_LIBBZ2=y at compilation)
>>> export.txt
>>> qseq.txt
>>> .sff (paired reads must be extracted manually)
>>> .csfasta (color-space reads)
>>> .csfa (color-space reads)
>>>
>>>
>>> Si votre est en C++, vous pouvez utiliser directement les classes de
>>> Ray.
>>>
>>> Sinon, je vous suggère de gérer initialement le fastq et le fastq.gz.
>>>
>>>> *SequencesLoader
>>>> Charger en mémoire les reads. Le maître calcule le partitionnement des
>>>> reads, puis chaque esclave charge sa partie en mémoire, dans
>>>> machine::ArrayOfReads.
>>>
>>> Oui.
>>>
>>> Il y a une partition sur les fichiers et une partitions sur les rangs.
>>> La première est envopyer à tout le monde et la seconde est calculée
>>> avec la
>>> première.
>>>
>>>>
>>>> *KmerAcademyBuilder
>>>> Chaque esclave compte le nombre de k-mers dans ses reads et l'envoie.
>>>
>>> Ce module calcul les sommets du graphe et utilise un filtre de Bloom.
>>>
>>>>
>>>> *CoverageGatherer
>>>> Les slaves envoient la distribution de la couverture, soit le nombre
>>>> de k-mers ayant la couverture donnée.
>>>>
>>>
>>> oui
>>>
>>>> *VerticesExtractor
>>>> RAY_MASTER_MODE_SEND_COVERAGE_VALUES
>>>> - Le maître envoie à tous les esclaves les valeurs maximales,
>>>> minimales et de répétition de la couverture.
>>>> RAY_SLAVE_MODE_ADD_EDGES
>>>> - Les esclaves fabriquent des edges. Lorsqu'ils en ont amassé un
>>>> certain nombre, ils les envoient. Pour trouver un edge, ils commencent
>>>> par décoder la séquence ADN d'un read. Si un k-mer a un sommet
>>>> précédent, il envoie le edge au bon rank à l'aide du BufferedData.
>>>> Sinon, il renverse le k-mer et refait la même vérification. Si n'en a
>>>> toujours pas un, il passe au prochain k-mer, qui aura comme sommet
>>>> précédent ce dernier.
>>>> -ligne 148 de VerticesExtractor.cpp: comment un DNA pourrait ne pas
>>>> être valide, ça provient de codeToChar() qui retourne que les bons
>>>> chars => ?
>>>
>>> Un ADN invalide contient autre chose que des A, T, C, G. En pratique,
>>> ça n'arrive pas je crois car
>>> il y a du code ailleur qui gère ça.
>>>
>>>>
>>>> *EdgePurger
>>>> RAY_MASTER_MODE_PURGE_NULL_EDGES
>>>> - passage en mode DO_NOTHING
>>>> - Avertit les autres de passer à cette étape via le message
>>>> RAY_MPI_TAG_PURGE_NULL_EDGES
>>>> RAY_SLAVE_MODE_PURGE_NULL_EDGES
>>>> -EdgePurger est un TaskCreator. Chaque worker demande aux autres la
>>>> couverture d'un certain sommet. Si la couverture est nulle, il élimine
>>>> le sommet.
>>>> -ligne 49 de EdgePurgeWorker.cpp: Pourquoi vérifie-t-on seulement la
>>>> première réponse et pas les suivantes => ?
>>>
>>> Parce que le message est démultiplexé déjà avant par le
>>> VirtualCommunicator (demande faite par le VirtualProcessor qui a
>>> réveillé
>>> le Worker.)
>>>
>>>>
>>>> MachineHelper.cpp
>>>> RAY_MASTER_MODE_WRITE_KMERS
>>>> - Chaque esclave ecrit dans le fichier kmers.txt les k-mers ainsi que
>>>> leur couverture. Ils envoient aussi la distribution des edges. Il
>>>> s'agit du nombre de k-mers avec un nombre de parents
>>>> et un nombre d'enfants donnés.
>>>
>>>
>>> (actif avec -write-kmers).
>>>
>>>> C'est le master qui écrira ceci dans le fichier degreeDistribution.txt.
>>>>
>>>> *SequencesIndexer
>>>> Chaque worker commence par demander la couverture des vertex. Il
>>>> sélectionne ensuite un vertex qui n'est pas une erreur selon la
>>>> couverture maximale et minimale. Il envoie ce vertex ainsi que sa
>>>> position dans le read. Il fait pareil pour son inverse. Selon les
>>>> options, il peut aussi écrire dans un fichier des marqueurs décrivant
>>>> les reads, soit leur position, leur coverage et autres.
>>>
>>> Exactement.
>>>
>>> Initialement, le code choisissait toujours index=0, mais ça donne de
>>> meilleurs résultats avec cet algorithme.
>>>
>>>>
>>>> *SeedingData
>>>> RAY_MASTER_MODE_PREPARE_SEEDING
>>>> - mise à jour de deux attributs
>>>> - closeMasterMode()
>>>> RAY_MASTER_MODE_TRIGGER_SEEDING
>>>> - envoi d'un message (RAY_MPI_TAG_START_SEEDING) à tous les ranks
>>>> RAY_SLAVE_MODE_START_SEEDING
>>>> - génération des seeds dans SeedWorker.cpp
>>>> RAY_SLAVE_MODE_SEND_SEED_LENGTHS
>>>> - envoi du message RAY_MPI_TAG_IS_DONE_SENDING_SEED_LENGTHS au rank 0
>>>> - passage en mode DO_NOTHING
>>>>
>>>> SpuriousSeedAnnihilator
>>>> RAY_MASTER_MODE_REGISTER_SEEDS & RAY_MASTER_MODE_FILTER_SEEDS &
>>>> RAY_MASTER_MODE_CLEAN_SEEDS & RAY_MASTER_MODE_PUSH_SEED_LENGTHS
>>>> - distribution non démarrée -> openMasterMode
>>>> - distribution terminée -> closeMasterMode
>>>> RAY_SLAVE_MODE_REGISTER_SEEDS
>>>> - si skip ou checkPoint déjà atteit -> return
>>>> - => ?
>>>> RAY_SLAVE_MODE_FILTER_SEEDS
>>>> - si checkPoint ou skip -> closeSlaveModeLocally()
>>>> - sinon TaskCreator.mainLoop() (cf FusionTaskCreator l.117)
>>>> RAY_SLAVE_MODE_CLEAN_SEEDS
>>>> - si checkPoint ou skip -> closeSeModeLocally()
>>>> - désallocation de pointeurs devenus inutiles après l'étape
>>>> d'extension des seeds
>>>> RAY_SLAVE_MODE_PUSH_SEED_LENGTHS
>>>> - récupération de "length" et "count" (deux attributs relatifs aux
>>>> seeds obtenus grâce à un itérateur sur map<int,int>)
>>>> - envoi de ces données au rank 0 via un message
>>>>
>>>> *Library
>>>> RAY_MASTER_MODE_TRIGGER_DETECTION
>>>> - envoi d'un message (RAY_MPI_TAG_AUTOMATIC_DISTANCE_DETECTION) à tous
>>>> les ranks
>>>> - passage en mode DO_NOTHING
>>>> RAY_SLAVE_MODE_AUTOMATIC_DISTANCE_DETECTION
>>>> - Si on a des paired reads, on termine tout de suite en envoyant
>>>> MASTER_RANK,RAY_MPI_TAG_AUTOMATIC_DISTANCE_DETECTION_IS_DONE
>>>> - Gestion du bug sur Guillimin
>>>
>>> Ce code est en commentaire et donc inactif. Le bug était dans Ray, pas
>>> causé par guillimin. Le bug arrivait seulement
>>> sur guillimin.clumeq.ca par contre.
>>>
>>>> - Les workers travaillent pour trouver les distances externes moyennes:
>>>> - Cela commence par les données des seeds dans lesquelles ils prennent
>>>> un vertex.
>>>> - Ils obtiennent les reads à partir du vertex.
>>>> - Pour chacun de ces reads:
>>>> - Ils trouvent l'autre lecture qui forme les paires de nucleotides en
>>>> faisant une demande (RAY_MPI_TAG_GET_READ_MATE)
>>>> - La distance = position read - position autre read + longueur read +
>>>> position strand gauche - position strand droit
>>>> - Finalement, la quantité d'une telle distance est incrementée.
>>>> RAY_MASTER_MODE_ASK_DISTANCES
>>>> - envoi d'un message (RAY_MPI_TAG_ASK_LIBRARY_DISTANCES) à tous les
>>>> ranks
>>>> RAY_SLAVE_MODE_SEND_LIBRARY_DISTANCES
>>>> - envoi au master la quantité de chaque distance pour chaque librairie
>>>> RAY_MASTER_MODE_START_UPDATING_DISTANCES
>>>> - Calcule les nouvelles distances moyennes à partir des résultats des
>>>> slaves
>>>> RAY_MASTER_MODE_UPDATE_DISTANCES
>>>> - Envoi des résultats à tous les ranks
>>>>
>>>> SeedExtender
>>>> RAY_MASTER_MODE_TRIGGER_EXTENSIONS
>>>> - envoi d'un message (RAY_MPI_TAG_ASK_EXTENSION) à tous les ranks
>>>> - passage en mode DO_NOTHING
>>>> RAY_SLAVE_MODE_EXTENSION
>>>> - vérifier checkPoint, si passé ("extensions")
>>>> - passage en mode DO_NOTHING
>>>> - message au rank 0 (RAY_MPI_TAG_EXTENSION_IS_DONE)
>>>> - si tous les seeds ont été gérés ou 0 seeds -> finalize()
>>>> - si initialisation pas faite -> initialize()
>>>> - si le vertex en cours (ou son revcomp) est déjà assemblé, on passe
>>>> au seed suivant
>>>> - sinon on le marque comme étant assemblé
>>>> - on "énumère les choix" puis on "fait un choix" => ?
>>>
>>> Ce code contient les heuristiques et la traversée du graphe.
>>>
>>>>
>>>> *FusionData
>>>> RAY_MASTER_MODE_TRIGGER_FUSIONS
>>>> - m_cycleNumber = 0;
>>>> - closeMasterMode()
>>>> RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS
>>>> - (*m_reductionOccured)=true;
>>>> - closeMasterMode()
>>>> RAY_MASTER_MODE_START_FUSION_CYCLE
>>>> - si cycle non démarré : envoi d'un message
>>>> (RAY_MPI_TAG_CLEAR_DIRECTIONS) à tous les ranks
>>>> - plusieurs cycles contenant les étapes suivantes :
>>>> 1 : envoi de RAY_MPI_TAG_DISTRIBUTE_FUSIONS
>>>> 2 : envoi de RAY_MPI_TAG_FINISH_FUSIONS
>>>> 3 : envoi de RAY_MPI_TAG_CLEAR_DIRECTIONS
>>>> 4 : envoi de RAY_MPI_TAG_DISTRIBUTE_FUSIONS
>>>> 5 : envoi de RAY_MPI_TAG_START_FUSION
>>>> 6 : reinitialisation pour cycle suivant => ?
>>>> RAY_SLAVE_MODE_DISTRIBUTE_FUSIONS
>>>>
>>>> FusionTaskCreator
>>>> RAY_SLAVE_MODE_FUSION
>>>> - TaskCreator -> mainLoop()
>>>> - initialisation()
>>>> - si il reste des tâches non assignées -> création de workers &
>>>> assignation tâche
>>>> - si plus de tâche non assignée -> noMoreTasksAreComing()
>>>> - on fait "travailler" un worker, on récupère le résultat de son
>>>> calcul puis on le détruit
>>>> - si tâches assignées accomplies & plus de tâche à assigner ->
>>>> finalize()
>>>>
>>>> JoinerTaskCreator
>>>> RAY_SLAVE_MODE_FINISH_FUSIONS
>>>> - TaskCreator -> mainLoop() (cf RAY_SLAVE_MODE_FUSION)
>>>>
>>>> PathEvaluator
>>>> (évaluation de la qualité d'un chemin)
>>>> RAY_MASTER_MODE_EVALUATE_PATHS
>>>> - openMasterMode() (boucle sur openSlaveMode(), une itération pour
>>>> chaque rank)
>>>> - si tous les ranks ont terminé -> closeMasterMode()
>>>> RAY_SLAVE_MODE_EVALUATE_PATHS
>>>> - writeCheckpointForContigPaths()
>>>> - écriture du nombre total de contigs (m_contig.size()) dans le
>>>> fichier checkpoint ContigPaths
>>>> - pour chaque contig, écriture dans ce fichier checkpoint de :
>>>> - "nom" du contig
>>>> - nombre de K-mers
>>>> - k-mers
>>>> - closeSlaveModeLocally() (envoi d'un message
>>>> (RAY_MPI_TAG_SWITCHMAN_COMPLETION_SIGNAL) au rank 0)
>>>>
>>>> MachineHelper.cpp
>>>> RAY_MASTER_MODE_ASK_EXTENSIONS
>>>> (demande aux ranks d'envoyer leurs extensions)
>>>> - étape 1 :
>>>> - initialisations
>>>> - envoi d'un message
>>>> (RAY_MPI_TAG_COMPUTE_REQUIRED_SPACE_FOR_EXTENSIONS) à tous les ranks
>>>> - étape 2 (quand tous les ranks ont terminé d'écrire leurs contigs) :
>>>> - si RAY configuré pour utiliser AMOS ->
>>>> setMasterMode(RAY_MASTER_MODE_AMOS);
>>>> - sinon -> closeMasterMode();
>>>> RAY_SLAVE_MODE_SEND_EXTENSION_DATA
>>>> - si autorisation (rank 0 ou message RAY_MPI_TAG_SEND_AUTHORIZATION
>>>> reçu) -> pour que deux ranks ne tentent pas d'écrire dans le fichier
>>>> en même temps
>>>> - ouverture fichier
>>>> - écriture de tous les contigs pour le rank en cours
>>>> - fermeture fichier
>>>> - passage en mode DO_NOTHING
>>>> - envoi d'un message (RAY_MPI_TAG_EXTENSION_DATA_END) au rank 0
>>>> - envoi d'un message (RAY_MPI_TAG_SEND_AUTHORIZATION) au rank suivant
>>>>
>>>> *Scaffolder
>>>> RAY_MASTER_MODE_SCAFFOLDER
>>>> - envoi d'un message (RAY_MPI_TAG_START_SCAFFOLDER) à tous les ranks
>>>> - passage en mode DO_NOTHING
>>>> RAY_SLAVE_MODE_SCAFFOLDER
>>>> - initialisations
>>>> - tant que tous les contigs n'ont pas été gérés -> processContig();
>>>> - envoi d'un message (RAY_MPI_TAG_I_FINISHED_SCAFFOLDING) au rank 0
>>>> - passage en mode DO_NOTHING
>>>> RAY_MASTER_MODE_WRITE_SCAFFOLDS
>>>> - initialisations (ouverture d'un fichier)
>>>> - tant qu'on a pas géré tous les scaffolds :
>>>> - tant qu'on a pas géré tous les contigs du scaffold en cours :
>>>> - getContigSequence();
>>>> - ecriture des scaffolds dans le fichier
>>>> - closeMasterMode();
>>>
>>>
>>> Pour votre premier milestone, vous pouvez probablement commencer par le
>>> support fastq seulement.
>>>
>>>
>>> Je pense que la partie de construction du graphe se fait sur un GPU ou
>>> sur un FPGA. Le reste, je ne suis pas certain du gain.
>>>
>>>
>>> Voir:
>>>
>>> Efficient Graph Based Assembly of Short-Read Sequences on Hybrid Core
>>> Architecture
>>> http://www.osti.gov/bridge/servlets/purl/1015333-q62bUB/1015333.pdf
>>> http://www.conveycomputer.com/files/2313/5095/9539/ConveyGraphConstructor_5172011.pdf
>>>
>>
>
>
> --
> Cyril MARPAUD
> Étudiant ingénieur à l'ENSICAEN
> 3ème année électronique et physique appliquée
> Signal, Automatique et Télécoms pour l'Embarqué
> Ligne fixe : 418 656-7777 poste 1-7839
> @ : cma...@gm...
>
|
|
From: Sébastien B. <seb...@ul...> - 2013-05-27 18:57:23
|
Combien de séquences as-tu ?
On 27/05/13 01:58 PM, Carl Poirier wrote:
> Bonjour Sébastien,
>
> J'ai une petite question concernant les ressources.
>
> Si on roule un gros problème d'assemblage sur un trop petit nombre de noeuds du super-calculateur (pas assez de RAM pour charger tous les reads, le graph et autres), nous croyons que le programme ne fait que planter. Sommes-nous justes?
>
> Merci,
>
> Carl
> ________________________________________
> De : Carl Poirier
> Date d'envoi : 24 mai 2013 14:12
> À : Sébastien Boisvert
> Cc : Cyril Marpaud; Jacques Corbeil; François Laviolette; den...@li...
> Objet : RE : RAY
>
> Bonjour Sébastien,
>
> Merci pour les clarifications. C'est très apprécié.
>
> Nous devrions commencer quelques expérimentations dès la semaine prochaine. Nous vous tiendrons au courant des développements.
>
> Carl Poirier
> ________________________________________
> De : Sébastien Boisvert [seb...@ul...]
> Date d'envoi : 23 mai 2013 11:30
> À : Marpaud@Gmail
> Cc : Carl Poirier; Cyril Marpaud; Jacques Corbeil; François Laviolette; den...@li...
> Objet : Re: RAY
>
> (J'ai mis en C.C. Jacques Corbeil et François Laviolette pour les laisser savoir
> que je mets du temps sur ce projet.)
>
> On 21/05/13 04:27 PM, Marpaud@Gmail wrote:
>> Rebonjour,
>>
>> j'ai joint à ce mail un .txt produit par Carl Poirier (l'étudiant en maîtrise qui m'a rejoint sur le projet) et moi-même. Puis-je vous demander d'y jeter un coup d'œil pour :
>>
>> * confirmer que notre analyse est correcte
>> * clarifier certains points
>>
>> si vous en avez le temps ? Les points à clarifier sont signalés dans le fichier par la présence de la chaine "=> ?".
>>
>> Encore une fois, merci !
>>
>> --
>> Cyril MARPAUD
>> Étudiant ingénieur à l'ENSICAEN
>> 3ème année électronique et physique appliquée
>> Signal, Automatique et Télécoms pour l'Embarqué
>> Ligne fixe : 418 656-7777 poste 1-7839
>> @ : cma...@gm...
>
>
> Reviewer: Sébastien Boisvert
> File: descriptionAlgorithme.txt
>
> Patch applies cleanly:
> [ ] Yes
> [ ] No
> [x] Not applicable
>
> The code works:
> [ ] Yes
> [ ] No
> [x] Not applicable
>
> Major concerns:
>
> Code Review:
>
>> * : les noms de plugins précédés d'une étoile sont ceux dont les master modes sont situés dans le fichier MachineHelper.cpp
>
> Dans le meilleur des mondes, ces adapteurs devraient être migrés dans d'autres plugins.
>
>>
>> Partitioner
>> Il s'agit de compter le nombre d'entrées dans les fichiers fournis au séquenceur.
>
> fournis à l'assembleur, pas au séquenceur.
>
>> Chaque esclave a un certain nombre de fichiers à compter.
>
> Exactement ! Ça comprend aussi le rang 0.
>
>>
>> Les fichiers peuvent être de 3 formats différents:
>> -SFF: Voir ray-master/code/SequencesLoader/SffLoader.cpp
>> -FASTA: Voir ray-master/code/SequencesLoader/FastaLoader.cpp
>> -FASTQ: Voir ray-master/code/SequencesLoader/FastqLoader.cpp
>>
>
> Il y a beaucoup plus de formats:
>
> .fasta
> .fa
> .fasta.gz (needs HAVE_LIBZ=y at compilation)
> .fa.gz (needs HAVE_LIBZ=y at compilation)
> .fasta.bz2 (needs HAVE_LIBBZ2=y at compilation)
> .fa.bz2 (needs HAVE_LIBBZ2=y at compilation)
> .fastq
> .fq
> .fastq.gz (needs HAVE_LIBZ=y at compilation)
> .fq.gz (needs HAVE_LIBZ=y at compilation)
> .fastq.bz2 (needs HAVE_LIBBZ2=y at compilation)
> .fq.bz2 (needs HAVE_LIBBZ2=y at compilation)
> export.txt
> qseq.txt
> .sff (paired reads must be extracted manually)
> .csfasta (color-space reads)
> .csfa (color-space reads)
>
>
> Si votre est en C++, vous pouvez utiliser directement les classes de Ray.
>
> Sinon, je vous suggère de gérer initialement le fastq et le fastq.gz.
>
>> *SequencesLoader
>> Charger en mémoire les reads. Le maître calcule le partitionnement des reads, puis chaque esclave charge sa partie en mémoire, dans machine::ArrayOfReads.
>
> Oui.
>
> Il y a une partition sur les fichiers et une partitions sur les rangs. La première est envopyer à tout le monde et la seconde est calculée avec la
> première.
>
>>
>> *KmerAcademyBuilder
>> Chaque esclave compte le nombre de k-mers dans ses reads et l'envoie.
>
> Ce module calcul les sommets du graphe et utilise un filtre de Bloom.
>
>>
>> *CoverageGatherer
>> Les slaves envoient la distribution de la couverture, soit le nombre de k-mers ayant la couverture donnée.
>>
>
> oui
>
>> *VerticesExtractor
>> RAY_MASTER_MODE_SEND_COVERAGE_VALUES
>> - Le maître envoie à tous les esclaves les valeurs maximales, minimales et de répétition de la couverture.
>> RAY_SLAVE_MODE_ADD_EDGES
>> - Les esclaves fabriquent des edges. Lorsqu'ils en ont amassé un certain nombre, ils les envoient. Pour trouver un edge, ils commencent par décoder la séquence ADN d'un read. Si un k-mer a un sommet précédent, il envoie le edge au bon rank à l'aide du BufferedData. Sinon, il renverse le k-mer et refait la même vérification. Si n'en a toujours pas un, il passe au prochain k-mer, qui aura comme sommet précédent ce dernier.
>> -ligne 148 de VerticesExtractor.cpp: comment un DNA pourrait ne pas être valide, ça provient de codeToChar() qui retourne que les bons chars => ?
>
> Un ADN invalide contient autre chose que des A, T, C, G. En pratique, ça n'arrive pas je crois car
> il y a du code ailleur qui gère ça.
>
>>
>> *EdgePurger
>> RAY_MASTER_MODE_PURGE_NULL_EDGES
>> - passage en mode DO_NOTHING
>> - Avertit les autres de passer à cette étape via le message RAY_MPI_TAG_PURGE_NULL_EDGES
>> RAY_SLAVE_MODE_PURGE_NULL_EDGES
>> -EdgePurger est un TaskCreator. Chaque worker demande aux autres la couverture d'un certain sommet. Si la couverture est nulle, il élimine le sommet.
>> -ligne 49 de EdgePurgeWorker.cpp: Pourquoi vérifie-t-on seulement la première réponse et pas les suivantes => ?
>
> Parce que le message est démultiplexé déjà avant par le VirtualCommunicator (demande faite par le VirtualProcessor qui a réveillé
> le Worker.)
>
>>
>> MachineHelper.cpp
>> RAY_MASTER_MODE_WRITE_KMERS
>> - Chaque esclave ecrit dans le fichier kmers.txt les k-mers ainsi que leur couverture. Ils envoient aussi la distribution des edges. Il s'agit du nombre de k-mers avec un nombre de parents
>> et un nombre d'enfants donnés.
>
>
> (actif avec -write-kmers).
>
>> C'est le master qui écrira ceci dans le fichier degreeDistribution.txt.
>>
>> *SequencesIndexer
>> Chaque worker commence par demander la couverture des vertex. Il sélectionne ensuite un vertex qui n'est pas une erreur selon la couverture maximale et minimale. Il envoie ce vertex ainsi que sa position dans le read. Il fait pareil pour son inverse. Selon les options, il peut aussi écrire dans un fichier des marqueurs décrivant les reads, soit leur position, leur coverage et autres.
>
> Exactement.
>
> Initialement, le code choisissait toujours index=0, mais ça donne de meilleurs résultats avec cet algorithme.
>
>>
>> *SeedingData
>> RAY_MASTER_MODE_PREPARE_SEEDING
>> - mise à jour de deux attributs
>> - closeMasterMode()
>> RAY_MASTER_MODE_TRIGGER_SEEDING
>> - envoi d'un message (RAY_MPI_TAG_START_SEEDING) à tous les ranks
>> RAY_SLAVE_MODE_START_SEEDING
>> - génération des seeds dans SeedWorker.cpp
>> RAY_SLAVE_MODE_SEND_SEED_LENGTHS
>> - envoi du message RAY_MPI_TAG_IS_DONE_SENDING_SEED_LENGTHS au rank 0
>> - passage en mode DO_NOTHING
>>
>> SpuriousSeedAnnihilator
>> RAY_MASTER_MODE_REGISTER_SEEDS & RAY_MASTER_MODE_FILTER_SEEDS & RAY_MASTER_MODE_CLEAN_SEEDS & RAY_MASTER_MODE_PUSH_SEED_LENGTHS
>> - distribution non démarrée -> openMasterMode
>> - distribution terminée -> closeMasterMode
>> RAY_SLAVE_MODE_REGISTER_SEEDS
>> - si skip ou checkPoint déjà atteit -> return
>> - => ?
>> RAY_SLAVE_MODE_FILTER_SEEDS
>> - si checkPoint ou skip -> closeSlaveModeLocally()
>> - sinon TaskCreator.mainLoop() (cf FusionTaskCreator l.117)
>> RAY_SLAVE_MODE_CLEAN_SEEDS
>> - si checkPoint ou skip -> closeSeModeLocally()
>> - désallocation de pointeurs devenus inutiles après l'étape d'extension des seeds
>> RAY_SLAVE_MODE_PUSH_SEED_LENGTHS
>> - récupération de "length" et "count" (deux attributs relatifs aux seeds obtenus grâce à un itérateur sur map<int,int>)
>> - envoi de ces données au rank 0 via un message
>>
>> *Library
>> RAY_MASTER_MODE_TRIGGER_DETECTION
>> - envoi d'un message (RAY_MPI_TAG_AUTOMATIC_DISTANCE_DETECTION) à tous les ranks
>> - passage en mode DO_NOTHING
>> RAY_SLAVE_MODE_AUTOMATIC_DISTANCE_DETECTION
>> - Si on a des paired reads, on termine tout de suite en envoyant MASTER_RANK,RAY_MPI_TAG_AUTOMATIC_DISTANCE_DETECTION_IS_DONE
>> - Gestion du bug sur Guillimin
>
> Ce code est en commentaire et donc inactif. Le bug était dans Ray, pas causé par guillimin. Le bug arrivait seulement
> sur guillimin.clumeq.ca par contre.
>
>> - Les workers travaillent pour trouver les distances externes moyennes:
>> - Cela commence par les données des seeds dans lesquelles ils prennent un vertex.
>> - Ils obtiennent les reads à partir du vertex.
>> - Pour chacun de ces reads:
>> - Ils trouvent l'autre lecture qui forme les paires de nucleotides en faisant une demande (RAY_MPI_TAG_GET_READ_MATE)
>> - La distance = position read - position autre read + longueur read + position strand gauche - position strand droit
>> - Finalement, la quantité d'une telle distance est incrementée.
>> RAY_MASTER_MODE_ASK_DISTANCES
>> - envoi d'un message (RAY_MPI_TAG_ASK_LIBRARY_DISTANCES) à tous les ranks
>> RAY_SLAVE_MODE_SEND_LIBRARY_DISTANCES
>> - envoi au master la quantité de chaque distance pour chaque librairie
>> RAY_MASTER_MODE_START_UPDATING_DISTANCES
>> - Calcule les nouvelles distances moyennes à partir des résultats des slaves
>> RAY_MASTER_MODE_UPDATE_DISTANCES
>> - Envoi des résultats à tous les ranks
>>
>> SeedExtender
>> RAY_MASTER_MODE_TRIGGER_EXTENSIONS
>> - envoi d'un message (RAY_MPI_TAG_ASK_EXTENSION) à tous les ranks
>> - passage en mode DO_NOTHING
>> RAY_SLAVE_MODE_EXTENSION
>> - vérifier checkPoint, si passé ("extensions")
>> - passage en mode DO_NOTHING
>> - message au rank 0 (RAY_MPI_TAG_EXTENSION_IS_DONE)
>> - si tous les seeds ont été gérés ou 0 seeds -> finalize()
>> - si initialisation pas faite -> initialize()
>> - si le vertex en cours (ou son revcomp) est déjà assemblé, on passe au seed suivant
>> - sinon on le marque comme étant assemblé
>> - on "énumère les choix" puis on "fait un choix" => ?
>
> Ce code contient les heuristiques et la traversée du graphe.
>
>>
>> *FusionData
>> RAY_MASTER_MODE_TRIGGER_FUSIONS
>> - m_cycleNumber = 0;
>> - closeMasterMode()
>> RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS
>> - (*m_reductionOccured)=true;
>> - closeMasterMode()
>> RAY_MASTER_MODE_START_FUSION_CYCLE
>> - si cycle non démarré : envoi d'un message (RAY_MPI_TAG_CLEAR_DIRECTIONS) à tous les ranks
>> - plusieurs cycles contenant les étapes suivantes :
>> 1 : envoi de RAY_MPI_TAG_DISTRIBUTE_FUSIONS
>> 2 : envoi de RAY_MPI_TAG_FINISH_FUSIONS
>> 3 : envoi de RAY_MPI_TAG_CLEAR_DIRECTIONS
>> 4 : envoi de RAY_MPI_TAG_DISTRIBUTE_FUSIONS
>> 5 : envoi de RAY_MPI_TAG_START_FUSION
>> 6 : reinitialisation pour cycle suivant => ?
>> RAY_SLAVE_MODE_DISTRIBUTE_FUSIONS
>>
>> FusionTaskCreator
>> RAY_SLAVE_MODE_FUSION
>> - TaskCreator -> mainLoop()
>> - initialisation()
>> - si il reste des tâches non assignées -> création de workers & assignation tâche
>> - si plus de tâche non assignée -> noMoreTasksAreComing()
>> - on fait "travailler" un worker, on récupère le résultat de son calcul puis on le détruit
>> - si tâches assignées accomplies & plus de tâche à assigner -> finalize()
>>
>> JoinerTaskCreator
>> RAY_SLAVE_MODE_FINISH_FUSIONS
>> - TaskCreator -> mainLoop() (cf RAY_SLAVE_MODE_FUSION)
>>
>> PathEvaluator
>> (évaluation de la qualité d'un chemin)
>> RAY_MASTER_MODE_EVALUATE_PATHS
>> - openMasterMode() (boucle sur openSlaveMode(), une itération pour chaque rank)
>> - si tous les ranks ont terminé -> closeMasterMode()
>> RAY_SLAVE_MODE_EVALUATE_PATHS
>> - writeCheckpointForContigPaths()
>> - écriture du nombre total de contigs (m_contig.size()) dans le fichier checkpoint ContigPaths
>> - pour chaque contig, écriture dans ce fichier checkpoint de :
>> - "nom" du contig
>> - nombre de K-mers
>> - k-mers
>> - closeSlaveModeLocally() (envoi d'un message (RAY_MPI_TAG_SWITCHMAN_COMPLETION_SIGNAL) au rank 0)
>>
>> MachineHelper.cpp
>> RAY_MASTER_MODE_ASK_EXTENSIONS
>> (demande aux ranks d'envoyer leurs extensions)
>> - étape 1 :
>> - initialisations
>> - envoi d'un message (RAY_MPI_TAG_COMPUTE_REQUIRED_SPACE_FOR_EXTENSIONS) à tous les ranks
>> - étape 2 (quand tous les ranks ont terminé d'écrire leurs contigs) :
>> - si RAY configuré pour utiliser AMOS -> setMasterMode(RAY_MASTER_MODE_AMOS);
>> - sinon -> closeMasterMode();
>> RAY_SLAVE_MODE_SEND_EXTENSION_DATA
>> - si autorisation (rank 0 ou message RAY_MPI_TAG_SEND_AUTHORIZATION reçu) -> pour que deux ranks ne tentent pas d'écrire dans le fichier en même temps
>> - ouverture fichier
>> - écriture de tous les contigs pour le rank en cours
>> - fermeture fichier
>> - passage en mode DO_NOTHING
>> - envoi d'un message (RAY_MPI_TAG_EXTENSION_DATA_END) au rank 0
>> - envoi d'un message (RAY_MPI_TAG_SEND_AUTHORIZATION) au rank suivant
>>
>> *Scaffolder
>> RAY_MASTER_MODE_SCAFFOLDER
>> - envoi d'un message (RAY_MPI_TAG_START_SCAFFOLDER) à tous les ranks
>> - passage en mode DO_NOTHING
>> RAY_SLAVE_MODE_SCAFFOLDER
>> - initialisations
>> - tant que tous les contigs n'ont pas été gérés -> processContig();
>> - envoi d'un message (RAY_MPI_TAG_I_FINISHED_SCAFFOLDING) au rank 0
>> - passage en mode DO_NOTHING
>> RAY_MASTER_MODE_WRITE_SCAFFOLDS
>> - initialisations (ouverture d'un fichier)
>> - tant qu'on a pas géré tous les scaffolds :
>> - tant qu'on a pas géré tous les contigs du scaffold en cours :
>> - getContigSequence();
>> - ecriture des scaffolds dans le fichier
>> - closeMasterMode();
>
>
> Pour votre premier milestone, vous pouvez probablement commencer par le support fastq seulement.
>
>
> Je pense que la partie de construction du graphe se fait sur un GPU ou sur un FPGA. Le reste, je ne suis pas certain du gain.
>
>
> Voir:
>
> Efficient Graph Based Assembly of Short-Read Sequences on Hybrid Core Architecture
> http://www.osti.gov/bridge/servlets/purl/1015333-q62bUB/1015333.pdf
> http://www.conveycomputer.com/files/2313/5095/9539/ConveyGraphConstructor_5172011.pdf
>
|
|
From: Sébastien B. <seb...@ul...> - 2013-05-23 15:42:23
|
(J'ai mis en C.C. Jacques Corbeil et François Laviolette pour les laisser savoir
que je mets du temps sur ce projet.)
On 21/05/13 04:27 PM, Marpaud@Gmail wrote:
> Rebonjour,
>
> j'ai joint à ce mail un .txt produit par Carl Poirier (l'étudiant en maîtrise qui m'a rejoint sur le projet) et moi-même. Puis-je vous demander d'y jeter un coup d'œil pour :
>
> * confirmer que notre analyse est correcte
> * clarifier certains points
>
> si vous en avez le temps ? Les points à clarifier sont signalés dans le fichier par la présence de la chaine "=> ?".
>
> Encore une fois, merci !
>
> --
> Cyril MARPAUD
> Étudiant ingénieur à l'ENSICAEN
> 3ème année électronique et physique appliquée
> Signal, Automatique et Télécoms pour l'Embarqué
> Ligne fixe : 418 656-7777 poste 1-7839
> @ : cma...@gm...
Reviewer: Sébastien Boisvert
File: descriptionAlgorithme.txt
Patch applies cleanly:
[ ] Yes
[ ] No
[x] Not applicable
The code works:
[ ] Yes
[ ] No
[x] Not applicable
Major concerns:
Code Review:
> * : les noms de plugins précédés d'une étoile sont ceux dont les master modes sont situés dans le fichier MachineHelper.cpp
Dans le meilleur des mondes, ces adapteurs devraient être migrés dans d'autres plugins.
>
> Partitioner
> Il s'agit de compter le nombre d'entrées dans les fichiers fournis au séquenceur.
fournis à l'assembleur, pas au séquenceur.
>Chaque esclave a un certain nombre de fichiers à compter.
Exactement ! Ça comprend aussi le rang 0.
>
> Les fichiers peuvent être de 3 formats différents:
> -SFF: Voir ray-master/code/SequencesLoader/SffLoader.cpp
> -FASTA: Voir ray-master/code/SequencesLoader/FastaLoader.cpp
> -FASTQ: Voir ray-master/code/SequencesLoader/FastqLoader.cpp
>
Il y a beaucoup plus de formats:
.fasta
.fa
.fasta.gz (needs HAVE_LIBZ=y at compilation)
.fa.gz (needs HAVE_LIBZ=y at compilation)
.fasta.bz2 (needs HAVE_LIBBZ2=y at compilation)
.fa.bz2 (needs HAVE_LIBBZ2=y at compilation)
.fastq
.fq
.fastq.gz (needs HAVE_LIBZ=y at compilation)
.fq.gz (needs HAVE_LIBZ=y at compilation)
.fastq.bz2 (needs HAVE_LIBBZ2=y at compilation)
.fq.bz2 (needs HAVE_LIBBZ2=y at compilation)
export.txt
qseq.txt
.sff (paired reads must be extracted manually)
.csfasta (color-space reads)
.csfa (color-space reads)
Si votre est en C++, vous pouvez utiliser directement les classes de Ray.
Sinon, je vous suggère de gérer initialement le fastq et le fastq.gz.
> *SequencesLoader
> Charger en mémoire les reads. Le maître calcule le partitionnement des reads, puis chaque esclave charge sa partie en mémoire, dans machine::ArrayOfReads.
Oui.
Il y a une partition sur les fichiers et une partitions sur les rangs. La première est envopyer à tout le monde et la seconde est calculée avec la
première.
>
> *KmerAcademyBuilder
> Chaque esclave compte le nombre de k-mers dans ses reads et l'envoie.
Ce module calcul les sommets du graphe et utilise un filtre de Bloom.
>
> *CoverageGatherer
> Les slaves envoient la distribution de la couverture, soit le nombre de k-mers ayant la couverture donnée.
>
oui
> *VerticesExtractor
> RAY_MASTER_MODE_SEND_COVERAGE_VALUES
> - Le maître envoie à tous les esclaves les valeurs maximales, minimales et de répétition de la couverture.
> RAY_SLAVE_MODE_ADD_EDGES
> - Les esclaves fabriquent des edges. Lorsqu'ils en ont amassé un certain nombre, ils les envoient. Pour trouver un edge, ils commencent par décoder la séquence ADN d'un read. Si un k-mer a un sommet précédent, il envoie le edge au bon rank à l'aide du BufferedData. Sinon, il renverse le k-mer et refait la même vérification. Si n'en a toujours pas un, il passe au prochain k-mer, qui aura comme sommet précédent ce dernier.
> -ligne 148 de VerticesExtractor.cpp: comment un DNA pourrait ne pas être valide, ça provient de codeToChar() qui retourne que les bons chars => ?
Un ADN invalide contient autre chose que des A, T, C, G. En pratique, ça n'arrive pas je crois car
il y a du code ailleur qui gère ça.
>
> *EdgePurger
> RAY_MASTER_MODE_PURGE_NULL_EDGES
> - passage en mode DO_NOTHING
> - Avertit les autres de passer à cette étape via le message RAY_MPI_TAG_PURGE_NULL_EDGES
> RAY_SLAVE_MODE_PURGE_NULL_EDGES
> -EdgePurger est un TaskCreator. Chaque worker demande aux autres la couverture d'un certain sommet. Si la couverture est nulle, il élimine le sommet.
> -ligne 49 de EdgePurgeWorker.cpp: Pourquoi vérifie-t-on seulement la première réponse et pas les suivantes => ?
Parce que le message est démultiplexé déjà avant par le VirtualCommunicator (demande faite par le VirtualProcessor qui a réveillé
le Worker.)
>
> MachineHelper.cpp
> RAY_MASTER_MODE_WRITE_KMERS
> - Chaque esclave ecrit dans le fichier kmers.txt les k-mers ainsi que leur couverture. Ils envoient aussi la distribution des edges. Il s'agit du nombre de k-mers avec un nombre de parents
>et un nombre d'enfants donnés.
(actif avec -write-kmers).
> C'est le master qui écrira ceci dans le fichier degreeDistribution.txt.
>
> *SequencesIndexer
> Chaque worker commence par demander la couverture des vertex. Il sélectionne ensuite un vertex qui n'est pas une erreur selon la couverture maximale et minimale. Il envoie ce vertex ainsi que sa position dans le read. Il fait pareil pour son inverse. Selon les options, il peut aussi écrire dans un fichier des marqueurs décrivant les reads, soit leur position, leur coverage et autres.
Exactement.
Initialement, le code choisissait toujours index=0, mais ça donne de meilleurs résultats avec cet algorithme.
>
> *SeedingData
> RAY_MASTER_MODE_PREPARE_SEEDING
> - mise à jour de deux attributs
> - closeMasterMode()
> RAY_MASTER_MODE_TRIGGER_SEEDING
> - envoi d'un message (RAY_MPI_TAG_START_SEEDING) à tous les ranks
> RAY_SLAVE_MODE_START_SEEDING
> - génération des seeds dans SeedWorker.cpp
> RAY_SLAVE_MODE_SEND_SEED_LENGTHS
> - envoi du message RAY_MPI_TAG_IS_DONE_SENDING_SEED_LENGTHS au rank 0
> - passage en mode DO_NOTHING
>
> SpuriousSeedAnnihilator
> RAY_MASTER_MODE_REGISTER_SEEDS & RAY_MASTER_MODE_FILTER_SEEDS & RAY_MASTER_MODE_CLEAN_SEEDS & RAY_MASTER_MODE_PUSH_SEED_LENGTHS
> - distribution non démarrée -> openMasterMode
> - distribution terminée -> closeMasterMode
> RAY_SLAVE_MODE_REGISTER_SEEDS
> - si skip ou checkPoint déjà atteit -> return
> - => ?
> RAY_SLAVE_MODE_FILTER_SEEDS
> - si checkPoint ou skip -> closeSlaveModeLocally()
> - sinon TaskCreator.mainLoop() (cf FusionTaskCreator l.117)
> RAY_SLAVE_MODE_CLEAN_SEEDS
> - si checkPoint ou skip -> closeSeModeLocally()
> - désallocation de pointeurs devenus inutiles après l'étape d'extension des seeds
> RAY_SLAVE_MODE_PUSH_SEED_LENGTHS
> - récupération de "length" et "count" (deux attributs relatifs aux seeds obtenus grâce à un itérateur sur map<int,int>)
> - envoi de ces données au rank 0 via un message
>
> *Library
> RAY_MASTER_MODE_TRIGGER_DETECTION
> - envoi d'un message (RAY_MPI_TAG_AUTOMATIC_DISTANCE_DETECTION) à tous les ranks
> - passage en mode DO_NOTHING
> RAY_SLAVE_MODE_AUTOMATIC_DISTANCE_DETECTION
> - Si on a des paired reads, on termine tout de suite en envoyant MASTER_RANK,RAY_MPI_TAG_AUTOMATIC_DISTANCE_DETECTION_IS_DONE
> - Gestion du bug sur Guillimin
Ce code est en commentaire et donc inactif. Le bug était dans Ray, pas causé par guillimin. Le bug arrivait seulement
sur guillimin.clumeq.ca par contre.
> - Les workers travaillent pour trouver les distances externes moyennes:
> - Cela commence par les données des seeds dans lesquelles ils prennent un vertex.
> - Ils obtiennent les reads à partir du vertex.
> - Pour chacun de ces reads:
> - Ils trouvent l'autre lecture qui forme les paires de nucleotides en faisant une demande (RAY_MPI_TAG_GET_READ_MATE)
> - La distance = position read - position autre read + longueur read + position strand gauche - position strand droit
> - Finalement, la quantité d'une telle distance est incrementée.
> RAY_MASTER_MODE_ASK_DISTANCES
> - envoi d'un message (RAY_MPI_TAG_ASK_LIBRARY_DISTANCES) à tous les ranks
> RAY_SLAVE_MODE_SEND_LIBRARY_DISTANCES
> - envoi au master la quantité de chaque distance pour chaque librairie
> RAY_MASTER_MODE_START_UPDATING_DISTANCES
> - Calcule les nouvelles distances moyennes à partir des résultats des slaves
> RAY_MASTER_MODE_UPDATE_DISTANCES
> - Envoi des résultats à tous les ranks
>
> SeedExtender
> RAY_MASTER_MODE_TRIGGER_EXTENSIONS
> - envoi d'un message (RAY_MPI_TAG_ASK_EXTENSION) à tous les ranks
> - passage en mode DO_NOTHING
> RAY_SLAVE_MODE_EXTENSION
> - vérifier checkPoint, si passé ("extensions")
> - passage en mode DO_NOTHING
> - message au rank 0 (RAY_MPI_TAG_EXTENSION_IS_DONE)
> - si tous les seeds ont été gérés ou 0 seeds -> finalize()
> - si initialisation pas faite -> initialize()
> - si le vertex en cours (ou son revcomp) est déjà assemblé, on passe au seed suivant
> - sinon on le marque comme étant assemblé
> - on "énumère les choix" puis on "fait un choix" => ?
Ce code contient les heuristiques et la traversée du graphe.
>
> *FusionData
> RAY_MASTER_MODE_TRIGGER_FUSIONS
> - m_cycleNumber = 0;
> - closeMasterMode()
> RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS
> - (*m_reductionOccured)=true;
> - closeMasterMode()
> RAY_MASTER_MODE_START_FUSION_CYCLE
> - si cycle non démarré : envoi d'un message (RAY_MPI_TAG_CLEAR_DIRECTIONS) à tous les ranks
> - plusieurs cycles contenant les étapes suivantes :
> 1 : envoi de RAY_MPI_TAG_DISTRIBUTE_FUSIONS
> 2 : envoi de RAY_MPI_TAG_FINISH_FUSIONS
> 3 : envoi de RAY_MPI_TAG_CLEAR_DIRECTIONS
> 4 : envoi de RAY_MPI_TAG_DISTRIBUTE_FUSIONS
> 5 : envoi de RAY_MPI_TAG_START_FUSION
> 6 : reinitialisation pour cycle suivant => ?
> RAY_SLAVE_MODE_DISTRIBUTE_FUSIONS
>
> FusionTaskCreator
> RAY_SLAVE_MODE_FUSION
> - TaskCreator -> mainLoop()
> - initialisation()
> - si il reste des tâches non assignées -> création de workers & assignation tâche
> - si plus de tâche non assignée -> noMoreTasksAreComing()
> - on fait "travailler" un worker, on récupère le résultat de son calcul puis on le détruit
> - si tâches assignées accomplies & plus de tâche à assigner -> finalize()
>
> JoinerTaskCreator
> RAY_SLAVE_MODE_FINISH_FUSIONS
> - TaskCreator -> mainLoop() (cf RAY_SLAVE_MODE_FUSION)
>
> PathEvaluator
> (évaluation de la qualité d'un chemin)
> RAY_MASTER_MODE_EVALUATE_PATHS
> - openMasterMode() (boucle sur openSlaveMode(), une itération pour chaque rank)
> - si tous les ranks ont terminé -> closeMasterMode()
> RAY_SLAVE_MODE_EVALUATE_PATHS
> - writeCheckpointForContigPaths()
> - écriture du nombre total de contigs (m_contig.size()) dans le fichier checkpoint ContigPaths
> - pour chaque contig, écriture dans ce fichier checkpoint de :
> - "nom" du contig
> - nombre de K-mers
> - k-mers
> - closeSlaveModeLocally() (envoi d'un message (RAY_MPI_TAG_SWITCHMAN_COMPLETION_SIGNAL) au rank 0)
>
> MachineHelper.cpp
> RAY_MASTER_MODE_ASK_EXTENSIONS
> (demande aux ranks d'envoyer leurs extensions)
> - étape 1 :
> - initialisations
> - envoi d'un message (RAY_MPI_TAG_COMPUTE_REQUIRED_SPACE_FOR_EXTENSIONS) à tous les ranks
> - étape 2 (quand tous les ranks ont terminé d'écrire leurs contigs) :
> - si RAY configuré pour utiliser AMOS -> setMasterMode(RAY_MASTER_MODE_AMOS);
> - sinon -> closeMasterMode();
> RAY_SLAVE_MODE_SEND_EXTENSION_DATA
> - si autorisation (rank 0 ou message RAY_MPI_TAG_SEND_AUTHORIZATION reçu) -> pour que deux ranks ne tentent pas d'écrire dans le fichier en même temps
> - ouverture fichier
> - écriture de tous les contigs pour le rank en cours
> - fermeture fichier
> - passage en mode DO_NOTHING
> - envoi d'un message (RAY_MPI_TAG_EXTENSION_DATA_END) au rank 0
> - envoi d'un message (RAY_MPI_TAG_SEND_AUTHORIZATION) au rank suivant
>
> *Scaffolder
> RAY_MASTER_MODE_SCAFFOLDER
> - envoi d'un message (RAY_MPI_TAG_START_SCAFFOLDER) à tous les ranks
> - passage en mode DO_NOTHING
> RAY_SLAVE_MODE_SCAFFOLDER
> - initialisations
> - tant que tous les contigs n'ont pas été gérés -> processContig();
> - envoi d'un message (RAY_MPI_TAG_I_FINISHED_SCAFFOLDING) au rank 0
> - passage en mode DO_NOTHING
> RAY_MASTER_MODE_WRITE_SCAFFOLDS
> - initialisations (ouverture d'un fichier)
> - tant qu'on a pas géré tous les scaffolds :
> - tant qu'on a pas géré tous les contigs du scaffold en cours :
> - getContigSequence();
> - ecriture des scaffolds dans le fichier
> - closeMasterMode();
Pour votre premier milestone, vous pouvez probablement commencer par le support fastq seulement.
Je pense que la partie de construction du graphe se fait sur un GPU ou sur un FPGA. Le reste, je ne suis pas certain du gain.
Voir:
Efficient Graph Based Assembly of Short-Read Sequences on Hybrid Core Architecture
http://www.osti.gov/bridge/servlets/purl/1015333-q62bUB/1015333.pdf
http://www.conveycomputer.com/files/2313/5095/9539/ConveyGraphConstructor_5172011.pdf
|
|
From: Sébastien B. <se...@bo...> - 2013-02-27 04:47:22
|
On 02/26/2013 10:01 PM, sa...@ho... wrote: >> >> And it's all or nothing -- you can not release SoftwareX v1.2.3 as >> proprietary software >> and as GPLv3-licensed software because that would infringe the GPLv3 >> license, and it does not make sense. >> > > > Yes you can, and that was the business model for MySQL. MySQL distributed > their database program for free under GPL license, and that resulted in > wide deployment of their program. However, GPL meant anyone doing > derivative work had to release all their own code as well, and that was > not acceptable for many software companies. However, those companies could > buy proprietary licenses for open source MySQL so that they could bundle > their own proprietary programs. > > http://stackoverflow.com/questions/620696/mysql-licensing-and-gpl > > > I used to think like you before, but was corrected by two readers in our > blog. Please check here - > > http://www.homolog.us/blogs/2012/10/19/software-licenses-in-bioinformatics-programs-and-their-legal-implications/ > > http://www.homolog.us/blogs/2012/10/26/more-on-gpl-licensing-of-bioinformatics-programs/ > > Wow ! I have read these two links with great interest. Thank you. > > > |
|
From: Sébastien B. <seb...@ul...> - 2013-02-27 03:07:51
|
On 02/26/2013 08:58 PM, sa...@ho... wrote: > Hi Sébastien, > > Thank you for the detailed response. Yes, I meant 'copyright' in my > original question. In US, graduate students and post-docs are usually > employees of the institutions, and sign various legal contracts giving > away their patent/copy rights. As a result, someone interested in > licensing their work needs to deal with both the inventor and the > university. The second part can get very bureaucratic, and that is why I > asked. > >> Anyway, copyright does not mean anything for a GPLv3-licensed program >> because the point is to not limit copies of the programs so that people >> hopefully build on top of that. > > That is not true. Being the copyright owner allows you to change mind on a > later date, or issue different license for 'preferred clients'. If your > brother asks for a proprietary copy of your GPLv3-licensed Ray code from > you, you can release it to him, because you are the copyright owner. I > cannot create a proprietary copy of Ray for my brother :) What I meant here is that even if all the copyright holders decide at some point that the product should now be proprietary, previous releases keep their respective license arrangements. And it's all or nothing -- you can not release SoftwareX v1.2.3 as proprietary software and as GPLv3-licensed software because that would infringe the GPLv3 license, and it does not make sense. > > I agree about what you said on patents. Whole patent thing is legacy of > feudal society of Europe, where the large guilds could buy monopoly rights > from the kings. Yes. I believe in brands, but patents are just silly. > > You probably know that US patent law is changing next month from 'first to > invent' to 'first to file'. > > http://www.preti.com/major-patent-law-changes-2013 > http://www.forbes.com/sites/robertjordan/2012/11/13/the-new-patent-law-end-of-entrepreneurship/ > Correct me if I am wrong, but this is a bad thing for everyone except patent trolls, right ? In my opinion, the competition for market share should occur on the "battlefield" with customers, not in patent offices where there is no product and no customer. Competition is good for customers, it increases the quality of products and it lowers the prices of goods. > > Best, > Manoj > > > >> Hi ! >> >> [I CC'ed denovoassembler-devel as this is a really interesting question] >> >> I am not a legal expert, but read on ! >> >> On 02/26/2013 11:36 AM, sa...@ho... wrote: >>> Hey Sébastien, >>> >>>> >>>> I only work on open source software projects. >>>> >>> >>> Open-source or closed, someone owns the license to the code. >> >> I think you mean the copyright over the code, not its license. >> >> >> I am not an employee of my university -- I am a doctoral student. I have >> no employer-employee >> relationship with my university for what I do in my doctoral projects. >> I work on whatever I want, which happens to be open stuff in genomics. >> >> Back in 2010, my director wanted Ray to be a proprietary software, >> but I convinced him otherwise by describing the pros of doing things this >> way. >> And he's quite happy with that decision now. >> >> Ray and Ray Cloud Browser are licensed under the GPLv3. Ray Platform is >> licensed under the >> LGPLv3. I own the copyright on most source code files since I wrote them, >> and because I am not an employee >> and I have never signed any contracts (or other legal documents) for these >> projects. >> >> Anyway, copyright does not mean anything >> for a GPLv3-licensed program because the point is to not limit copies of >> the programs so that people >> hopefully build on top of that. >> >> To decide what license to release the source code (or to change the >> license), all copyright holders must >> agree. For closed-source programs, the license is not on the source code >> (since it's confidential), it's usually a end user license agreement >> (EULA) that is renewed >> with contracts (there can be a EULA over a product based on open source >> software too, see Red Hat). >> >> For instance, Applied Maths NV. ships Ray in their BioNumerics product >> >> => >> http://www.applied-maths.com/news/new-de-novo-assembler-next-generation-sequencing-data-bionumerics >> >> >> More and more, proprietary software are no longer licensed as a goods >> (which is bad for the consumers). >> Instead, you get a license to use the software (as a service) over a >> period. If you played Diablo III and got a sword in the game, >> you can actually (probably) send a mail to Blizzard and ask them for your >> license to use the sword. >> For each Diablo III item, the player gets a exclusive-from-Blizzard >> license to use it over the lifetime of his/her character. >> >> >> I think there is a clear distinction between: >> >> * license for using a software vs. license for distributing source code; >> >> => GPLv3 covers both of these. GPLv3 tells you that you have the 4 >> liberties and that you can use the software for whatever use. >> >> * ownership (intellectual property) vs. copyright >> >> => ownership is usually on ideas (patents), which is ridiculous by >> definition (see Apple v. Samsung) or brands >> (which make sense because a brand is for a product, not for an >> idea). >> >> => I think that in the GPLv3 (warning, I am not a legal expert), >> there is a clause that says that patents can not be >> added after releases as open source releases are prior art. >> Regardless, Everything novel in Ray is not patentable because >> there's is public >> prior art: Ray source code files and scientific papers describing >> methods. >> Furthermore, this prior art is easily demonstrated with git source >> code logs and publications. >> >> >> -Séb >> >>> That is the >>> part I was asking about. >>> >>> Best, >>> Manoj >>> >>> >> >> > > |
|
From: <sa...@ho...> - 2013-02-27 03:03:52
|
Hi Sébastien, Thank you for the detailed response. Yes, I meant 'copyright' in my original question. In US, graduate students and post-docs are usually employees of the institutions, and sign various legal contracts giving away their patent/copy rights. As a result, someone interested in licensing their work needs to deal with both the inventor and the university. The second part can get very bureaucratic, and that is why I asked. > Anyway, copyright does not mean anything for a GPLv3-licensed program > because the point is to not limit copies of the programs so that people > hopefully build on top of that. That is not true. Being the copyright owner allows you to change mind on a later date, or issue different license for 'preferred clients'. If your brother asks for a proprietary copy of your GPLv3-licensed Ray code from you, you can release it to him, because you are the copyright owner. I cannot create a proprietary copy of Ray for my brother :) I agree about what you said on patents. Whole patent thing is legacy of feudal society of Europe, where the large guilds could buy monopoly rights from the kings. You probably know that US patent law is changing next month from 'first to invent' to 'first to file'. http://www.preti.com/major-patent-law-changes-2013 http://www.forbes.com/sites/robertjordan/2012/11/13/the-new-patent-law-end-of-entrepreneurship/ Best, Manoj > Hi ! > > [I CC'ed denovoassembler-devel as this is a really interesting question] > > I am not a legal expert, but read on ! > > On 02/26/2013 11:36 AM, sa...@ho... wrote: >> Hey Sébastien, >> >>> >>> I only work on open source software projects. >>> >> >> Open-source or closed, someone owns the license to the code. > > I think you mean the copyright over the code, not its license. > > > I am not an employee of my university -- I am a doctoral student. I have > no employer-employee > relationship with my university for what I do in my doctoral projects. > I work on whatever I want, which happens to be open stuff in genomics. > > Back in 2010, my director wanted Ray to be a proprietary software, > but I convinced him otherwise by describing the pros of doing things this > way. > And he's quite happy with that decision now. > > Ray and Ray Cloud Browser are licensed under the GPLv3. Ray Platform is > licensed under the > LGPLv3. I own the copyright on most source code files since I wrote them, > and because I am not an employee > and I have never signed any contracts (or other legal documents) for these > projects. > > Anyway, copyright does not mean anything > for a GPLv3-licensed program because the point is to not limit copies of > the programs so that people > hopefully build on top of that. > > To decide what license to release the source code (or to change the > license), all copyright holders must > agree. For closed-source programs, the license is not on the source code > (since it's confidential), it's usually a end user license agreement > (EULA) that is renewed > with contracts (there can be a EULA over a product based on open source > software too, see Red Hat). > > For instance, Applied Maths NV. ships Ray in their BioNumerics product > > => > http://www.applied-maths.com/news/new-de-novo-assembler-next-generation-sequencing-data-bionumerics > > > More and more, proprietary software are no longer licensed as a goods > (which is bad for the consumers). > Instead, you get a license to use the software (as a service) over a > period. If you played Diablo III and got a sword in the game, > you can actually (probably) send a mail to Blizzard and ask them for your > license to use the sword. > For each Diablo III item, the player gets a exclusive-from-Blizzard > license to use it over the lifetime of his/her character. > > > I think there is a clear distinction between: > > * license for using a software vs. license for distributing source code; > > => GPLv3 covers both of these. GPLv3 tells you that you have the 4 > liberties and that you can use the software for whatever use. > > * ownership (intellectual property) vs. copyright > > => ownership is usually on ideas (patents), which is ridiculous by > definition (see Apple v. Samsung) or brands > (which make sense because a brand is for a product, not for an > idea). > > => I think that in the GPLv3 (warning, I am not a legal expert), > there is a clause that says that patents can not be > added after releases as open source releases are prior art. > Regardless, Everything novel in Ray is not patentable because > there's is public > prior art: Ray source code files and scientific papers describing > methods. > Furthermore, this prior art is easily demonstrated with git source > code logs and publications. > > > -Séb > >> That is the >> part I was asking about. >> >> Best, >> Manoj >> >> > > -- Manoj Pratim Samanta, Ph. D. http://www.homolog.us |
|
From: <sa...@ho...> - 2013-02-27 03:02:38
|
> > And it's all or nothing -- you can not release SoftwareX v1.2.3 as > proprietary software > and as GPLv3-licensed software because that would infringe the GPLv3 > license, and it does not make sense. > Yes you can, and that was the business model for MySQL. MySQL distributed their database program for free under GPL license, and that resulted in wide deployment of their program. However, GPL meant anyone doing derivative work had to release all their own code as well, and that was not acceptable for many software companies. However, those companies could buy proprietary licenses for open source MySQL so that they could bundle their own proprietary programs. http://stackoverflow.com/questions/620696/mysql-licensing-and-gpl I used to think like you before, but was corrected by two readers in our blog. Please check here - http://www.homolog.us/blogs/2012/10/19/software-licenses-in-bioinformatics-programs-and-their-legal-implications/ http://www.homolog.us/blogs/2012/10/26/more-on-gpl-licensing-of-bioinformatics-programs/ -- Manoj Pratim Samanta, Ph. D. http://www.homolog.us |
|
From: Sébastien B. <seb...@ul...> - 2013-02-26 18:02:35
|
Hi ! [I CC'ed denovoassembler-devel as this is a really interesting question] I am not a legal expert, but read on ! On 02/26/2013 11:36 AM, sa...@ho... wrote: > Hey Sébastien, > >> >> I only work on open source software projects. >> > > Open-source or closed, someone owns the license to the code. I think you mean the copyright over the code, not its license. I am not an employee of my university -- I am a doctoral student. I have no employer-employee relationship with my university for what I do in my doctoral projects. I work on whatever I want, which happens to be open stuff in genomics. Back in 2010, my director wanted Ray to be a proprietary software, but I convinced him otherwise by describing the pros of doing things this way. And he's quite happy with that decision now. Ray and Ray Cloud Browser are licensed under the GPLv3. Ray Platform is licensed under the LGPLv3. I own the copyright on most source code files since I wrote them, and because I am not an employee and I have never signed any contracts (or other legal documents) for these projects. Anyway, copyright does not mean anything for a GPLv3-licensed program because the point is to not limit copies of the programs so that people hopefully build on top of that. To decide what license to release the source code (or to change the license), all copyright holders must agree. For closed-source programs, the license is not on the source code (since it's confidential), it's usually a end user license agreement (EULA) that is renewed with contracts (there can be a EULA over a product based on open source software too, see Red Hat). For instance, Applied Maths NV. ships Ray in their BioNumerics product => http://www.applied-maths.com/news/new-de-novo-assembler-next-generation-sequencing-data-bionumerics More and more, proprietary software are no longer licensed as a goods (which is bad for the consumers). Instead, you get a license to use the software (as a service) over a period. If you played Diablo III and got a sword in the game, you can actually (probably) send a mail to Blizzard and ask them for your license to use the sword. For each Diablo III item, the player gets a exclusive-from-Blizzard license to use it over the lifetime of his/her character. I think there is a clear distinction between: * license for using a software vs. license for distributing source code; => GPLv3 covers both of these. GPLv3 tells you that you have the 4 liberties and that you can use the software for whatever use. * ownership (intellectual property) vs. copyright => ownership is usually on ideas (patents), which is ridiculous by definition (see Apple v. Samsung) or brands (which make sense because a brand is for a product, not for an idea). => I think that in the GPLv3 (warning, I am not a legal expert), there is a clause that says that patents can not be added after releases as open source releases are prior art. Regardless, Everything novel in Ray is not patentable because there's is public prior art: Ray source code files and scientific papers describing methods. Furthermore, this prior art is easily demonstrated with git source code logs and publications. -Séb > That is the > part I was asking about. > > Best, > Manoj > > |
|
From: Sébastien B. <seb...@ul...> - 2013-02-22 17:34:05
|
merged
On 02/22/2013 12:18 PM, Maxime Déraspe wrote:
> Signed-off-by: Maxime Déraspe <max...@gm...>
> ---
> FastDemultiplexer.py | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/FastDemultiplexer.py b/FastDemultiplexer.py
> index fa1afe3..e104985 100755
> --- a/FastDemultiplexer.py
> +++ b/FastDemultiplexer.py
> @@ -405,10 +405,10 @@ class Demultiplexer:
>
> def main():
> sheet=sys.argv[1]
> - inputDir=sys.argv[2].replace("\/","")
> - outputDir=sys.argv[3].replace("\/","")
> + inputDir=sys.argv[2]
> + outputDir=sys.argv[3]
>
> - lane=inputDir.split("lane")[1].strip()
> + lane=inputDir.split("lane")[1].strip().replace("/","")
>
> demultiplexer=Demultiplexer(sheet,inputDir,outputDir,lane)
>
>
|
|
From: Maxime D. <max...@gm...> - 2013-02-22 17:19:57
|
Signed-off-by: Maxime Déraspe <max...@gm...>
---
FastDemultiplexer.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/FastDemultiplexer.py b/FastDemultiplexer.py
index fa1afe3..e104985 100755
--- a/FastDemultiplexer.py
+++ b/FastDemultiplexer.py
@@ -405,10 +405,10 @@ class Demultiplexer:
def main():
sheet=sys.argv[1]
- inputDir=sys.argv[2].replace("\/","")
- outputDir=sys.argv[3].replace("\/","")
+ inputDir=sys.argv[2]
+ outputDir=sys.argv[3]
- lane=inputDir.split("lane")[1].strip()
+ lane=inputDir.split("lane")[1].strip().replace("/","")
demultiplexer=Demultiplexer(sheet,inputDir,outputDir,lane)
--
1.8.1.4
|
|
From: Maxime D. <max...@gm...> - 2013-02-22 17:08:54
|
From: Maxime Deraspe <deraspem@colosse1.(none)>
Signed-off-by: Maxime Deraspe <deraspem@colosse1.(none)>
---
FastDemultiplexer.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/FastDemultiplexer.py b/FastDemultiplexer.py
index fa1afe3..e104985 100755
--- a/FastDemultiplexer.py
+++ b/FastDemultiplexer.py
@@ -405,10 +405,10 @@ class Demultiplexer:
def main():
sheet=sys.argv[1]
- inputDir=sys.argv[2].replace("\/","")
- outputDir=sys.argv[3].replace("\/","")
+ inputDir=sys.argv[2]
+ outputDir=sys.argv[3]
- lane=inputDir.split("lane")[1].strip()
+ lane=inputDir.split("lane")[1].strip().replace("/","")
demultiplexer=Demultiplexer(sheet,inputDir,outputDir,lane)
--
1.7.4.1
|
|
From: Sébastien B. <seb...@ul...> - 2013-02-22 17:03:59
|
Can you resubmit your patch with your email in the signed-off and in the committer ?
git config --global user.email "me...@he..."
On 02/22/2013 11:37 AM, Maxime Déraspe wrote:
> From: Maxime Deraspe <deraspem@colosse1.(none)>
>
> Signed-off-by: Maxime Deraspe <deraspem@colosse1.(none)>
> ---
> FastDemultiplexer.py | 6 +++---
> 1 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/FastDemultiplexer.py b/FastDemultiplexer.py
> index fa1afe3..e104985 100755
> --- a/FastDemultiplexer.py
> +++ b/FastDemultiplexer.py
> @@ -405,10 +405,10 @@ class Demultiplexer:
>
> def main():
> sheet=sys.argv[1]
> - inputDir=sys.argv[2].replace("\/","")
> - outputDir=sys.argv[3].replace("\/","")
> + inputDir=sys.argv[2]
> + outputDir=sys.argv[3]
>
> - lane=inputDir.split("lane")[1].strip()
> + lane=inputDir.split("lane")[1].strip().replace("/","")
>
> demultiplexer=Demultiplexer(sheet,inputDir,outputDir,lane)
>
>
|
|
From: Maxime D. <max...@gm...> - 2013-02-22 16:39:18
|
From: Maxime Deraspe <deraspem@colosse1.(none)>
Signed-off-by: Maxime Deraspe <deraspem@colosse1.(none)>
---
FastDemultiplexer.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/FastDemultiplexer.py b/FastDemultiplexer.py
index fa1afe3..e104985 100755
--- a/FastDemultiplexer.py
+++ b/FastDemultiplexer.py
@@ -405,10 +405,10 @@ class Demultiplexer:
def main():
sheet=sys.argv[1]
- inputDir=sys.argv[2].replace("\/","")
- outputDir=sys.argv[3].replace("\/","")
+ inputDir=sys.argv[2]
+ outputDir=sys.argv[3]
- lane=inputDir.split("lane")[1].strip()
+ lane=inputDir.split("lane")[1].strip().replace("/","")
demultiplexer=Demultiplexer(sheet,inputDir,outputDir,lane)
--
1.7.4.1
|
|
From: Maxime D. <max...@gm...> - 2013-02-21 22:21:16
|
From: Maxime Deraspe <deraspem@colosse1.(none)>
Signed-off-by: Maxime Deraspe <deraspem@colosse1.(none)>
---
FastDemultiplexer.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/FastDemultiplexer.py b/FastDemultiplexer.py
index fa1afe3..e104985 100755
--- a/FastDemultiplexer.py
+++ b/FastDemultiplexer.py
@@ -405,10 +405,10 @@ class Demultiplexer:
def main():
sheet=sys.argv[1]
- inputDir=sys.argv[2].replace("\/","")
- outputDir=sys.argv[3].replace("\/","")
+ inputDir=sys.argv[2]
+ outputDir=sys.argv[3]
- lane=inputDir.split("lane")[1].strip()
+ lane=inputDir.split("lane")[1].strip().replace("/","")
demultiplexer=Demultiplexer(sheet,inputDir,outputDir,lane)
--
1.7.4.1
|
|
From: Sébastien B. <se...@bo...> - 2013-02-03 16:19:18
|
(Please CC denovoassembler-devel)
I merged the patch.
However, the methods registerPlugin and resolveSymbols should be the two
last in the .cpp file of any plugin. I fixed that.
Sébastien
On 02/02/2013 10:03 PM, Charles Joly Beauparlant wrote:
> This plugin comes from RayPlatform-example.
>
> The example is activated with the -example option. Will work without input sequences.
>
> Signed-off-by: Charles Joly Beauparlant <Cha...@cr...>
> ---
> code/application_core/Machine.cpp | 4 +-
> code/application_core/Machine.h | 5 +-
> code/plugin_Example/Example.cpp | 273 +++++++++++++++++++++++++++
> code/plugin_Example/Example.h | 117 ++++++++++++
> code/plugin_Example/Makefile | 3 +
> code/plugin_MachineHelper/MachineHelper.cpp | 64 ++++---
> code/plugin_MachineHelper/MachineHelper.h | 9 +-
> 7 files changed, 443 insertions(+), 32 deletions(-)
> create mode 100644 code/plugin_Example/Example.cpp
> create mode 100644 code/plugin_Example/Example.h
> create mode 100644 code/plugin_Example/Makefile
>
> diff --git a/code/application_core/Machine.cpp b/code/application_core/Machine.cpp
> index 704c477..dc5c184 100644
> --- a/code/application_core/Machine.cpp
> +++ b/code/application_core/Machine.cpp
> @@ -1,6 +1,7 @@
> /*
> - Ray
> + Ray -- Parallel genome assemblies for parallel DNA sequencing
> Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
> + Copyright (C) 2013 Charles Joly Beauparlant
>
> http://DeNovoAssembler.SourceForge.Net/
>
> @@ -813,6 +814,7 @@ void Machine::registerPlugins(){
> m_computeCore.registerPlugin(&m_genomeNeighbourhood);
> m_computeCore.registerPlugin(&m_ontologyPlugin);
> m_computeCore.registerPlugin(&m_mock);
> + m_computeCore.registerPlugin(&m_example);
>
> // resolve the symbols
> // this is done here because we want to write a summary for
> diff --git a/code/application_core/Machine.h b/code/application_core/Machine.h
> index a65a0a5..65ea98c 100644
> --- a/code/application_core/Machine.h
> +++ b/code/application_core/Machine.h
> @@ -1,6 +1,7 @@
> /*
> - Ray
> + Ray -- Parallel genome assemblies for parallel DNA sequencing
> Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
> + Copyright (C) 2013 Charles Joly Beauparlant
>
> http://DeNovoAssembler.SourceForge.Net/
>
> @@ -63,6 +64,7 @@
> #include <code/plugin_GenomeNeighbourhood/GenomeNeighbourhood.h>
> #include <code/plugin_GeneOntology/GeneOntology.h>
> #include <code/plugin_Mock/Mock.h>
> +#include <code/plugin_Example/Example.h>
>
>
>
> @@ -159,6 +161,7 @@ class Machine : public MiniRank{
> EdgePurger m_edgePurger;
> PhylogenyViewer m_phylogeny;
> GeneOntology m_ontologyPlugin;
> + Example m_example;
>
> KmerAcademyBuilder m_kmerAcademyBuilder;
> bool m_initialisedAcademy;
> diff --git a/code/plugin_Example/Example.cpp b/code/plugin_Example/Example.cpp
> new file mode 100644
> index 0000000..d3ee9a4
> --- /dev/null
> +++ b/code/plugin_Example/Example.cpp
> @@ -0,0 +1,273 @@
> +/*
> + Ray -- Parallel genome assemblies for parallel DNA sequencing
> + Copyright (C) 2013 Charles Joly Beauparlant
> +
> + http://DeNovoAssembler.SourceForge.Net/
> +
> + This program is free software: you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation, version 3 of the License.
> +
> + This program is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You have received a copy of the GNU General Public License
> + along with this program (gpl-3.0.txt).
> + see <http://www.gnu.org/licenses/>
> +
> + This is derived from RayPlatform-example.
> +
> +*/
> +
> +#include "Example.h"
> +
> +__CreatePlugin(Example);
> +
> +__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_A);
> +__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_B);
> +__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_C);
> +
> +__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_A);
> +__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_B);
> +__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_C);
> +
> +__CreateMessageTagAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
> +__CreateMessageTagAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
> +
> +Example::Example(){
> + m_doneA=false;
> + m_doneB=false;
> + m_doneC=false;
> +}
> +
> +void Example::call_RAY_MASTER_MODE_STEP_A(){
> + if(m_doneA==false){
> + m_doneA=true;
> + cout<<"Rank "<<m_core->getRank()<<" call_RAY_MASTER_MODE_STEP_A"<<endl;
> +
> + m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank());
> + }else if(m_core->getSwitchMan()->allRanksAreReady()){
> + m_core->getSwitchMan()->closeMasterMode();
> + }
> +}
> +
> +void Example::call_RAY_MASTER_MODE_STEP_B(){
> + if(m_doneB==false){
> + m_doneB=true;
> + cout<<"Rank "<<MASTER_RANK<<" call_RAY_MASTER_MODE_STEP_B"<<endl;
> + m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank());
> + }else if(m_core->getSwitchMan()->allRanksAreReady()){
> + m_core->getSwitchMan()->closeMasterMode();
> + }
> +}
> +
> +void Example::call_RAY_MASTER_MODE_STEP_C(){
> + if(m_doneC==false){
> + m_doneC=true;
> + cout<<"Rank "<<MASTER_RANK<<" call_RAY_MASTER_MODE_STEP_C"<<endl;
> + m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank());
> + }else if(m_core->getSwitchMan()->allRanksAreReady()){
> + m_core->getSwitchMan()->closeMasterMode();
> + }
> +}
> +
> +void Example::call_RAY_SLAVE_MODE_STEP_A(){
> + cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_A"<<endl;
> +
> + m_core->getSwitchMan()->closeSlaveModeLocally(m_core->getOutbox(),m_core->getRank());
> +}
> +
> +void Example::call_RAY_SLAVE_MODE_STEP_B(){
> + cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_B"<<endl;
> +
> + m_core->getSwitchMan()->closeSlaveModeLocally(m_core->getOutbox(),m_core->getRank());
> +}
> +
> +void Example::call_RAY_SLAVE_MODE_STEP_C(){
> +
> + cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_C, now I die"<<endl;
> + cout<<"This is over "<<endl;
> +
> + if(m_core->getRank()==0){
> + uint64_t*buffer=(uint64_t*)m_core->getOutboxAllocator()->allocate(1*sizeof(uint64_t));
> + buffer[0]=100;
> +
> + // compute the next destination
> + Rank destination=m_core->getRank()+1;
> + if(destination==m_core->getSize())
> + destination=0;
> +
> + Message aMessage(buffer,1,destination,RAY_MPI_TAG_TIME_BOMB,m_core->getRank());
> +
> + // send the bomb to another rank
> + m_core->getOutbox()->push_back(&aMessage);
> + }
> +
> + // do nothing now
> + m_core->getSwitchMan()->setSlaveMode(RAY_SLAVE_MODE_DO_NOTHING);
> +}
> +
> +void Example::call_RAY_MPI_TAG_TIME_BOMB(Message*message){
> + uint64_t*buffer=message->getBuffer();
> +
> + if(buffer[0]==0){
> + cout<<"The bomb exploded on rank "<<m_core->getRank()<<" !"<<endl;
> +
> + // kill everyone
> +
> + m_core->sendEmptyMessageToAll(RAY_MPI_TAG_STOP_AND_DIE);
> + }else{
> + uint64_t*bufferOut=(uint64_t*)m_core->getOutboxAllocator()->allocate(1*sizeof(uint64_t));
> + bufferOut[0]=buffer[0]-1;
> +
> + cout<<"Remaining time before the explosion is "<<bufferOut[0]<<" according to rank "<<m_core->getRank()<<endl;
> +
> + // compute the next destination
> + Rank destination=m_core->getRank()+1;
> + if(destination==m_core->getSize())
> + destination=0;
> +
> + Message aMessage(bufferOut,1,destination,RAY_MPI_TAG_TIME_BOMB,m_core->getRank());
> +
> + // send the bomb to another rank
> + m_core->getOutbox()->push_back(&aMessage);
> + }
> +}
> +
> +void Example::call_RAY_MPI_TAG_STOP_AND_DIE(Message*message){
> +
> + cout<<"rank "<<m_core->getRank()<<" received message RAY_MPI_TAG_STOP_AND_DIE, this kills the batman"<<endl;
> +
> + m_core->stop();
> +}
> +
> +void Example::registerPlugin(ComputeCore*core){
> +
> + /** register the m_plugin with the core **/
> +
> + m_plugin=core->allocatePluginHandle();
> +
> + core->setPluginName(m_plugin,"Example");
> + core->setPluginDescription(m_plugin,"Minimal plugin example");
> + core->setPluginAuthors(m_plugin,"Charles Joly Beauparlant");
> + core->setPluginLicense(m_plugin,"GNU General Public License version 3");
> +
> + // for each master mode, we allocate a handle after that, we register a handler for it
> + //
> + // allocateMasterModeHandle takes 1 arguments
> + // - the plugin handle
> +
> + RAY_MASTER_MODE_STEP_A=core->allocateMasterModeHandle(m_plugin);
> + core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_A,
> + __GetAdapter(Example,RAY_MASTER_MODE_STEP_A));
> +
> + core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_A,"RAY_MASTER_MODE_STEP_A");
> +
> + core->setMasterModePublicAccess(m_plugin, RAY_MASTER_MODE_STEP_A);
> +
> + RAY_MASTER_MODE_STEP_B=core->allocateMasterModeHandle(m_plugin);
> + core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_B,
> + __GetAdapter(Example,RAY_MASTER_MODE_STEP_B));
> +
> + core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_B,"RAY_MASTER_MODE_STEP_B");
> +
> + RAY_MASTER_MODE_STEP_C=core->allocateMasterModeHandle(m_plugin);
> + core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_C,
> + __GetAdapter(Example,RAY_MASTER_MODE_STEP_C));
> +
> + core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_C,"RAY_MASTER_MODE_STEP_C");
> +
> + // for each slave mode, we allocate a handle after that, we register a handler for it
> +
> + RAY_SLAVE_MODE_STEP_A=core->allocateSlaveModeHandle(m_plugin);
> + core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_A,
> + __GetAdapter(Example,RAY_SLAVE_MODE_STEP_A));
> +
> + core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_A,"RAY_SLAVE_MODE_STEP_A");
> +
> + RAY_SLAVE_MODE_STEP_B=core->allocateSlaveModeHandle(m_plugin);
> + core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_B,
> + __GetAdapter(Example,RAY_SLAVE_MODE_STEP_B));
> +
> + core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_B,"RAY_SLAVE_MODE_STEP_B");
> +
> + RAY_SLAVE_MODE_STEP_C=core->allocateSlaveModeHandle(m_plugin);
> + core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_C,
> + __GetAdapter(Example,RAY_SLAVE_MODE_STEP_C));
> +
> + core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_C,"RAY_SLAVE_MODE_STEP_C");
> +
> + RAY_MPI_TAG_START_STEP_A=core->allocateMessageTagHandle(m_plugin);
> + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_A, "RAY_MPI_TAG_START_STEP_A");
> + RAY_MPI_TAG_START_STEP_B=core->allocateMessageTagHandle(m_plugin);
> + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_B, "RAY_MPI_TAG_START_STEP_B");
> + RAY_MPI_TAG_START_STEP_C=core->allocateMessageTagHandle(m_plugin);
> + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_C, "RAY_MPI_TAG_START_STEP_C");
> +
> + // now, we register the order of the master modes
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_STEP_A,RAY_MASTER_MODE_STEP_B);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_STEP_B,RAY_MASTER_MODE_STEP_C);
> +
> + // configure which control message to send given a master mode
> + core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_A,RAY_MPI_TAG_START_STEP_A);
> + core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_B,RAY_MPI_TAG_START_STEP_B);
> + core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_C,RAY_MPI_TAG_START_STEP_C);
> +
> + // configure which slave mode to set given a message tag
> + core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_A,RAY_SLAVE_MODE_STEP_A);
> + core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_B,RAY_SLAVE_MODE_STEP_B);
> + core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_C,RAY_SLAVE_MODE_STEP_C);
> +
> + m_core=core;
> +
> + // configure the two message tags
> + RAY_MPI_TAG_TIME_BOMB=core->allocateMessageTagHandle(m_plugin);
> + core->setMessageTagObjectHandler(m_plugin,RAY_MPI_TAG_TIME_BOMB,
> + __GetAdapter(Example,RAY_MPI_TAG_TIME_BOMB));
> +
> + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_TIME_BOMB, "RAY_MPI_TAG_TIME_BOMB");
> +
> + RAY_MPI_TAG_STOP_AND_DIE=core->allocateMessageTagHandle(m_plugin);
> + core->setMessageTagObjectHandler(m_plugin,RAY_MPI_TAG_STOP_AND_DIE,
> + __GetAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE));
> +
> + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_STOP_AND_DIE, "RAY_MPI_TAG_STOP_AND_DIE");
> +
> + /* a plugin can share any of its object with other plugins **/
> + /** other plugins have to resolve the symbol. **/
> + void*object=&m_doneA;
> + core->setObjectSymbol(m_plugin,object,"BooleanState");
> +}
> +
> +void Example::resolveSymbols(ComputeCore*core){
> +
> + // here, we resolve symbols owned by other m_plugins
> + // but needed by the current m_plugin
> + // obviously, this is not needed here because
> + // we only have one m_plugin
> +
> + RAY_SLAVE_MODE_STEP_A=core->getSlaveModeFromSymbol(m_plugin,"RAY_SLAVE_MODE_STEP_A");
> +
> + // this slave mode is registered somewhere in RayPlatform
> + RAY_SLAVE_MODE_DO_NOTHING=core->getSlaveModeFromSymbol(m_plugin,"RAY_SLAVE_MODE_DO_NOTHING");
> +
> + bool*example=(bool*) core->getObjectFromSymbol(m_plugin,"BooleanState");
> +
> + if(*example)
> + *example=false;
> +
> + __BindPlugin(Example);
> +
> + __BindAdapter(Example,RAY_MASTER_MODE_STEP_A);
> + __BindAdapter(Example,RAY_MASTER_MODE_STEP_B);
> + __BindAdapter(Example,RAY_MASTER_MODE_STEP_C);
> +
> + __BindAdapter(Example,RAY_SLAVE_MODE_STEP_A);
> + __BindAdapter(Example,RAY_SLAVE_MODE_STEP_B);
> + __BindAdapter(Example,RAY_SLAVE_MODE_STEP_C);
> +
> + __BindAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
> + __BindAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
> +}
> diff --git a/code/plugin_Example/Example.h b/code/plugin_Example/Example.h
> new file mode 100644
> index 0000000..01fb444
> --- /dev/null
> +++ b/code/plugin_Example/Example.h
> @@ -0,0 +1,117 @@
> +/*
> + Ray -- Parallel genome assemblies for parallel DNA sequencing
> + Copyright (C) 2013 Charles Joly Beauparlant
> +
> + http://DeNovoAssembler.SourceForge.Net/
> +
> + This program is free software: you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation, version 3 of the License.
> +
> + This program is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You have received a copy of the GNU General Public License
> + along with this program (gpl-3.0.txt).
> + see <http://www.gnu.org/licenses/>
> +
> + This is derived from RayPlatform-example.
> +
> +*/
> +
> +#ifndef _Example_h
> +#define _Example_h
> +
> +#include <RayPlatform/core/ComputeCore.h>
> +
> +__DeclarePlugin(Example);
> +
> +__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_A);
> +__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_B);
> +__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_C);
> +
> +__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_A);
> +__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_B);
> +__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_C);
> +
> +__DeclareMessageTagAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
> +__DeclareMessageTagAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
> +
> +/**
> + * The plugin Example
> + *
> + * \author Charles Joly Beauparlant
> + **/
> +class Example: public CorePlugin{
> +
> + __AddAdapter(Example,RAY_MASTER_MODE_STEP_A);
> + __AddAdapter(Example,RAY_MASTER_MODE_STEP_B);
> + __AddAdapter(Example,RAY_MASTER_MODE_STEP_C);
> +
> + __AddAdapter(Example,RAY_SLAVE_MODE_STEP_A);
> + __AddAdapter(Example,RAY_SLAVE_MODE_STEP_B);
> + __AddAdapter(Example,RAY_SLAVE_MODE_STEP_C);
> +
> + __AddAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
> + __AddAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
> +
> +/**
> + * A list of master modes
> + */
> + MasterMode RAY_MASTER_MODE_STEP_A;
> + MasterMode RAY_MASTER_MODE_STEP_B;
> + MasterMode RAY_MASTER_MODE_STEP_C;
> +
> +/** slave modes **/
> + SlaveMode RAY_SLAVE_MODE_STEP_A;
> + SlaveMode RAY_SLAVE_MODE_STEP_B;
> + SlaveMode RAY_SLAVE_MODE_STEP_C;
> + SlaveMode RAY_SLAVE_MODE_DO_NOTHING;
> +
> +
> + MessageTag RAY_MPI_TAG_START_STEP_A;
> + MessageTag RAY_MPI_TAG_START_STEP_B;
> + MessageTag RAY_MPI_TAG_START_STEP_C;
> +
> + MessageTag RAY_MPI_TAG_TIME_BOMB;
> + MessageTag RAY_MPI_TAG_STOP_AND_DIE;
> +
> + ComputeCore*m_core;
> +
> +/** states for progression **/
> + bool m_doneA;
> + bool m_doneB;
> + bool m_doneC;
> + bool m_doneExample;
> +
> +/** Example state **/
> + bool m_example;
> +public:
> +
> + Example();
> +
> +/** callbacks for master modes **/
> + void call_RAY_MASTER_MODE_STEP_A();
> + void call_RAY_MASTER_MODE_STEP_B();
> + void call_RAY_MASTER_MODE_STEP_C();
> +
> +/** callbacks for slave modes **/
> + void call_RAY_SLAVE_MODE_STEP_A();
> + void call_RAY_SLAVE_MODE_STEP_B();
> + void call_RAY_SLAVE_MODE_STEP_C();
> +
> + void call_RAY_MPI_TAG_TIME_BOMB(Message*message);
> + void call_RAY_MPI_TAG_STOP_AND_DIE(Message*message);
> +
> +/** the following two methods are required by the interface CorePlugin **/
> +
> +/** register the plugin with the core **/
> + void registerPlugin(ComputeCore*core);
> +
> +/** resolve symbols not owned by the current plugin **/
> + void resolveSymbols(ComputeCore*core);
> +};
> +
> +#endif
> diff --git a/code/plugin_Example/Makefile b/code/plugin_Example/Makefile
> new file mode 100644
> index 0000000..50b1ba4
> --- /dev/null
> +++ b/code/plugin_Example/Makefile
> @@ -0,0 +1,3 @@
> +Example-y += code/plugin_Example/Example.o
> +
> +obj-y += $(Example-y)
> diff --git a/code/plugin_MachineHelper/MachineHelper.cpp b/code/plugin_MachineHelper/MachineHelper.cpp
> index fbe52a5..04ec687 100644
> --- a/code/plugin_MachineHelper/MachineHelper.cpp
> +++ b/code/plugin_MachineHelper/MachineHelper.cpp
> @@ -1,6 +1,7 @@
> /*
> - Ray
> - Copyright (C) 2012 Sébastien Boisvert
> + Ray -- Parallel genome assemblies for parallel DNA sequencing
> + Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
> + Copyright (C) 2013 Charles Joly Beauparlant
>
> http://DeNovoAssembler.SourceForge.Net/
>
> @@ -1225,6 +1226,7 @@ void MachineHelper::resolveSymbols(ComputeCore*core){
> RAY_MASTER_MODE_UPDATE_DISTANCES=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_UPDATE_DISTANCES");
> RAY_MASTER_MODE_WRITE_KMERS=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_WRITE_KMERS");
> RAY_MASTER_MODE_WRITE_SCAFFOLDS=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_WRITE_SCAFFOLDS");
> + RAY_MASTER_MODE_STEP_A=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_STEP_A");
>
> RAY_MPI_TAG_FINISH_FUSIONS=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_FINISH_FUSIONS");
> RAY_MPI_TAG_GET_CONTIG_CHUNK=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GET_CONTIG_CHUNK");
> @@ -1312,31 +1314,13 @@ void MachineHelper::resolveSymbols(ComputeCore*core){
> RAY_MPI_TAG_SET_FILE_ENTRIES=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_SET_FILE_ENTRIES");
> RAY_MPI_TAG_SET_FILE_ENTRIES_REPLY=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_SET_FILE_ENTRIES_REPLY");
>
> - core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG);
> -
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG, RAY_MASTER_MODE_TEST_NETWORK);
> -
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_SEEDING,RAY_MASTER_MODE_TRIGGER_SEEDING);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_SEEDING,RAY_MASTER_MODE_START_UPDATING_DISTANCES);
> -
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_SEQUENCES,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS,RAY_MASTER_MODE_SEND_COVERAGE_VALUES);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SEND_COVERAGE_VALUES,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING,RAY_MASTER_MODE_PURGE_NULL_EDGES);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PURGE_NULL_EDGES,RAY_MASTER_MODE_WRITE_KMERS);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_WRITE_KMERS,RAY_MASTER_MODE_TRIGGER_INDEXING);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_INDEXING,RAY_MASTER_MODE_PREPARE_SEEDING);
> -
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_UPDATING_DISTANCES,RAY_MASTER_MODE_UPDATE_DISTANCES);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FUSIONS,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS,RAY_MASTER_MODE_START_FUSION_CYCLE);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_FUSION_CYCLE,RAY_MASTER_MODE_ASK_EXTENSIONS);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_ASK_EXTENSIONS,RAY_MASTER_MODE_SCAFFOLDER);
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SCAFFOLDER,RAY_MASTER_MODE_WRITE_SCAFFOLDS);
> -
> - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_KILL_RANKS,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS);
> + if (m_parameters->hasOption("-example")) {
> + cout << "************** Example Mode **************" << endl;
> + core->setFirstMasterMode(m_plugin, RAY_MASTER_MODE_STEP_A);
> + }
> + else {
> + performAssemblyWorkflow(core);
> + }
>
> RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON");
> RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON_REPLY=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON_REPLY");
> @@ -1382,4 +1366,30 @@ void MachineHelper::resolveSymbols(ComputeCore*core){
> m_startedToSendCounts=false;
> }
>
> +void MachineHelper::performAssemblyWorkflow(ComputeCore*core) {
> + core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG);
> +
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG, RAY_MASTER_MODE_TEST_NETWORK);
> +
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_SEEDING,RAY_MASTER_MODE_TRIGGER_SEEDING);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_SEEDING,RAY_MASTER_MODE_START_UPDATING_DISTANCES);
> +
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_SEQUENCES,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS,RAY_MASTER_MODE_SEND_COVERAGE_VALUES);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SEND_COVERAGE_VALUES,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING,RAY_MASTER_MODE_PURGE_NULL_EDGES);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PURGE_NULL_EDGES,RAY_MASTER_MODE_WRITE_KMERS);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_WRITE_KMERS,RAY_MASTER_MODE_TRIGGER_INDEXING);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_INDEXING,RAY_MASTER_MODE_PREPARE_SEEDING);
> +
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_UPDATING_DISTANCES,RAY_MASTER_MODE_UPDATE_DISTANCES);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FUSIONS,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS,RAY_MASTER_MODE_START_FUSION_CYCLE);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_FUSION_CYCLE,RAY_MASTER_MODE_ASK_EXTENSIONS);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_ASK_EXTENSIONS,RAY_MASTER_MODE_SCAFFOLDER);
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SCAFFOLDER,RAY_MASTER_MODE_WRITE_SCAFFOLDS);
>
> + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_KILL_RANKS,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS);
> +}
> diff --git a/code/plugin_MachineHelper/MachineHelper.h b/code/plugin_MachineHelper/MachineHelper.h
> index c58e680..c2b94d6 100644
> --- a/code/plugin_MachineHelper/MachineHelper.h
> +++ b/code/plugin_MachineHelper/MachineHelper.h
> @@ -1,6 +1,7 @@
> /*
> - Ray
> - Copyright (C) 2012 Sébastien Boisvert
> + Ray -- Parallel genome assemblies for parallel DNA sequencing
> + Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
> + Copyright (C) 2013 Charles Joly Beauparlant
>
> http://DeNovoAssembler.SourceForge.Net/
>
> @@ -24,6 +25,7 @@
> #include <code/plugin_Mock/Parameters.h>
> #include <code/plugin_FusionData/FusionData.h>
> #include <code/plugin_NetworkTest/NetworkTest.h>
> +#include <code/plugin_Example/Example.h>
> #include <code/plugin_Partitioner/Partitioner.h>
> #include <code/plugin_SequencesLoader/SequencesLoader.h>
> #include <code/plugin_SeedingData/SeedingData.h>
> @@ -251,6 +253,7 @@ class MachineHelper: public CorePlugin{
> MasterMode RAY_MASTER_MODE_UPDATE_DISTANCES;
> MasterMode RAY_MASTER_MODE_WRITE_KMERS;
> MasterMode RAY_MASTER_MODE_WRITE_SCAFFOLDS;
> + MasterMode RAY_MASTER_MODE_STEP_A;
>
> SlaveMode RAY_SLAVE_MODE_EXTENSION;
> SlaveMode RAY_SLAVE_MODE_ADD_COLORS;
> @@ -359,7 +362,7 @@ class MachineHelper: public CorePlugin{
>
> int getRank();
> int getSize();
> -
> + void performAssemblyWorkflow(ComputeCore*core);
>
> public:
> void constructor(int argc,char**argv,Parameters*parameters,
>
|
|
From: Sébastien B. <seb...@ul...> - 2013-02-02 08:16:50
|
I edited the following bits in your patch and merged your work. 1. screen and canvas size -> screen and canvas sizes 2. I converted spaces to tabular symbols for indentation 3. I appended my signed-off line to your patch Your work is at git://github.com/sebhtml/Ray-Cloud-Browser.git 4030605c42194accefdc2e3c6b8dbcb70ad46da0 or at https://github.com/sebhtml/Ray-Cloud-Browser/commit/4030605c42194accefdc2e3c6b8dbcb70ad46da0 Thank you. On 02/01/2013 06:21 PM, Pier-Luc Plante wrote: > Fixed? > > commit: 36e44fc52d3b1013f6990e4d3b3dfb44e5f79160 > git://github.com/plpla/Ray-Cloud-Browser.git master > > PL > > Le 2013-02-01 16:43, Sébastien Boisvert a écrit : >> This code should probably go in draw() in Screen because the iterate() method >> runs the physics engine and has nothing to do with rendering, >> >> On 02/01/2013 04:24 PM, Pier-Luc Plante wrote: >>> Signed-off-by: Pier-Luc Plante <pie...@ul...> >>> --- >>> client/Screen.js | 20 ++++++++++++++++++++ >>> 1 file changed, 20 insertions(+) >>> >>> diff --git a/client/Screen.js b/client/Screen.js >>> index 97bd068..119cdf7 100644 >>> --- a/client/Screen.js >>> +++ b/client/Screen.js >>> @@ -583,6 +583,9 @@ Screen.prototype.iterate=function(){ >>> } >>> >>> this.performZoomOperations(); >>> + if(!this.checkScreenSize()){ >>> + this.performScreenSizeChange(); >>> + } >>> >>> var end=this.getMilliseconds(); >>> this.gameMilliseconds+=(end-start); >>> @@ -591,6 +594,23 @@ Screen.prototype.iterate=function(){ >>> this.globalGameFrameNumber++; >>> } >>> >>> +Screen.prototype.checkScreenSize=function(){ >>> + return(this.width==window.innerWidth && this.height==window.innerHeight && >>> + this.canvas.width==this.width && this.canvas.heidth==this.height && >>> + this.renderingCanvas.width==this.width && this.renderingCanvas.height==this.height); >>> +} >>> + >>> +Screen.prototype.performScreenSizeChange=function(){ >>> + var availableWidth=window.innerWidth; >>> + var availableHeight=window.innerHeight; >>> + this.width=availableWidth; >>> + this.height=availableHeight; >>> + this.canvas.width=this.width; >>> + this.canvas.height=this.height; >> This resets the canvas. So this method should be called somewhere before the drawing. >> >>> + this.renderingCanvas.width=this.width; >>> + this.renderingCanvas.height=this.height; >> OK. >> >>> +} >>> + >>> Screen.prototype.getMilliseconds=function(){ >>> return new Date()*1; >>> } >>> >> > |
|
From: Sébastien B. <seb...@ul...> - 2013-02-01 22:36:53
|
3 minor concerns: * remove white spaces in your commit message * remove trailing white spaces (git diff --color to spot them * move RayPlatform API calls in a new method for the classical assembly workflow. Details below: On 02/01/2013 05:13 PM, Charles Joly Beauparlant wrote: > This plugin comes from RayPlatform-example. > > The example is activated with the -example option. Will work without input sequences. > Remove these spaces at the beginning of lines. > Signed-off-by: Charles Joly Beauparlant <Cha...@cr...> > --- > code/application_core/Machine.cpp | 4 +- > code/application_core/Machine.h | 5 +- > code/plugin_Example/Example.cpp | 278 +++++++++++++++++++++++++++ > code/plugin_Example/Example.h | 117 +++++++++++ > code/plugin_Example/Makefile | 3 + > code/plugin_MachineHelper/MachineHelper.cpp | 63 ++++--- > code/plugin_MachineHelper/MachineHelper.h | 7 +- > 7 files changed, 446 insertions(+), 31 deletions(-) > create mode 100644 code/plugin_Example/Example.cpp > create mode 100644 code/plugin_Example/Example.h > create mode 100644 code/plugin_Example/Makefile > > diff --git a/code/application_core/Machine.cpp b/code/application_core/Machine.cpp > index 704c477..dc5c184 100644 > --- a/code/application_core/Machine.cpp > +++ b/code/application_core/Machine.cpp > @@ -1,6 +1,7 @@ > /* > - Ray > + Ray -- Parallel genome assemblies for parallel DNA sequencing > Copyright (C) 2010, 2011, 2012 Sébastien Boisvert > + Copyright (C) 2013 Charles Joly Beauparlant > > http://DeNovoAssembler.SourceForge.Net/ > > @@ -813,6 +814,7 @@ void Machine::registerPlugins(){ > m_computeCore.registerPlugin(&m_genomeNeighbourhood); > m_computeCore.registerPlugin(&m_ontologyPlugin); > m_computeCore.registerPlugin(&m_mock); > + m_computeCore.registerPlugin(&m_example); > > // resolve the symbols > // this is done here because we want to write a summary for > diff --git a/code/application_core/Machine.h b/code/application_core/Machine.h > index a65a0a5..65ea98c 100644 > --- a/code/application_core/Machine.h > +++ b/code/application_core/Machine.h > @@ -1,6 +1,7 @@ > /* > - Ray > + Ray -- Parallel genome assemblies for parallel DNA sequencing > Copyright (C) 2010, 2011, 2012 Sébastien Boisvert > + Copyright (C) 2013 Charles Joly Beauparlant > > http://DeNovoAssembler.SourceForge.Net/ > > @@ -63,6 +64,7 @@ > #include <code/plugin_GenomeNeighbourhood/GenomeNeighbourhood.h> > #include <code/plugin_GeneOntology/GeneOntology.h> > #include <code/plugin_Mock/Mock.h> > +#include <code/plugin_Example/Example.h> > > > > @@ -159,6 +161,7 @@ class Machine : public MiniRank{ > EdgePurger m_edgePurger; > PhylogenyViewer m_phylogeny; > GeneOntology m_ontologyPlugin; > + Example m_example; > > KmerAcademyBuilder m_kmerAcademyBuilder; > bool m_initialisedAcademy; > diff --git a/code/plugin_Example/Example.cpp b/code/plugin_Example/Example.cpp > new file mode 100644 > index 0000000..baf949b > --- /dev/null > +++ b/code/plugin_Example/Example.cpp > @@ -0,0 +1,278 @@ > +/* > + Ray -- Parallel genome assemblies for parallel DNA sequencing > + Copyright (C) 2013 Charles Joly Beauparlant remove extra white space at the end of this line > + > + http://DeNovoAssembler.SourceForge.Net/ > + > + This program is free software: you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation, version 3 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You have received a copy of the GNU General Public License > + along with this program (gpl-3.0.txt). remove extra white space at the end of this line > + see <http://www.gnu.org/licenses/> > + > + This is derived from RayPlatform-example. > + > +*/ > + > +#include "Example.h" > + > +__CreatePlugin(Example); > + > +__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_A); > +__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_B); > +__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_C); > + > +__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_A); > +__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_B); > +__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_C); > + > +__CreateMessageTagAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE); > +__CreateMessageTagAdapter(Example,RAY_MPI_TAG_TIME_BOMB); > + > +Example::Example(){ > + m_doneA=false; > + m_doneB=false; > + m_doneC=false; > +} > + > +void Example::call_RAY_MASTER_MODE_STEP_A(){ > + if(m_doneA==false){ > + m_doneA=true; > + cout<<"Rank "<<m_core->getRank()<<" call_RAY_MASTER_MODE_STEP_A"<<endl; > + > + m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank()); > + }else if(m_core->getSwitchMan()->allRanksAreReady()){ > + m_core->getSwitchMan()->closeMasterMode(); > + } > +} > + > +void Example::call_RAY_MASTER_MODE_STEP_B(){ > + if(m_doneB==false){ > + m_doneB=true; > + cout<<"Rank "<<MASTER_RANK<<" call_RAY_MASTER_MODE_STEP_B"<<endl; > + m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank()); > + }else if(m_core->getSwitchMan()->allRanksAreReady()){ > + m_core->getSwitchMan()->closeMasterMode(); > + } > +} > + > +void Example::call_RAY_MASTER_MODE_STEP_C(){ > + if(m_doneC==false){ > + m_doneC=true; > + cout<<"Rank "<<MASTER_RANK<<" call_RAY_MASTER_MODE_STEP_C"<<endl; > + m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank()); > + }else if(m_core->getSwitchMan()->allRanksAreReady()){ > + m_core->getSwitchMan()->closeMasterMode(); > + } > +} > + > +void Example::call_RAY_SLAVE_MODE_STEP_A(){ > + cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_A"<<endl; > + > + m_core->getSwitchMan()->closeSlaveModeLocally(m_core->getOutbox(),m_core->getRank()); > +} > + > +void Example::call_RAY_SLAVE_MODE_STEP_B(){ > + cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_B"<<endl; > + > + m_core->getSwitchMan()->closeSlaveModeLocally(m_core->getOutbox(),m_core->getRank()); > +} > + > +void Example::call_RAY_SLAVE_MODE_STEP_C(){ > + > + cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_C, now I die"<<endl; > + cout<<"This is over "<<endl; > + > + if(m_core->getRank()==0){ > + uint64_t*buffer=(uint64_t*)m_core->getOutboxAllocator()->allocate(1*sizeof(uint64_t)); > + buffer[0]=100; > + > + // compute the next destination > + Rank destination=m_core->getRank()+1; > + if(destination==m_core->getSize()) > + destination=0; > + > + Message aMessage(buffer,1,destination,RAY_MPI_TAG_TIME_BOMB,m_core->getRank()); > + > + // send the bomb to another rank > + m_core->getOutbox()->push_back(&aMessage); > + } > + > + // do nothing now > + m_core->getSwitchMan()->setSlaveMode(RAY_SLAVE_MODE_DO_NOTHING); > +} > + > +void Example::call_RAY_MPI_TAG_TIME_BOMB(Message*message){ > + uint64_t*buffer=message->getBuffer(); > + > + if(buffer[0]==0){ > + cout<<"The bomb exploded on rank "<<m_core->getRank()<<" !"<<endl; > + > + // kill everyone > + > + m_core->sendEmptyMessageToAll(RAY_MPI_TAG_STOP_AND_DIE); > + }else{ > + uint64_t*bufferOut=(uint64_t*)m_core->getOutboxAllocator()->allocate(1*sizeof(uint64_t)); > + bufferOut[0]=buffer[0]-1; > + > + cout<<"Remaining time before the explosion is "<<bufferOut[0]<<" according to rank "<<m_core->getRank()<<endl; > + > + // compute the next destination > + Rank destination=m_core->getRank()+1; > + if(destination==m_core->getSize()) > + destination=0; > + > + Message aMessage(bufferOut,1,destination,RAY_MPI_TAG_TIME_BOMB,m_core->getRank()); > + > + // send the bomb to another rank > + m_core->getOutbox()->push_back(&aMessage); > + } > +} > + > +void Example::call_RAY_MPI_TAG_STOP_AND_DIE(Message*message){ > + > + cout<<"rank "<<m_core->getRank()<<" received message RAY_MPI_TAG_STOP_AND_DIE, this kills the batman"<<endl; > + > + m_core->stop(); > +} > + > +void Example::registerPlugin(ComputeCore*core){ > + > + /** register the m_plugin with the core **/ > + > + m_plugin=core->allocatePluginHandle(); > + > + core->setPluginName(m_plugin,"Example"); > + core->setPluginDescription(m_plugin,"Minimal plugin example"); > + core->setPluginAuthors(m_plugin,"Charles Joly Beauparlant"); > + core->setPluginLicense(m_plugin,"GNU General Public License version 3"); > + > + // for each master mode, we allocate a handle remove extra white space at the end of this line > + // after that, we register a handler for it > + // > + // allocateMasterModeHandle takes 1 arguments > + // - the plugin handle > + > + RAY_MASTER_MODE_STEP_A=core->allocateMasterModeHandle(m_plugin); > + core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_A, > + __GetAdapter(Example,RAY_MASTER_MODE_STEP_A)); > + > + core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_A,"RAY_MASTER_MODE_STEP_A"); > + > + core->setMasterModePublicAccess(m_plugin, RAY_MASTER_MODE_STEP_A); > + > + RAY_MASTER_MODE_STEP_B=core->allocateMasterModeHandle(m_plugin); > + core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_B, > + __GetAdapter(Example,RAY_MASTER_MODE_STEP_B)); > + > + core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_B,"RAY_MASTER_MODE_STEP_B"); > + > + RAY_MASTER_MODE_STEP_C=core->allocateMasterModeHandle(m_plugin); > + core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_C, > + __GetAdapter(Example,RAY_MASTER_MODE_STEP_C)); > + > + core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_C,"RAY_MASTER_MODE_STEP_C"); > + > + // for each slave mode, we allocate a handle > + // after that, we register a handler for it > + > + RAY_SLAVE_MODE_STEP_A=core->allocateSlaveModeHandle(m_plugin); > + core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_A, > + __GetAdapter(Example,RAY_SLAVE_MODE_STEP_A)); > + > + core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_A,"RAY_SLAVE_MODE_STEP_A"); > + > + RAY_SLAVE_MODE_STEP_B=core->allocateSlaveModeHandle(m_plugin); > + core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_B, > + __GetAdapter(Example,RAY_SLAVE_MODE_STEP_B)); > + > + core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_B,"RAY_SLAVE_MODE_STEP_B"); > + > + RAY_SLAVE_MODE_STEP_C=core->allocateSlaveModeHandle(m_plugin); > + core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_C, > + __GetAdapter(Example,RAY_SLAVE_MODE_STEP_C)); > + > + core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_C,"RAY_SLAVE_MODE_STEP_C"); > + > + RAY_MPI_TAG_START_STEP_A=core->allocateMessageTagHandle(m_plugin); > + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_A, "RAY_MPI_TAG_START_STEP_A"); > + RAY_MPI_TAG_START_STEP_B=core->allocateMessageTagHandle(m_plugin); > + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_B, "RAY_MPI_TAG_START_STEP_B"); > + RAY_MPI_TAG_START_STEP_C=core->allocateMessageTagHandle(m_plugin); > + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_C, "RAY_MPI_TAG_START_STEP_C"); > + > + // now, we register the order of the master modes > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_STEP_A,RAY_MASTER_MODE_STEP_B); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_STEP_B,RAY_MASTER_MODE_STEP_C); > + > + // configure which control message to send given a master mode > + core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_A,RAY_MPI_TAG_START_STEP_A); > + core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_B,RAY_MPI_TAG_START_STEP_B); > + core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_C,RAY_MPI_TAG_START_STEP_C); > + > + // configure which slave mode to set given a message tag > + core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_A,RAY_SLAVE_MODE_STEP_A); > + core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_B,RAY_SLAVE_MODE_STEP_B); > + core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_C,RAY_SLAVE_MODE_STEP_C); > + > + // finally, we define an entry point > +// core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_STEP_A); remove this > + > + m_core=core; > + > + // configure the two message tags > + RAY_MPI_TAG_TIME_BOMB=core->allocateMessageTagHandle(m_plugin); > + core->setMessageTagObjectHandler(m_plugin,RAY_MPI_TAG_TIME_BOMB, > + __GetAdapter(Example,RAY_MPI_TAG_TIME_BOMB)); > + > + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_TIME_BOMB, "RAY_MPI_TAG_TIME_BOMB"); > + > + RAY_MPI_TAG_STOP_AND_DIE=core->allocateMessageTagHandle(m_plugin); > + core->setMessageTagObjectHandler(m_plugin,RAY_MPI_TAG_STOP_AND_DIE, > + __GetAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE)); > + > + core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_STOP_AND_DIE, "RAY_MPI_TAG_STOP_AND_DIE"); > + > + /* a plugin can share any of its object with other plugins **/ > + /** other plugins have to resolve the symbol. **/ > + void*object=&m_doneA; > + core->setObjectSymbol(m_plugin,object,"BooleanState"); > +} > + > +void Example::resolveSymbols(ComputeCore*core){ > + > + // here, we resolve symbols owned by other m_plugins > + // but needed by the current m_plugin > + // obviously, this is not needed here because > + // we only have one m_plugin > + remove extra white space at the end of this line > + RAY_SLAVE_MODE_STEP_A=core->getSlaveModeFromSymbol(m_plugin,"RAY_SLAVE_MODE_STEP_A"); > + > + // this slave mode is registered somewhere in RayPlatform > + RAY_SLAVE_MODE_DO_NOTHING=core->getSlaveModeFromSymbol(m_plugin,"RAY_SLAVE_MODE_DO_NOTHING"); > + > + bool*example=(bool*) core->getObjectFromSymbol(m_plugin,"BooleanState"); > + > + if(*example) > + *example=false; > + > + __BindPlugin(Example); > + > + __BindAdapter(Example,RAY_MASTER_MODE_STEP_A); > + __BindAdapter(Example,RAY_MASTER_MODE_STEP_B); > + __BindAdapter(Example,RAY_MASTER_MODE_STEP_C); > + > + __BindAdapter(Example,RAY_SLAVE_MODE_STEP_A); > + __BindAdapter(Example,RAY_SLAVE_MODE_STEP_B); > + __BindAdapter(Example,RAY_SLAVE_MODE_STEP_C); > + > + __BindAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE); > + __BindAdapter(Example,RAY_MPI_TAG_TIME_BOMB); > +} > diff --git a/code/plugin_Example/Example.h b/code/plugin_Example/Example.h > new file mode 100644 > index 0000000..600afe8 > --- /dev/null > +++ b/code/plugin_Example/Example.h > @@ -0,0 +1,117 @@ > +/* > + Ray -- Parallel genome assemblies for parallel DNA sequencing > + Copyright (C) 2013 Charles Joly Beauparlant remove extra white space at the end of this line > + > + http://DeNovoAssembler.SourceForge.Net/ > + > + This program is free software: you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation, version 3 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You have received a copy of the GNU General Public License > + along with this program (gpl-3.0.txt). remove extra white space at the end of this line > + see <http://www.gnu.org/licenses/> > + > + This is derived from RayPlatform-example. > + > +*/ > + > +#ifndef _Example_h > +#define _Example_h > + > +#include <RayPlatform/core/ComputeCore.h> > + > +__DeclarePlugin(Example); > + > +__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_A); > +__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_B); > +__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_C); > + > +__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_A); > +__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_B); > +__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_C); > + > +__DeclareMessageTagAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE); > +__DeclareMessageTagAdapter(Example,RAY_MPI_TAG_TIME_BOMB); > + > +/** > + * The plugin Example > + * > + * \author Charles Joly Beauparlant > + **/ > +class Example: public CorePlugin{ > + > + __AddAdapter(Example,RAY_MASTER_MODE_STEP_A); > + __AddAdapter(Example,RAY_MASTER_MODE_STEP_B); > + __AddAdapter(Example,RAY_MASTER_MODE_STEP_C); > + > + __AddAdapter(Example,RAY_SLAVE_MODE_STEP_A); > + __AddAdapter(Example,RAY_SLAVE_MODE_STEP_B); > + __AddAdapter(Example,RAY_SLAVE_MODE_STEP_C); > + > + __AddAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE); > + __AddAdapter(Example,RAY_MPI_TAG_TIME_BOMB); > + > +/** > + * A list of master modes > + */ > + MasterMode RAY_MASTER_MODE_STEP_A; > + MasterMode RAY_MASTER_MODE_STEP_B; > + MasterMode RAY_MASTER_MODE_STEP_C; > + > +/** slave modes **/ > + SlaveMode RAY_SLAVE_MODE_STEP_A; > + SlaveMode RAY_SLAVE_MODE_STEP_B; > + SlaveMode RAY_SLAVE_MODE_STEP_C; > + SlaveMode RAY_SLAVE_MODE_DO_NOTHING; > + > + > + MessageTag RAY_MPI_TAG_START_STEP_A; > + MessageTag RAY_MPI_TAG_START_STEP_B; > + MessageTag RAY_MPI_TAG_START_STEP_C; > + > + MessageTag RAY_MPI_TAG_TIME_BOMB; > + MessageTag RAY_MPI_TAG_STOP_AND_DIE; > + > + ComputeCore*m_core; > + > +/** states for progression **/ > + bool m_doneA; > + bool m_doneB; > + bool m_doneC; > + bool m_doneExample; > + > +/** Example state **/ > + bool m_example; > +public: > + > + Example(); > + > +/** callbacks for master modes **/ > + void call_RAY_MASTER_MODE_STEP_A(); > + void call_RAY_MASTER_MODE_STEP_B(); > + void call_RAY_MASTER_MODE_STEP_C(); > + > +/** callbacks for slave modes **/ > + void call_RAY_SLAVE_MODE_STEP_A(); > + void call_RAY_SLAVE_MODE_STEP_B(); > + void call_RAY_SLAVE_MODE_STEP_C(); > + > + void call_RAY_MPI_TAG_TIME_BOMB(Message*message); > + void call_RAY_MPI_TAG_STOP_AND_DIE(Message*message); > + > +/** the following two methods are required by the interface CorePlugin **/ > + > +/** register the plugin with the core **/ > + void registerPlugin(ComputeCore*core); > + > +/** resolve symbols not owned by the current plugin **/ > + void resolveSymbols(ComputeCore*core); > +}; > + > +#endif > diff --git a/code/plugin_Example/Makefile b/code/plugin_Example/Makefile > new file mode 100644 > index 0000000..50b1ba4 > --- /dev/null > +++ b/code/plugin_Example/Makefile > @@ -0,0 +1,3 @@ > +Example-y += code/plugin_Example/Example.o > + > +obj-y += $(Example-y) > diff --git a/code/plugin_MachineHelper/MachineHelper.cpp b/code/plugin_MachineHelper/MachineHelper.cpp > index fbe52a5..afc6335 100644 > --- a/code/plugin_MachineHelper/MachineHelper.cpp > +++ b/code/plugin_MachineHelper/MachineHelper.cpp > @@ -1,6 +1,7 @@ > /* > - Ray > - Copyright (C) 2012 Sébastien Boisvert > + Ray -- Parallel genome assemblies for parallel DNA sequencing > + Copyright (C) 2010, 2011, 2012 Sébastien Boisvert > + Copyright (C) 2013 Charles Joly Beauparlant > > http://DeNovoAssembler.SourceForge.Net/ > > @@ -1225,6 +1226,7 @@ void MachineHelper::resolveSymbols(ComputeCore*core){ > RAY_MASTER_MODE_UPDATE_DISTANCES=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_UPDATE_DISTANCES"); > RAY_MASTER_MODE_WRITE_KMERS=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_WRITE_KMERS"); > RAY_MASTER_MODE_WRITE_SCAFFOLDS=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_WRITE_SCAFFOLDS"); > + RAY_MASTER_MODE_STEP_A=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_STEP_A"); > > RAY_MPI_TAG_FINISH_FUSIONS=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_FINISH_FUSIONS"); > RAY_MPI_TAG_GET_CONTIG_CHUNK=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GET_CONTIG_CHUNK"); > @@ -1312,31 +1314,38 @@ void MachineHelper::resolveSymbols(ComputeCore*core){ > RAY_MPI_TAG_SET_FILE_ENTRIES=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_SET_FILE_ENTRIES"); > RAY_MPI_TAG_SET_FILE_ENTRIES_REPLY=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_SET_FILE_ENTRIES_REPLY"); > Maybe you can put the lines below in a new method called performAssemblyWorkflow(). > - core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG); > - > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG, RAY_MASTER_MODE_TEST_NETWORK); > - > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_SEEDING,RAY_MASTER_MODE_TRIGGER_SEEDING); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_SEEDING,RAY_MASTER_MODE_START_UPDATING_DISTANCES); > - > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_SEQUENCES,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS,RAY_MASTER_MODE_SEND_COVERAGE_VALUES); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SEND_COVERAGE_VALUES,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING,RAY_MASTER_MODE_PURGE_NULL_EDGES); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PURGE_NULL_EDGES,RAY_MASTER_MODE_WRITE_KMERS); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_WRITE_KMERS,RAY_MASTER_MODE_TRIGGER_INDEXING); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_INDEXING,RAY_MASTER_MODE_PREPARE_SEEDING); > - > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_UPDATING_DISTANCES,RAY_MASTER_MODE_UPDATE_DISTANCES); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FUSIONS,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS,RAY_MASTER_MODE_START_FUSION_CYCLE); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_FUSION_CYCLE,RAY_MASTER_MODE_ASK_EXTENSIONS); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_ASK_EXTENSIONS,RAY_MASTER_MODE_SCAFFOLDER); > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SCAFFOLDER,RAY_MASTER_MODE_WRITE_SCAFFOLDS); > - > - core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_KILL_RANKS,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS); > + if (m_parameters->hasOption("-example")) { > + cout << "************** Example Mode **************" << endl; > +// core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS); > + core->setFirstMasterMode(m_plugin, RAY_MASTER_MODE_STEP_A); > + } > + else { Here call performAssemblyWorkflow() instead of these indented lines below. > + core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG); > + > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG, RAY_MASTER_MODE_TEST_NETWORK); > + > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_SEEDING,RAY_MASTER_MODE_TRIGGER_SEEDING); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_SEEDING,RAY_MASTER_MODE_START_UPDATING_DISTANCES); > + > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_SEQUENCES,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS,RAY_MASTER_MODE_SEND_COVERAGE_VALUES); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SEND_COVERAGE_VALUES,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING,RAY_MASTER_MODE_PURGE_NULL_EDGES); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PURGE_NULL_EDGES,RAY_MASTER_MODE_WRITE_KMERS); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_WRITE_KMERS,RAY_MASTER_MODE_TRIGGER_INDEXING); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_INDEXING,RAY_MASTER_MODE_PREPARE_SEEDING); > + > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_UPDATING_DISTANCES,RAY_MASTER_MODE_UPDATE_DISTANCES); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FUSIONS,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS,RAY_MASTER_MODE_START_FUSION_CYCLE); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_FUSION_CYCLE,RAY_MASTER_MODE_ASK_EXTENSIONS); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_ASK_EXTENSIONS,RAY_MASTER_MODE_SCAFFOLDER); > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SCAFFOLDER,RAY_MASTER_MODE_WRITE_SCAFFOLDS); > + > + core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_KILL_RANKS,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS); > + } > > RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON"); > RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON_REPLY=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON_REPLY"); > diff --git a/code/plugin_MachineHelper/MachineHelper.h b/code/plugin_MachineHelper/MachineHelper.h > index c58e680..459d21c 100644 > --- a/code/plugin_MachineHelper/MachineHelper.h > +++ b/code/plugin_MachineHelper/MachineHelper.h > @@ -1,6 +1,7 @@ > /* > - Ray > - Copyright (C) 2012 Sébastien Boisvert > + Ray -- Parallel genome assemblies for parallel DNA sequencing > + Copyright (C) 2010, 2011, 2012 Sébastien Boisvert > + Copyright (C) 2013 Charles Joly Beauparlant > > http://DeNovoAssembler.SourceForge.Net/ > > @@ -24,6 +25,7 @@ > #include <code/plugin_Mock/Parameters.h> > #include <code/plugin_FusionData/FusionData.h> > #include <code/plugin_NetworkTest/NetworkTest.h> > +#include <code/plugin_Example/Example.h> > #include <code/plugin_Partitioner/Partitioner.h> > #include <code/plugin_SequencesLoader/SequencesLoader.h> > #include <code/plugin_SeedingData/SeedingData.h> > @@ -251,6 +253,7 @@ class MachineHelper: public CorePlugin{ > MasterMode RAY_MASTER_MODE_UPDATE_DISTANCES; > MasterMode RAY_MASTER_MODE_WRITE_KMERS; > MasterMode RAY_MASTER_MODE_WRITE_SCAFFOLDS; > + MasterMode RAY_MASTER_MODE_STEP_A; > > SlaveMode RAY_SLAVE_MODE_EXTENSION; > SlaveMode RAY_SLAVE_MODE_ADD_COLORS; > |
|
From: Charles J. B. <Cha...@cr...> - 2013-02-01 21:44:06
|
This plugin comes from RayPlatform-example.
The example is activated with the -example option. Will work without input sequences.
Signed-off-by: Charles Joly Beauparlant <Cha...@cr...>
---
code/application_core/Machine.cpp | 4 +-
code/application_core/Machine.h | 5 +-
code/plugin_Example/Example.cpp | 278 +++++++++++++++++++++++++++
code/plugin_Example/Example.h | 117 +++++++++++
code/plugin_Example/Makefile | 3 +
code/plugin_MachineHelper/MachineHelper.cpp | 63 ++++---
code/plugin_MachineHelper/MachineHelper.h | 7 +-
7 files changed, 446 insertions(+), 31 deletions(-)
create mode 100644 code/plugin_Example/Example.cpp
create mode 100644 code/plugin_Example/Example.h
create mode 100644 code/plugin_Example/Makefile
diff --git a/code/application_core/Machine.cpp b/code/application_core/Machine.cpp
index 704c477..dc5c184 100644
--- a/code/application_core/Machine.cpp
+++ b/code/application_core/Machine.cpp
@@ -1,6 +1,7 @@
/*
- Ray
+ Ray -- Parallel genome assemblies for parallel DNA sequencing
Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
+ Copyright (C) 2013 Charles Joly Beauparlant
http://DeNovoAssembler.SourceForge.Net/
@@ -813,6 +814,7 @@ void Machine::registerPlugins(){
m_computeCore.registerPlugin(&m_genomeNeighbourhood);
m_computeCore.registerPlugin(&m_ontologyPlugin);
m_computeCore.registerPlugin(&m_mock);
+ m_computeCore.registerPlugin(&m_example);
// resolve the symbols
// this is done here because we want to write a summary for
diff --git a/code/application_core/Machine.h b/code/application_core/Machine.h
index a65a0a5..65ea98c 100644
--- a/code/application_core/Machine.h
+++ b/code/application_core/Machine.h
@@ -1,6 +1,7 @@
/*
- Ray
+ Ray -- Parallel genome assemblies for parallel DNA sequencing
Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
+ Copyright (C) 2013 Charles Joly Beauparlant
http://DeNovoAssembler.SourceForge.Net/
@@ -63,6 +64,7 @@
#include <code/plugin_GenomeNeighbourhood/GenomeNeighbourhood.h>
#include <code/plugin_GeneOntology/GeneOntology.h>
#include <code/plugin_Mock/Mock.h>
+#include <code/plugin_Example/Example.h>
@@ -159,6 +161,7 @@ class Machine : public MiniRank{
EdgePurger m_edgePurger;
PhylogenyViewer m_phylogeny;
GeneOntology m_ontologyPlugin;
+ Example m_example;
KmerAcademyBuilder m_kmerAcademyBuilder;
bool m_initialisedAcademy;
diff --git a/code/plugin_Example/Example.cpp b/code/plugin_Example/Example.cpp
new file mode 100644
index 0000000..baf949b
--- /dev/null
+++ b/code/plugin_Example/Example.cpp
@@ -0,0 +1,278 @@
+/*
+ Ray -- Parallel genome assemblies for parallel DNA sequencing
+ Copyright (C) 2013 Charles Joly Beauparlant
+
+ http://DeNovoAssembler.SourceForge.Net/
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, version 3 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You have received a copy of the GNU General Public License
+ along with this program (gpl-3.0.txt).
+ see <http://www.gnu.org/licenses/>
+
+ This is derived from RayPlatform-example.
+
+*/
+
+#include "Example.h"
+
+__CreatePlugin(Example);
+
+__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_A);
+__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_B);
+__CreateMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_C);
+
+__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_A);
+__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_B);
+__CreateSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_C);
+
+__CreateMessageTagAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
+__CreateMessageTagAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
+
+Example::Example(){
+ m_doneA=false;
+ m_doneB=false;
+ m_doneC=false;
+}
+
+void Example::call_RAY_MASTER_MODE_STEP_A(){
+ if(m_doneA==false){
+ m_doneA=true;
+ cout<<"Rank "<<m_core->getRank()<<" call_RAY_MASTER_MODE_STEP_A"<<endl;
+
+ m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank());
+ }else if(m_core->getSwitchMan()->allRanksAreReady()){
+ m_core->getSwitchMan()->closeMasterMode();
+ }
+}
+
+void Example::call_RAY_MASTER_MODE_STEP_B(){
+ if(m_doneB==false){
+ m_doneB=true;
+ cout<<"Rank "<<MASTER_RANK<<" call_RAY_MASTER_MODE_STEP_B"<<endl;
+ m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank());
+ }else if(m_core->getSwitchMan()->allRanksAreReady()){
+ m_core->getSwitchMan()->closeMasterMode();
+ }
+}
+
+void Example::call_RAY_MASTER_MODE_STEP_C(){
+ if(m_doneC==false){
+ m_doneC=true;
+ cout<<"Rank "<<MASTER_RANK<<" call_RAY_MASTER_MODE_STEP_C"<<endl;
+ m_core->getSwitchMan()->openMasterMode(m_core->getOutbox(),m_core->getRank());
+ }else if(m_core->getSwitchMan()->allRanksAreReady()){
+ m_core->getSwitchMan()->closeMasterMode();
+ }
+}
+
+void Example::call_RAY_SLAVE_MODE_STEP_A(){
+ cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_A"<<endl;
+
+ m_core->getSwitchMan()->closeSlaveModeLocally(m_core->getOutbox(),m_core->getRank());
+}
+
+void Example::call_RAY_SLAVE_MODE_STEP_B(){
+ cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_B"<<endl;
+
+ m_core->getSwitchMan()->closeSlaveModeLocally(m_core->getOutbox(),m_core->getRank());
+}
+
+void Example::call_RAY_SLAVE_MODE_STEP_C(){
+
+ cout<<"I am "<<m_core->getRank()<<" doing call_RAY_SLAVE_MODE_STEP_C, now I die"<<endl;
+ cout<<"This is over "<<endl;
+
+ if(m_core->getRank()==0){
+ uint64_t*buffer=(uint64_t*)m_core->getOutboxAllocator()->allocate(1*sizeof(uint64_t));
+ buffer[0]=100;
+
+ // compute the next destination
+ Rank destination=m_core->getRank()+1;
+ if(destination==m_core->getSize())
+ destination=0;
+
+ Message aMessage(buffer,1,destination,RAY_MPI_TAG_TIME_BOMB,m_core->getRank());
+
+ // send the bomb to another rank
+ m_core->getOutbox()->push_back(&aMessage);
+ }
+
+ // do nothing now
+ m_core->getSwitchMan()->setSlaveMode(RAY_SLAVE_MODE_DO_NOTHING);
+}
+
+void Example::call_RAY_MPI_TAG_TIME_BOMB(Message*message){
+ uint64_t*buffer=message->getBuffer();
+
+ if(buffer[0]==0){
+ cout<<"The bomb exploded on rank "<<m_core->getRank()<<" !"<<endl;
+
+ // kill everyone
+
+ m_core->sendEmptyMessageToAll(RAY_MPI_TAG_STOP_AND_DIE);
+ }else{
+ uint64_t*bufferOut=(uint64_t*)m_core->getOutboxAllocator()->allocate(1*sizeof(uint64_t));
+ bufferOut[0]=buffer[0]-1;
+
+ cout<<"Remaining time before the explosion is "<<bufferOut[0]<<" according to rank "<<m_core->getRank()<<endl;
+
+ // compute the next destination
+ Rank destination=m_core->getRank()+1;
+ if(destination==m_core->getSize())
+ destination=0;
+
+ Message aMessage(bufferOut,1,destination,RAY_MPI_TAG_TIME_BOMB,m_core->getRank());
+
+ // send the bomb to another rank
+ m_core->getOutbox()->push_back(&aMessage);
+ }
+}
+
+void Example::call_RAY_MPI_TAG_STOP_AND_DIE(Message*message){
+
+ cout<<"rank "<<m_core->getRank()<<" received message RAY_MPI_TAG_STOP_AND_DIE, this kills the batman"<<endl;
+
+ m_core->stop();
+}
+
+void Example::registerPlugin(ComputeCore*core){
+
+ /** register the m_plugin with the core **/
+
+ m_plugin=core->allocatePluginHandle();
+
+ core->setPluginName(m_plugin,"Example");
+ core->setPluginDescription(m_plugin,"Minimal plugin example");
+ core->setPluginAuthors(m_plugin,"Charles Joly Beauparlant");
+ core->setPluginLicense(m_plugin,"GNU General Public License version 3");
+
+ // for each master mode, we allocate a handle
+ // after that, we register a handler for it
+ //
+ // allocateMasterModeHandle takes 1 arguments
+ // - the plugin handle
+
+ RAY_MASTER_MODE_STEP_A=core->allocateMasterModeHandle(m_plugin);
+ core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_A,
+ __GetAdapter(Example,RAY_MASTER_MODE_STEP_A));
+
+ core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_A,"RAY_MASTER_MODE_STEP_A");
+
+ core->setMasterModePublicAccess(m_plugin, RAY_MASTER_MODE_STEP_A);
+
+ RAY_MASTER_MODE_STEP_B=core->allocateMasterModeHandle(m_plugin);
+ core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_B,
+ __GetAdapter(Example,RAY_MASTER_MODE_STEP_B));
+
+ core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_B,"RAY_MASTER_MODE_STEP_B");
+
+ RAY_MASTER_MODE_STEP_C=core->allocateMasterModeHandle(m_plugin);
+ core->setMasterModeObjectHandler(m_plugin,RAY_MASTER_MODE_STEP_C,
+ __GetAdapter(Example,RAY_MASTER_MODE_STEP_C));
+
+ core->setMasterModeSymbol(m_plugin,RAY_MASTER_MODE_STEP_C,"RAY_MASTER_MODE_STEP_C");
+
+ // for each slave mode, we allocate a handle
+ // after that, we register a handler for it
+
+ RAY_SLAVE_MODE_STEP_A=core->allocateSlaveModeHandle(m_plugin);
+ core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_A,
+ __GetAdapter(Example,RAY_SLAVE_MODE_STEP_A));
+
+ core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_A,"RAY_SLAVE_MODE_STEP_A");
+
+ RAY_SLAVE_MODE_STEP_B=core->allocateSlaveModeHandle(m_plugin);
+ core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_B,
+ __GetAdapter(Example,RAY_SLAVE_MODE_STEP_B));
+
+ core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_B,"RAY_SLAVE_MODE_STEP_B");
+
+ RAY_SLAVE_MODE_STEP_C=core->allocateSlaveModeHandle(m_plugin);
+ core->setSlaveModeObjectHandler(m_plugin,RAY_SLAVE_MODE_STEP_C,
+ __GetAdapter(Example,RAY_SLAVE_MODE_STEP_C));
+
+ core->setSlaveModeSymbol(m_plugin,RAY_SLAVE_MODE_STEP_C,"RAY_SLAVE_MODE_STEP_C");
+
+ RAY_MPI_TAG_START_STEP_A=core->allocateMessageTagHandle(m_plugin);
+ core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_A, "RAY_MPI_TAG_START_STEP_A");
+ RAY_MPI_TAG_START_STEP_B=core->allocateMessageTagHandle(m_plugin);
+ core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_B, "RAY_MPI_TAG_START_STEP_B");
+ RAY_MPI_TAG_START_STEP_C=core->allocateMessageTagHandle(m_plugin);
+ core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_START_STEP_C, "RAY_MPI_TAG_START_STEP_C");
+
+ // now, we register the order of the master modes
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_STEP_A,RAY_MASTER_MODE_STEP_B);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_STEP_B,RAY_MASTER_MODE_STEP_C);
+
+ // configure which control message to send given a master mode
+ core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_A,RAY_MPI_TAG_START_STEP_A);
+ core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_B,RAY_MPI_TAG_START_STEP_B);
+ core->setMasterModeToMessageTagSwitch(m_plugin,RAY_MASTER_MODE_STEP_C,RAY_MPI_TAG_START_STEP_C);
+
+ // configure which slave mode to set given a message tag
+ core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_A,RAY_SLAVE_MODE_STEP_A);
+ core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_B,RAY_SLAVE_MODE_STEP_B);
+ core->setMessageTagToSlaveModeSwitch(m_plugin,RAY_MPI_TAG_START_STEP_C,RAY_SLAVE_MODE_STEP_C);
+
+ // finally, we define an entry point
+// core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_STEP_A);
+
+ m_core=core;
+
+ // configure the two message tags
+ RAY_MPI_TAG_TIME_BOMB=core->allocateMessageTagHandle(m_plugin);
+ core->setMessageTagObjectHandler(m_plugin,RAY_MPI_TAG_TIME_BOMB,
+ __GetAdapter(Example,RAY_MPI_TAG_TIME_BOMB));
+
+ core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_TIME_BOMB, "RAY_MPI_TAG_TIME_BOMB");
+
+ RAY_MPI_TAG_STOP_AND_DIE=core->allocateMessageTagHandle(m_plugin);
+ core->setMessageTagObjectHandler(m_plugin,RAY_MPI_TAG_STOP_AND_DIE,
+ __GetAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE));
+
+ core->setMessageTagSymbol(m_plugin, RAY_MPI_TAG_STOP_AND_DIE, "RAY_MPI_TAG_STOP_AND_DIE");
+
+ /* a plugin can share any of its object with other plugins **/
+ /** other plugins have to resolve the symbol. **/
+ void*object=&m_doneA;
+ core->setObjectSymbol(m_plugin,object,"BooleanState");
+}
+
+void Example::resolveSymbols(ComputeCore*core){
+
+ // here, we resolve symbols owned by other m_plugins
+ // but needed by the current m_plugin
+ // obviously, this is not needed here because
+ // we only have one m_plugin
+
+ RAY_SLAVE_MODE_STEP_A=core->getSlaveModeFromSymbol(m_plugin,"RAY_SLAVE_MODE_STEP_A");
+
+ // this slave mode is registered somewhere in RayPlatform
+ RAY_SLAVE_MODE_DO_NOTHING=core->getSlaveModeFromSymbol(m_plugin,"RAY_SLAVE_MODE_DO_NOTHING");
+
+ bool*example=(bool*) core->getObjectFromSymbol(m_plugin,"BooleanState");
+
+ if(*example)
+ *example=false;
+
+ __BindPlugin(Example);
+
+ __BindAdapter(Example,RAY_MASTER_MODE_STEP_A);
+ __BindAdapter(Example,RAY_MASTER_MODE_STEP_B);
+ __BindAdapter(Example,RAY_MASTER_MODE_STEP_C);
+
+ __BindAdapter(Example,RAY_SLAVE_MODE_STEP_A);
+ __BindAdapter(Example,RAY_SLAVE_MODE_STEP_B);
+ __BindAdapter(Example,RAY_SLAVE_MODE_STEP_C);
+
+ __BindAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
+ __BindAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
+}
diff --git a/code/plugin_Example/Example.h b/code/plugin_Example/Example.h
new file mode 100644
index 0000000..600afe8
--- /dev/null
+++ b/code/plugin_Example/Example.h
@@ -0,0 +1,117 @@
+/*
+ Ray -- Parallel genome assemblies for parallel DNA sequencing
+ Copyright (C) 2013 Charles Joly Beauparlant
+
+ http://DeNovoAssembler.SourceForge.Net/
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, version 3 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You have received a copy of the GNU General Public License
+ along with this program (gpl-3.0.txt).
+ see <http://www.gnu.org/licenses/>
+
+ This is derived from RayPlatform-example.
+
+*/
+
+#ifndef _Example_h
+#define _Example_h
+
+#include <RayPlatform/core/ComputeCore.h>
+
+__DeclarePlugin(Example);
+
+__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_A);
+__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_B);
+__DeclareMasterModeAdapter(Example,RAY_MASTER_MODE_STEP_C);
+
+__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_A);
+__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_B);
+__DeclareSlaveModeAdapter(Example,RAY_SLAVE_MODE_STEP_C);
+
+__DeclareMessageTagAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
+__DeclareMessageTagAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
+
+/**
+ * The plugin Example
+ *
+ * \author Charles Joly Beauparlant
+ **/
+class Example: public CorePlugin{
+
+ __AddAdapter(Example,RAY_MASTER_MODE_STEP_A);
+ __AddAdapter(Example,RAY_MASTER_MODE_STEP_B);
+ __AddAdapter(Example,RAY_MASTER_MODE_STEP_C);
+
+ __AddAdapter(Example,RAY_SLAVE_MODE_STEP_A);
+ __AddAdapter(Example,RAY_SLAVE_MODE_STEP_B);
+ __AddAdapter(Example,RAY_SLAVE_MODE_STEP_C);
+
+ __AddAdapter(Example,RAY_MPI_TAG_STOP_AND_DIE);
+ __AddAdapter(Example,RAY_MPI_TAG_TIME_BOMB);
+
+/**
+ * A list of master modes
+ */
+ MasterMode RAY_MASTER_MODE_STEP_A;
+ MasterMode RAY_MASTER_MODE_STEP_B;
+ MasterMode RAY_MASTER_MODE_STEP_C;
+
+/** slave modes **/
+ SlaveMode RAY_SLAVE_MODE_STEP_A;
+ SlaveMode RAY_SLAVE_MODE_STEP_B;
+ SlaveMode RAY_SLAVE_MODE_STEP_C;
+ SlaveMode RAY_SLAVE_MODE_DO_NOTHING;
+
+
+ MessageTag RAY_MPI_TAG_START_STEP_A;
+ MessageTag RAY_MPI_TAG_START_STEP_B;
+ MessageTag RAY_MPI_TAG_START_STEP_C;
+
+ MessageTag RAY_MPI_TAG_TIME_BOMB;
+ MessageTag RAY_MPI_TAG_STOP_AND_DIE;
+
+ ComputeCore*m_core;
+
+/** states for progression **/
+ bool m_doneA;
+ bool m_doneB;
+ bool m_doneC;
+ bool m_doneExample;
+
+/** Example state **/
+ bool m_example;
+public:
+
+ Example();
+
+/** callbacks for master modes **/
+ void call_RAY_MASTER_MODE_STEP_A();
+ void call_RAY_MASTER_MODE_STEP_B();
+ void call_RAY_MASTER_MODE_STEP_C();
+
+/** callbacks for slave modes **/
+ void call_RAY_SLAVE_MODE_STEP_A();
+ void call_RAY_SLAVE_MODE_STEP_B();
+ void call_RAY_SLAVE_MODE_STEP_C();
+
+ void call_RAY_MPI_TAG_TIME_BOMB(Message*message);
+ void call_RAY_MPI_TAG_STOP_AND_DIE(Message*message);
+
+/** the following two methods are required by the interface CorePlugin **/
+
+/** register the plugin with the core **/
+ void registerPlugin(ComputeCore*core);
+
+/** resolve symbols not owned by the current plugin **/
+ void resolveSymbols(ComputeCore*core);
+};
+
+#endif
diff --git a/code/plugin_Example/Makefile b/code/plugin_Example/Makefile
new file mode 100644
index 0000000..50b1ba4
--- /dev/null
+++ b/code/plugin_Example/Makefile
@@ -0,0 +1,3 @@
+Example-y += code/plugin_Example/Example.o
+
+obj-y += $(Example-y)
diff --git a/code/plugin_MachineHelper/MachineHelper.cpp b/code/plugin_MachineHelper/MachineHelper.cpp
index fbe52a5..afc6335 100644
--- a/code/plugin_MachineHelper/MachineHelper.cpp
+++ b/code/plugin_MachineHelper/MachineHelper.cpp
@@ -1,6 +1,7 @@
/*
- Ray
- Copyright (C) 2012 Sébastien Boisvert
+ Ray -- Parallel genome assemblies for parallel DNA sequencing
+ Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
+ Copyright (C) 2013 Charles Joly Beauparlant
http://DeNovoAssembler.SourceForge.Net/
@@ -1225,6 +1226,7 @@ void MachineHelper::resolveSymbols(ComputeCore*core){
RAY_MASTER_MODE_UPDATE_DISTANCES=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_UPDATE_DISTANCES");
RAY_MASTER_MODE_WRITE_KMERS=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_WRITE_KMERS");
RAY_MASTER_MODE_WRITE_SCAFFOLDS=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_WRITE_SCAFFOLDS");
+ RAY_MASTER_MODE_STEP_A=core->getMasterModeFromSymbol(m_plugin,"RAY_MASTER_MODE_STEP_A");
RAY_MPI_TAG_FINISH_FUSIONS=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_FINISH_FUSIONS");
RAY_MPI_TAG_GET_CONTIG_CHUNK=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GET_CONTIG_CHUNK");
@@ -1312,31 +1314,38 @@ void MachineHelper::resolveSymbols(ComputeCore*core){
RAY_MPI_TAG_SET_FILE_ENTRIES=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_SET_FILE_ENTRIES");
RAY_MPI_TAG_SET_FILE_ENTRIES_REPLY=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_SET_FILE_ENTRIES_REPLY");
- core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG);
-
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG, RAY_MASTER_MODE_TEST_NETWORK);
-
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_SEEDING,RAY_MASTER_MODE_TRIGGER_SEEDING);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_SEEDING,RAY_MASTER_MODE_START_UPDATING_DISTANCES);
-
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_SEQUENCES,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS,RAY_MASTER_MODE_SEND_COVERAGE_VALUES);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SEND_COVERAGE_VALUES,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING,RAY_MASTER_MODE_PURGE_NULL_EDGES);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PURGE_NULL_EDGES,RAY_MASTER_MODE_WRITE_KMERS);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_WRITE_KMERS,RAY_MASTER_MODE_TRIGGER_INDEXING);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_INDEXING,RAY_MASTER_MODE_PREPARE_SEEDING);
-
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_UPDATING_DISTANCES,RAY_MASTER_MODE_UPDATE_DISTANCES);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FUSIONS,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS,RAY_MASTER_MODE_START_FUSION_CYCLE);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_FUSION_CYCLE,RAY_MASTER_MODE_ASK_EXTENSIONS);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_ASK_EXTENSIONS,RAY_MASTER_MODE_SCAFFOLDER);
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SCAFFOLDER,RAY_MASTER_MODE_WRITE_SCAFFOLDS);
-
- core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_KILL_RANKS,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS);
+ if (m_parameters->hasOption("-example")) {
+ cout << "************** Example Mode **************" << endl;
+// core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS);
+ core->setFirstMasterMode(m_plugin, RAY_MASTER_MODE_STEP_A);
+ }
+ else {
+ core->setFirstMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG);
+
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_CONFIG, RAY_MASTER_MODE_TEST_NETWORK);
+
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_SEEDING,RAY_MASTER_MODE_TRIGGER_SEEDING);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_SEEDING,RAY_MASTER_MODE_START_UPDATING_DISTANCES);
+
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_LOAD_SEQUENCES,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_VERTICE_DISTRIBUTION,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PREPARE_DISTRIBUTIONS_WITH_ANSWERS,RAY_MASTER_MODE_SEND_COVERAGE_VALUES);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SEND_COVERAGE_VALUES,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_GRAPH_BUILDING,RAY_MASTER_MODE_PURGE_NULL_EDGES);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_PURGE_NULL_EDGES,RAY_MASTER_MODE_WRITE_KMERS);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_WRITE_KMERS,RAY_MASTER_MODE_TRIGGER_INDEXING);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_INDEXING,RAY_MASTER_MODE_PREPARE_SEEDING);
+
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_UPDATING_DISTANCES,RAY_MASTER_MODE_UPDATE_DISTANCES);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FUSIONS,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_TRIGGER_FIRST_FUSIONS,RAY_MASTER_MODE_START_FUSION_CYCLE);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_START_FUSION_CYCLE,RAY_MASTER_MODE_ASK_EXTENSIONS);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_ASK_EXTENSIONS,RAY_MASTER_MODE_SCAFFOLDER);
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_SCAFFOLDER,RAY_MASTER_MODE_WRITE_SCAFFOLDS);
+
+ core->setMasterModeNextMasterMode(m_plugin,RAY_MASTER_MODE_KILL_RANKS,RAY_MASTER_MODE_KILL_ALL_MPI_RANKS);
+ }
RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON");
RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON_REPLY=core->getMessageTagFromSymbol(m_plugin,"RAY_MPI_TAG_GOOD_JOB_SEE_YOU_SOON_REPLY");
diff --git a/code/plugin_MachineHelper/MachineHelper.h b/code/plugin_MachineHelper/MachineHelper.h
index c58e680..459d21c 100644
--- a/code/plugin_MachineHelper/MachineHelper.h
+++ b/code/plugin_MachineHelper/MachineHelper.h
@@ -1,6 +1,7 @@
/*
- Ray
- Copyright (C) 2012 Sébastien Boisvert
+ Ray -- Parallel genome assemblies for parallel DNA sequencing
+ Copyright (C) 2010, 2011, 2012 Sébastien Boisvert
+ Copyright (C) 2013 Charles Joly Beauparlant
http://DeNovoAssembler.SourceForge.Net/
@@ -24,6 +25,7 @@
#include <code/plugin_Mock/Parameters.h>
#include <code/plugin_FusionData/FusionData.h>
#include <code/plugin_NetworkTest/NetworkTest.h>
+#include <code/plugin_Example/Example.h>
#include <code/plugin_Partitioner/Partitioner.h>
#include <code/plugin_SequencesLoader/SequencesLoader.h>
#include <code/plugin_SeedingData/SeedingData.h>
@@ -251,6 +253,7 @@ class MachineHelper: public CorePlugin{
MasterMode RAY_MASTER_MODE_UPDATE_DISTANCES;
MasterMode RAY_MASTER_MODE_WRITE_KMERS;
MasterMode RAY_MASTER_MODE_WRITE_SCAFFOLDS;
+ MasterMode RAY_MASTER_MODE_STEP_A;
SlaveMode RAY_SLAVE_MODE_EXTENSION;
SlaveMode RAY_SLAVE_MODE_ADD_COLORS;
--
1.7.7.6
|
|
From: Sébastien B. <seb...@ul...> - 2013-02-01 21:43:07
|
This code should probably go in draw() in Screen because the iterate() method
runs the physics engine and has nothing to do with rendering,
On 02/01/2013 04:24 PM, Pier-Luc Plante wrote:
> Signed-off-by: Pier-Luc Plante <pie...@ul...>
> ---
> client/Screen.js | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
>
> diff --git a/client/Screen.js b/client/Screen.js
> index 97bd068..119cdf7 100644
> --- a/client/Screen.js
> +++ b/client/Screen.js
> @@ -583,6 +583,9 @@ Screen.prototype.iterate=function(){
> }
>
> this.performZoomOperations();
> + if(!this.checkScreenSize()){
> + this.performScreenSizeChange();
> + }
>
> var end=this.getMilliseconds();
> this.gameMilliseconds+=(end-start);
> @@ -591,6 +594,23 @@ Screen.prototype.iterate=function(){
> this.globalGameFrameNumber++;
> }
>
> +Screen.prototype.checkScreenSize=function(){
> + return(this.width==window.innerWidth && this.height==window.innerHeight &&
> + this.canvas.width==this.width && this.canvas.heidth==this.height &&
> + this.renderingCanvas.width==this.width && this.renderingCanvas.height==this.height);
> +}
> +
> +Screen.prototype.performScreenSizeChange=function(){
> + var availableWidth=window.innerWidth;
> + var availableHeight=window.innerHeight;
> + this.width=availableWidth;
> + this.height=availableHeight;
> + this.canvas.width=this.width;
> + this.canvas.height=this.height;
This resets the canvas. So this method should be called somewhere before the drawing.
> + this.renderingCanvas.width=this.width;
> + this.renderingCanvas.height=this.height;
OK.
> +}
> +
> Screen.prototype.getMilliseconds=function(){
> return new Date()*1;
> }
>
|
|
From: Sébastien B. <seb...@ul...> - 2013-02-01 21:35:17
|
I get intense flickering.
This should probably done only a few times per second.
I am not sure flickering occurs though.
On 02/01/2013 04:24 PM, Pier-Luc Plante wrote:
> Signed-off-by: Pier-Luc Plante <pie...@ul...>
> ---
> client/Screen.js | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
>
> diff --git a/client/Screen.js b/client/Screen.js
> index 97bd068..119cdf7 100644
> --- a/client/Screen.js
> +++ b/client/Screen.js
> @@ -583,6 +583,9 @@ Screen.prototype.iterate=function(){
> }
>
> this.performZoomOperations();
> + if(!this.checkScreenSize()){
> + this.performScreenSizeChange();
> + }
>
> var end=this.getMilliseconds();
> this.gameMilliseconds+=(end-start);
> @@ -591,6 +594,23 @@ Screen.prototype.iterate=function(){
> this.globalGameFrameNumber++;
> }
>
> +Screen.prototype.checkScreenSize=function(){
> + return(this.width==window.innerWidth && this.height==window.innerHeight &&
> + this.canvas.width==this.width && this.canvas.heidth==this.height &&
> + this.renderingCanvas.width==this.width && this.renderingCanvas.height==this.height);
> +}
This method name is not really informative.
> +
> +Screen.prototype.performScreenSizeChange=function(){
> + var availableWidth=window.innerWidth;
> + var availableHeight=window.innerHeight;
> + this.width=availableWidth;
> + this.height=availableHeight;
> + this.canvas.width=this.width;
> + this.canvas.height=this.height;
> + this.renderingCanvas.width=this.width;
> + this.renderingCanvas.height=this.height;
> +}
> +
> Screen.prototype.getMilliseconds=function(){
> return new Date()*1;
> }
>
|
|
From: Sébastien B. <seb...@ul...> - 2013-02-01 21:35:17
|
I am getting heavy flickering on Chrome Version 24.0.1312.56 in Fedora 17.
checkScreenSize may be buggy.
Maybe this should be done a maximum of, let' say, 1 time per second.
On 02/01/2013 04:24 PM, Pier-Luc Plante wrote:
> Signed-off-by: Pier-Luc Plante <pie...@ul...>
> ---
> client/Screen.js | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
>
> diff --git a/client/Screen.js b/client/Screen.js
> index 97bd068..119cdf7 100644
> --- a/client/Screen.js
> +++ b/client/Screen.js
> @@ -583,6 +583,9 @@ Screen.prototype.iterate=function(){
> }
>
> this.performZoomOperations();
> + if(!this.checkScreenSize()){
> + this.performScreenSizeChange();
> + }
>
> var end=this.getMilliseconds();
> this.gameMilliseconds+=(end-start);
> @@ -591,6 +594,23 @@ Screen.prototype.iterate=function(){
> this.globalGameFrameNumber++;
> }
>
> +Screen.prototype.checkScreenSize=function(){
> + return(this.width==window.innerWidth && this.height==window.innerHeight &&
> + this.canvas.width==this.width && this.canvas.heidth==this.height &&
> + this.renderingCanvas.width==this.width && this.renderingCanvas.height==this.height);
> +}
This method tells yes if the screen have not changed ?
If so, its name should reflect that.
> +
> +Screen.prototype.performScreenSizeChange=function(){
> + var availableWidth=window.innerWidth;
> + var availableHeight=window.innerHeight;
> + this.width=availableWidth;
> + this.height=availableHeight;
> + this.canvas.width=this.width;
> + this.canvas.height=this.height;
> + this.renderingCanvas.width=this.width;
> + this.renderingCanvas.height=this.height;
> +}
> +
> Screen.prototype.getMilliseconds=function(){
> return new Date()*1;
> }
>
|
