[Denovoassembler-devel] Write a single file from many source actors

[Sébastien B. <se...@bo...>]

Hi Maxime,


Option 1: store keepers send messages to a single actor who will write 
the file
in real time. Be sure to buffer the I/O.


StoreKeeper  ------------>   MatrixOwner       --------->   
KmerFeatureFile

StoreKeeper  ------------>

StoreKeeper  ------------>

StoreKeeper  ------------>




One issue with this solution is that two runs lead to two different 
KmerFeatureFile
because the order of the kmers will be different. However, Both files 
will contain the
same kmers though, so strictly speaking that's the same data.



Option 2: use MPI I/O

The other approach would be to use MPI I/O (there is one StoreKeeper 
actor per MPI rank, so it would work).
You would basically need to know how much bytes are required by each 
StoreKeeper in order to set the
MPI File views (offsets, in some way).



I prefer option 1 because it is more natural with regards to the actor 
model.
I think that the MPI I/O option will lead to a better performance on 
file systems like GPFS or Lustre.


Anyway, that's pretty much the idea.