Re: [Moosefs-users] Optimal writes for MooseFS

[Michal B. <mic...@ge...>]

Hi!

 

This is very interesting question. Our experience shows that when you have a
big file to write (eg. 100 GB) it is good to logically divide it into blocks
of one chunk size (64MB) and write these blocks to different files and later
merge these files into one by using mfsappendchunks tool. This allows us to
optimally use the throughput of the network. Mind that the last block may be
filled with "zeros" up to 64MB.

 

Another option is to also divide a big file into chunks and write them to
the target file simultaneously do different positions of the file (using
seek). So each "writer" saves a portion of data of [ chunksize*i,
chunksize*(i+1)] range to different position.

 

Unfortunately I am not sure if this could be implemented for optimal writing
of vm images.

 

 

Regards

Michal

 

 

From: Thomas S Hatch [mailto:tha...@gm...] 
Sent: Thursday, January 27, 2011 9:41 PM
To: moosefs-users
Subject: [Moosefs-users] Optimal writes for MooseFS

 

We already know that vm image writes are not the optimal way to write to
MooseFS, and randomIO is obviously not the best, so let me ask, what it the
optimal way to write to a MooseFS filesystem? just sequentially? Is an
append to an existing file faster?

 

Thanks!

 

-Thomas S Hatch