Re: [Moosefs-users] Optimal writes for MooseFS

[Thomas S H. <tha...@gm...>]

This helps a lot! Thanks Michal

2011/2/8 Michal Borychowski <mic...@ge...>

> Hi!
>
>
>
> This is very interesting question. Our experience shows that when you have
> a big file to write (eg. 100 GB) it is good to logically divide it into
> blocks of one chunk size (64MB) and write these blocks to different files
> and later merge these files into one by using *mfsappendchunks* tool. This
> allows us to optimally use the throughput of the network. Mind that the last
> block may be filled with “zeros” up to 64MB.
>
>
>
> Another option is to also divide a big file into chunks and write them to
> the target file simultaneously do different positions of the file (using
> seek). So each “writer” saves a portion of data of [ chunksize*i,
> chunksize*(i+1)] range to different position.
>
>
>
> Unfortunately I am not sure if this could be implemented for optimal
> writing of vm images…
>
>
>
>
>
> Regards
>
> Michal
>
>
>
>
>
> *From:* Thomas S Hatch [mailto:tha...@gm...]
> *Sent:* Thursday, January 27, 2011 9:41 PM
> *To:* moosefs-users
> *Subject:* [Moosefs-users] Optimal writes for MooseFS
>
>
>
> We already know that vm image writes are not the optimal way to write to
> MooseFS, and randomIO is obviously not the best, so let me ask, what it the
> optimal way to write to a MooseFS filesystem? just sequentially? Is an
> append to an existing file faster?
>
>
>
> Thanks!
>
>
>
> -Thomas S Hatch
>