Re: [Moosefs-users] Write starvation

[Robert S. <rsa...@ne...>]

Hi Michal,

In our case we never modify the files once we have received them. It is 
also very rare for multiple processes to read the same file at the same 
time. What is not rare is hundreds of simultaneous reads on hundreds of 
unique files randomly distributed through the file system.

We plan on adding more chunkservers as we move content from traditional 
file systems to MFS. A dedicated master may be some time away. Getting a 
machine with 64+ GB of RAM is not always easy to get past the budget 
monsters.

Robert

On 7/4/11 3:30 AM, Michal Borychowski wrote:
> Hi Robert!
>
> Parallel reading and writing to the same file indeed causes some write
> delays. If it is not your case, adding 1-2 chunkservers and having a
> dedicated machine for the master should help.
>
>
> Kind regards
> Michał Borychowski
> MooseFS Support Manager
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> Gemius S.A.
> ul. Wołoska 7, 02-672 Warszawa
> Budynek MARS, klatka D
> Tel.: +4822 874-41-00
> Fax : +4822 874-41-01
>
>
>
> -----Original Message-----
> From: Robert Sandilands [mailto:rsa...@ne...]
> Sent: Wednesday, June 29, 2011 4:30 AM
> To: moo...@li...
> Subject: [Moosefs-users] Write starvation
>
> I have been moving data from existing non-distributed file systems onto
> a MooseFS file system.
>
> I am using 1.6.20 on Centos 5.6.
>
> While moving the data I have also transfered some of the normal
> read-only traffic load to use the data already moved onto the MFS volume.
>
> What I can see is when there is any significant read traffic then write
> traffic slows down to a crawl.
>
> When I look at the server charts for any of the chunk servers generated
> by mfscgiserv then it seems like read and write traffic seems to alternate.
>
> Write traffic does not stop completely, but seems to slow down to<  10
> kB per second under high read traffic conditions. When the read traffic
> decreases the write traffic will increase to normal levels.
>
> Is this a known problem? Is there something I can do to ensure that
> write traffic is not starved by read traffic?
>
> Robert
>
>
>
>
>
> ----------------------------------------------------------------------------
> --
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>