Re: [Moosefs-users] Write starvation

[Robert D. <ro...@in...>]

As Ricardo mentioned below, you could be hitting the metadata, a lot.  When
I ran benchmarks for enumerations in a directory with more than 10,000
files, it was an obvious slow down in comparison to a directory with only a
few files.

 

Not sure which OS you are running, but I would tweak the many IO/Net/Process
parameters.  From personal experience, I had a huge slow down with
mfsmasterserver when running on CentOS.  I have since moved to Ubuntu, which
appears to be faster out of the box.

 

-Rob

 

  _____  

From: Robert [mailto:rsa...@ne...] 
Sent: Wednesday, June 29, 2011 11:29 AM
To: ric...@da...; moo...@li...
Subject: Re: [Moosefs-users] Write starvation

 

Yes, the master server is working hard. But I would still expect a somewhat
fair distribution of load between read and write.

The specs:

2 x quad core Xeon E5405 @ 2GHz.
64 GB of RAM
32 x 2 TB 7200 RPM SATA disks
68 million file system objects
65.4 million files

No swap is being used
mfsmaster is using 23 GB of RAM.

Robert

 

-----Original Message-----
From: Ricardo J. Barberis <ric...@da...>
To: moosefs-users <moo...@li...>
Sent: Wed, Jun 29, 2011 12:08 pm
Subject: Re: [Moosefs-users] Write starvation

El Martes 28 Junio 2011, Robert Sandilands escribió:
> Write traffic does not stop completely, but seems to slow down to < 10
> kB per second under high read traffic conditions. When the read traffic
> decreases the write traffic will increase to normal levels.
> 
> Is this a known problem? Is there something I can do to ensure that
> write traffic is not starved by read traffic?
> 
> Robert
 
What are the master machine specs, and how many files do you have already on

MFS?
 
You might be hiting the master too hard with metadata.
 
Regards,
-- 
Ricardo J. Barberis
Senior SysAdmin / ITI
Dattatec.com :: Soluciones de Web Hosting
Tu Hosting hecho Simple!
 
----------------------------------------------------------------------------
--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users