Re: [Moosefs-users] cache in moosefs

[Michał B. <mic...@ge...>]

> -O mfscachefiles
>  preserve file data in cache (default is not to use data cache)

Now this option should be used:
-o mfscachemode=X

where X can be:
YES - all files are stored in a local cache
NO - no files are stored in a local cache
AUTO - storing in a local cache depends on 'allowdatacache' flag which can
be set individually for every file

MooseFS is a network file system and generally speaking files should not be
kept in cache because from the moment of putting a file in cache another
computer may change it. So cache should be used only there where you are
sure about this.


 
> -O mfsattrcacheto=SEC
> set attributes cache timeout in seconds (default: 1.0)
This is time of keeping file attributes in cache which are used by kernel by
simple operations like "rm", "mv", etc. The system tests many times
attributes of the same i-nodes and that's why we should allow for using
cache here. The time should be relevantly short but not shorter than time
needed for an individual operation like "rm" or "mv". We suggest leaving the
default value.

 
 
> -O mfsentrycacheto=SEC
> set file entry cache timeout in seconds (default: 0.0, I.e. no
> cache)
  
> -O mfsdirentrycacheto=SEC
> set directory entry cache timeout in seconds (default: 1.0)
 
Both options affect keeping entry in a folder ("lookup" operation - mapping
file or folder name to its i-node). This situation is similar to the above.
Every operation in which we refer to the full path causes that the system
every time scans file system in order to locate the given object. Most of
these operations concerns folders and folder's attributes do not change
often. So we decided to store in cache subfolders entries. That's why there
are two options: "mfsdirentrycacheto" for subfolders and "mfsentrycacheto"
for other objects. We also suggest leaving the default values here.

 

> -O mfswritecachesize=N
> specify write cache size in MiB (in range: 16..2048 - default:
> 250)
In order to increase performance writing in MooseFS is doen by "mfsmount" in
background. That's why some part of RAM has to be used as a write buffer.
This option allows to change this buffer. Lower value means lower memory
usage but also potentially slower writing speed. Greater value means higher
memory usage and in some cases can increase write operations. If there is no
need to keep this value smaller, we recommend not to change it. In pratice
increasing this buffer value does not influence a lot.


So the most important is possibility to control cache of file content (which
is important for the real system performance) with these options:

mfsseteattr -f allowdatacache FILE_1 [ FILE_2 ... ]
mfsdeleattr -f allowdatacache FILE_1 [ FILE_2 ... ]



Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01



> -----Original Message-----
> From: Steve [mailto:st...@bo...]
> Sent: Monday, January 25, 2010 3:17 PM
> To: Michał Borychowski
> Subject: RE: [Moosefs-users] mfs mount generates high trafic from
> chunks
> 
> I understand the outline of what a cache does but under what situation
> would
> it be useful ? What is the downside ?
> 
> Any guidance on choosing parameters or is a case of trying values and
> measuring results ?
> 
> 
> 
> 
> 
> 
> 
> -------Original Message-------
> 
> 
> 
> From: Michał Borychowski
> 
> Date: 25/01/2010 10:48:36
> 
> To: 'Steve'
> 
> Subject: RE: [Moosefs-users] mfs mount generates high traffic from
> chunks
> 
> 
> 
> Steve, let me know more precisely what did you mean by your question
> below??
> 
> 
> 
> 
> 
> 
> Regards
> 
> Michał
> 
> 
> 
> -----Original Message-----
> 
> From: Steve [mailto:st...@bo...]
> 
> Sent: Thursday, January 21, 2010 3:27 PM
> 
> To: moo...@li...
> 
> Subject: Re: [Moosefs-users] mfs mount generates high traffic from
> chunks
> 
> 
> 
> Hi,
> 
> 
> 
> 
> 
> 
> 
> Is there any extra docs on the cache pro's and con's, guidelines for
> usage ?
> 
> 
> more explicit than the man page.