Re: [Moosefs-users] [SPAM] questions mfs 1.6.x

[Michał B. <mic...@ge...>]

Hola Jose!

Estoy muy contento que tenemos usuarios en Espana :) 


> -----Original Message-----
> From: logs [mailto:lo...@se...]
> Sent: Sunday, January 24, 2010 8:36 AM
> To: moo...@li...
> Subject: [Moosefs-users] [SPAM] questions mfs 1.6.x
 

[...]

> Questions:
> 
> 1 .- In the chunservers is best to mount a raid5 with one big chunk or
> 6 individually mounted hard drives?
It should work better with 6 individually mounted hard drives.

 
> 2 .- Very important, vsftpd can write directly on the MFS mount point,
> or
> need to mount --bind. A better idea? commercial code, other
> implementation
> of mfsmount, native module?
We are not sure if we fully understand the question, but we'll try to answer.

There is no ready ftp server which would communicate directly with the master server and chunkservers so the solution is to "go through" the normal mfsmount. It gives better results than creating a special code for communication - kernel really supports I/O operations very efficiently (eg. by using read-ahead, buffering, writing in background, etc.). That's why we also use classic file systems for storing data in chunkservers rather than using the disk directly as a block-device. 


> 
> 3.- Reduce to 1 or 2 client machines to mount the MFS cluster would
> increase performance?
Mount devices does not influence performance. It is rather the contrary - for the overall performance the more clients the better.

 
> 4.- stored millions of small files, backup software cript. compress and
> split
> files 5/30 Mb, any advice on file systems to use ext3-ext4 or block
> size. kernel
> elevator, another tuning?
(Again we are not fully sure about the question)

At our company we do not use any tuning, just normal ext3.

How many milions of files do you have? Generally speaking 1 million files takes approximately 300 MiB of RAM. Installation of 25 million files requires about 8GiB of RAM and 25GiB space on HDD.


> 5.- ¿Any suggestions for the network, MTU, bandwitch Agrégation,
> windows
> size, etc ... on sysctl.conf?
We strongly recommend using jumbo-frames (MTU=9000). With the greater amount of chunkservers switches have to be connected through optical fiber or links aggregated. 

 
> 6.- I want to store 3 copies of the files, is there any possibility to
> specify that one of them is stored in another group of chunkservers
> remote
> but connected by optical fiber on the same subnet?
> ¿planned in de future?
There is no such a possibility now. But maybe it will be introduced in the future. 


 
> 7.- I maintain high retention time of deleted files. What is the
> technique that
> uses MFS?. ¿It retains the three copies scheduling their final
> deletion?
> ¿Delete two copies and keep one?. if so, ¿how that machine is stored?
All the copies in the trash are retained. And still for the files in the trash you can set the goal (ie. number of copies), so you can easily write a simple script which would for example change goal for all files in the trash from 3 to 2 (and later to 1).



> 8.- What would be the best option on the clients automounter or
> desconecction, autofs, daemon guard, cron script?
We have not used mfsmount with automounter nor autofs, we think you have to just monitor its activity. If the master server or chunkservers are termporarily not available, mfsmount tries to reconnect automatically.


> 
> 9 .- Do you know of any recent future risk of abandoning the project?
No, we do not plan to abandon the project :) The system is supported by a big Polish research company with several branches abroad. We have about 1 petabyte of data kept on MooseFS. And it is an open source project.



Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01


> 
> * Any suggestions on hardware, configurations, usability ...... will be
> greatly
> appreciated.
> 
> Thank you very much everyone and especially to the developers of MFS.
> 
> jose maria
> 
> 
> -----------------------------------------------------------------------
> -------
> Throughout its 18-year history, RSA Conference consistently attracts
> the
> world's best and brightest in the field, creating opportunities for
> Conference
> attendees to learn about information security's most important issues
> through
> interactions with peers, luminaries and emerging and established
> companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users