Re: [MooseFS-Users] slow lookup

[Ben H. <bj...@ba...>]

I've experienced weirdness when using various raid setups. It's stated in the documentation that a non raid setup is preferred - at least from what I've remembered. MFS is a fuse filesystem that's bound to have a performance penalty somewhere when it comes to huge numbers of small files due to it's distributed nature.

 I've mitigated this somewhat by introducing an SSD based tier of storage that holds files smaller than a certain size. This one's definitely helped, and you can also set it as the C (create) tier for when files get created if you get creative with your storage classes - the initial writes to SSD are much quicker and can make your setup feel much faster when writing files. 

Hope that's helpful!

On May 18, 2017 07:06, Eugene Diatlov <it...@da...> wrote:
Hi,
I have a hardware server with latest moosefs. HDD's are in a hardware RAID5. Debian 8.
I have a lot of small files in folders (100k+). When I enter the folder I have a big delay while the system reading directory's files list.
Using moosefs metrics I found that the speed of lookup operations is equal to 1,8k operations per minute at maximum.
Why this could happen? I have another server with moosefs and the speed of lookup could be 300k per minute and higher.  



-- 
Best regards,
Eugene