Re: [MooseFS-Users] Dealing with millions of files

[Matt W. <mat...@gm...>]

To ask the obvious ... you don't explicitly state that the chunk servers
have plenty of memory and are not hitting swap, are they fine?

Can you tell from the logs or cgi web view if the slowness comes from
metadata queries or from retrieving the chunks? Are other operations
equally slow? E.g. if you change to one of the directories that you are
tarring up, do an ls and then cat a file to /dev/null are both the ls and
cat slow or just one or the other?

On Tue, Sep 12, 2017 at 9:48 AM, Wilson, Steven M <st...@pu...> wrote:

> Hi,
>
>
> We have eight different MooseFS installations at our site and they are all
> doing terrific (thanks, MooseFS developers!) except for the one that
> hosts around 180 million files.  The performance on this installation is
> quite dismal.  I expect to see some decrease in performance due to the
> sheer number of files but not to the degree that I'm experiencing.  For
> example, I am in the process of tarring/gzipping 176 directories that
> together contain 15M files and occupy 5.5TB of space.  I am about 78% done
> after 25 days.
>
>
> Are there any users on this list who have MooseFS installations with large
> numbers of files and have some hints they can share about how to improve
> performance?
>
>
> Here is some basic information about this installation:
>
>     * MooseFS master is running on a Xeon E5-1630v3 (3.7GHz) and 132GB of
> memory.  Its system disk (where /var/lib/mfs is located) is a 256GB Samsung
> 850 Pro SSD.
>
>     * Four chunk servers, two with 16 SATA disks ranging from 3TB to 10TB
> each.  And two chunk servers with two 10TB SATA disks each.  All disks are
> JBOD and most are formatted using XFS (some are ext4).
>
>     * Disk schedulers on chunk servers are set to use deadline.  Tried
> different settings for read_ahead_kb and nr_requests without much
> performance gain.
>
>     * Network is gigabit Ethernet
>
>     * Master and chunk servers are running a mix of MooseFS
> versions 3.0.90 and 3.0.92.
>
>
> Thanks in advance for any suggestions!
>
>
> Steve
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>