Re: [MooseFS-Users] Performance suggestions for millions of small files

[Wilson, S. M <st...@pu...>]

We have a mix of 3.0.97 and 3.0.101.

Steve

________________________________________
From: Marco Milano <mar...@gm...>
Sent: Thursday, October 18, 2018 5:43 PM
To: moo...@li...
Subject: Re: [MooseFS-Users] Performance suggestions for millions of small files

Steve,

What is the version of the MooseFS ?

-- Marco

On 10/18/18 4:47 PM, Wilson, Steven M wrote:
> Hi,
>
>
> We have ten different MooseFS installations in our research group and
> one, in particular, is struggling with poor I/O performance.  This
> installation currently has 315 million files occupying 170TB of disk
> space (goal = 2).  If anyone else has a similar installation, I would
> like to hear what you have done to maintain performance at a reasonable
> level.
>
>
> Here are some metrics to give a basic idea of the performance
> characteristics.  I'll include in parentheses the range of measurements
> from other MFS installations with far fewer files for comparison.
>
>      * tar xf linux-4.9-rc3.tar: 1185 secs (220 - 296 secs)
>
>      * smallfile test, create MB/s: 0.8 (2.3 - 4.8)  <== Ouch!
>
>      * smallfile test, read MB/s: 10.7 (12.8 - 15.4)
>
>      * smallfile test, append MB/s: 6.1 (3.0 - 7.7)
>
>
> It looks file creation is where I'm losing most of my performance
> compared to the other installations.  My master server has a Xeon
> E5-1630v3 3.7GHz CPU with 256GB of DDR4 2133MHz memory.
>
>
> I tried several mfsmount options but the only one that showed any
> significant improvement was the mfsfsyncmintime option
> ("mfsfsyncmintime=5").  As to be expected, the improvement gained was
> during the write/append operation.  Here are the results using the same
> tests as above:
>
>      * tar xf linux-4.9-rc3.tar: 683 secs
>      * smallfile test, create MB/s: 1.2
>      * smallfile test, read MB/s: 11.7
>      * smallfile test, append MB/s: 11.4  <== Dramatic improvement over
> 6.1 MB/s
>
>
> The smallfile benchmark test I used is ​from
> https://github.com/distributed-system-analysis/smallfile.
>
>
> Thanks for any suggestions you might have!
>
>
> Regards,
>
> Steve
>
>
>
> _________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>


_________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users