Re: [fuse-devel] NFS export

[Miklos S. <mi...@sz...>]

On Thu, 29 Oct 2009, Fredrik Widlund wrote:
> Hi,
> 
> I'm currently building a HA high-performance storage back-end using
> Arch Linux (2.6.31.4), GlusterFS 2.0.7, Fuse 2.8, XFS, PERC6 HW, and
> (currently only) 4 SATA drives in a RAID-5 setup.
> 
> Semi-random I/O (40 concurrent large dd reads of 100MB files) gives
> the following numbers:
> XFS:  150 MB/s
> XFS exported over NFS: 120 MB/s (1Gbps fully saturated)
> GlusterFS/XFS locally: 120 MB/s
> 
> So, life is grand. I am really impressed by these numbers in
> general. However, here comes the downer:
> GlusterFS/XFS exported over NFS: 30 MB/s
> 
> I know this isn't a new problem. Sadly this kind of strikes out the
> high-performance goals, so I need to try to address this somehow.
> 
> I noticed a thread earlier this year about trying to bump the
> syscall limit from 128kB to 4MB for example. Any pointers regarding
> if this the way to try to move forward?
> 

I don't think the 128kB limit is the culprit here.  Can you try
running gluster with debugging option (not sure if that's possible) to
see what size requests go to the filesystem?

Equivalently a "strace" on the glusterfs would also be useful.

Thanks,
Miklos