Re: [fuse-devel] Read performance of NFS shared fuse FS

[Miklos S. <mi...@sz...>]

On Sat, 16 May 2009, Philip Langdale wrote:
> Hi all,
> 
> First off, I apologise if this has been covered before, but my
> searches through google and the archives have not been successful.
> 
> I'm trying to NFS export a fuse filesystem using fuse 2.7.4 and
> kernel 2.6.30-rc5. Specifically, this is dvdreadfs which does
> transparent decryption of DVDs. As such, the data is read-only
> but can change when the DVD is switched out.
> 
> When I use the kernel server, I'm observing abysmal performance
> due to a stubborn insistence on doing the reads 4K at a time,
> regardless of the NFS rsize passed by the client. On my beefy
> quad-core desktop, it leads to me burning 25% of a core just
> doing reads in the filesystem and on the ARM machine that I
> really want to run this code on, I saturate the CPU and can't
> get a good enough transfer rate for real-time viewing. :-(
> 
> If I pass the kernel_cache option, the reads happen in larger
> blocks, solving the performance problem, but the cache is not
> flushed when the DVD is changed - resulting in garbage data
> being sent to the client.
> 
> I've tried unfs3 which is able to do the reads in larger blocks.
> This fixes the performance problem but unfs3 doesn't react well
> to DVD changes and makes the client very unhappy.
> 
> Is there any way to get the kernel server to do sanely sized reads
> without kernel_cache?

This probably something to do with the fact that the kernel NFS server
basically does a

  open();
  read();
  close();

sequence for _each_ read request.  And unless the "kernel_cache"
option is given fuse will flush the cache on each open.  This in
itself shouldn't cause small read requests, but my guess is that the
cache flushing somehow interacts badly with the readahead logic,
effectively turning it off.

Since you are experimenting with -rc kernels you can try if the
"direct_io" option helps, it should now work with NFS exports.

The proper way to fix this would be to make dvdreadfs only flush the
cache after a media change.

Thanks,
Miklos