Re: [fuse-devel] Kernel page cache and FUSE

[Miklos S. <mi...@sz...>]

(Sorry about the late answer, it must have slipped my attention).

On Fri, 18 Jul 2008, Archie Cobbs wrote:
> I really like FUSE and have I've written my first FUSE filesystem called
> s3backer <http://code.google.com/p/s3backer/>. All this filesystem does is
> contain a single normal file which is backed by a network remote data store
> (Amazon S3). The file is divided up into blocks (typically will be same size
> as kernel page size) and then you do a loopback mount of a normal filesystem
> on top of this file.

Note, that loop over fuse is something not "supported" in the sense
that I haven't really thought about all the nasty corner cases that
happen when the machine is out of memory and is trying to free some up
by writing out dirty data through the loopback device and then through
fuse.

This is generally a difficult problem, and even you suggested "open
file with O_DIRECT" thing wouldn't solve it, as the cached data
belongs to the filesystem, not to the loop device.

> As the "upper" filesystem reads and writes blocks, the "lower" s3backer
> filesystem reads and writes over the network. I'm sure you've seen a similar
> arrangement before in other FUSE filesystems. The result is that you treat
> the single file in the FUSE filesystem more like a hard disk type block
> device, where the "hard disk" storage is remotely located over the network.

Why not NBD?  That has been designed especially for this.

> My questions all relate to kernel caching of the file data in this scenario.
> I'm mostly ignorant about how exactly Linux kernel caching works. And this
> is a complicated scenario because it involves two filesystems (the "upper"
> one and the "lower" one) and a loopback mount...
> 
> How do the kernel page cache, and the data blocks read from/written to the
> FUSE filesystem (on behalf of the "upper" filesystem) interact?
> 
> How does the kernel handle caching of the underlying file data blocks when
> doing a loopback mount? Does the fact that the underlying file is within a
> FUSE filesystem matter at all?

Yes, it matters, when writing out file backed dirty data to free up
memory, the kernel has complicated mechanisms to prevent deadlocks
when more memory is needed to complete the write.

When fuse is involved, the kernel doesn't have any idea that the
allocation by the filesystem is special and needs to complete in order
to complete the original write request.

Miklos