Re: [fuse-devel] vanished files after rename

[Miklos S. <mi...@sz...>]

On Tue, Oct 20, 2015 at 2:15 PM, Wolfgang Bumiller
<w.b...@pr...> wrote:
> The same issue seems to also affect link(). Specifically linkat() with
> AT_EMPTY_PATH, and none of the available flags, not even that dirty
> hidden-file workaround tackle this problem at all.
>
> What basically happens is the kernel starts off with a `lookup` on the
> file, getting an inode for the path without creating a full reference
> (in the eyes of libfuse). Then the other task gets to execute its
> `rename()`, thereby deleting the old inode's *path*. It's *not*
> hidden, because this hack only applies when the file is actually
> opened, but it's not yet open, it has only been looked up. After
> this, the `open()` on the old *inode* fails because it fails to
> lookup the *path* for said inode, which has been removed by
> `rename()`.
>
> Similar behavior can be observed with `linkat()`: Create a file and
> make a hard-link to it, then try to open one, delete it, and use
> `linkat()` with AT_EMPTY_PATH. (the hardlink is there because
> AT_EMPTY_PATH isn't required to work on files with a zero reference
> count). Eg:
>
> # echo stuff > file
> # cp -l file another_link
>
> Now do the following sequence:
>
> int f = open("file", O_PATH);
> unlink("file");
> linkat(f, "", AT_FDCWD, "file", AT_EMPTY_PATH);
>
> This works without fuse, but with fuse `link()` fails to rebuild a
> path for the inode of `f` and thus can't execute the userspace's
> link() function. (get_path()).
>
> Tell me if I'm on a completely wrong track here, but the problem
> seems to be that the kernel works with inodes, but the userspace
> doesn't. The current API is fine for functions where the 'nopath'
> option applies, but does not work for `open()` and `link()`.
>
> So how should we solve this? This affects *all* fuse filesystems
> regardless of mount-options.
> The way I see it, this is a design-bug.
> So we either need an update to the user-side API to pass through
> lookup/forget (since we can't simply open the file at `lookup()` time
> since we don't know the open-mode yet - lookup just does
> `getattr()`),

This is already there:  see <fuse_lowlevel.h>.

> or the kernel needs to make the combination of
> lookup+open atomic, or the "hidden-file" hack needs to be improved (if
> you just remove the is_open() checks you end up moving from ENOENT to
> EBUSY because it performs multiple actions which each lock and unlock
> rather than taking one big lock over the entire process which would
> seem more reasonable).

Possibly doable.  The problem is: we only get FORGET when the kernel
flushes the inode from the cache for some reason.  This event can only
happen after all references to the dentry and inode  inside the kernel
have been dropped, but it won't happen immediately after that,
possibly only after a very long time (if the cache is large and the
kernel doesn't feel the need to free up the memory associated with the
cached structures).

What we really need here is some indication about the case when the
inode/dentry is not just sitting in the cache, but there's an active
reference to either at the point of unlink/rename.  Only in that
relatively rare case do we need to extend the hidden-file hack.

> Or get rid of `lookup()` on the kernel-side
> altogether?

Huh?

Thanks,
Miklos