Re: [fuse-devel] posix OpenDir/Readdir behavior?

[Han-Wen N. <ha...@gm...>]

On Wed, Aug 31, 2011 at 4:10 PM, Kevin Fox <Kev...@pn...> wrote:
> On Wed, 2011-08-31 at 11:34 -0700, Han-Wen Nienhuys wrote:
>> On Wed, Aug 31, 2011 at 3:20 PM, Kevin Fox <Kev...@pn...> wrote:
>> > On Wed, 2011-08-31 at 10:59 -0700, Han-Wen Nienhuys wrote:
>> >> On Wed, Aug 31, 2011 at 2:43 PM, Kevin Fox <Kev...@pn...> wrote:
>> >> > On Tue, 2011-08-30 at 19:57 -0700, Han-Wen Nienhuys wrote:
>> >> >> Hi there,
>> >> >>
>> >> >> my Go-FUSE UnionFs fails when running "rm -rf" on it on Ubuntu Lucid
>> >> >> 64 bits. (Fedora 32 works fine).
>> >> >>
>> >> >> I strongly suspect that this is because Go-FUSE does not correctly
>> >> >> implement directory reading, and rm has sophisticated code for
>> >> >> traversing directories. In order to fix this, I would like to know
>> >> >> what exactly directory reading should support.
>> >> >>
>> >> >> For context, the following is the current approach:
>> >> >>
>> >> >> * On OpenDir(), construct a list of all directory entries.
>> >> >>
>> >> >> * On ensuing ReadDir()s, feed bits of the preconstructed list back to
>> >> >> the kernel.
>> >> >>
>> >> >> * It does not support seeking.
>> >> >
>> >> > If you have the list built up front, it should be fairly easy to
>> >> > implement seeking.
>> >> >
>> >> >> I am wondering about two issues:
>> >> >>
>> >> >> - is it correct to already construct the list on Open? In other words,
>> >> >> if an application does
>> >> >>
>> >> >>   d = open("dir")
>> >> >>   unlink("dir/file")
>> >> >>   readdir(d)
>> >> >>
>> >> >> is it within spec to return an entry for "file" from the readdir?
>> >> >
>> >> > I think the posix spec says renames and redirs are fairly undefined
>> >> > behavior.
>> >>
>> >> Are you sure ?   I took a closer look again at the failure I observed,
>> >
>> > http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir.html
>> >
>> > "If a file is removed from or added to the directory after the most
>> > recent call to opendir() or rewinddir(), whether a subsequent call to
>> > readdir() returns an entry for that file is unspecified."
>>
>> >> diff fail.txt success.txt
>> >>
>> >>   @@ -104,12 +104,12 @@
>> >>  fcntl(3, F_GETFL)                       = 0xhex (flags O_RDONLY|O_LARGEFILE)
>> >>  fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
>> >>  fstat(3, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
>> >> -getdents(3, /* 3 entries */, 32768)     = 80
>> >> -openat(3, "subdir", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW) = -1
>> >> ENOENT (No such file or directory)
>> >> +getdents(3, /* 2 entries */, 32768)     = 48
>> >>  getdents(3, /* 0 entries */, 32768)     = 0
>> >>  fcntl(3, F_GETFD)                       = 0xhex (flags FD_CLOEXEC)
>> >>  close(3)                                = 0
>> >>
>> >> This is a diff of strace rm -rf on a normal FS vs a FUSE fs.
>> >>
>> >> The context is that the program does this
>> >>
>> >> openat(4, "..", O_RDONLY)               = 3
>> >> close(4)                                = 0
>> >> fstat(3, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
>> >> unlinkat(3, "subdir", AT_REMOVEDIR)     = 0
>
>     ^----- Its unlinking "subdir". To know it should unlink "subdir",
> it must have seen "subdir" in a previous readdir.

Yes. However, fd 3 is the one returned here, at the start of the snippet

 openat(4, "..", O_RDONLY)               = 3

ie. the entries below are from a fresh opendir(), and it would be
valid to return "subdir" once again.

I think rm gets confused that a file returned from a readdir cannot be stat()ed.

I upgraded coreutils to 8.12, and the problem went away.


>> >> fstat(3, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
>> >> fcntl(3, F_GETFL)                       = 0xhex (flags O_RDONLY|O_LARGEFILE)
>> >> fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
>> >> fstat(3, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
>> >> getdents(3, /* 3 entries */, 32768)     = 80
>
>     ^----- Reading more entries from directory
>
>> >> openat(3, "subdir", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW) = -1
>> >> ENOENT (No such file or directory)
>
>     ^----- Trying to access from newly readdired item that appears to
> have been returned from a previous readdir.
>
> I could be wrong though.
>

-- 
Han-Wen Nienhuys - ha...@xs... - http://www.xs4all.nl/~hanwen