From: Chris F. <ch...@fr...> - 2009-03-27 17:00:37
|
I believe that the highlevel fuse api does not guarantee that, after a vfs rename call, there will exist a directory entry for the destination file name. For example, given: $ touch foo bar $ tail -f foo & And then this rename: $ mv bar foo Then, if the fuse server is using the highlevel api, the fuse library will issue the following two calls to the server to implement the rename: 1. rename(foo, fusehiddenname) 2. rename(bar, foo) A crash could cause there to not exist a directory entry named foo. (However, the hidden and bar directory entries should still both exist.) As far as I am aware, posix applications are generally designed with the assumption that this is an invalid result of the rename. And, in particular, ext3 and ext4 guarantee that either both the foo and bar directory entries will exist or that only foo will exist (and refer to bar's inode). The above issue does not exist when the the hard_remove option is enabled. Also, as far as I can tell, the lowlevel library code does not have this issue. One approach to fix the issue, if the fuse server implements hardlinks, may be for the fuse library to instead issue these two calls to implement rename: 1. link(foo, fusehiddenname) 2. rename(bar, foo) If the rename fails, the fuse library could try to unlink the hidden file. This does require hardlink support, but perhaps it is acceptable to provide this rename guarantee only for such file systems. Important question: I know that fuse renames foo to a hidden fuse file before renaming bar to foo because of kernel or fuse interface issues with open files being unlinked. Does having another name for foo permit the proposed step two? I have attached some of an implementation of the above solution as a patch against HEAD. "some of" because: - I believe that the implementation needs to create the fuse library node for the hidden file, but it does not yet do so. - I've not yet tested the patch. (It seems that I can't use my compiled fuse library with my distribution's 2.7.2 mount.fuse.) But I wanted to pass along this email now in case the 2.8.0 release is near. Any thoughts on this issue, proposed solution, or patch? cheers! -- Chris Frost http://www.frostnet.net/chris/ |
From: Chris F. <ch...@fr...> - 2009-03-27 17:33:59
Attachments:
rename.patch
|
On Fri, Mar 27, 2009 at 09:59:57AM -0700, Chris Frost wrote: > I have attached some of an implementation of the above solution as a > patch against HEAD. "some of" because: > - I believe that the implementation needs to create the fuse library > node for the hidden file, but it does not yet do so. > - I've not yet tested the patch. (It seems that I can't use my compiled > fuse library with my distribution's 2.7.2 mount.fuse.) > But I wanted to pass along this email now in case the 2.8.0 release is near. It's actually attached this time. -- Chris Frost http://www.frostnet.net/chris/ |
From: Roman S. <rv...@su...> - 2009-03-28 04:38:02
|
On Mar 27, 2009, at 9:59 AM, Chris Frost wrote: > I believe that the highlevel fuse api does not guarantee that, after > a vfs > rename call, there will exist a directory entry for the destination > file name. This aspect of POSIX is hard not only of fuse, but on most networked filesystems (NFS is a good example there). And strictly speaking POSIX had an exception to the "there will exist a directory entry for the destination file name" guarantee -- if EIO is returned all bets are off: http://www.opengroup.org/onlinepubs/009695399/ > For example, given: > $ touch foo bar > $ tail -f foo & > And then this rename: > $ mv bar foo > > Then, if the fuse server is using the highlevel api, the fuse > library will > issue the following two calls to the server to implement the rename: > 1. rename(foo, fusehiddenname) > 2. rename(bar, foo) > A crash could cause there to not exist a directory entry named foo. > (However, the hidden and bar directory entries should still both > exist.) What's the error that gets returned if there's a crash? > The above issue does not exist when the the hard_remove option is > enabled. Yeah, that's what I was going to suggest. > One approach to fix the issue, if the fuse server implements > hardlinks, may > be for the fuse library to instead issue these two calls to > implement rename: > 1. link(foo, fusehiddenname) > 2. rename(bar, foo) > If the rename fails, the fuse library could try to unlink the hidden > file. I guess the real question (for Miklos) would be to come up with the #1 that is easy for fuse to back out of if #2 fails. Your solution seems to be better than what's happening right now, but I can't comment any further. Thanks,. Roman. |
From: Miklos S. <mi...@sz...> - 2009-04-01 10:42:42
|
On Fri, 27 Mar 2009, Chris Frost wrote: > I believe that the highlevel fuse api does not guarantee that, after a vfs > rename call, there will exist a directory entry for the destination file name. > > For example, given: > $ touch foo bar > $ tail -f foo & > And then this rename: > $ mv bar foo > > Then, if the fuse server is using the highlevel api, the fuse library will > issue the following two calls to the server to implement the rename: > 1. rename(foo, fusehiddenname) > 2. rename(bar, foo) > A crash could cause there to not exist a directory entry named foo. > (However, the hidden and bar directory entries should still both exist.) Right. > As far as I am aware, posix applications are generally designed with the > assumption that this is an invalid result of the rename. And, in particular, > ext3 and ext4 guarantee that either both the foo and bar directory entries > will exist or that only foo will exist (and refer to bar's inode). > > The above issue does not exist when the the hard_remove option is enabled. > Also, as far as I can tell, the lowlevel library code does not have this > issue. > > > One approach to fix the issue, if the fuse server implements hardlinks, may > be for the fuse library to instead issue these two calls to implement rename: > 1. link(foo, fusehiddenname) > 2. rename(bar, foo) > If the rename fails, the fuse library could try to unlink the hidden file. Sounds sane. > This does require hardlink support, but perhaps it is acceptable to provide > this rename guarantee only for such file systems. > > Important question: I know that fuse renames foo to a hidden fuse file > before renaming bar to foo because of kernel or fuse interface issues > with open files being unlinked. Does having another name for foo permit > the proposed step two? I don't understand the question. > I have attached some of an implementation of the above solution as a > patch against HEAD. "some of" because: > - I believe that the implementation needs to create the fuse library > node for the hidden file, but it does not yet do so. No, on the node level it should still be just a rename. The old link _will_ go away unless fuse crashes, and if it crashes then the in-memory structures doesn't matter anyway. > - I've not yet tested the patch. (It seems that I can't use my compiled > fuse library with my distribution's 2.7.2 mount.fuse.) I would prefer if the old behavior remained the default (for backward compatibility) and this feature would be turned on via a flag in 'struct fuse_operations', similar to 'flag_nullpath_ok'. > But I wanted to pass along this email now in case the 2.8.0 release is near. It's not very near yet :( Thanks, Miklos |
From: Miklos S. <mi...@sz...> - 2009-04-01 10:48:25
|
On Fri, 27 Mar 2009, Roman Shaposhnik wrote: > On Mar 27, 2009, at 9:59 AM, Chris Frost wrote: > > Then, if the fuse server is using the highlevel api, the fuse > > library will > > issue the following two calls to the server to implement the rename: > > 1. rename(foo, fusehiddenname) > > 2. rename(bar, foo) > > A crash could cause there to not exist a directory entry named foo. > > (However, the hidden and bar directory entries should still both > > exist.) > > What's the error that gets returned if there's a crash? ECONNABORTED or ENOTCONN, depending on whether the operation was in progress during the crash, or if it was issued after the crash. > > One approach to fix the issue, if the fuse server implements > > hardlinks, may > > be for the fuse library to instead issue these two calls to > > implement rename: > > 1. link(foo, fusehiddenname) > > 2. rename(bar, foo) > > If the rename fails, the fuse library could try to unlink the hidden > > file. > > I guess the real question (for Miklos) would be to come up with the > #1 that is easy for fuse to back out of if #2 fails. Your solution > seems to be > better than what's happening right now, but I can't comment any > further. We could add a new rename_with_hide() operation to the high level API, but I think it goes against the philosophy of having only primitive file system operations. Thanks, Miklos |
From: Goswin v. B. <gos...@we...> - 2009-04-01 15:27:40
|
Miklos Szeredi <mi...@sz...> writes: > On Fri, 27 Mar 2009, Roman Shaposhnik wrote: >> On Mar 27, 2009, at 9:59 AM, Chris Frost wrote: >> > One approach to fix the issue, if the fuse server implements >> > hardlinks, may >> > be for the fuse library to instead issue these two calls to >> > implement rename: >> > 1. link(foo, fusehiddenname) >> > 2. rename(bar, foo) >> > If the rename fails, the fuse library could try to unlink the hidden >> > file. >> >> I guess the real question (for Miklos) would be to come up with the >> #1 that is easy for fuse to back out of if #2 fails. Your solution >> seems to be >> better than what's happening right now, but I can't comment any >> further. > > We could add a new rename_with_hide() operation to the high level API, > but I think it goes against the philosophy of having only primitive > file system operations. > > Thanks, > Miklos Why do we need the hidden name at all? Specifically if null path is allowed shouldn't a open but removed file have a null path? Then the highlevel internaly records the old foo as null path and calls rename. Or am I missing something? MfG Goswin |
From: Roman V S. <rv...@su...> - 2009-04-01 15:10:15
|
On Wed, 2009-04-01 at 12:48 +0200, Miklos Szeredi wrote: > On Fri, 27 Mar 2009, Roman Shaposhnik wrote: > > On Mar 27, 2009, at 9:59 AM, Chris Frost wrote: > > > Then, if the fuse server is using the highlevel api, the fuse > > > library will > > > issue the following two calls to the server to implement the rename: > > > 1. rename(foo, fusehiddenname) > > > 2. rename(bar, foo) > > > A crash could cause there to not exist a directory entry named foo. > > > (However, the hidden and bar directory entries should still both > > > exist.) > > > > What's the error that gets returned if there's a crash? > > ECONNABORTED or ENOTCONN, depending on whether the operation was in > progress during the crash, or if it was issued after the crash. I see. Wouldn't it be a problem (not that I have a good solution) for POSIX aware applications testing against EIO as a condition for low level errors? > > I guess the real question (for Miklos) would be to come up with the > > #1 that is easy for fuse to back out of if #2 fails. Your solution > > seems to be > > better than what's happening right now, but I can't comment any > > further. > > We could add a new rename_with_hide() operation to the high level API, ------------------------------------------------------------------------ > but I think it goes against the philosophy of having only primitive > file system operations. Good point. Thanks, Roman. |
From: Miklos S. <mi...@sz...> - 2009-04-14 19:05:59
|
On Wed, 01 Apr 2009, Goswin von Brederlow wrote: > Miklos Szeredi <mi...@sz...> writes: > > We could add a new rename_with_hide() operation to the high level API, > > but I think it goes against the philosophy of having only primitive > > file system operations. > Why do we need the hidden name at all? Specifically if null path is > allowed shouldn't a open but removed file have a null path? Then the > highlevel internaly records the old foo as null path and calls rename. > > Or am I missing something? a) currently fstat(), fchmod() and fchown() on a deleted file cannot be implemented without a hidden file (this is an internal API issue in the linux kernel, and it's difficult to get acceptance for a change). b) some filesystems may not want to use the file handle at all for filesystem operations. In that case hiding the file on unlink is still the right thing to do and passing a NULL path is not acceptable. Thanks, Miklos |