From: Daniel B. <db...@vm...> - 2008-07-16 22:07:00
|
Hi, Should readdir fail if the result buffer fills up? The example file system doesn't, the api documentation on the site doesn't say anything about it, and the man page for readdir(2) lists it as an error condition. Thanks, Dan |
From: Miklos S. <mi...@sz...> - 2008-07-18 14:41:07
|
On Wed, 16 Jul 2008, Daniel Benamy wrote: > Should readdir fail if the result buffer fills up? The example file > system doesn't, the api documentation on the site doesn't say > anything about it, and the man page for readdir(2) lists it as an > error condition. Which example? fusexmp and fusexmp_fh use different strategies, both of which are pretty well documented in fuse.h, I think. |
From: Daniel B. <db...@vm...> - 2008-07-18 18:01:52
|
On Friday 18 July 2008 07:41:08 Miklos Szeredi wrote: > On Wed, 16 Jul 2008, Daniel Benamy wrote: > > Should readdir fail if the result buffer fills up? The example file > > system doesn't, the api documentation on the site doesn't say > > anything about it, and the man page for readdir(2) lists it as an > > error condition. > > Which example? fusexmp and fusexmp_fh use different strategies, both > of which are pretty well documented in fuse.h, I think. I'm looking at fusexmp. The documentation for this mode says: * 1) The readdir implementation ignores the offset parameter, and * passes zero to the filler function's offset. The filler * function will not return '1' (unless an error happens), so the * whole directory is read in a single readdir operation. This * works just like the old getdir() method. (Filler is supposed to only return 1 if the buffer is full.) In fusexmp, readdir always returns 0, so how can the fuse kernel module know if the buffer filled up? Thanks, Dan |
From: Miklos S. <mi...@sz...> - 2008-07-19 10:50:08
|
On Fri, 18 Jul 2008, Daniel Benamy wrote: > In fusexmp, readdir always returns 0, so how can the fuse kernel module know > if the buffer filled up? It fills a big buffer in memory, and later copies from this big buffer to the small buffer requested by the fuse kernel module. If the directory is so large it doesn't fit in the memory, then this method won't work. Miklos |
From: Daniel B. <db...@vm...> - 2008-07-23 02:47:17
|
On Saturday 19 July 2008 03:50:07 Miklos Szeredi wrote: > On Fri, 18 Jul 2008, Daniel Benamy wrote: > > In fusexmp, readdir always returns 0, so how can the fuse kernel module > > know if the buffer filled up? > > It fills a big buffer in memory, and later copies from this big buffer > to the small buffer requested by the fuse kernel module. If the > directory is so large it doesn't fit in the memory, then this method > won't work. So filler() won't ever return 1 when used in this mode? Dan |
From: Miklos S. <mi...@sz...> - 2008-07-23 09:51:35
|
On Tue, 22 Jul 2008, Daniel Benamy wrote: > On Saturday 19 July 2008 03:50:07 Miklos Szeredi wrote: > > On Fri, 18 Jul 2008, Daniel Benamy wrote: > > > In fusexmp, readdir always returns 0, so how can the fuse kernel module > > > know if the buffer filled up? > > > > It fills a big buffer in memory, and later copies from this big buffer > > to the small buffer requested by the fuse kernel module. If the > > directory is so large it doesn't fit in the memory, then this method > > won't work. > > So filler() won't ever return 1 when used in this mode? It will only return 1 if it failed to allocate memory for the buffer. Miklos |
From: Daniel B. <db...@vm...> - 2008-07-25 17:24:37
|
On Wednesday 23 July 2008 02:51:37 Miklos Szeredi wrote: > On Tue, 22 Jul 2008, Daniel Benamy wrote: > > On Saturday 19 July 2008 03:50:07 Miklos Szeredi wrote: > > > On Fri, 18 Jul 2008, Daniel Benamy wrote: > > > > In fusexmp, readdir always returns 0, so how can the fuse kernel > > > > module know if the buffer filled up? > > > > > > It fills a big buffer in memory, and later copies from this big buffer > > > to the small buffer requested by the fuse kernel module. If the > > > directory is so large it doesn't fit in the memory, then this method > > > won't work. > > > > So filler() won't ever return 1 when used in this mode? > > It will only return 1 if it failed to allocate memory for the buffer. If it can ever return 1 on error, doesn't readdir need to indicate that? Like so: --- fusexmp.c 2006-02-02 09:04:51.000000000 -0800 +++ fusexmp-readdir-error.c 2008-07-25 10:22:36.000000000 -0700 @@ -78,7 +78,7 @@ st.st_ino = de->d_ino; st.st_mode = de->d_type << 12; if (filler(buf, de->d_name, &st, 0)) - break; + return -EINVAL; } closedir(dp); Am I missing something? Dan |
From: Miklos S. <mi...@sz...> - 2008-07-25 21:26:25
|
On Fri, 25 Jul 2008, Daniel Benamy wrote: > On Wednesday 23 July 2008 02:51:37 Miklos Szeredi wrote: > > On Tue, 22 Jul 2008, Daniel Benamy wrote: > > > So filler() won't ever return 1 when used in this mode? > > > > It will only return 1 if it failed to allocate memory for the buffer. > > If it can ever return 1 on error, doesn't readdir need to indicate > that? Like so: > > --- fusexmp.c 2006-02-02 09:04:51.000000000 -0800 > +++ fusexmp-readdir-error.c 2008-07-25 10:22:36.000000000 -0700 > @@ -78,7 +78,7 @@ > st.st_ino = de->d_ino; > st.st_mode = de->d_type << 12; > if (filler(buf, de->d_name, &st, 0)) > - break; > + return -EINVAL; > } > > closedir(dp); > > Am I missing something? The filler function will store the error value in the buffer, and the caller will check that and return the stored error. So the original code is correct, and the above patch would actually make it worse: - the wrong error value is returned (EINVAL instead of ENOMEM) - closedir() is not called, causing memory and a file descriptor to leak Miklos |
From: Daniel B. <db...@vm...> - 2008-07-29 05:06:28
|
On Friday 25 July 2008 14:26:22 Miklos Szeredi wrote: > On Fri, 25 Jul 2008, Daniel Benamy wrote: > > On Wednesday 23 July 2008 02:51:37 Miklos Szeredi wrote: > > > On Tue, 22 Jul 2008, Daniel Benamy wrote: > > > > So filler() won't ever return 1 when used in this mode? > > > > > > It will only return 1 if it failed to allocate memory for the buffer. > > > > If it can ever return 1 on error, doesn't readdir need to indicate > > that? Like so: > > > > --- fusexmp.c 2006-02-02 09:04:51.000000000 -0800 > > +++ fusexmp-readdir-error.c 2008-07-25 10:22:36.000000000 -0700 > > @@ -78,7 +78,7 @@ > > st.st_ino = de->d_ino; > > st.st_mode = de->d_type << 12; > > if (filler(buf, de->d_name, &st, 0)) > > - break; > > + return -EINVAL; > > } > > > > closedir(dp); > > > > Am I missing something? > > The filler function will store the error value in the buffer, and the > caller will check that and return the stored error. So the original > code is correct, and the above patch would actually make it worse: > > - the wrong error value is returned (EINVAL instead of ENOMEM) > - closedir() is not called, causing memory and a file descriptor to leak Ignore the broken details of my patch for a second. If filler can't allocate memory, there's no buffer for it to store an error value in. So it returns 1. With the original code, readdir will still return 0 in that case. So how will fuse know that there was an error? Dan |
From: Miklos S. <mi...@sz...> - 2008-07-29 10:15:06
|
On Mon, 28 Jul 2008, Daniel Benamy wrote: > Ignore the broken details of my patch for a second. If filler can't > allocate memory, there's no buffer for it to store an error value > in. So it returns 1. With the original code, readdir will still > return 0 in that case. So how will fuse know that there was an > error? It stores the error in the buffer (second argument) passed to ->readdir(). See extend_contents() and readdir_fill() functions in lib/fuse.c for the implementation. Miklos |
From: Daniel B. <db...@vm...> - 2008-07-29 18:24:46
|
On Tuesday 29 July 2008 03:15:05 Miklos Szeredi wrote: > On Mon, 28 Jul 2008, Daniel Benamy wrote: > > Ignore the broken details of my patch for a second. If filler can't > > allocate memory, there's no buffer for it to store an error value > > in. So it returns 1. With the original code, readdir will still > > return 0 in that case. So how will fuse know that there was an > > error? > > It stores the error in the buffer (second argument) passed to > ->readdir(). See extend_contents() and readdir_fill() functions in > lib/fuse.c for the implementation. Got it. Thanks for the patient explanation. Dan |