From: Joshua J. B. <con...@co...> - 2005-07-30 21:30:06
|
Miklos -- I noticed that you added a fuse_lowlevel API to CVS. Thanks for this! I'v= e=20 started playing with an implementation for my project. I obviously don't=20 expect anything to work right now, but I want to have my code ready to go b= y=20 the time fuse_lowlevel is ready to be released. Are you interested in any debugging feedback (bug reports and the like), or= is=20 it still too early in fuse_lowlevel's life for that? Thanks again. =2D- Josh =2D-=20 Joshua J. Berry "I haven't lost my mind -- it's backed up on tape somewhere." -- /usr/games/fortune |
From: Miklos S. <mi...@sz...> - 2005-07-31 08:27:50
|
> I noticed that you added a fuse_lowlevel API to CVS. Thanks for > this! I've started playing with an implementation for my project. > I obviously don't expect anything to work right now, but I want to > have my code ready to go by the time fuse_lowlevel is ready to be > released. > > Are you interested in any debugging feedback (bug reports and the > like), or is it still too early in fuse_lowlevel's life for that? Yes, any feedback is most welcome. If you don't like something in the API, please let me know. Now is the time to get it right. Thanks, Miklos |
From: Joshua J. B. <con...@co...> - 2005-07-31 19:12:18
|
On Sunday 31 July 2005 01:27, Miklos Szeredi wrote: > > Are you interested in any debugging feedback (bug reports and the > > like), or is it still too early in fuse_lowlevel's life for that? > > Yes, any feedback is most welcome. If you don't like something in the > API, please let me know. Now is the time to get it right. The one weak spot/suggestion I have currently for the API is the *dir()=20 functions. It was hard to figure out how to handle readdir(), specifically: =2D What does "offset" mean? Is offset specified in bytes, or directory ent= ries? =2D What format should we use for the buffer? I eventually got it figured out, but the interface seems very difficult to= =20 use. Perhaps it would be good to have a fuse_dirlist_t or something, and use=20 fuse_add_dirent() to append to that list (and handle memory allocation, etc= ). =20 Then make a new fuse_reply_dirlist() for responding. Also, make "offset" a= nd=20 "size" be in terms of directory entries, not bytes. So maybe something lik= e=20 this: struct fuse_dirlist_t { ... }; // Pass NULL initially to create a new dirlist // Returns the dirlist with the item appended fuse_dirlist_t * fuse_dirlist_add(fuse_dirlist_t * list, char * name, struc= t=20 stat * st, ...); void fuse_dirlist_free(fuse_dirlist_t * list); void fuse_reply_dirlist(fuse_req_t * req, fuse_dirlist_t * dirents); Also, I noticed that the libfuse versionscript left off a few functions,=20 specifically fuse_ll_loop_mt() and fuse_req_ctx(). I had to add those to g= et=20 my code to link properly. Hope this helps. =2D-=20 Joshua J. Berry "I haven't lost my mind -- it's backed up on tape somewhere." -- /usr/games/fortune |
From: Miklos S. <mi...@sz...> - 2005-08-01 09:02:05
|
> The one weak spot/suggestion I have currently for the API is the *dir() > functions. It was hard to figure out how to handle readdir(), specifically: > > - What does "offset" mean? Is offset specified in bytes, or > - directory entries? Either. Or something else entirely. The important thing is, that the filesystem can find the next directory entry by what it filled in as offset. > - What format should we use for the buffer? That's an internal detail of the FUSE interface, that the filesystem should not need to know. > I eventually got it figured out, but the interface seems very > difficult to use. Yes, leaving the memory management to the filesystem makes it a bit awkward. However it also let's the filesystem do caching and reusing the raw directory data (the high level FUSE library uses that feature). > Perhaps it would be good to have a fuse_dirlist_t or something, and > use fuse_add_dirent() to append to that list (and handle memory > allocation, etc). Then make a new fuse_reply_dirlist() for > responding. Also, make "offset" and "size" be in terms of directory > entries, not bytes. So maybe something like this: Size cannot be the number of entries, since that is not known in advance (the size of entries varies because of variable length filenames). > struct fuse_dirlist_t { ... }; > > // Pass NULL initially to create a new dirlist > // Returns the dirlist with the item appended > fuse_dirlist_t * fuse_dirlist_add(fuse_dirlist_t * list, char * name, struct > stat * st, ...); > void fuse_dirlist_free(fuse_dirlist_t * list); > void fuse_reply_dirlist(fuse_req_t * req, fuse_dirlist_t * dirents); It would be easy to implement such an interface _on top_ of the current one. I'm not averse to that. Also fuse_dirlist_add() needs to be passed the size parameter, and return NULL when the buffer is filled. > Also, I noticed that the libfuse versionscript left off a few > functions, specifically fuse_ll_loop_mt() and fuse_req_ctx(). I had > to add those to get my code to link properly. Good find. Added them. > Hope this helps. Thanks, Miklos |
From: Joshua J. B. <con...@co...> - 2005-08-05 00:37:58
|
On Monday 01 August 2005 02:01, Miklos Szeredi wrote: > > Also, I noticed that the libfuse versionscript left off a few > > functions, specifically fuse_ll_loop_mt() and fuse_req_ctx(). I had > > to add those to get my code to link properly. > > Good find. Added them. Spelling error -- fuse_ll_loop_mNt :) Thanks. =2D- Josh =2D-=20 Joshua J. Berry "I haven't lost my mind -- it's backed up on tape somewhere." -- /usr/games/fortune |
From: Miklos S. <mi...@sz...> - 2005-08-05 08:40:04
|
> > > Also, I noticed that the libfuse versionscript left off a few > > > functions, specifically fuse_ll_loop_mt() and fuse_req_ctx(). I had > > > to add those to get my code to link properly. > > > > Good find. Added them. > > Spelling error -- fuse_ll_loop_mNt :) My fingers seem to think think 'mnt' makes more sense than 'mt' :) Thanks, Miklos |
From: Joshua J. B. <con...@co...> - 2005-08-01 17:21:33
|
On Monday 01 August 2005 02:01, Miklos Szeredi wrote: > > The one weak spot/suggestion I have currently for the API is the *dir() > > functions. It was hard to figure out how to handle readdir(), > > specifically: > > > > - What does "offset" mean? Is offset specified in bytes, or > > - directory entries? > > Either. Or something else entirely. The important thing is, that the > filesystem can find the next directory entry by what it filled in as > offset. But isn't the kernel the one specifying the offset? > Yes, leaving the memory management to the filesystem makes it a bit > awkward. However it also let's the filesystem do caching and reusing > the raw directory data (the high level FUSE library uses that > feature). That makes a certain amount of sense. I tend to cache things in a name->in= ode=20 map, though, because I find the caching also helps for handling lookups. I= t=20 seems like it would be more useful, generally, to cache at a higher level. > > Perhaps it would be good to have a fuse_dirlist_t or something, and > > use fuse_add_dirent() to append to that list (and handle memory > > allocation, etc). Then make a new fuse_reply_dirlist() for > > responding. Also, make "offset" and "size" be in terms of directory > > entries, not bytes. So maybe something like this: > > Size cannot be the number of entries, since that is not known in > advance (the size of entries varies because of variable length > filenames). I was thinking the kernel could request N entries, and then receive back a= =20 total length (in bytes), because the filesystem knows how long each entry i= s. =20 I guess this would make memory management in the kernel more complicated,=20 though. Are there any other implementation barriers in the kernel (aside from the=20 memory management issues) to making "size" and "off" be in terms of entries= =20 instead of bytes? > > struct fuse_dirlist_t { ... }; > > > > // Pass NULL initially to create a new dirlist > > // Returns the dirlist with the item appended > > fuse_dirlist_t * fuse_dirlist_add(fuse_dirlist_t * list, char * name, > > struct stat * st, ...); > > void fuse_dirlist_free(fuse_dirlist_t * list); > > void fuse_reply_dirlist(fuse_req_t * req, fuse_dirlist_t * dirents); > > It would be easy to implement such an interface _on top_ of the > current one. I'm not averse to that. > > Also fuse_dirlist_add() needs to be passed the size parameter, and > return NULL when the buffer is filled. I was thinking fuse_dirlist would be a linked list whose memory was managed= by=20 the library, thus no need to mess with sizes. (Of course, having a=20 fuse_dirlist_t like that implies that the filesystem knows how many entries= =20 to return, as opposed to how big a buffer to fill.) =2D- Josh =2D-=20 Joshua J. Berry "I haven't lost my mind -- it's backed up on tape somewhere." -- /usr/games/fortune |
From: Valient G. <vg...@po...> - 2005-08-01 21:52:42
|
On Monday 01 August 2005 19:21, Joshua J. Berry wrote: > On Monday 01 August 2005 02:01, Miklos Szeredi wrote: > > The one weak spot/suggestion I have currently for the > > Either. =C2=A0Or something else entirely. =C2=A0The important thing is,= that the > > filesystem can find the next directory entry by what it filled in as > > offset. > > But isn't the kernel the one specifying the offset? The kernel is just passing back what was given to it by the filesystem. I= =20 don't think it interprets the offset value. =20 I haven't played with the low-level interface yet, but at least in the=20 standard libfuse api, the offset is provided by the filesystem. Whenever y= ou=20 call the filldir function, you're providing the next offset. That way if y= ou=20 weren't able to return the entire directory contents at once (because the=20 buffer size has been reached), then the kernel can request starting at the= =20 offset provided by the last entry. > > Size cannot be the number of entries, since that is not known in > > advance (the size of entries varies because of variable length > > filenames). >=20 > I was thinking the kernel could request N entries, and then receive back = a=20 > total length (in bytes), because the filesystem knows how long each entry= =20 is. =C2=A0 But that would require two round-trips in order to get even a small listing= =2E =20 One to find out the size and the next to get the data. And what would you= =20 gain from this? > Are there any other implementation barriers in the kernel (aside from the= =20 > memory management issues) to making "size" and "off" be in terms of entri= es=20 > instead of bytes? I think you misunderstand how offset is used -- it is already much more=20 flexable then you suspect. The two are independent, the units for 'size' a= re=20 defined by the kernel, and the units for 'offset' are defined by the=20 filesystem. =46or the offset, you can store any value you want - whatever is convenient= =2E If=20 you want to store the entry number into an array, that is fine. For me, I= =20 might want to store the off_t from telldir(3), because my filesystem builds= =20 on top of an existing filesystem. That way when I get a new request, I can= =20 use seekdir() to seek to the correct location and continue reading entries.= =20 Or you could even store a pointer to the next element in a linked list. Ve= ry=20 flexable. Size is in bytes and is sent from the kernel. In kernel/dir.c, fuse_readdi= r=20 seems to send PAGE_SIZE as the requested size, so one page worth of bytes a= re=20 being requested at once. regards, Valient |
From: Joshua J. B. <con...@co...> - 2005-08-01 22:55:34
|
On Monday 01 August 2005 14:52, Valient Gough wrote: > > But isn't the kernel the one specifying the offset? > > The kernel is just passing back what was given to it by the filesystem. I > don't think it interprets the offset value. > > I haven't played with the low-level interface yet, but at least in the > standard libfuse api, the offset is provided by the filesystem. Whenever > you call the filldir function, you're providing the next offset. That way > if you weren't able to return the entire directory contents at once > (because the buffer size has been reached), then the kernel can request > starting at the offset provided by the last entry. In the low-level interface, readdir() gets an offset and a size. When=20 generating the response, you create a buffer and fill it using=20 fuse_add_dirent(). The reply is just the buffer, and the length of the=20 buffer (in bytes) using fuse_reply_buf(). I would assume that if a listing isn't big enough to fit into a buffer=20 of /size/ bytes, the kernel will issue another readdir() request starting a= t=20 offset(previous)+size(previous). > > I was thinking the kernel could request N entries, and then receive back > > a total length (in bytes), because the filesystem knows how long each > > entry is. =C2=A0 > > But that would require two round-trips in order to get even a small > listing. One to find out the size and the next to get the data. And what > would you gain from this? I don't see how it would. Userspace already sends back a buffer size. > I think you misunderstand how offset is used -- it is already much more > flexable then you suspect. The two are independent, the units for 'size' > are defined by the kernel, and the units for 'offset' are defined by the > filesystem. > > For the offset, you can store any value you want - whatever is convenient= =2E=20 > If you want to store the entry number into an array, that is fine. For m= e, > I might want to store the off_t from telldir(3), because my filesystem > builds on top of an existing filesystem. That way when I get a new > request, I can use seekdir() to seek to the correct location and continue > reading entries. Or you could even store a pointer to the next element in= a > linked list. Very flexable. How do I store an offset, though? The only way for me to respond to the=20 kernel is to give it a buffer full of packed dirents, and a size. Poking=20 through fuse_lowlevel.h, I don't see any way to give it an offset. But yes, I think if I could set my own offset (or the "next expected offset= ",=20 or something similar), that would work well. =2D- Josh =2D-=20 Joshua J. Berry "I haven't lost my mind -- it's backed up on tape somewhere." -- /usr/games/fortune |
From: Valient G. <vg...@po...> - 2005-08-01 23:36:18
|
On Tuesday 02 August 2005 00:55, Joshua J. Berry wrote: > How do I store an offset, though? The only way for me to respond to the > kernel is to give it a buffer full of packed dirents, and a size. Poking > through fuse_lowlevel.h, I don't see any way to give it an offset. When you add dirent entries, you use fuse_add_dirent(). The last argument is "off_t off". That is the offset. Valient |
From: Joshua J. B. <con...@co...> - 2005-08-01 23:51:35
|
On Monday 01 August 2005 16:36, Valient Gough wrote: > On Tuesday 02 August 2005 00:55, Joshua J. Berry wrote: > > How do I store an offset, though? The only way for me to respond to the > > kernel is to give it a buffer full of packed dirents, and a size. Poki= ng > > through fuse_lowlevel.h, I don't see any way to give it an offset. > > When you add dirent entries, you use fuse_add_dirent(). The last argument > is "off_t off". That is the offset. D'oh. I see it now. I was also looking at my code, and I just stick the=20 current buffer size in that parameter, which explains why I was confused. ::adds a long comment talking about offsets to that part of his code:: Thanks for pointing it out to me. =2D- Josh =2D-=20 Joshua J. Berry "I haven't lost my mind -- it's backed up on tape somewhere." -- /usr/games/fortune |