From: Leandro F. <leo...@gm...> - 2005-12-28 11:54:00
|
Hi all, I am having a couple of problems and I would like to know if someone has an interesting idea to solve them ( as you always do :) ). 1. The ugliest one: I have a FS that enforces quotas. The idea is simple, when I write to a file I increase the "used" variable and when I delete a file I decrease the variable (a bit more than that but is not really difficult). However, I have a tricky scenario where things get messy and I end up with negative used space. - create file "test" - truncate(test,10) used=3D=3D10 - link(test, link_to_test) used=3D=3D20 - truncate(link_to_test,20) used=3D=3D30 (here both increase !!!) - unlink(test) used=3D=3D10 (Will say "test" is 20 not 10 !!!) - unlink(link_to_test) used=3D=3D-10 !!!!!!!!! I only know the basics about hard links and I dont see how to solve the problem in a simple way... :/ 2. Very ugly: The same idea than the last one but with directories.... When there are too many files in a directory it will increase (in size), and I dont know a way to track it... so we get a negative "used" space when we delete it. 3. Not so ugly but seems to be a fuse bug: - mkdir("dir") - open("dir/file", O_CREAT) - unlink("dir/file") - rmdir("dir") !!!! Error, directory not emp= ty - close("dir/file"); the .fuse is deleted here I saw it using fsstress and apart from that I don't know if it could be dangerous... I suppose the error comes from the .fuse_hidden file so I dont think it will be easy to solve.... All the comments are welcome and thanks a lot for the great work Leandro Franco |
From: <pr...@po...> - 2005-12-28 20:12:40
|
Leandro Franco <leo...@gm...> wrote: > - create file "test" > - truncate(test,10) used==10 > - link(test, link_to_test) used==20 If your fs implements hard links in the usual way, then adding a hard link doesn't actually double the amount of space used. So you shouldn't increase the "used" amount for link(). Even if "used" is also supposed to account for metadata, the increase would not generally be the same as the file size. > - truncate(link_to_test,20) used==30 (here both increase !!!) Here "used" would become 20. > - unlink(test) used==10 (Will say "test" is 20 not 10 !!!) Just as link() should not increase the "used" amount, unlink() should not decrease it, unless you are removing the last link. So here "used" would still be 20. > - unlink(link_to_test) used==-10 !!!!!!!!! Here, the last link to the file is removed, and the file's size is 20, so "used" would decrease by 20 to become 0. > When there are too many files in a directory it will increase (in > size), and I dont know a way to track it... so we get a negative > "used" space when we delete it. Can you call your own getattr() function to see how big the directory is, before and after adding a file, and then update "used"? > - mkdir("dir") > - open("dir/file", O_CREAT) > - unlink("dir/file") > - rmdir("dir") !!!! Error, directory not empty > - close("dir/file"); the .fuse is deleted here I think you can use the hard_remove mount option to fix that, but it may cause other problems. paul |
From: Leandro F. <leo...@gm...> - 2005-12-29 11:16:15
|
> If your fs implements hard links in the usual way, then adding a hard > link doesn't actually double the amount of space used. So you > shouldn't increase the "used" amount for link(). Even if "used" is > also supposed to account for metadata, the increase would not > generally be the same as the file size. > Just as link() should not increase the "used" amount, unlink() should > not decrease it, unless you are removing the last link. Yeap... you are right, my approach was wrong form the beginning. link() shouldnt add the size of the file and unlink() should take it out only when unlinking the last link. I only have a silly question... how do I know the size of the metadata taken by a hard link?.... or is it included in directory entry that contains this file?... > > Can you call your own getattr() function to see how big the directory > is, before and after adding a file, and then update "used"? yes... I suppose I need to _monitor_ it in all the calls that can increase the directory size ... it should be mknod() mkdir() symlink() link() and since the directory does not shrink i dont have to bother about tracking the size of unlink() or the others.... Thanks a lot, Leandro Franco |
From: J. K. <ja...@co...> - 2006-01-02 03:10:09
|
I am using fuse to implement a NFS type application. The write calls are 4K in size. Is there any way I can increase this size? Does this depend on write call or page size? Thanks Jana |
From: Miklos S. <mi...@sz...> - 2006-01-02 13:47:23
|
> I am using fuse to implement a NFS type application. > > The write calls are 4K in size. Is there any way I can increase this size? > Does this depend on write call or page size? Both. In fact it's a minimum of the page size and the write size. Also 2.5.0-pre1 and later support bigger write requests for 'direct_io' mounts. Doing it with normal mounts is harder but maybe possible. It's on the todo list. Miklos |
From: <pr...@po...> - 2005-12-29 11:32:29
|
Leandro Franco <leo...@gm...> wrote: > I only have a silly question... how do I know the size of the metadata > taken by a hard link?.... or is it included in directory entry that > contains this file?... Some of it (the name) is in the directory's data. The rest (inode, extended attributes if you provide them) is separate. If one user creates an entry in a directory owned by another user, you might choose either user to bear the cost of the name. Or you might exclude metadata from your quota calculations to make life easier. :) > I suppose I need to _monitor_ it in all the calls that can > increase the directory size ... If your fs is multi-threaded, you'll need a mutex for the directory to stop other threads from changing the directory before fetching the initial size, and keep it locked until after fetching the final size. > and since the directory does not shrink i dont have to bother about > tracking the size of unlink() or the others.... True, but it's nicer if unused space is released when directory entries are removed. Otherwise, you have a (typically very slow) resource leak. paul |
From: Leandro F. <leo...@gm...> - 2005-12-29 11:46:07
|
> Some of it (the name) is in the directory's data. The rest (inode, > extended attributes if you provide them) is separate. > > If one user creates an entry in a directory owned by another user, you > might choose either user to bear the cost of the name. Or you might > exclude metadata from your quota calculations to make life easier. :) > mmm... since the inode is shared and the name is in the dir entry, i can basically ignore the metadata (if no ext. attr) ... :) actually.... is there more information than that? > If your fs is multi-threaded, you'll need a mutex for the directory to > stop other threads from changing the directory before fetching the > initial size, and keep it locked until after fetching the final size. yes.. you are right... funny things can happen with those funky threads ;) = .... > True, but it's nicer if unused space is released when directory > entries are removed. Otherwise, you have a (typically very slow) > resource leak. Yeap... I was saying that since the whole size of the dir entry will be release at rmdir() I dont need to track it when files are deleted inside it.... thanks, Leandro |
From: Hisham M. <his...@gm...> - 2005-12-29 14:42:12
|
On 12/29/05, Leandro Franco wrote: > > Some of it (the name) is in the directory's data. The rest (inode, > > extended attributes if you provide them) is separate. > > > > If one user creates an entry in a directory owned by another user, you > > might choose either user to bear the cost of the name. Or you might > > exclude metadata from your quota calculations to make life easier. :) > > > > mmm... since the inode is shared and the name is in the dir entry, i > can basically ignore the metadata (if no ext. attr) ... :) It's also advisable to set a limit on the number of files a user can have -- even if hard links take no considerable disk space (it will take at least *some* space in the dir entry and/or a little bit of memory in the libfuse hash), you don't want a user to attack your system by creating a gazillion hard links. -- Hisham |