From: Szabolcs S. <sz...@nt...> - 2008-04-02 21:30:06
|
Hello file system developers, There are several POSIX file system test suites: closed source, commercial, one which needs reading 174 pages installation guide, etc. Because of these frustrations when Pawel Jakub Dawidek ported ZFS to FreeBSD, he also wrote such a test suite quickly. Last year the NTFS-3G team ported it to Linux/ext3 and Linux/NTFS-3G to validate Jean-Pierre Andre's full file permissions and ownership support for NTFS-3G. We sent our patches to Pawel for integration but this doesn't seem to happen him (he didn't see problems but is busy). Since this topic regularly appears on several lists, we are also often asked about it and NTFS-3G does need it to be maintained, hence we decided to release it and if nobody else would like to maintain it then we will do so. The test suite mostly checks POSIX compliance and works for FreeBSD, Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, rmdir, symlink, truncate, unlink. There are currently 1950 regression tests. Availability: http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz and in the NTFS-3G CVS as pjd-fstest module: http://sourceforge.net/cvs/?group_id=181143 The usage is extremely simple: # tar czf pjd-fstest-20080402.tgz # cd pjd-fstest-20080402 # vi tests/conf Change 'fs' to file system type you want to test (UFS, ZFS, ext3, ntfs-3g). # make It will compile fstest utility which is used by regression tests. # cd /path/to/file/system/you/want/to/test/ The test must be run as root user and requires a few basic Perl modules. # prove -r /path/to/fstest/ It's also possible to run individual set of tests: # /path/to/fstest/tests/chown/00.t Or make single system call tests: # fstest mkdir foo 0750 0 # fstest mkdir foo 0750 mkdir returned -1 EEXIST The test suite is easy to understand, modify and extend. For instance doing a test cases for the above examples is only expect 0 fstest mkdir foo 0750 expect EEXIST fstest mkdir foo 0750 The default file system type is ext3 and it passes all tests. NTFS-3G also passes all the tests if the latest PERMISSION_HANDLING_BRANCH CVS branch is used with the below UserMapping file placed in the .NTFS-3G control directory: ---------------------------------------------------------------> :500:S-1-5-21-2271520284-214583110-2989893066-513 500::S-1-5-21-2271520284-214583110-2989893066-1007 # default mapping pattern ::S-1-5-21-2271520284-214583110-2989893066-10000 <-------------------------------------------------------------- Many thanks to Pawel Jakub Dawidek for writing this fantastic test suite, to Jean-Pierre Andre for tirelessly working on the port and fixing countless file system problems over the last half year and to Szeredi Miklos for his exceptionally rapid FUSE fixes. Enjoy, Szaka -- NTFS-3G: http://ntfs-3g.org |
From: Gerard J. C. <gj...@ci...> - 2008-04-02 21:59:30
|
Szabolcs Szakacsits wrote: > Last year the NTFS-3G team ported it to Linux/ext3 and Linux/NTFS-3G to > validate Jean-Pierre Andre's full file permissions and ownership support > for NTFS-3G. We sent our patches to Pawel for integration but this doesn't > seem to happen him (he didn't see problems but is busy). > > Since this topic regularly appears on several lists, we are also often > asked about it and NTFS-3G does need it to be maintained, hence we decided > to release it and if nobody else would like to maintain it then we will do > so. > > The test suite mostly checks POSIX compliance and works for FreeBSD, > Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list > of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, > rmdir, symlink, truncate, unlink. There are currently 1950 regression > tests. > Thanks Szaka, I added pointers to the FUSE wiki FAQ -Gerard |
From: Joel B. <Joe...@or...> - 2008-04-02 22:24:16
|
On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote: > The test suite mostly checks POSIX compliance and works for FreeBSD, > Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list > of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, > rmdir, symlink, truncate, unlink. There are currently 1950 regression > tests. <snip> > Availability: > > http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz Very interesting. ocfs2, running as 'ext3' mode, gets: Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% okay. Thanks! Joel -- "In the beginning, the universe was created. This has made a lot of people very angry, and is generally considered to have been a bad move." - Douglas Adams Joel Becker Principal Software Developer Oracle E-mail: joe...@or... Phone: (650) 506-8127 |
From: Szabolcs S. <sz...@nt...> - 2008-04-03 04:12:16
|
On Wed, 2 Apr 2008, Joel Becker wrote: > On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote: > > The test suite mostly checks POSIX compliance and works for FreeBSD, > > Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list > > of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, > > rmdir, symlink, truncate, unlink. There are currently 1950 regression > > tests. > <snip> > > Availability: > > > > http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz > > Very interesting. ocfs2, running as 'ext3' mode, gets: > > Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% okay. That's not bad as a start and it doesn't necessarily mean that there is anything wrong with ocfs2. There are many cases when SuS says that the behavior can be implementation specific. But we did find that following ext3 as close as possible will reduce the number of bug reports. We started from "574/1950 subtests failed, 70.56% okay." Regards, Szaka -- NTFS-3G: http://ntfs-3g.org |
From: Joel B. <Joe...@or...> - 2008-04-03 16:49:59
|
On Thu, Apr 03, 2008 at 07:12:12AM +0300, Szabolcs Szakacsits wrote: > > > http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz > > > > Very interesting. ocfs2, running as 'ext3' mode, gets: > > > > Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% okay. > > That's not bad as a start and it doesn't necessarily mean that there is > anything wrong with ocfs2. There are many cases when SuS says that the > behavior can be implementation specific. Yeah, I haven't looked into the failures. Some day when I have free time :-) Joel -- "When I am working on a problem I never think about beauty. I only think about how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." - Buckminster Fuller Joel Becker Principal Software Developer Oracle E-mail: joe...@or... Phone: (650) 506-8127 |
From: Amar S. T. <am...@zr...> - 2008-04-03 16:51:44
|
Hi, Thanks for the test tool. It really helps us to very our posix compliance level. GlusterFS running as 'ext3' gets: Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------- /tmp/pjd-fstest-20080402/tests/ch 171 5 2.92% 69 141 145 149 153 Failed 1/184 test scripts, 99.46% okay. 5/1950 subtests failed, 99.74% okay. Regards, Amar ----- > > > > Very interesting. ocfs2, running as 'ext3' mode, gets: > > > > Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% > okay. > > That's not bad as a start and it doesn't necessarily mean that there is > anything wrong with ocfs2. There are many cases when SuS says that the > behavior can be implementation specific. > > But we did find that following ext3 as close as possible will reduce the > number of bug reports. We started from "574/1950 subtests failed, 70.56% > okay." > > Regards, > Szaka > > -- > NTFS-3G: http://ntfs-3g.org > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > fuse-devel mailing list > fus...@li... > https://lists.sourceforge.net/lists/listinfo/fuse-devel > > -- Amar Tumballi Gluster/GlusterFS Hacker [bulde on #gluster/irc.gnu.org] http://www.zresearch.com - Commoditizing Supercomputing and Superstorage! |
From: Badari P. <pb...@gm...> - 2008-04-03 17:27:00
|
On Thu, 2008-04-03 at 00:29 +0300, Szabolcs Szakacsits wrote: > Hello file system developers, > > There are several POSIX file system test suites: closed source, commercial, > one which needs reading 174 pages installation guide, etc. Because of these > frustrations when Pawel Jakub Dawidek ported ZFS to FreeBSD, he also wrote > such a test suite quickly. > > Last year the NTFS-3G team ported it to Linux/ext3 and Linux/NTFS-3G to > validate Jean-Pierre Andre's full file permissions and ownership support > for NTFS-3G. We sent our patches to Pawel for integration but this doesn't > seem to happen him (he didn't see problems but is busy). > > Since this topic regularly appears on several lists, we are also often > asked about it and NTFS-3G does need it to be maintained, hence we decided > to release it and if nobody else would like to maintain it then we will do > so. > > The test suite mostly checks POSIX compliance and works for FreeBSD, > Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list > of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, > rmdir, symlink, truncate, unlink. There are currently 1950 regression > tests. > > Availability: > > http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz > > and in the NTFS-3G CVS as pjd-fstest module: > > http://sourceforge.net/cvs/?group_id=181143 > > The usage is extremely simple: > > # tar czf pjd-fstest-20080402.tgz > # cd pjd-fstest-20080402 > # vi tests/conf > Change 'fs' to file system type you want to test (UFS, ZFS, ext3, ntfs-3g). > # make > It will compile fstest utility which is used by regression tests. > # cd /path/to/file/system/you/want/to/test/ > The test must be run as root user and requires a few basic Perl modules. > # prove -r /path/to/fstest/ > > It's also possible to run individual set of tests: > > # /path/to/fstest/tests/chown/00.t > > Or make single system call tests: > > # fstest mkdir foo 0750 > 0 > # fstest mkdir foo 0750 > mkdir returned -1 > EEXIST > > The test suite is easy to understand, modify and extend. For instance doing > a test cases for the above examples is only > > expect 0 fstest mkdir foo 0750 > expect EEXIST fstest mkdir foo 0750 > > The default file system type is ext3 and it passes all tests. Hmm.. I ran it against ext2, ext3, jfs, btrfs. I don't see all "pass" on ext3. What am I missing ? btrfs seems to have little more failures. Thanks, Badari ext2: ==== Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153 /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10 /root/posix/tests/open/05.t 12 2 16.67% 5 9 /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 39 42 /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 /root/posix/tests/truncate/12.t 3 1 33.33% 2 /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay. ext3: ==== Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10 /root/posix/tests/open/05.t 12 2 16.67% 5 9 /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 39 42 /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 /root/posix/tests/truncate/12.t 3 1 33.33% 2 /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 Failed 8/184 test scripts, 95.65% okay. 29/1950 subtests failed, 98.51% okay. jfs: === Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153 /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10 /root/posix/tests/open/05.t 12 2 16.67% 5 9 /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 39 42 /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 /root/posix/tests/truncate/12.t 3 1 33.33% 2 /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay. btrfs: ===== Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153 /root/posix/tests/link/00.t 82 8 9.76% 3 5-6 8-10 56 63 /root/posix/tests/open/05.t 12 2 16.67% 5 9 /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 39 42 /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 /root/posix/tests/truncate/00.t 21 1 4.76% 15 /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 /root/posix/tests/truncate/12.t 3 1 33.33% 2 /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 /root/posix/tests/unlink/00.t 55 3 5.45% 17 22 53 Failed 11/184 test scripts, 94.02% okay. 39/1950 subtests failed, 98.00% okay. |
From: Szabolcs S. <sz...@nt...> - 2008-04-06 23:34:48
|
On Thu, 3 Apr 2008, Badari Pulavarty wrote: > On Thu, 2008-04-03 at 00:29 +0300, Szabolcs Szakacsits wrote: > > > The default file system type is ext3 and it passes all tests. > > Hmm.. I ran it against ext2, ext3, jfs, btrfs. I don't see all "pass" > on ext3. What am I missing ? Your unique, consistently failing test cases for all file systems suggest that you have a buggy private kernel or some other individual issue in your test environment. You could use the -x debug shell option in the test scripts, rerun the failing ones and they will show why these test cases exactly fail. > btrfs seems to have little more failures. If you find the reason for the unexpected failures then the btrfs result will be quite good. Apparently it has only a few link, truncate, and unlink ctimes update problems. I think that's quite impressive in its state of development. Szaka > ext2: > ==== > > Failed Test Stat Wstat Total Fail Failed List of Failed > ------------------------------------------------------------------------------- > /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 > /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153 > /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10 > /root/posix/tests/open/05.t 12 2 16.67% 5 9 > /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 > 39 42 > /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 > /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 > /root/posix/tests/truncate/12.t 3 1 33.33% 2 > /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 > Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay. > > > ext3: > ==== > > Failed Test Stat Wstat Total Fail Failed List of Failed > ------------------------------------------------------------------------------- > /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 > /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10 > /root/posix/tests/open/05.t 12 2 16.67% 5 9 > /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 > 39 42 > /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 > /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 > /root/posix/tests/truncate/12.t 3 1 33.33% 2 > /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 > Failed 8/184 test scripts, 95.65% okay. 29/1950 subtests failed, 98.51% okay. > > > jfs: > === > > Failed Test Stat Wstat Total Fail Failed List of Failed > ------------------------------------------------------------------------------- > /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 > /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153 > /root/posix/tests/link/00.t 82 6 7.32% 3 5-6 8-10 > /root/posix/tests/open/05.t 12 2 16.67% 5 9 > /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 > 39 42 > /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 > /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 > /root/posix/tests/truncate/12.t 3 1 33.33% 2 > /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 > Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay. > > > btrfs: > ===== > > Failed Test Stat Wstat Total Fail Failed List of Failed > ------------------------------------------------------------------------------- > /root/posix/tests/chmod/00.t 58 2 3.45% 3 19 > /root/posix/tests/chown/00.t 171 4 2.34% 141 145 149 153 > /root/posix/tests/link/00.t 82 8 9.76% 3 5-6 8-10 56 63 > /root/posix/tests/open/05.t 12 2 16.67% 5 9 > /root/posix/tests/rename/00.t 79 9 11.39% 3 6 8-9 11 13 37 > 39 42 > /root/posix/tests/symlink/00.t 14 2 14.29% 2 5 > /root/posix/tests/truncate/00.t 21 1 4.76% 15 > /root/posix/tests/truncate/05.t 15 5 33.33% 5-6 10-12 > /root/posix/tests/truncate/12.t 3 1 33.33% 2 > /root/posix/tests/truncate/13.t 4 2 50.00% 2-3 > /root/posix/tests/unlink/00.t 55 3 5.45% 17 22 53 > Failed 11/184 test scripts, 94.02% okay. 39/1950 subtests failed, 98.00% okay. > > > -- NTFS-3G: http://ntfs-3g.org |
From: David C. <dg...@sg...> - 2008-04-04 00:33:48
|
On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote: > The test must be run as root user and requires a few basic Perl modules. And openssl, it appears. > # prove -r /path/to/fstest/ The current xfs-dev tree: Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- /root/posix/tests/chown/00.t 171 2 1.17% 84 88 /root/posix/tests/symlink/02.t 7 2 28.57% 6-7 Failed 2/184 test scripts, 98.91% okay. 4/1950 subtests failed, 99.79% okay. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group |
From: David C. <dg...@sg...> - 2008-04-04 06:51:29
|
On Fri, Apr 04, 2008 at 10:33:30AM +1000, David Chinner wrote: > On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote: > > The test must be run as root user and requires a few basic Perl modules. > > And openssl, it appears. > > > # prove -r /path/to/fstest/ > > The current xfs-dev tree: > > Failed Test Stat Wstat Total Fail Failed List of Failed > ------------------------------------------------------------------------------- > /root/posix/tests/chown/00.t 171 2 1.17% 84 88 > /root/posix/tests/symlink/02.t 7 2 28.57% 6-7 > Failed 2/184 test scripts, 98.91% okay. 4/1950 subtests failed, 99.79% okay. Symlink tests 6 and 7: expect 0 symlink ${name256} ${n0} expect 0 unlink ${n0} Test 6 is failing with ENAMETOOLONG Test 7 is failing (correctly) with ENOENT because test 6 failed. So there's only one failure here, and that is that that we're rejecting ${name256} as too long. I think that getname() is doing this. Seems sane to me to disallow symlinking to pathnames that can't be constructed, even if POSIX apparently allows it. Chown tests 84 and 88: Test 84 appears to be checking the result of test 83: expect 0 -u 65534 -g 65533,65532 chown ${n0} 65534 65532 case "${os}" in Linux) expect 06555,65534,65532 lstat ${n0} mode,uid,gid ;; *) expect 0555,65534,65532 lstat ${n0} mode,uid,gid ;; esac And running these manually I get: # /root/posix/fstest -u 65534 -g 65533,65532 chown z 65534 65532 changing groups to 65533,65532 changing uid to 65534 0 # /root/posix/fstest lstat z mode,uid,gid 0555,65534,65532 Which matches the "non-Linux" output. Looks like bits 06000 are the set-uid and set-gid bits. Ok, Posix says: "If the chown() function is successfully invoked on a file that is not a regular file and one or more of the S_IXUSR, S_IXGRP, or S_IXOTH bits of the file mode are set, the set-user-ID and set-group-ID bits may be cleared." So, either result is valid. Hence i suggest that test 84 and test 88 (same failure) are special cased to "ext3" behaviour. That means XFS is not failing any tests at all. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group |
From: Szabolcs S. <sz...@nt...> - 2008-04-06 23:51:15
|
On Fri, 4 Apr 2008, David Chinner wrote: > On Fri, Apr 04, 2008 at 10:33:30AM +1000, David Chinner wrote: > > On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote: > > > The test must be run as root user and requires a few basic Perl modules. > > > > And openssl, it appears. Openssl is replaced with md5sum+cut in the CVS. It would be also nice to eliminate the Perl dependency ... > > The current xfs-dev tree: > > > > Failed Test Stat Wstat Total Fail Failed List of Failed > > ------------------------------------------------------------------------------- > > /root/posix/tests/chown/00.t 171 2 1.17% 84 88 > > /root/posix/tests/symlink/02.t 7 2 28.57% 6-7 > > Failed 2/184 test scripts, 98.91% okay. 4/1950 subtests failed, 99.79% okay. > > Symlink tests 6 and 7: > > expect 0 symlink ${name256} ${n0} > expect 0 unlink ${n0} > > Test 6 is failing with ENAMETOOLONG > Test 7 is failing (correctly) with ENOENT because test 6 failed. > > So there's only one failure here, and that is that that we're rejecting > ${name256} as too long. I think that getname() is doing this. Seems sane > to me to disallow symlinking to pathnames that can't be constructed, > even if POSIX apparently allows it. As Christoph noted, I also noticed XFS is unique in this behavior. > Chown tests 84 and 88: [...] > So, either result is valid. Hence i suggest that test 84 and test 88 > (same failure) are special cased to "ext3" behaviour. Done in the CVS. > That means XFS is not failing any tests at all. I added the xfs target but left the symlink Test 6 fail because POSIX says "The string pointed to by path1 shall be treated only as a character string and shall not be validated as a pathname" and "the length of the path1 argument is longer than {SYMLINK_MAX}" where {SYMLINK_MAX} is typically not defined on Linux or it's {PATH_MAX}. Szaka -- NTFS-3G: http://ntfs-3g.org |
From: Christoph H. <hc...@in...> - 2008-04-04 15:48:23
|
On Fri, Apr 04, 2008 at 04:51:09PM +1000, David Chinner wrote: > expect 0 symlink ${name256} ${n0} > expect 0 unlink ${n0} > > Test 6 is failing with ENAMETOOLONG > Test 7 is failing (correctly) with ENOENT because test 6 failed. > > So there's only one failure here, and that is that that we're rejecting > ${name256} as too long. I think that getname() is doing this. Seems sane > to me to disallow symlinking to pathnames that can't be constructed, > even if POSIX apparently allows it. i'd rather expect this to be the component validation in xfs_symlink. It's superflous and not done by any other fiesystem. |
From: David C. <dg...@sg...> - 2008-04-07 02:08:03
|
On Fri, Apr 04, 2008 at 11:48:18AM -0400, Christoph Hellwig wrote: > On Fri, Apr 04, 2008 at 04:51:09PM +1000, David Chinner wrote: > > expect 0 symlink ${name256} ${n0} > > expect 0 unlink ${n0} > > > > Test 6 is failing with ENAMETOOLONG > > Test 7 is failing (correctly) with ENOENT because test 6 failed. > > > > So there's only one failure here, and that is that that we're rejecting > > ${name256} as too long. I think that getname() is doing this. Seems sane > > to me to disallow symlinking to pathnames that can't be constructed, > > even if POSIX apparently allows it. > > i'd rather expect this to be the component validation in xfs_symlink. > It's superflous and not done by any other fiesystem. Ah yes, you are right - PATH_MAX != NAME_MAX... Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group |
From: Marc A. T. <ma...@br...> - 2008-04-04 20:38:34
|
On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote: > > Hello file system developers, > > There are several POSIX file system test suites: closed source, commercial, > one which needs reading 174 pages installation guide, etc. Because of these > frustrations when Pawel Jakub Dawidek ported ZFS to FreeBSD, he also wrote > such a test suite quickly. > > Last year the NTFS-3G team ported it to Linux/ext3 and Linux/NTFS-3G to > validate Jean-Pierre Andre's full file permissions and ownership support > for NTFS-3G. We sent our patches to Pawel for integration but this doesn't > seem to happen him (he didn't see problems but is busy). > > Since this topic regularly appears on several lists, we are also often > asked about it and NTFS-3G does need it to be maintained, hence we decided > to release it and if nobody else would like to maintain it then we will do > so. Hi, Thanks for the release. I have a general question regarding FUSE. Why do all the tests pass on a regular ext3 file system while there are quite a lot of errors (~80% success) when run on top of the fuse example file system as found in the fuse tarball? I thought the fuse filesystem just passes every access to the underlying file system. So what's causing the failures? Or am i doing something wrong? Thanks, Marc -- Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0 |
From: Szabolcs S. <sz...@nt...> - 2008-04-07 00:40:48
|
On Fri, 4 Apr 2008, Marc Andre Tanner wrote: > Why do all the tests pass on a regular ext3 file system while there are > quite a lot of errors (~80% success) when run on top of the fuse example > file system as found in the fuse tarball? I thought the fuse filesystem > just passes every access to the underlying file system. So what's causing > the failures? Or am i doing something wrong? I don't think you're doing anything wrong. I guess the FUSE file system examples weren't supposed to fully pass a posix test. Moreover FUSE restricts some things as a security measure. For instance if you (correctly) use the allow_other and default_permissions mount options then 92+% of the tests will pass. Szaka -- NTFS-3G: http://ntfs-3g.org |
From: Marc A. T. <ma...@br...> - 2008-04-07 20:46:46
|
On Mon, Apr 07, 2008 at 03:40:50AM +0300, Szabolcs Szakacsits wrote: > > On Fri, 4 Apr 2008, Marc Andre Tanner wrote: > > > Why do all the tests pass on a regular ext3 file system while there are > > quite a lot of errors (~80% success) when run on top of the fuse example > > file system as found in the fuse tarball? I thought the fuse filesystem > > just passes every access to the underlying file system. So what's causing > > the failures? Or am i doing something wrong? > > I don't think you're doing anything wrong. > > I guess the FUSE file system examples weren't supposed to fully pass a > posix test. > > Moreover FUSE restricts some things as a security measure. For instance if > you (correctly) use the allow_other and default_permissions mount options > then 92+% of the tests will pass. I see. It just seemed strange to me that the tests passed on a normal ext3 file system but not on the example fuse file system which does nothing more than pass all operations one layer down. I have now taken a closer look and after reading the POSIX comments within the test cases it makes more sense. For example from tests/open/00.t: # POSIX: (If O_CREAT is specified and the file doesn't exist) [...] the user ID # of the file shall be set to the effective user ID of the process; the group ID # of the file shall be set to the group ID of the file's parent directory or to # the effective group ID of the process [...] expect 0 chown . 65535 65535 expect 0 -u 65535 -g 65535 open ${n0} O_CREAT,O_WRONLY 0644 expect 65535,65535 lstat ${n0} uid,gid This doesn't work for the fuse example file system because the open syscall which actually creates the file isn't executed with the requested uid,gid but in the context of the user who mounted the fuse file system. So you end up with different values. The other test failures are probably similar things. Marc -- Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0 |
From: Szabolcs S. <sz...@nt...> - 2008-04-07 21:21:47
|
On Mon, 7 Apr 2008, Marc Andre Tanner wrote: > I have now taken a closer look and after reading the POSIX comments > within the test cases it makes more sense. For example from > tests/open/00.t: > > # POSIX: (If O_CREAT is specified and the file doesn't exist) [...] the user ID > # of the file shall be set to the effective user ID of the process; the group ID > # of the file shall be set to the group ID of the file's parent directory or to > # the effective group ID of the process [...] > > expect 0 chown . 65535 65535 > expect 0 -u 65535 -g 65535 open ${n0} O_CREAT,O_WRONLY 0644 > expect 65535,65535 lstat ${n0} uid,gid > > This doesn't work for the fuse example file system because the open > syscall which actually creates the file isn't executed with the requested > uid,gid but in the context of the user who mounted the fuse file system. > So you end up with different values. Yes. These could be supported by using fuse_get_context()->uid, fuse_get_context()->gid, seteuid() and setegid(). > The other test failures are probably similar things. I think so. I checked out some still failing fuse example test cases and all of them had the same reason. Szaka -- NTFS-3G: http://ntfs-3g.org |
From: Marc A. T. <ma...@br...> - 2008-04-08 12:17:39
|
Szabolcs Szakacsits wrote: > On Mon, 7 Apr 2008, Marc Andre Tanner wrote: > >> I have now taken a closer look and after reading the POSIX comments >> within the test cases it makes more sense. For example from >> tests/open/00.t: >> >> # POSIX: (If O_CREAT is specified and the file doesn't exist) [...] the user ID >> # of the file shall be set to the effective user ID of the process; the group ID >> # of the file shall be set to the group ID of the file's parent directory or to >> # the effective group ID of the process [...] >> >> expect 0 chown . 65535 65535 >> expect 0 -u 65535 -g 65535 open ${n0} O_CREAT,O_WRONLY 0644 >> expect 65535,65535 lstat ${n0} uid,gid >> >> This doesn't work for the fuse example file system because the open >> syscall which actually creates the file isn't executed with the requested >> uid,gid but in the context of the user who mounted the fuse file system. >> So you end up with different values. > > Yes. These could be supported by using fuse_get_context()->uid, > fuse_get_context()->gid, seteuid() and setegid(). So when you want a fuse file system with correct permission semantics for multiple users you basically have to wrap every operation with: setegid(fuse_get_context()->gid); seteuid(fuse_get_context()->uid); /* do some work */ seteuid(getuid()); setegid(getgid()); Or am i missing something? But this only works when the file system is mounted by root. Also are the euid, egid stored per thread? If not then this will cause all kind of problems with race conditions. So in my opinion -o allow_other when used as a non root user and not intended for read only access is basically useless because new files will be owned by the user who mounted the fs. Cheers, Marc -- Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0 |
From: Miklos S. <mi...@sz...> - 2008-04-08 12:26:51
|
> So when you want a fuse file system with correct permission semantics > for multiple users you basically have to wrap every operation with: > > setegid(fuse_get_context()->gid); > seteuid(fuse_get_context()->uid); > /* do some work */ > seteuid(getuid()); > setegid(getgid()); > > Or am i missing something? But this only works when the file system is > mounted by root. Also are the euid, egid stored per thread? Yes. You could also use setfsuid() and setfsgid(). > If not then > this will cause all kind of problems with race conditions. > > So in my opinion -o allow_other when used as a non root user and not > intended for read only access is basically useless because new files > will be owned by the user who mounted the fs. I'd rather say that fusexmp* are basically useless whatever options you give them ;) The "-oallow_other" option is usually not for users to play with, as it could have unintended consequences. That's why it's not allowed for unprivileged mounts by default. Miklos |
From: Marc A. T. <ma...@br...> - 2008-04-08 14:15:30
|
Miklos Szeredi wrote: >> So in my opinion -o allow_other when used as a non root user and not >> intended for read only access is basically useless because new files >> will be owned by the user who mounted the fs. > > I'd rather say that fusexmp* are basically useless whatever options > you give them ;) Ok, i am trying to build a case insensitive file system which basically works like fusexmp* but well case insensitive. But if this turns out to be unusable then it doesn't really matter. My main motivation is to learn a thing or two about fuse and file systems in general. > The "-oallow_other" option is usually not for users to play with, as > it could have unintended consequences. That's why it's not allowed > for unprivileged mounts by default. Could you please elaborate a bit on the "unintended consequences"? Thanks, Marc -- Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0 |
From: Marc A. T. <ma...@br...> - 2008-04-09 18:52:07
|
On Tue, Apr 08, 2008 at 02:26:41PM +0200, Miklos Szeredi wrote: > > So when you want a fuse file system with correct permission semantics > > for multiple users you basically have to wrap every operation with: > > > > setegid(fuse_get_context()->gid); > > seteuid(fuse_get_context()->uid); > > /* do some work */ > > seteuid(getuid()); > > setegid(getgid()); > > > > Or am i missing something? But this only works when the file system is > > mounted by root. Also are the euid, egid stored per thread? > > Yes. You could also use setfsuid() and setfsgid(). Ok, I have now added sete{gid,uid} calls around the file system operations. And when I mount the fs with: -o allow_other,default_permissions,use_ino,attr_timeout=0 I only get the errors listed below: Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- ../fstest/tests/chown/00.t 171 10 5.85% 36-37 68-69 83-84 141 145 149 153 ../fstest/tests/chown/05.t 15 2 13.33% 11-12 Failed 2/184 test scripts, 98.91% okay. 12/1950 subtests failed, 99.38% okay The reason for those failure is mostly that I don't know the supplementary group IDs from the calling process. Is this information available somewhere? Or could it be added to fuse_context? So i could change it with setgroups(2). Marc -- Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0 |
From: Miklos S. <mi...@sz...> - 2008-04-09 19:08:16
|
> The reason for those failure is mostly that I don't know the > supplementary group IDs from the calling process. Is this information > available somewhere? Or could it be added to fuse_context? So i could > change it with setgroups(2). You can get the group list by reading '/proc/<tid>/task/<tid>/status'. The task ID <tid> is avaiable in context->pid. Arguably not the most elegant way to pass the group list to the filesystem, but it should at least work correctly. Miklos |
From: Marc A. T. <ma...@br...> - 2008-04-09 19:44:31
|
On Wed, Apr 09, 2008 at 09:07:59PM +0200, Miklos Szeredi wrote: > > The reason for those failure is mostly that I don't know the > > supplementary group IDs from the calling process. Is this information > > available somewhere? Or could it be added to fuse_context? So i could > > change it with setgroups(2). > > You can get the group list by reading '/proc/<tid>/task/<tid>/status'. > The task ID <tid> is avaiable in context->pid. > > Arguably not the most elegant way to pass the group list to the > filesystem, but it should at least work correctly. Ok thanks that's indeed not elegant. I should have searched the mailing list a bit better, in another thread you mentioned that it's on your TODO list to implement this in fuse. Would be nice if it could make it into one of the next releases. Thanks, Marc -- Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0 |
From: John M. <mu...@no...> - 2008-04-09 19:52:32
|
On 9-Apr-08, at 3:07 PM, Miklos Szeredi wrote: >> The reason for those failure is mostly that I don't know the >> supplementary group IDs from the calling process. Is this information >> available somewhere? Or could it be added to fuse_context? So i >> could >> change it with setgroups(2). > > You can get the group list by reading '/proc/<tid>/task/<tid>/status'. > The task ID <tid> is avaiable in context->pid. > > Arguably not the most elegant way to pass the group list to the > filesystem, but it should at least work correctly. Correct me if I'm wrong, but since the 'allow_other', and 'default_permissions' mount options are used, it is not necessary to check permissions within the file-system. My file-system assumes this, and simply sets the file ownership according to whatever is given in the arguments to the setattr, mknod, mkdir, and create operations, and I don't implement 'access'. John. |
From: Brian W. <ywa...@ho...> - 2008-04-10 05:22:14
|
I think the "ESTALE" problem is well understood for FUSE based FS because of the lack of persistent inode number. fusexmp uses the underneath filesystem, if fusexmp is exported via NFS (with some popular underneath fs such as EXT3), would "use_ino" option makes it "perfect" NFS export? ( or at least can be done if ext3/xfs etc. are the underneath file system?) I have been struggling with NFS export since the ESTALE problem is a big issue. I have got the NFS patch from John but it requires the work at fuse_lowlevel interface. |