From: Sandon V. N. <sa...@va...> - 2010-06-01 16:24:12
|
I installed OpenSSH_5.3p1 on my opensolaris file-server as the default version didn't seem to support disk space usage at all (showed 1000G with 0 used) and now it shows the correct percentage but it is giving me out of wack values (in the Petabyte range): root@sabayonx86-64: 09:21 AM :~# df -H Filesystem Size Used Avail Use% Mounted on /dev/sdc2 35G 32G 3.3G 91% / udev 13G 283k 13G 1% /dev none 2.1M 324k 1.8M 16% /lib/rcscripts/init.d /dev/sdc1 100M 89M 12M 89% /boot /dev/sda1 84G 60G 25G 71% /winxp tmpfs 13G 0 13G 0% /dev/shm /dev/sdc3 18T 15T 3.6T 80% /data box:/data 8.9T 7.9T 1.1T 89% /colo osol:/data 4.5P 3.2P 1.3P 72% /osol This is not the correct size of the volume as it should be 18TB with df -H: root@opensolaris: 09:24 AM :~# df -H /data Filesystem Size Used Avail Use% Mounted on data 18T 13T 5.1T 72% /data The percentage it lists; however, is ok. Also when mounting the near same size file-sytem on a linux host to another linux machine I see the correct values: hptc ~ # df -H /sshfs Filesystem Size Used Avail Use% Mounted on 1.1.1.3:/data 18T 15T 3.6T 80% /sshfs Anyone have an idea of which end (sshfs or openssh) this bug is on? |
From: Miklos S. <mi...@sz...> - 2010-06-01 20:22:47
|
On Tue, 01 Jun 2010, Sandon Van Ness wrote: > I installed OpenSSH_5.3p1 on my opensolaris file-server as the default > version didn't seem to support disk space usage at all (showed 1000G > with 0 used) and now it shows the correct percentage but it is giving me > out of wack values (in the Petabyte range): > > root@sabayonx86-64: 09:21 AM :~# df -H > Filesystem Size Used Avail Use% Mounted on > /dev/sdc2 35G 32G 3.3G 91% / > udev 13G 283k 13G 1% /dev > none 2.1M 324k 1.8M 16% /lib/rcscripts/init.d > /dev/sdc1 100M 89M 12M 89% /boot > /dev/sda1 84G 60G 25G 71% /winxp > tmpfs 13G 0 13G 0% /dev/shm > /dev/sdc3 18T 15T 3.6T 80% /data > box:/data 8.9T 7.9T 1.1T 89% /colo > osol:/data 4.5P 3.2P 1.3P 72% /osol I believe the problem is that "df" uses statfs, not statvfs. If f_frsize and f_bsize differ then df will give the wrong answer, as statfs doesn't have an f_frsize field. Try "stat -f /osol" to verify this theory. > > This is not the correct size of the volume as it should be 18TB with df -H: > > root@opensolaris: 09:24 AM :~# df -H /data > Filesystem Size Used Avail Use% Mounted on > data 18T 13T 5.1T 72% /data > > The percentage it lists; however, is ok. Also when mounting the near > same size file-sytem on a linux host to another linux machine I see the > correct values: > > hptc ~ # df -H /sshfs > Filesystem Size Used Avail Use% Mounted on > 1.1.1.3:/data 18T 15T 3.6T 80% /sshfs Yep, linux sets f_frsize and f_bsize to the same values (presumably not to break df). Thanks, Miklos |
From: Sandon V. N. <sa...@va...> - 2010-06-01 20:39:22
|
On 06/01/2010 01:22 PM, Miklos Szeredi wrote: > On Tue, 01 Jun 2010, Sandon Van Ness wrote: > >> I installed OpenSSH_5.3p1 on my opensolaris file-server as the default >> version didn't seem to support disk space usage at all (showed 1000G >> with 0 used) and now it shows the correct percentage but it is giving me >> out of wack values (in the Petabyte range): >> >> root@sabayonx86-64: 09:21 AM :~# df -H >> Filesystem Size Used Avail Use% Mounted on >> /dev/sdc2 35G 32G 3.3G 91% / >> udev 13G 283k 13G 1% /dev >> none 2.1M 324k 1.8M 16% /lib/rcscripts/init.d >> /dev/sdc1 100M 89M 12M 89% /boot >> /dev/sda1 84G 60G 25G 71% /winxp >> tmpfs 13G 0 13G 0% /dev/shm >> /dev/sdc3 18T 15T 3.6T 80% /data >> box:/data 8.9T 7.9T 1.1T 89% /colo >> osol:/data 4.5P 3.2P 1.3P 72% /osol >> > I believe the problem is that "df" uses statfs, not statvfs. If > f_frsize and f_bsize differ then df will give the wrong answer, as > statfs doesn't have an f_frsize field. > > Try "stat -f /osol" to verify this theory. > > Yep, linux sets f_frsize and f_bsize to the same values (presumably > not to break df). > > Thanks, > Miklos > > I changed my mount point to /sshfs as I changed over to NFS to try seeing how that reported (which reports ok as expected). Here is what I get with stat: NFS: root@sabayonx86-64: 01:27 PM :~# stat -f /osol File: "/osol" ID: 0 Namelen: 255 Type: nfs Block size: 32768 Fundamental block size: 32768 Blocks: Total: 532021184 Free: 109079823 Available: 109079823 Inodes: Total: 6987112837 Free: 6981108616 SSHFS: root@sabayonx86-64: 01:27 PM :~# stat -f /sshfs File: "/sshfs" ID: 0 Namelen: 255 Type: UNKNOWN (0x65735546) Block size: 131072 Fundamental block size: 131072 Blocks: Total: 34049356570 Free: 6980647748 Available: 6980647748 Inodes: Total: 6986651970 Free: 6980647748 |
From: Sandon V. N. <sa...@va...> - 2010-06-02 01:46:43
|
On 06/01/2010 01:38 PM, Sandon Van Ness wrote: > On 06/01/2010 01:22 PM, Miklos Szeredi wrote: > >> On Tue, 01 Jun 2010, Sandon Van Ness wrote: >> >> >>> I installed OpenSSH_5.3p1 on my opensolaris file-server as the default >>> version didn't seem to support disk space usage at all (showed 1000G >>> with 0 used) and now it shows the correct percentage but it is giving me >>> out of wack values (in the Petabyte range): >>> >>> root@sabayonx86-64: 09:21 AM :~# df -H >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/sdc2 35G 32G 3.3G 91% / >>> udev 13G 283k 13G 1% /dev >>> none 2.1M 324k 1.8M 16% /lib/rcscripts/init.d >>> /dev/sdc1 100M 89M 12M 89% /boot >>> /dev/sda1 84G 60G 25G 71% /winxp >>> tmpfs 13G 0 13G 0% /dev/shm >>> /dev/sdc3 18T 15T 3.6T 80% /data >>> box:/data 8.9T 7.9T 1.1T 89% /colo >>> osol:/data 4.5P 3.2P 1.3P 72% /osol >>> >>> >> I believe the problem is that "df" uses statfs, not statvfs. If >> f_frsize and f_bsize differ then df will give the wrong answer, as >> statfs doesn't have an f_frsize field. >> >> Try "stat -f /osol" to verify this theory. >> >> Yep, linux sets f_frsize and f_bsize to the same values (presumably >> not to break df). >> >> Thanks, >> Miklos >> >> >> > I changed my mount point to /sshfs as I changed over to NFS to try > seeing how that reported (which reports ok as expected). Here is what I > get with stat: > > NFS: > > root@sabayonx86-64: 01:27 PM :~# stat -f /osol > File: "/osol" > ID: 0 Namelen: 255 Type: nfs > Block size: 32768 Fundamental block size: 32768 > Blocks: Total: 532021184 Free: 109079823 Available: 109079823 > Inodes: Total: 6987112837 Free: 6981108616 > > SSHFS: > root@sabayonx86-64: 01:27 PM :~# stat -f /sshfs > File: "/sshfs" > ID: 0 Namelen: 255 Type: UNKNOWN (0x65735546) > Block size: 131072 Fundamental block size: 131072 > Blocks: Total: 34049356570 Free: 6980647748 Available: 6980647748 > Inodes: Total: 6986651970 Free: 6980647748 > Also here is the stats on the solaris box itself vs ssshfs: opensolaris: root@opensolaris: 06:38 PM :/usr/local/etc# stat -f /data File: "/data" ID: 1690006 Namelen: 255 Type: zfs Block size: 131072 Fundamental block size: 512 Blocks: Total: 34049348764 Free: 6056839257 Available: 6056839257 Inodes: Total: 6064436485 Free: 6056839257 sshfs: root@sabayonx86-64: 06:35 PM :~# stat -f /sshfs/ File: "/sshfs/" ID: 0 Namelen: 255 Type: UNKNOWN (0x65735546) Block size: 131072 Fundamental block size: 131072 Blocks: Total: 34049348764 Free: 6056839257 Available: 6056839257 Inodes: Total: 6064436485 Free: 6056839257 I noticed the block counts are equal but the block size is 131072 on sshfs and 512 on the solaris machine. |
From: Miklos S. <mi...@sz...> - 2010-06-02 10:40:24
|
[bug-coreutils@... added to CC] On Tue, 01 Jun 2010, Sandon Van Ness wrote: > > NFS: > > > > root@sabayonx86-64: 01:27 PM :~# stat -f /osol > > File: "/osol" > > ID: 0 Namelen: 255 Type: nfs > > Block size: 32768 Fundamental block size: 32768 > > Blocks: Total: 532021184 Free: 109079823 Available: 109079823 > > Inodes: Total: 6987112837 Free: 6981108616 > > > > SSHFS: > > root@sabayonx86-64: 01:27 PM :~# stat -f /sshfs > > File: "/sshfs" > > ID: 0 Namelen: 255 Type: UNKNOWN (0x65735546) > > Block size: 131072 Fundamental block size: 131072 > > Blocks: Total: 34049356570 Free: 6980647748 Available: 6980647748 > > Inodes: Total: 6986651970 Free: 6980647748 > > > Also here is the stats on the solaris box itself vs ssshfs: > > opensolaris: > > > root@opensolaris: 06:38 PM :/usr/local/etc# stat -f /data > File: "/data" > ID: 1690006 Namelen: 255 Type: zfs > Block size: 131072 Fundamental block size: 512 > Blocks: Total: 34049348764 Free: 6056839257 Available: 6056839257 > Inodes: Total: 6064436485 Free: 6056839257 > > sshfs: > root@sabayonx86-64: 06:35 PM :~# stat -f /sshfs/ > File: "/sshfs/" > ID: 0 Namelen: 255 Type: UNKNOWN (0x65735546) > Block size: 131072 Fundamental block size: 131072 > Blocks: Total: 34049348764 Free: 6056839257 Available: 6056839257 > Inodes: Total: 6064436485 Free: 6056839257 > > I noticed the block counts are equal but the block size is 131072 on > sshfs and 512 on the solaris machine. > I looked more deeply into this issue and it appears that "df" and "stat" on linux both use the statfs libc function which doesn't have f_frsize. To verify this, try doing strace -o /tmp/strace stat -f /sshfs You should get a line like this in the strace output: statfs("/sshfs/", {f_type=0x65735546, ..., f_frsize=512}) = 0 Notice that the real statfs syscall does have an f_frsize field, but for some historical reason it is not exported in the library API. Options to fix this behavior would be: 1) fix df and stat in coreutils to use statvfs 2) make sshfs set f_bsize to f_frsize 3) make sshfs set f_frsize to f_bsize and recalculate stats 4) change both f_frsize and f_bsize to a common value and recalculate stats (this is what NFS appears to be doing) There is some difficulty with 1) as the libc implementation of statvfs has problems: "Do not use statvfs on systems with GNU libc on Linux, because that function stats all preceding entries in /proc/mounts, and that makes df hang if even one of the corresponding file systems is hard-mounted, but not available. statvfs in GNU libc on Hurd, BeOS, Haiku operates differently: it only makes a system call." This might have been fixed in libc since, so it's possible that coreutils developers are willing to using statvfs on linux. Another option would be to use the direct statfs (and statfs64) syscall interface on linux, bypassing the unnecessary cruft that libc put there. But this adds some complexity. Thanks, Miklos |
From: Jim M. <ji...@me...> - 2010-06-03 08:26:12
|
Miklos Szeredi wrote: ... > Notice that the real statfs syscall does have an f_frsize field, but > for some historical reason it is not exported in the library API. > > Options to fix this behavior would be: > > 1) fix df and stat in coreutils to use statvfs > 2) make sshfs set f_bsize to f_frsize > 3) make sshfs set f_frsize to f_bsize and recalculate stats > 4) change both f_frsize and f_bsize to a common value and > recalculate stats (this is what NFS appears to be doing) > > There is some difficulty with 1) as the libc implementation of statvfs > has problems: > > "Do not use statvfs on systems with GNU libc on Linux, because that function > stats all preceding entries in /proc/mounts, and that makes df hang if even > one of the corresponding file systems is hard-mounted, but not available. > statvfs in GNU libc on Hurd, BeOS, Haiku operates differently: it only makes > a system call." > > This might have been fixed in libc since, so it's possible that > coreutils developers are willing to using statvfs on linux. glibc's statvfs still appears to have this problem, so coreutils' df must not use that function. To demonstrate, I recompiled fsusage.c with -DSTAT_STATVFS, relinked df, and ran this: $ strace -e open,stat ./df . ... open("/proc/mounts", O_RDONLY) = 3 stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 stat(".", {st_mode=S_IFDIR|0755, st_size=65536, ...}) = 0 stat("/", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 stat("/boot", {st_mode=S_IFDIR|0555, st_size=3072, ...}) = 0 stat("/full", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0 stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 It still opens /proc/mounts and stats mount point directories until it finds one with a matching dev/inode. Note that it does limit its search, when possible, to file systems that are known to be of the same file system type, but that doesn't help when the target is NFS mounted. |
From: Miklos S. <mi...@sz...> - 2010-06-03 08:41:11
|
On Thu, 03 Jun 2010, Jim Meyering wrote: > Miklos Szeredi wrote: > ... > > Notice that the real statfs syscall does have an f_frsize field, but > > for some historical reason it is not exported in the library API. > > > > Options to fix this behavior would be: > > > > 1) fix df and stat in coreutils to use statvfs > > 2) make sshfs set f_bsize to f_frsize > > 3) make sshfs set f_frsize to f_bsize and recalculate stats > > 4) change both f_frsize and f_bsize to a common value and > > recalculate stats (this is what NFS appears to be doing) > > > > There is some difficulty with 1) as the libc implementation of statvfs > > has problems: > > > > "Do not use statvfs on systems with GNU libc on Linux, because that function > > stats all preceding entries in /proc/mounts, and that makes df hang if even > > one of the corresponding file systems is hard-mounted, but not available. > > statvfs in GNU libc on Hurd, BeOS, Haiku operates differently: it only makes > > a system call." > > > > This might have been fixed in libc since, so it's possible that > > coreutils developers are willing to using statvfs on linux. > > glibc's statvfs still appears to have this problem, so coreutils' df > must not use that function. To demonstrate, I recompiled fsusage.c > with -DSTAT_STATVFS, relinked df, and ran this: > > $ strace -e open,stat ./df . > ... > open("/proc/mounts", O_RDONLY) = 3 > stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 > stat(".", {st_mode=S_IFDIR|0755, st_size=65536, ...}) = 0 > stat("/", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 > stat("/boot", {st_mode=S_IFDIR|0555, st_size=3072, ...}) = 0 > stat("/full", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0 > stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 > > It still opens /proc/mounts and stats mount point directories > until it finds one with a matching dev/inode. Note that it > does limit its search, when possible, to file systems that are known > to be of the same file system type, but that doesn't help when > the target is NFS mounted. > Hmm, actually "struct statfs" on linux does have f_frsize, only the manpage doesn't document it. So correct thing would be to use that, no? Thanks, Miklos |
From: Miklos S. <mi...@sz...> - 2010-06-15 13:14:21
|
On Thu, 03 Jun 2010, Miklos Szeredi wrote: > Hmm, actually "struct statfs" on linux does have f_frsize, only the > manpage doesn't document it. So correct thing would be to use that, > no? Here's a patch that allows df(1) to correctly calculate the disk usage and displays f_frsize in stat(1). Thanks, Miklos --- gnulib/lib/fsusage.c | 7 +++++++ m4/stat-prog.m4 | 2 +- src/stat.c | 6 +++++- 3 files changed, 13 insertions(+), 2 deletions(-) Index: coreutils/m4/stat-prog.m4 =================================================================== --- coreutils.orig/m4/stat-prog.m4 2010-06-15 13:08:56.000000000 +0200 +++ coreutils/m4/stat-prog.m4 2010-06-15 13:47:51.000000000 +0200 @@ -71,7 +71,7 @@ AC_INCLUDES_DEFAULT [AC_DEFINE([STRUCT_STATVFS_F_FSID_IS_INTEGER], [1], [Define to 1 if the f_fsid member of struct statvfs is an integer.])]) else - AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type],,, + AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type, struct statfs.f_frsize],,, [$statfs_includes]) if test $ac_cv_header_OS_h != yes; then AC_COMPILE_IFELSE( Index: coreutils/src/stat.c =================================================================== --- coreutils.orig/src/stat.c 2010-06-15 13:08:56.000000000 +0200 +++ coreutils/src/stat.c 2010-06-15 13:09:02.000000000 +0200 @@ -118,7 +118,11 @@ statfs (char const *filename, struct fs_ # else # define STRUCT_STATVFS struct statfs # define STRUCT_STATXFS_F_FSID_IS_INTEGER STRUCT_STATFS_F_FSID_IS_INTEGER -# define STATFS_FRSIZE(S) 0 +# if HAVE_STRUCT_STATFS_F_FRSIZE +# define STATFS_FRSIZE(S) ((S)->f_frsize) +# else +# define STATFS_FRSIZE(S) 0 +# endif # endif #endif Index: coreutils/gnulib/lib/fsusage.c =================================================================== --- coreutils.orig/gnulib/lib/fsusage.c 2010-06-15 13:46:49.000000000 +0200 +++ coreutils/gnulib/lib/fsusage.c 2010-06-15 13:46:56.000000000 +0200 @@ -172,7 +172,14 @@ get_fs_usage (char const *file, char con if (statfs (file, &fsd) < 0) return -1; +#ifdef HAVE_STRUCT_STATFS_F_FRSIZE + /* f_frsize isn't guaranteed to be supported. */ + fsp->fsu_blocksize = (fsd.f_frsize + ? PROPAGATE_ALL_ONES (fsd.f_frsize) + : PROPAGATE_ALL_ONES (fsd.f_bsize)); +#else fsp->fsu_blocksize = PROPAGATE_ALL_ONES (fsd.f_bsize); +#endif # ifdef STATFS_TRUNCATES_BLOCK_COUNTS |
From: Jim M. <ji...@me...> - 2010-06-16 05:54:51
|
Miklos Szeredi wrote: > On Thu, 03 Jun 2010, Miklos Szeredi wrote: >> Hmm, actually "struct statfs" on linux does have f_frsize, only the >> manpage doesn't document it. So correct thing would be to use that, >> no? > > Here's a patch that allows df(1) to correctly calculate the disk usage > and displays f_frsize in stat(1). Hi Miklos, [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib] Thank you for the patch. However, I'm not sure it's portable enough. Do you know when the f_frsize member began to be useful? Without knowing that, I cannot judge whether this introduces a portability problem on older glibc, kernels or file systems. Even merely knowing that is probably not enough. Given the lack of documentation for that member, I suspect that any safe change would involve a run-time check to verify that infrastructure is new enough that the member is usable. > --- > gnulib/lib/fsusage.c | 7 +++++++ > m4/stat-prog.m4 | 2 +- > src/stat.c | 6 +++++- > 3 files changed, 13 insertions(+), 2 deletions(-) > > Index: coreutils/m4/stat-prog.m4 > =================================================================== > --- coreutils.orig/m4/stat-prog.m4 2010-06-15 13:08:56.000000000 +0200 > +++ coreutils/m4/stat-prog.m4 2010-06-15 13:47:51.000000000 +0200 > @@ -71,7 +71,7 @@ AC_INCLUDES_DEFAULT > [AC_DEFINE([STRUCT_STATVFS_F_FSID_IS_INTEGER], [1], > [Define to 1 if the f_fsid member of struct statvfs is an integer.])]) > else > - AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type],,, > + AC_CHECK_MEMBERS([struct statfs.f_namelen, struct statfs.f_type, struct statfs.f_frsize],,, > [$statfs_includes]) > if test $ac_cv_header_OS_h != yes; then > AC_COMPILE_IFELSE( > Index: coreutils/src/stat.c > =================================================================== > --- coreutils.orig/src/stat.c 2010-06-15 13:08:56.000000000 +0200 > +++ coreutils/src/stat.c 2010-06-15 13:09:02.000000000 +0200 > @@ -118,7 +118,11 @@ statfs (char const *filename, struct fs_ > # else > # define STRUCT_STATVFS struct statfs > # define STRUCT_STATXFS_F_FSID_IS_INTEGER STRUCT_STATFS_F_FSID_IS_INTEGER > -# define STATFS_FRSIZE(S) 0 > +# if HAVE_STRUCT_STATFS_F_FRSIZE > +# define STATFS_FRSIZE(S) ((S)->f_frsize) > +# else > +# define STATFS_FRSIZE(S) 0 > +# endif > # endif > #endif > > Index: coreutils/gnulib/lib/fsusage.c > =================================================================== > --- coreutils.orig/gnulib/lib/fsusage.c 2010-06-15 13:46:49.000000000 +0200 > +++ coreutils/gnulib/lib/fsusage.c 2010-06-15 13:46:56.000000000 +0200 > @@ -172,7 +172,14 @@ get_fs_usage (char const *file, char con > if (statfs (file, &fsd) < 0) > return -1; > > +#ifdef HAVE_STRUCT_STATFS_F_FRSIZE > + /* f_frsize isn't guaranteed to be supported. */ > + fsp->fsu_blocksize = (fsd.f_frsize > + ? PROPAGATE_ALL_ONES (fsd.f_frsize) > + : PROPAGATE_ALL_ONES (fsd.f_bsize)); > +#else > fsp->fsu_blocksize = PROPAGATE_ALL_ONES (fsd.f_bsize); > +#endif > > # ifdef STATFS_TRUNCATES_BLOCK_COUNTS > |
From: Miklos S. <mi...@sz...> - 2010-06-16 09:36:00
|
On Wed, 16 Jun 2010, Jim Meyering wrote: > Miklos Szeredi wrote: > > On Thu, 03 Jun 2010, Miklos Szeredi wrote: > >> Hmm, actually "struct statfs" on linux does have f_frsize, only the > >> manpage doesn't document it. So correct thing would be to use that, > >> no? > > > > Here's a patch that allows df(1) to correctly calculate the disk usage > > and displays f_frsize in stat(1). > > Hi Miklos, > > [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib] > > Thank you for the patch. > However, I'm not sure it's portable enough. > Do you know when the f_frsize member began to be useful? It was introduced in the linux-2.5 timeframe, so from linux-2.6.0 on it contains useful information. In 2.4.* and earlier it was part of the f_spare array and was zeroed out by the kernel. Checking f_spare value for zero and using f_bsize instead works fine for 2.4 and earlier kernels. > Without knowing that, I cannot judge whether this introduces > a portability problem on older glibc, kernels or file systems. statfs() is (and probably always was) a straight syscall wrapper in glibc. There's also statfs64 (used in case of _FILE_OFFSET_BITS=64) which is not necessarily a straight syscall, but has the same spare fields at the end of the structure. > Even merely knowing that is probably not enough. > > Given the lack of documentation for that member, I suspect > that any safe change would involve a run-time check to verify > that infrastructure is new enough that the member is usable. Already talked with the man pages maintainer about lack of documentation, and this will be fixed in new versions of the manual. In the end it's your call, but based on the level of backward compatibility built into the statfs structure I'd say such a change is perfectly safe. And there's a real bug that needs to be fixed or worked around that boils down to df(1) using the wrong field for blocksize on linux. Thanks, Miklos |
From: Jim M. <ji...@me...> - 2010-06-16 10:08:34
|
Miklos Szeredi wrote: > On Wed, 16 Jun 2010, Jim Meyering wrote: >> Miklos Szeredi wrote: >> > On Thu, 03 Jun 2010, Miklos Szeredi wrote: >> >> Hmm, actually "struct statfs" on linux does have f_frsize, only the >> >> manpage doesn't document it. So correct thing would be to use that, >> >> no? >> > >> > Here's a patch that allows df(1) to correctly calculate the disk usage >> > and displays f_frsize in stat(1). >> >> Hi Miklos, >> >> [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib] >> >> Thank you for the patch. >> However, I'm not sure it's portable enough. >> Do you know when the f_frsize member began to be useful? > > It was introduced in the linux-2.5 timeframe, so from linux-2.6.0 on > it contains useful information. In 2.4.* and earlier it was part of > the f_spare array and was zeroed out by the kernel. > > Checking f_spare value for zero and using f_bsize instead works fine > for 2.4 and earlier kernels. That sounds sufficient. Can you confirm that this works on a 2.4.x kernel? If so, I'll apply your changes to gnulib and coreutils. >> Without knowing that, I cannot judge whether this introduces >> a portability problem on older glibc, kernels or file systems. > > statfs() is (and probably always was) a straight syscall wrapper in > glibc. There's also statfs64 (used in case of _FILE_OFFSET_BITS=64) > which is not necessarily a straight syscall, but has the same spare > fields at the end of the structure. > >> Even merely knowing that is probably not enough. >> >> Given the lack of documentation for that member, I suspect >> that any safe change would involve a run-time check to verify >> that infrastructure is new enough that the member is usable. > > Already talked with the man pages maintainer about lack of > documentation, and this will be fixed in new versions of the manual. > > In the end it's your call, but based on the level of backward > compatibility built into the statfs structure I'd say such a change is > perfectly safe. And there's a real bug that needs to be fixed or > worked around that boils down to df(1) using the wrong field for > blocksize on linux. Thanks. |
From: Miklos S. <mi...@sz...> - 2010-06-16 15:43:12
|
On Wed, 16 Jun 2010, Jim Meyering wrote: > Miklos Szeredi wrote: > > On Wed, 16 Jun 2010, Jim Meyering wrote: > >> Miklos Szeredi wrote: > >> > On Thu, 03 Jun 2010, Miklos Szeredi wrote: > >> >> Hmm, actually "struct statfs" on linux does have f_frsize, only the > >> >> manpage doesn't document it. So correct thing would be to use that, > >> >> no? > >> > > >> > Here's a patch that allows df(1) to correctly calculate the disk usage > >> > and displays f_frsize in stat(1). > >> > >> Hi Miklos, > >> > >> [Cc'ing bug-gnulib, since all but src/stat.c are from gnulib] > >> > >> Thank you for the patch. > >> However, I'm not sure it's portable enough. > >> Do you know when the f_frsize member began to be useful? > > > > It was introduced in the linux-2.5 timeframe, so from linux-2.6.0 on > > it contains useful information. In 2.4.* and earlier it was part of > > the f_spare array and was zeroed out by the kernel. > > > > Checking f_spare value for zero and using f_bsize instead works fine > > for 2.4 and earlier kernels. > > That sounds sufficient. > Can you confirm that this works on a 2.4.x kernel? Okay, after some unsuccessful attempts to run a current system with a 2.4 kernel (userspace bootup immediately fails with "FATAL: kernel too old" message) I turned to a virtualized environment. I compiled the patched coreutils under debian/sarge (linux-2.4.27, glibc-2.3.6). This glibc already has f_frsize and configure detects it. I then verified that df(1) and stat(1) work correctly on this system. Then took the binaries to a 2.6 kernel and glibc-2.10.1 and verified that they indeed take a differing f_frsize and f_bsize into account. Thanks, Miklos |
From: Miklos S. <mi...@sz...> - 2011-09-27 14:12:16
|
Jim, Any news about this bug? This problem was again reported against a recent Debian release here: http://article.gmane.org/gmane.comp.file-systems.fuse.sshfs/1200 Thanks, Miklos |