From: Miklos S. <mi...@sz...> - 2007-08-13 08:32:10
|
> Great news! I think I nailed this one. As Andrew Morton noted opening at > Linuxworld, Linux bugs do rarely reproduce... > > The debug output can be found here http://q.gotdns.com/shout2.bz2 > That is a 3.7 MiB archive Thanks for the report. > Test Setup =============== > The client running sshfs with four command windows and a single sshfs > mount. The server has one 9.6 GiB file and five 2.1 GiB files on the > sshfs mount. > > > window 1: reading 9.6 GiB, then re-reading same file again > foxyboy@foxyboy-laptop:~$ cp /x/data/vistau Desktop/ > foxyboy@foxyboy-laptop:~$ cp /x/data/vistau Desktop/ > cp: reading `/x/data/vistau': Transport endpoint is not connected > cp: closing `/x/data/vistau': Transport endpoint is not connected > > > Window 2: sshd logging > ... > sshdebug script, single line: > sshfs foxyboy@1.0.0.8:/home/foxyboy/Desktop /x/data -o > allow_other,debug,sshfs_debug,loglevel=debug > ... > root@foxyboy-laptop:~# ./sshdebug 2>&1 | cat >shout2 > foxyboy@1.0.0.8's password: > root@foxyboy-laptop:~# > (when copy failed, this window back to prompt) > > > Windows 3: reading a sequence of 2.1 GiB files > root@foxyboy-laptop:~# cp /x/data/Vista_EN_Ultimat* Desktop/ > cp: reading `/x/data/Vista_EN_Ultimateab': Input/output error > cp: cannot stat `/x/data/Vista_EN_Ultimateac': Input/output error > cp: cannot stat `/x/data/Vista_EN_Ultimatead': Input/output error > cp: cannot stat `/x/data/Vista_EN_Ultimateae': Input/output error > root@foxyboy-laptop:~# > > > window 4: writing a read 2.1 GiB file back to the server folder under a > new name > foxyboy@foxyboy-laptop:~$ cp Desktop/Vista_EN_Ultimateaa /x/data/cpcp > cp: writing `/x/data/cpcp': Software caused connection abort > cp: closing `/x/data/cpcp': Transport endpoint is not connected > foxyboy@foxyboy-laptop:~$ This looks like a known bug in the sftp-server, that's been worked around in sshfs-1.8. Here's the relevant changelog entry: * OpenSSH sftp-server can read requests faster, than it processes them, when it's buffer is full it aborts. This can happen on a large upload to a slow server. Work around this by limiting the total size of outstanding reqests. Debian bug #365541. Tracked down by Thue Janus Kristensen Can you please try repeating the test with sshfs-1.8? > The timeline as command given to the client: > Window2 mounts sshfs to /x/data > Phase a. window 1 reads 9.6 GiB, concluded successfully > Phase b-1. window 1 repeats read of 9.6 GiB, in and outfiles the same as > first time > Phase b-2. window 3 reads sequence of 2.1 GiB files simultaneous with > b-1, between same folders, fails when on second file > Phase b-3. simultaneously with b-1 and b-2, window 4: when first 2.1 GiB > file read, that file is written back in the other direction to the > server under another name > Event c. at this point, with concurrently two reads and one write, sshfs > fails > > > More data ========== > Problem 1: all machines (below), when sequentially reading or writing a > file of 3.7 GiB or larger, an i/o error occurs > > Problem 2: On ubuntu 7.04, when the i/o error occurs, all sshfs mounts > are lost and you may get "endpoint disconnected" Both of these may be due to the sftp-server problem. > > Problem 3: On Ubuntu, if three parallel reads and writes are initiated, > sshfs may hang. Any sshfs access (such as filename completion) from a > shell then hangs that shell. exit: unmount -f of all sshfs mounts by > root. This doesn't ring a bell. Is it reproducible with sshfs-1.8? > Note: In all cases, it appears sshfs fully recovers once all sshfs > mounts have been unmounted. That's good. It means, that there's no problem with the kernel part of fuse, only in userspace. > Additional note (a client not used above): Pentium-4 HT, CentOS5, > 2.6.18-8.1.8.el5 #1 SMP: sshfs 1.8/2.6.5/2.6.5/7.8 corrupts the kernel > causing system crash (likely known issue) Looks like it. Can you try compliling the fuse kernel module from the fuse-2.6.5.tar.gz package: tar xfz fuse-2.6.5.tar.gz cd fuse-2.6.5 ./configure --enable-kernel-module make as root: make install rmmod fuse modprobe fuse dmesg | tail The dmesg should give you something like: fuse init (API version 7.8) fuse distribution version: 2.6.5 Thanks, Miklos |