Re: [sshfs] sshfs crashing and core dumps

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

On Tue, 10 Apr 2007 23:54:03 +0200
Miklos Szeredi <mi...@sz...> wrote:

> > sshfs is repeatedly crashing on me with segmentation fault, after
> > about a day of intense read-only operations.
> 
> Which fuse version?

fuse 2.6.3

> 
> > This happens even after applying the latest cvs patches (revisions
> > 1.79 and 1.81 of sshfs.c).
> > 
> > The main problem is that I can't run sshfs under gdb (it crashes
> > immediately for some reason) and I can't get a core dump, so I can't
> > help with debugging. Finally I figured out that sshfs can't write a
> > core dump because it chdir()'s to / where it has no write
> > permissions. Is there a particular reason for doing that?
> 
> Yes, daemons usually change their working directory to / so that they
> let the current directory be unmounted.
> 
> You can enable corefiles by doing something like this:
> 
>   echo "/var/tmp/core.%p" > /proc/sys/kernel/core_pattern
> 
> Thanks,
> Miklos

Thanks Miklos, I have already done that (although I still don't agree with this behaviour because if I hadn't root access I wouldn't be able to do it) and got 2 core dumps, which are identically cryptic to me. First of all here is how I call sshfs: 

$ /var/abs/local/sshfs/src/sshfs-fuse-1.7/sshfs -f -o workaround=all -o ro -o allow_other -o MACs=hmac-md5-96 -o Ciphers=arcfour user@host:/remote/dir /local/dir/
user@host's password:
(after about 1 day of 5MB/s read-only traffic...)
Segmentation fault (core dumped)

The first core file is 216 MB and the other is 31 MB, I don't know why they differ so much in size, the only thing I changed on the second case is doubling the stack size from 8MB to 16MB. I should note that the executables and the libraries are not stripped (at least 'file' command reports so). Anyway here is what I did, please tell me if I can do more: 

$ ls -l /tmp/core.*
-rw------- 1 jimis users 225640448 2007-04-10 10:37 /tmp/core.15212
-rw------- 1 jimis users  31981568 2007-04-11 05:53 /tmp/core.3482

$ gdb -c /tmp/core.15212
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
Core was generated by `/var/abs/local/sshfs/src/sshfs-fuse-1.7/sshfs -f -o workaround=all -o ro -o all'.
Program terminated with signal 11, Segmentation fault.
#0  0xb7e9bd87 in ?? ()
(gdb) info thread
  12 process 15212  0xb7f11410 in __kernel_vsyscall ()
  11 process 15218  0xb7f11410 in __kernel_vsyscall ()
  10 process 26948  0xb7f11410 in __kernel_vsyscall ()
  9 process 26952  0xb7f11410 in __kernel_vsyscall ()
  8 process 26955  0xb7f11410 in __kernel_vsyscall ()
  7 process 26963  0xb7f11410 in __kernel_vsyscall ()
  6 process 26972  0xb7f11410 in __kernel_vsyscall ()
  5 process 26980  0xb7f11410 in __kernel_vsyscall ()
  4 process 26981  0xb7f11410 in __kernel_vsyscall ()
  3 process 26982  0xb7f11410 in __kernel_vsyscall ()
  2 process 27003  0xb7f11410 in __kernel_vsyscall ()
* 1 process 26967  0xb7e9bd87 in ?? ()
(gdb) thread apply all bt

Thread 12 (process 15212):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4ee31 in ?? ()

Thread 11 (process 15218):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 10 (process 26948):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 9 (process 26952):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 9 (process 26952):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 8 (process 26955):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4d87e in ?? ()

Thread 7 (process 26963):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4d87e in ?? ()

Thread 6 (process 26972):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 5 (process 26980):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4d87e in ?? ()

Thread 4 (process 26981):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 3 (process 26982):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4e3fb in ?? ()

Thread 2 (process 27003):
#0  0xb7f11410 in __kernel_vsyscall ()
#1  0xb7e4d87e in ?? ()
---Type <return> to continue, or q <return> to quit---

Thread 1 (process 26967):
#0  0xb7e9bd87 in ?? ()
(gdb) info registers
eax            0x8054888        134563976
ecx            0xda6cc2e4       -630406428
edx            0x8054898        134563992
ebx            0xb7eeb3e0       -1209093152
esp            0xb52fe080       0xb52fe080
ebp            0xb52fe0b8       0xb52fe0b8
esi            0xa2627b0        170272688
edi            0x1      1
eip            0xb7e9bd87       0xb7e9bd87
eflags         0x10286  [ PF SF IF RF ]
cs             0x73     115
ss             0x7b     123
ds             0xffff007b       -65413
es             0xc010007b       -1072693125
fs             0x0      0
gs             0xc0100033       -1072693197
(gdb) quit

$ gdb -c /tmp/core.3482 
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
Failed to read a valid object file image from memory.
Core was generated by `/var/abs/local/sshfs/src/sshfs-fuse-1.7/sshfs -f -o workaround=all -o ro -o all'.
Program terminated with signal 11, Segmentation fault.
#0  0xb7e96ff8 in ?? ()
(gdb) info thread
  12 process 3482  0xb7f0d410 in ?? ()
  11 process 3488  0xb7f0d410 in ?? ()
  10 process 20925  0xb7f0d410 in ?? ()
  9 process 21064  0xb7f0d410 in ?? ()
  8 process 21067  0xb7f0d410 in ?? ()
  7 process 21068  0xb7f0d410 in ?? ()
  6 process 21072  0xb7f0d410 in ?? ()
  5 process 21073  0xb7f0d410 in ?? ()
  4 process 21075  0xb7f0d410 in ?? ()
  3 process 21076  0xb7f0d410 in ?? ()
  2 process 21077  0xb7f0d410 in ?? ()
* 1 process 21069  0xb7e96ff8 in ?? ()

(gdb) thread apply all bt

Thread 12 (process 3482):
#0  0xb7f0d410 in ?? ()

Thread 11 (process 3488):
#0  0xb7f0d410 in ?? ()

Thread 10 (process 20925):
#0  0xb7f0d410 in ?? ()

Thread 9 (process 21064):
#0  0xb7f0d410 in ?? ()

Thread 8 (process 21067):
#0  0xb7f0d410 in ?? ()

Thread 7 (process 21068):
#0  0xb7f0d410 in ?? ()

Thread 6 (process 21072):
#0  0xb7f0d410 in ?? ()

Thread 5 (process 21073):
#0  0xb7f0d410 in ?? ()

Thread 4 (process 21075):
#0  0xb7f0d410 in ?? ()

Thread 3 (process 21076):
#0  0xb7f0d410 in ?? ()

Thread 2 (process 21077):
#0  0xb7f0d410 in ?? ()

Thread 1 (process 21069):
#0  0xb7e96ff8 in ?? ()
(gdb) info registers 
eax            0x0      0
ecx            0xaf8c57d0       -1349756976
edx            0xb217cc30       -1307063248
ebx            0xb7ee73e0       -1209109536
esp            0xb4cc8f60       0xb4cc8f60
ebp            0xb4cc8fb8       0xb4cc8fb8
esi            0xb23f3a10       -1304479216
edi            0xb2142f20       -1307300064
eip            0xb7e96ff8       0xb7e96ff8
eflags         0x10292  [ AF SF IF RF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0xc0100033       -1072693197

Dimitris