From: Hans B. <han...@gm...> - 2011-05-30 09:34:28
Attachments:
fuse.txt
|
I have a problem with my FUSE application mounted over network using Netatalk/AFP. One of the users of the files system reports crashes in the AFP deamon on server side. It seems that using some other FUSE based filesystem, such as sshfs, work fine. The is the user reported system configuration: Fuse version: fusefs-libs-2.7.4 server os: FreeBSD client OSX 10.6.7 Crash is server side, afpd (netatalk) Just tested with sshfs (fuse) and that seems to work fine, more or less. No afpd crashes. Does anyone have any experience with the combination Netatalk and FUSE ? I read a few articles on the net and it seems some of them mention that there might be a compatibilty issue. But then, why does it work for sshfs ? I checked the FUSE log and can not see any obvious reasons to why the afpd crash with a signal 8. Also, the application log does not indicate any issues what so ever. Attached is the FUSE log from a -d run when the crash happens. The use case is a simple directory access, opened Finder and clicked the share announced with netatalk. Is there something I have missed in my application? I have both read and write support. Hans |
From: Hans B. <han...@gm...> - 2011-05-30 19:03:44
|
Ok, some progress. The file system did not implement statfs which was needed by Netatalk/AFP. But now there is a new problem, doing a simple 'cat' on the client side fails. Below is the FUSE log from the session, what could possibly be wrong this time ? There are only OPEN calls made but no READ calls?? And I can not see that any operation fails, really. The cat fails with: vandalon$ cat somefile cat: somefile: Invalid argument Hans unique: 0, opcode: LOOKUP (1), nodeid: 27, insize: 73 LOOKUP /Movies/somefolder/somefile NODEID: 40 unique: 0, error: 0 (Unknown error: 0), outsize: 136 unique: 1, opcode: LOOKUP (1), nodeid: 27, insize: 53 LOOKUP /Movies/somefolder/.AppleDouble unique: 1, error: -2 (No such file or directory), outsize: 16 unique: 0, opcode: LOOKUP (1), nodeid: 27, insize: 73 LOOKUP /Movies/somefolder/somefile NODEID: 40 unique: 0, error: 0 (Unknown error: 0), outsize: 136 unique: 1, opcode: GETATTR (3), nodeid: 27, insize: 40 unique: 1, error: 0 (Unknown error: 0), outsize: 112 unique: 0, opcode: LOOKUP (1), nodeid: 27, insize: 73 LOOKUP /Movies/somefolder/somefile NODEID: 40 unique: 0, error: 0 (Unknown error: 0), outsize: 136 unique: 1, opcode: OPEN (14), nodeid: 40, insize: 48 unique: 1, error: 0 (Unknown error: 0), outsize: 32 OPEN[9] flags: 0x102 /Movies/somefolder/somefile unique: 0, opcode: LOOKUP (1), nodeid: 27, insize: 53 LOOKUP /Movies/somefolder/.AppleDouble unique: 0, error: -2 (No such file or directory), outsize: 16 unique: 1, opcode: RELEASE (18), nodeid: 40, insize: 64 RELEASE+FLUSH[9] flags: 0x2 unique: 1, error: 0 (Unknown error: 0), outsize: 16 unique: 0, opcode: LOOKUP (1), nodeid: 27, insize: 73 LOOKUP /Movies/somefolder/somefile NODEID: 40 unique: 0, error: 0 (Unknown error: 0), outsize: 136 unique: 1, opcode: OPEN (14), nodeid: 40, insize: 48 unique: 1, error: 0 (Unknown error: 0), outsize: 32 OPEN[9] flags: 0x102 /Movies/somefolder/somefile unique: 0, opcode: RELEASE (18), nodeid: 40, insize: 64 RELEASE+FLUSH[9] flags: 0x2 unique: 0, error: 0 (Unknown error: 0), outsize: 16 Hans Beckérus skrev 2011-05-30 11:34: > I have a problem with my FUSE application mounted over network using > Netatalk/AFP. > One of the users of the files system reports crashes in the AFP deamon > on server side. > It seems that using some other FUSE based filesystem, such as sshfs, > work fine. > The is the user reported system configuration: > Fuse version: fusefs-libs-2.7.4 > server os: FreeBSD > client OSX 10.6.7 > Crash is server side, afpd (netatalk) > Just tested with sshfs (fuse) and that seems to work fine, more or > less. No afpd crashes. > Does anyone have any experience with the combination Netatalk and FUSE ? > I read a few articles on the net and it seems some of them mention > that there might be a compatibilty issue. > But then, why does it work for sshfs ? I checked the FUSE log and can > not see any obvious reasons to > why the afpd crash with a signal 8. > Also, the application log does not indicate any issues what so ever. > Attached is the FUSE log from a -d run when the crash happens. The use > case is a simple directory access, > opened Finder and clicked the share announced with netatalk. > Is there something I have missed in my application? I have both read > and write support. > Hans |
From: Hans B. <han...@gm...> - 2011-05-31 07:55:34
|
Seems the same problem exists for eg. sshfs too. Vicarius:tmp vandalon$ cat bla cat: bla: Invalid argument Does FUSE not provide support for Netatalk/AFP ? Or does AFP (or FUSE?) need to be configured in some special way to make it work ? |
From: Stef B. <st...@gm...> - 2011-05-31 08:16:22
|
I guess the open call does use a flag which is not supported, that's probably the reason you get a EINVAL error. This is just a guess, Stef 2011/5/31 Hans Beckérus <han...@gm...>: > Seems the same problem exists for eg. sshfs too. > > Vicarius:tmp vandalon$ cat bla > cat: bla: Invalid argument > Does FUSE not provide support for Netatalk/AFP ? Or does AFP (or FUSE?) need > to be configured in some special way to make it work ? > ------------------------------------------------------------------------------ > Simplify data backup and recovery for your virtual environment with vRanger. > Installation's a snap, and flexible recovery options mean your data is safe, > secure and there when you need it. Data protection magic? > Nope - It's vRanger. Get your free trial download today. > http://p.sf.net/sfu/quest-sfdev2dev > _______________________________________________ > fuse-devel mailing list > fus...@li... > https://lists.sourceforge.net/lists/listinfo/fuse-devel > |
From: Hans B. <han...@gm...> - 2011-05-31 08:30:36
|
Your guess is most probably close to the truth, but the flags used are 0x102. I interpret that as O_RDWR|O_CREAT, surely that should be supported? And I do not see this -EINVAL coming from the FUSE log. To me it looks like FUSE is accepting the OPEN call, or? If OPEN throws a EINVAL is must be happening behind the scene somewhere. Hans 2011/5/31 Stef Bon <st...@gm...> > I guess the open call does use a flag which is not supported, that's > probably the reason you get a EINVAL error. > > This is just a guess, > > Stef > > 2011/5/31 Hans Beckérus <han...@gm...>: > > Seems the same problem exists for eg. sshfs too. > > > > Vicarius:tmp vandalon$ cat bla > > cat: bla: Invalid argument > > Does FUSE not provide support for Netatalk/AFP ? Or does AFP (or FUSE?) > need > > to be configured in some special way to make it work ? > > > ------------------------------------------------------------------------------ > > Simplify data backup and recovery for your virtual environment with > vRanger. > > Installation's a snap, and flexible recovery options mean your data is > safe, > > secure and there when you need it. Data protection magic? > > Nope - It's vRanger. Get your free trial download today. > > http://p.sf.net/sfu/quest-sfdev2dev > > _______________________________________________ > > fuse-devel mailing list > > fus...@li... > > https://lists.sourceforge.net/lists/listinfo/fuse-devel > > > |
From: Hans B. <han...@gm...> - 2011-05-31 09:02:47
|
I think we are getting closer to the real issue here, Apple Double feature. Looking back at the log again made we wonder what is the story behind the .AppleDouble file, that returns -2. I think that is needed for Mac OS X, and if it can not find it, nor create it, I guess it might internally fail with EINVAL. But that is going to be difficult to work around, my file system can fallback to native mode and write to the underlying fs, but the core of the functionaltity works with virtual nodes that does not exits and that part of the fs is read only! That means that even if O_CREAT is used, it will always fail :( I guess I then need to create the .AppleDouble folder and place it somewhere that can be later picked up also for the virtual nodes. Sounds like some hard work ahead :( Hans 2011/5/31 Hans Beckérus <han...@gm...> > Your guess is most probably close to the truth, but the flags used are > 0x102. > I interpret that as O_RDWR|O_CREAT, surely that should be supported? > And I do not see this -EINVAL coming from the FUSE log. To me it looks like > FUSE is accepting the OPEN call, or? > If OPEN throws a EINVAL is must be happening behind the scene somewhere. > > Hans > 2011/5/31 Stef Bon <st...@gm...> > >> I guess the open call does use a flag which is not supported, that's >> probably the reason you get a EINVAL error. >> >> This is just a guess, >> >> Stef >> >> 2011/5/31 Hans Beckérus <han...@gm...>: >> > Seems the same problem exists for eg. sshfs too. >> > >> > Vicarius:tmp vandalon$ cat bla >> > cat: bla: Invalid argument >> > Does FUSE not provide support for Netatalk/AFP ? Or does AFP (or FUSE?) >> need >> > to be configured in some special way to make it work ? >> > >> ------------------------------------------------------------------------------ >> > Simplify data backup and recovery for your virtual environment with >> vRanger. >> > Installation's a snap, and flexible recovery options mean your data is >> safe, >> > secure and there when you need it. Data protection magic? >> > Nope - It's vRanger. Get your free trial download today. >> > http://p.sf.net/sfu/quest-sfdev2dev >> > _______________________________________________ >> > fuse-devel mailing list >> > fus...@li... >> > https://lists.sourceforge.net/lists/listinfo/fuse-devel >> > >> > > |
From: Stef B. <st...@gm...> - 2011-05-31 10:36:48
|
Please add some more info. A .AppleDouble file? Is it something like a .directory file, providing information for the desktop environment? And the backend/server is the OS X machine? Stef 2011/5/31 Hans Beckérus <han...@gm...>: > I think we are getting closer to the real issue here, Apple Double feature. |
From: Hans B. <han...@gm...> - 2011-05-31 10:45:15
|
Yes, the .AppleDouble folder is used by OS X to simulate the data fork and a resource fork for file systems not supporting it. The resource fork contains metadata about the file. The server is a OpenBSD machine, OS X is the client here. OS X implements AFP natively, but it is supported by the Netatalk/AFP implementation on OpenBSD. Hans 2011/5/31 Stef Bon <st...@gm...> > Please add some more info. A .AppleDouble file? > > Is it something like a .directory file, providing information for the > desktop environment? > And the backend/server is the OS X machine? > > Stef > > 2011/5/31 Hans Beckérus <han...@gm...>: > > I think we are getting closer to the real issue here, Apple Double > feature. > |
From: Charles B. <cc...@ac...> - 2011-05-31 10:53:53
|
Some backround info from lurker-land: .AppleDouble is a Netatalk-ism. Traditionally Apple filesystems have a "data fork" (what UNIX guys are used to) and a "resource fork" which is something like extended-attributes but potentially way bigger. Every object written by a Mac program will potentially have both parts. If you are on HFS or HFS+, not a problem. If you are trying to support this on a "normal" filesystem, something's gotta give. Enter the .AppleDouble directory. Every directory gets one. Each file in the directory is actually only half of the file from Apple's perspective: the data half. The resource half gets stuffed in a corresponding location in .AppleDouble. This means of course that it's very challenging to manipulate (move, copy) a file server-side without accidentally screwing up the connection to the appropriate resource fork. ccb On Tue, 2011-05-31 at 12:36 +0200, Stef Bon wrote: > Please add some more info. A .AppleDouble file? > > Is it something like a .directory file, providing information for the > desktop environment? > And the backend/server is the OS X machine? > > Stef > > 2011/5/31 Hans Beckérus <han...@gm...>: > > I think we are getting closer to the real issue here, Apple Double feature. > > ------------------------------------------------------------------------------ > Simplify data backup and recovery for your virtual environment with vRanger. > Installation's a snap, and flexible recovery options mean your data is safe, > secure and there when you need it. Data protection magic? > Nope - It's vRanger. Get your free trial download today. > http://p.sf.net/sfu/quest-sfdev2dev > _______________________________________________ > fuse-devel mailing list > fus...@li... > https://lists.sourceforge.net/lists/listinfo/fuse-devel |
From: Hans B. <han...@gm...> - 2011-05-31 11:14:09
|
Yes, this is what I also concluded. But you expressed it a way lot better ;) The problem now is that Netatalk (or actually AFP on the OS X) side seems to refuse opening the file properly if this .AppleDouble folder and "resource fork" is missing :( I think the user turned off that feature in Netatalk, but still, would not AFP on the OS X side still require it ? At least something makes it back-off in not finding the .AppleDouble folder. Or maybe it is Netatalk/AFP on the server side that actually refuse the connection? But the latter does not make sense if it is true what the user says and that he switched that feature off? Hans 2011/5/31 Charles Bennett <cc...@ac...> > > Some backround info from lurker-land: > > .AppleDouble is a Netatalk-ism. Traditionally Apple filesystems have a > "data fork" (what UNIX guys are used to) and a "resource fork" which is > something like extended-attributes but potentially way bigger. > > Every object written by a Mac program will potentially have both parts. > If you are on HFS or HFS+, not a problem. If you are trying to support > this on a "normal" filesystem, something's gotta give. Enter > the .AppleDouble directory. Every directory gets one. Each file in the > directory is actually only half of the file from Apple's perspective: > the data half. The resource half gets stuffed in a corresponding > location in .AppleDouble. > > This means of course that it's very challenging to manipulate (move, > copy) a file server-side without accidentally screwing up the connection > to the appropriate resource fork. > > > ccb > > > On Tue, 2011-05-31 at 12:36 +0200, Stef Bon wrote: > > Please add some more info. A .AppleDouble file? > > > > Is it something like a .directory file, providing information for the > > desktop environment? > > And the backend/server is the OS X machine? > > > > Stef > > > > 2011/5/31 Hans Beckérus <han...@gm...>: > > > I think we are getting closer to the real issue here, Apple Double > feature. > > > > > ------------------------------------------------------------------------------ > > Simplify data backup and recovery for your virtual environment with > vRanger. > > Installation's a snap, and flexible recovery options mean your data is > safe, > > secure and there when you need it. Data protection magic? > > Nope - It's vRanger. Get your free trial download today. > > http://p.sf.net/sfu/quest-sfdev2dev > > _______________________________________________ > > fuse-devel mailing list > > fus...@li... > > https://lists.sourceforge.net/lists/listinfo/fuse-devel > > > > > ------------------------------------------------------------------------------ > Simplify data backup and recovery for your virtual environment with > vRanger. > Installation's a snap, and flexible recovery options mean your data is > safe, > secure and there when you need it. Data protection magic? > Nope - It's vRanger. Get your free trial download today. > http://p.sf.net/sfu/quest-sfdev2dev > _______________________________________________ > fuse-devel mailing list > fus...@li... > https://lists.sourceforge.net/lists/listinfo/fuse-devel > |
From: Hans B. <han...@gm...> - 2011-05-31 16:04:58
|
I am really puzzled! This is from a tracing of the afpd calls: 11622 afpd CALL open(0x574fe0,O_RDWR|O_NOFOLLOW,<unused>0) 11622 afpd NAMI "somefile" 11622 afpd RET open 10/0xa 11622 afpd CALL open(0x57bba0,O_RDWR|O_NOFOLLOW,<unused>0) 11622 afpd NAMI ".AppleDouble/somefile" 11622 afpd RET open 11/0xb But why does FUSE present these as flags = 0x102 in OPEN ? That would be O_RDWR|O_CREAT. And, from what I understand FUSE should never call open() with O_CREAT? But to me it looks like it actually does that, or? Anyway, seems that one reason for the access to fail is this 11622 afpd RET writev 160/0xa0 11622 afpd CALL fcntl(0xb,F_GETLK,0xffffffffffffe6f0) 11622 afpd RET fcntl -1 errno 9 Bad file descriptor 11622 afpd CALL close(0xa) 11622 afpd RET close 0 11622 afpd CALL close(0xb) 11622 afpd RET close 0 So I need to implement the lock callback suddenly ? Or, is FUSE not taking care of this properly ? Hans 2011/5/31 Hans Beckérus <han...@gm...> > > Yes, this is what I also concluded. But you expressed it a way lot better ;) > The problem now is that Netatalk (or actually AFP on the OS X) side seems to refuse opening the file properly if this .AppleDouble folder and "resource fork" is missing :( I think the user turned off that feature in Netatalk, but still, would not AFP on the OS X side still require it ? > At least something makes it back-off in not finding the .AppleDouble folder. Or maybe it is Netatalk/AFP on the server side that actually refuse the connection? But the latter does not make sense if it is true what the user says and that he switched that feature off? > > Hans > 2011/5/31 Charles Bennett <cc...@ac...> >> >> Some backround info from lurker-land: >> >> .AppleDouble is a Netatalk-ism. Traditionally Apple filesystems have a >> "data fork" (what UNIX guys are used to) and a "resource fork" which is >> something like extended-attributes but potentially way bigger. >> >> Every object written by a Mac program will potentially have both parts. >> If you are on HFS or HFS+, not a problem. If you are trying to support >> this on a "normal" filesystem, something's gotta give. Enter >> the .AppleDouble directory. Every directory gets one. Each file in the >> directory is actually only half of the file from Apple's perspective: >> the data half. The resource half gets stuffed in a corresponding >> location in .AppleDouble. >> >> This means of course that it's very challenging to manipulate (move, >> copy) a file server-side without accidentally screwing up the connection >> to the appropriate resource fork. >> >> >> ccb >> >> >> On Tue, 2011-05-31 at 12:36 +0200, Stef Bon wrote: >> > Please add some more info. A .AppleDouble file? >> > >> > Is it something like a .directory file, providing information for the >> > desktop environment? >> > And the backend/server is the OS X machine? >> > >> > Stef >> > >> > 2011/5/31 Hans Beckérus <han...@gm...>: >> > > I think we are getting closer to the real issue here, Apple Double feature. >> > >> > ------------------------------------------------------------------------------ >> > Simplify data backup and recovery for your virtual environment with vRanger. >> > Installation's a snap, and flexible recovery options mean your data is safe, >> > secure and there when you need it. Data protection magic? >> > Nope - It's vRanger. Get your free trial download today. >> > http://p.sf.net/sfu/quest-sfdev2dev >> > _______________________________________________ >> > fuse-devel mailing list >> > fus...@li... >> > https://lists.sourceforge.net/lists/listinfo/fuse-devel >> >> >> >> ------------------------------------------------------------------------------ >> Simplify data backup and recovery for your virtual environment with vRanger. >> Installation's a snap, and flexible recovery options mean your data is safe, >> secure and there when you need it. Data protection magic? >> Nope - It's vRanger. Get your free trial download today. >> http://p.sf.net/sfu/quest-sfdev2dev >> _______________________________________________ >> fuse-devel mailing list >> fus...@li... >> https://lists.sourceforge.net/lists/listinfo/fuse-devel > |
From: Hans B. <han...@gm...> - 2011-05-31 17:11:27
|
This is geating more and more strange: These two (2) open calls I can see from afpd: 16402 afpd CALL open(0x57bba0,O_RDWR|O_CREAT,S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH) 16402 afpd NAMI "./.AppleDouble/.Parent" 16402 afpd RET open -1 errno 13 Permission denied 16402 afpd CALL open(0x57bba0,O_RDWR|O_CREAT,S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH) 16402 afpd NAMI ".AppleDouble/somefile" 16402 afpd RET open -1 errno 13 Permission denied But there is only one (1) OPEN call as can be seen from the FUSE log below. These two open calls never made it all the way for FUSE to see them ? unique: 0, opcode: OPEN (14), nodeid: 32, insize: 48 open flags: 0x102 /Movies/somefolder/somefile open[13] flags: 0x102 /Movies/somefolder/somefile unique: 0, success, outsize: 32 unique: 0, success, outsize: 32 Here the open flags seems to actually match, but... It says success (!?), but the path does not match so I do not think these are entries for the same open call. It rather think the below calls is what actually match the FUSE log. I have implemented the lock callback, but I never saw an entry for F_GETLK in the FUSE log. 16402 afpd CALL open(0x574fe0,O_RDWR|O_NOFOLLOW,<unused>0) 16402 afpd NAMI "somefile" 16402 afpd RET open 8 16402 afpd CALL open(0x57bba0,O_RDWR|O_NOFOLLOW,<unused>0) 16402 afpd NAMI ".AppleDouble/somefile" 16402 afpd RET open -1 errno 2 No such file or directory 16402 afpd CALL close(0x8) 16402 afpd RET close 0 16402 afpd CALL open(0x574fe0,O_RDWR|O_NOFOLLOW,<unused>0) 16402 afpd NAMI "somefile" 16402 afpd RET open 8 16402 afpd CALL fcntl(0x8,F_GETLK,0xffffffffffffe740) 16402 afpd RET fcntl -1 errno 9 Bad file descriptor 16402 afpd CALL close(0x8) 16402 afpd RET close 0 16402 afpd CALL fcntl(0x6,F_GETFL,0) 16402 afpd RET fcntl 2 16402 afpd CALL fcntl(0x6,F_SETFL,O_RDWR|O_NONBLOCK) 16402 afpd RET fcntl 0 |
From: Hans B. <han...@gm...> - 2011-05-31 22:03:49
|
The issue with .AppleDouble is solved (sort of), it was a permission issue with the .AppleDouble folder being created by the uid of the calling process. But, now the _real_ issue seems to be file locking. It plain and simple does not seem to work!? I get these logs from afpd 4941 afpd RET pread 741/0x2e5 4941 afpd CALL fcntl(0x8,F_GETLK,0xffffffffffffe110) 4941 afpd RET fcntl -1 errno 9 Bad file descriptor 4941 afpd CALL fcntl(0x8,F_GETLK,0xffffffffffffe110) 4941 afpd RET fcntl -1 errno 9 Bad file descriptor 4941 afpd CALL fcntl(0x8,F_GETLK,0xffffffffffffe110) 4941 afpd RET fcntl -1 errno 9 Bad file descriptor But I since I implement the lock callback I receive these requests and the fcntl() succeed and my file system returns 0. The command I get is 12, which means F_GETLK64. The flock structure looks like this in the call to lock and is the same (no modification) after the fcntl() call. struct flock { l_type = 2 l_whence = 0 l_start = 0 l_len = 0 l_pid = 0 } It's odd that l_type is received as 2 since that mean F_UNLCK (remove lock). But if this is supposed to work, why is afpd interpreting this as -1 with errno = 9 (EBADF) when I return 0 from the lock callback? Really hope someone can shed some light on this. Hans |
From: Hans B. <han...@gm...> - 2011-06-01 11:10:49
|
As I suspected, the lock call as received by the file system is just a side effect of some flush+release calls. The failed lock calls as seen from the afpd trace does not seem to ever enter the FUSE domain, or at least not the application of it. Also, the debug output from FUSE seems broken since these two traces comes hand in hand FUSE >>> "lock[9] F_SETLK F_UNLCK start: 0 len: 0 pid: 0 " appl >>> "bytes "lock cmd = 12, fd=4 " ans do not make sense, cmd = 12 is not F_SETLK it is F_GETLK64. Probably just a minor detail since the return of 0 indicates success in any case. So, still no answers to why afpd believes that the fcntl() call fails with errno 9 :( Hans |
From: Hans B. <han...@gm...> - 2011-06-05 07:35:14
|
Everything points in the direction of this being a problem unique to fuse4freebsd. |