From: SourceForge.net <no...@so...> - 2004-04-30 16:16:09
|
Bugs item #929534, was opened at 2004-04-05 13:43 Message generated for change (Comment added) made by das You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=110894&aid=929534&group_id=10894 Category: 36. File System Group: current: 8.4.6 Status: Open Resolution: None Priority: 5 Submitted By: tauvan (tauvan) Assigned to: Daniel A. Steffen (das) Summary: OS X: filesystem FAILED link normalisation Initial Comment: filesystem-1.2 FAILED filesystem-1.3 FAILED filesystem-1.4 FAILED filesystem-1.7 FAILED filesystem-1.9 FAILED filesystem-1.10 FAILED filesystem-1.11 FAILED these occur both local and root You can see more detailed description under 20 failures bug machine: mac ppc G4 (1st PCI 400Mhz) OSX 10.1.5 My solution attempt at trying to be helpfull attached but needs to be "Tcl compatable" However, since I'm trying if I have questions "not bugs" where do I go? ---------------------------------------------------------------------- >Comment By: Daniel A. Steffen (das) Date: 2004-05-01 02:16 Message: Logged In: YES user_id=90580 it should be easy, just do the following in Terminal cvs -d:pserver:ano...@cv...:/cvsroot/tcl co tcl open tcl #apply your changes cd tcl cvs diff -u -p -N > ../tauvan.patch ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2004-05-01 02:07 Message: Logged In: NO I'll try, but never done before so it may take a while. Sorry. But it will be a great experience, I love new knowledge! ---------------------------------------------------------------------- Comment By: Daniel A. Steffen (das) Date: 2004-05-01 01:48 Message: Logged In: YES user_id=90580 Could you provide a cvs diff against tcl HEAD please? otherwise this is unnecessarily painful to test, e.g. I'm not sure what to do with the attached files... I want to be able to do cd tcl && patch -Np0 < tauvan.patch && cd unix && make test && cd .. && patch -Rp0 < tauvan.patch ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-05-01 01:44 Message: Logged In: YES user_id=1011552 Here is a wrapper code for inserting into the current TclpObjNormalizePath(). ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-05-01 01:38 Message: Logged In: YES user_id=1011552 Here's the code not a patch for TclpObjNormalizePath(). One normalized difference to note is in cmdAH.test "home:test" is passed, returned result is "/home/test". As this is the function pointer for unix and osx, and os9 is being dropped, I didn't think a problem, both cases returned 0 as startAt. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2004-04-30 17:09 Message: Logged In: NO I think we're very happy to test a different implementation of TclpObjNormalizePath() which doesn't need realpath(), especially with the speed improvements you cite, provided it works on all Unix systems. I'm actually away for the next week, so can't test anything, but perhaps someone else can give this a go? ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-30 15:31 Message: Logged In: YES user_id=1011552 The errno for both realpath fails is 13, EACCES. The second thing to note is that realpath doesn't necessarily point to the offending path segment in its return, sometimes it does and sometimes it doesn't. Realpath depends on several unix functions that are access sensitive. fchdir, chdir, and getcwd are access sensitive. If you are willing to create a patch and try something a little different, I've got some code that fixes the fileSystem.test fails, normalizes paths, consistently points to the last good path segment, always has nextCheckpoint point to the last good separator, runs at minimum 10% faster with threads enabled and in the 20s with threads disabled. It solves at least 4 realpath issues, removes the security issue of access(), etc, etc. And if you wish to keep the two realpaths and the slow way, I can provide a wrapper to insert code which only reduces speed increases by about 1%. ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2004-04-30 08:08 Message: Logged In: YES user_id=32170 First off, thanks for the *very* clear summary! My first question would be that if '/tmp/tcl.foo.dir' is the first thing passed to realpath, it ought to return '/private/tmp/tcl.foo.dir', and if it returns NULL then we have a problem (I wonder what 'errno' is set to?). The second issue is that it sounds as if file normalization (in the current code) is affected by what permissions are available on the sub-directories of the path, in that calls to 'access()' may fail, not because the given directory doesn't exist (which would signal that normalization is complete), but rather because there are inadequate permissions.... Is that true? I'm not sure what to do about that (I'm not particularly Unix-savvy, so it would be helpful to get someone else's input here, since what I'm writing might be totally wrong). Daniel, do you know much about this? ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-30 06:08 Message: Logged In: YES user_id=1011552 /tmp is a symlink to /private/tmp. /tmp -> private/tmp At the time when Tcl_EvalEx: script = file exists /tmp/tcl.foo.dir/file]\n\n /private/tmp/tcl.foo.dir/file exists but can only be read as root. /private/tmp can be read as local and root. /tmp can be read as local and root. The nativePath in TclpObjNormalizePath passed to the first realpath is /tmp/tcl.foo.dir. It returns NULL with normPath set to /tmp. The slow way then pieces /tmp/tcl.foo.dir/file with accesses = OK. The second realpath fails with normPath set to /tmp. This leaves fsPathPtr->normPathPtr = /tmp/tcl.foo.dir/file. The true normalized name is /private/tmp/tcl.foo.dir/file and is not being reported. The test succeeds but normalization fails as /private/tmp/tcl.foo.dir/ file and /tmp/tcl.foo.dir/file are the same file. ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2004-04-30 04:57 Message: Logged In: YES user_id=32170 Can you explain further? Does /private/tmp exist? Is /tmp a symlink to /private/tmp? If so, then either both: /tmp/tcl.foo.dir/file and /private/tmp/tcl.foo.dir/file exist, or neither of them exists. Which of these cases is it? What Tcl command in the test is triggering the issue you describe? test cmdAH-19.11 {Tcl_FileObjCmd: exists} {unixOnly notRoot} { file delete -force /tmp/tcl.foo.dir/file file delete -force /tmp/tcl.foo.dir makeDirectory /tmp/tcl.foo.dir makeFile 12345 /tmp/tcl.foo.dir/file file attributes /tmp/tcl.foo.dir -permissions 0000 set result [file exists /tmp/tcl.foo.dir/file] file attributes /tmp/tcl.foo.dir -permissions 0775 removeFile /tmp/tcl.foo.dir/file removeDirectory /tmp/tcl.foo.dir set result } 0 When you say 'it fails', what is it? Sorry, but there's just not enough information here to really help much.... ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-30 04:45 Message: Logged In: YES user_id=1011552 Not sure if this is a seperate bug, or even a bug, but the only dumb question is the one not asked. In cmdAH.test, test 19.11, TclpObjNormalizePath is asked to normalize "/tmp/tcl.foo.dir/file" from CheckAccess. Currently, TclpObjNormalizePath returns "/tmp/tcl.foo.dir/file" because realpath fails, assumed due to permission. But, if you do normalize the path to "/private/tmp/tcl.foo.dir/file" it fails because TclpObjAccess fails. So I'm not sure if TclpObjNormalizePath should be checking access or provide a normalized path? ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2004-04-22 00:44 Message: Logged In: YES user_id=32170 Ok, in that case you should return the full path (including /foo), but the 'startAt' that should be returned should point to the '/' before foo -- thereby saying that the path was understand (and normalized) up to that point. In fact 'startAt', as returned, should always point to a '/'. (I think that's correct - no time to look at the code/docs right now, sorry). ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-21 23:38 Message: Logged In: YES user_id=1011552 Maybe I phrase wrong? The path to normalize entering TclpObjNormalizePath is "/Users/ steven/Desktop/tclmyproj/tcl8.4.6/macosx/build/gorp.file/foo". At that instant it is passed there is a "/Users/steven/Desktop/ tclmyproj/tcl8.4.6/macosx/build/gorp.file", but my new code errors on "/Users/steven/Desktop/tclmyproj/tcl8.4.6/macosx/build/gorp.file/ foo" exits with error "ENOTDIR" because gorp.file is not a directory. It returns a NULL string with startAt of 53. It is then passed "/Users/ steven/Desktop/tclmyproj/tcl8.4.6/macosx/build/link.file/foo" exiting "ENOTDIR" with NULL and startAt of 53. So I am trying to be sure of what is being requested a that precise instant in the program. Is it supposed to return "/Users/steven/ Desktop/tclmyproj/tcl8.4.6/macosx/build/gorp.file/foo" possibily allowing someone to try an operation on that file? If gorp.file was a directory and link.file a directory it passes, hence, should I be passing a string ignoring the information found and/or what should be the startAt? ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2004-04-21 17:09 Message: Logged In: YES user_id=32170 As you say, you're better off trying to create a replacement for realpath, since the rest of the code is known to work ok. As for: [file normalize [file join gorp.file foo]] Well, file join will return gorp.file/foo, and that must then be normalized partly by other functions and partly by the code you're trying to write. You probably want to read the documentation of 'file join'... I don't see how the code that you write should even notice whether it's being given: file normalize gorp.file/foo or file normalize [file join gorp.file/foo] Since your code is only inside the 'file normalize' routines.... Anyway, 'file join' is very well documented (more so than we can explain here is a few lines), so please do read that, and good luck with the Realpath implementation. ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-21 13:34 Message: Logged In: YES user_id=1011552 Hi I was doing some testing, I haven't mention the code I sent you has one small bug, but it tests identical to tests with using realpath, except for startAt return value. Now, I created new code that handles exceptions, a realpath replacement, and luckily got some FAILS. So I need some clarifaction on what the tests are asking or what join is permitted to do. As an example (from test filesystem-1.2): [file normalize [file join gorp.file foo]] questions: should grop.file be turned into a dir to yield gorp.file/foo? : if gorp.file is turned into a dir is link still valid? : should it be gorp.filefoo? :am I just loonie? my return values from TclpObjNormalizePath show ....../gorp.file/foo Thanks ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-20 23:58 Message: Logged In: YES user_id=1011552 Not sure if this helps, but as tcl is mentioned on a list of realpath users BSD issued a notice about a security issue, and the new realpath mentions something about thread safe. ---------------------------------------------------------------------- Comment By: Daniel A. Steffen (das) Date: 2004-04-20 17:14 Message: Logged In: YES user_id=90580 sorry for the delay this a known problem with threaded tcl on OSX; the OSX realpath is not threadsafe (it uses chdir internally and the current dir is process global...), so we've had to disable it in the threaded build: 2003-05-13 Daniel Steffen <da...@us...> * unix/tclUnixPort.h: worked around the issue of realpath() not being thread-safe on Mac OS X by defining NO_REALPATH for threaded builds on Mac OS X. [Bug 711232] see https://sourceforge.net/tracker/? func=detail&atid=110894&aid=711232&group_id=10894 at the time, Zoran said he'd submitted a bugreport with Apple about this, don't know the current status of that though; Jim? to fix this in tcl, TclpObjNormalizePath() would need to be implemented properly for systems without realpath /* * We should really now convert this to a canonical path. We do * that with 'realpath' if we have it available. Otherwise we could * step through every single path component, checking whether it is a * symlink, but that would be a lot of work, and most modern OSes * have 'realpath'. */ (in fact realpath on OSX does exactly this, but in an thread-unsafe manner...) or we could add a crossplatform realpath() to compat. tauvan, for questions that are "not bugs" use the tcl-mac mailing list: https://lists.sourceforge.net/lists/listinfo/tcl-mac ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2004-04-08 19:15 Message: Logged In: YES user_id=32170 This is somewhat beyond me, but I think Daniel will be able to shed some light (when he gets back from cycling next week ;-). ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-08 17:21 Message: Logged In: YES user_id=1011552 Sorry to do this but I just found so new data that may or may not just make this a OSX issue. In trying to gather more info on another bug, I had this idea of doing a straight unix build, not using macosx makefile nor my build system. Well I accedently forgot to "--enable-threads". After running the tests, I noticed there was not 27 failures. All fileSystem failures disappeared. I then ran "--enable-threads" they appeared. Then, tried without enabled again, disappeared. I also verified that the test did not skip because of testthread. I only ran this as local user, but thought I should pass this on immediatly, because it changes things. ---------------------------------------------------------------------- Comment By: tauvan (tauvan) Date: 2004-04-06 00:38 Message: Logged In: YES user_id=1011552 ac_cv_func_realpath=${ac_cv_func_realpath=yes} However, in tclUnixPort.h; #ifdef MAC_OSX_TCL /* * On Mac OS X, realpath is currently not * thread safe, c.f. SF bug # 711232. */ #define NO_REALPATH #endif ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2004-04-05 18:58 Message: Logged In: YES user_id=32170 Thanks for the report -- we'll look into these. The first thing that occurs to me is whether 'realpath' is supported or not on OS X 10.1.5. Anyway, if you have questions, either comp.lang.tcl or the mac-tcl mailing list is the place. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=110894&aid=929534&group_id=10894 |