Menu
▾
▴
#2778 file isdirectory causing variable corruption?
closed-fixed
5
2004-07-02
2004-06-25
No
After a call to "file isdirectory" certain paths no longer
seem to exist. I observed this with Tcl 8.4.x on Win32,
but not with Tcl 8.3.x. Something must be happening to
the variable that is passed to file isdirectory. If I make
sure that it's just a temporary string (as in TestFind2)
the problem does not occur.
Example:
proc TestFind1 {d f} {
set r1 [file exists [file join $d $f]]
puts "TF1: [file join $d $f] found: $r1"
puts "TF1: is dir a dir? [file isdirectory $d]"
set r2 [file exists [file join $d $f]]
puts "TF1: [file join $d $f] found: $r2"
puts ""
}
proc TestFind2 {d f} {
set r1 [file exists [file join $d $f]]
puts "TF2: [file join $d $f] found: $r1"
puts "TF2: is dir a dir? [file isdirectory [file join $d]]"
set r2 [file exists [file join $d $f]]
puts "TF2: [file join $d $f] found: $r2"
puts ""
}
file mkdir [file join a b c]
TestFind1 a [file join b . c]
TestFind2 a [file join b . c]
My output:
TF1: a/b/./c found: 1
TF1: is dir a dir? 1
TF1: a/b/./c found: 0
TF2: a/b/./c found: 1
TF2: is dir a dir? 1
TF2: a/b/./c found: 1
Discussion
Logged In: YES
user_id=113501
Results from NON-THREADED build of core-8-4-branch on
FreeBSD 4.10-STABLE:
TF1: a/b/./c found: 1
TF1: is dir a dir? 1
TF1: a/b/./c found: 1
TF2: a/b/./c found: 1
TF2: is dir a dir? 1
TF2: a/b/./c found: 1
Results from THREADED build of core-8-4-branch on FreeBSD
4.10-STABLE:
TF1: a/b/./c found: 1
TF1: is dir a dir? 1
TF1: a/b/./c found: 1
TF2: a/b/./c found: 1
TF2: is dir a dir? 1
TF2: a/b/./c found: 1
Results from NON-THREADED build of HEAD on FreeBSD 4.10-
STABLE:
TF1: a/b/./c found: 1
TF1: is dir a dir? 1
TF1: a/b/./c found: 1
TF2: a/b/./c found: 1
TF2: is dir a dir? 1
TF2: a/b/./c found: 1
Results from THREADED build of HEAD on FreeBSD 4.10-
STABLE:
TF1: a/b/./c found: 1
TF1: is dir a dir? 1
TF1: a/b/./c found: 1
TF2: a/b/./c found: 1
TF2: is dir a dir? 1
TF2: a/b/./c found: 1
Logged In: YES
user_id=32170
It looks from mistachkin's comments that this, if it was a
bug, has been fixed in both 8.4 and 8.5 branches.
Logged In: YES
user_id=113501
Is there any way we can find out what was causing this and
how it was fixed (in case the root problem has just been
obscured and not really fixed)?
Submitter: Was this in a released build and if so, what build?
Logged In: YES
user_id=1055638
My earlier tests were from the 8.4.5 and 8.4.6 releases on
Windows 2000, using threaded builds from both msys-mingw
and MSVC. I just tried the same test using the 8.5a1 release
(threaded builds on both compilers), and I got the following
output (even worse than before):
TF1: a/b/./c found: 1
TF1: is dir a dir? 1
TF1: a/b/./c found: 0
TF2: a/b/./c found: 1
TF2: is dir a dir? 1
TF2: a/b/./c found: 0
This is really strange. The way I found this was that I had a
set of scripts that was assembling paths from multiple
sources, and when I upgraded from 8.3 to 8.4 it stopped
working. This is the smallest test that still showed the
problem.
I cannot access the CVS repository from here, but this
evening I can pull it, and I can test using Linux as well.
Maybe this only happens on Win32.
Logged In: YES
user_id=79902
Sounds offhand like something is getting cached in the
internal rep of $d that is semantics-altering. This is
really really bad.
Logged In: YES
user_id=79902
BTW, the fact that it doesn't happen under Joe Mistachkin's
config might just mean that the problem is Win specific.
(It works for me with 8.4.1, but that doesn't necessarily
mean very much except for the fact that I ought to upgrade
sometime!)
Logged In: YES
user_id=32170
I'll look into this. I can reproduce it, so shouldn't be
too hard to solve. Only subtlety is that it fails in
different ways on 8.4 vs 8.5 (8.4 = 110 111, 8.5 = 110 110)
Logged In: YES
user_id=1055638
Just FYI: looks like 8.4.1 acts normally (111 111) and 8.4.2 is
when the problem started showing up (110 111)
Logged In: YES
user_id=72656
Confirmed that this is a bug in 8-4-head and HEAD of
2004-06-28 on Windows, but not on Linux (and that HEAD fails
with 110 110 as Vince noted).
Logged In: YES
user_id=79902
I suspect that the work done on path normalization fixes in
the HEAD allows the fault results to propagate further, but
that the problem is not that at all.
Currently, #1 suspicion points at bad information getting
attached to the Tcl_Obj inside $d, and that this bad info is
coming from a mistake in the Win native FS.
Logged In: YES
user_id=32170
In Tcl 8.5 the problem is caused here in tclPathObj.c (line
1590):
Tcl_AppendObjToObj(copy, fsPathPtr->normPathPtr);
/*
* Normalize the combined string, but only starting after
* the end of the previously normalized 'dir'. This should
* be much faster! We use 'cwdLen-1' so that we are
* already pointing at the dir-separator that we know about.
* The normalization code will actually start off directly
* after that separator.
*/
TclFSNormalizeToUniquePath(interp, copy, cwdLen-1,
(fsPathPtr->nativePathPtr == NULL ? &clientData : NULL));
where we are calling TclFSNormalizeToUniquePath contrary to
its documentation, with a string which may still contain
'./' or '../' sequences (from fsPathPtr->normPathPtr).
It's not really clear where the fix should be -- perhaps the
special 'joined' pathObj representation shouldn't be allowed
if the second part has '.' or '..' sequences, or else this
bit of code shown above should be cleverer, or else the
native normalization code should actually cope with the
existence of such sequences.
The problem occurs because the TclpObjNormalizePath routine
tries to find the 'long, unique path name' for each path
segment (one by one), and therefore turns 'a/b/./c' into
'a/b/b/c' (since the real name of '.' is 'b' in this case).
Similar bugs exist with '..'.
Lastly I should add that turning on 'TclNORM_LONG_PATH' in
tclWinFile.c seems to fix the problem, at least for the
cases I've tested (it activates a different version of the
normalization), but I think that other cases might still
trigger a failure.
Anyway, this is too hard for a quick fix, so it'll take me
some time (i.e. until I have some time) -- others are very
welcome to look into it.
diff -u with windows eols
Logged In: YES
user_id=32170
Here's a proposed patch for Tcl 8.5. Something similar (if
not identical) should work for 8.4
Logged In: YES
user_id=1055638
That patch seems to work for me with both the 8.4 and 8.5
versions. I didn't try anything other than that test yet
though, so hopefully it doesn't break anything else.
Logged In: YES
user_id=32170
Checked in fix for Tcl 8.5.
Leaving open since fix still needs applying to 8.4
Logged In: YES
user_id=32170
Backported to 8.4 branch.