#5217 Incompatible behaviour of "file exists"

current: 8.6.0
closed-fixed
8
2013-03-19
2013-03-18
Uwe Schmitz
No

There is an incompatibilty concerning "file exists"
in 8.6:

% puts $tcl_patchLevel
8.6.0
% glob *
x.tcl
% file exists *.tcl
1

Whereas in 8.5:
% puts $tcl_patchLevel
8.5.12
% glob *
x.tcl
% file exists *.tcl
0

platform: Win7_x64,
tcl-binaries: ActiveState.

This breaks backward compatibility.

Discussion

  • Donal K. Fellows

    • assigned_to: dkf --> nijtmans
    • priority: 5 --> 8
     
  • Donal K. Fellows

    Definitely a problem in implementation of NativeAccess() in tclWinFile.c (the generic layers over that do the right thing) but I really don't know the relevant API. Is this related to 2893771? (Committed on 2009-11-24 in https://core.tcl.tk/tcl/info/02224b9ef68c so it fits with being an 8.6.* issue.) We're feeding a filename into FindFirstFile which is the directory entry enumerator, which in turn is the only thing that can do that sort of thing with a glob patter AIUI. My knowledge of the deep ins-and-outs of Win32 is too small for me to go further.

    Assigning to someone who knows more about the relevant bits than me. Perhaps just testing for glob metacharacters in the name would be enough? I vaguely remember them as being always forbidden in filenames...

     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18

    I will have a look. Thanks!

     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18

    On UNIX (actually Cygwin):
    $ tclsh8.5
    % puts $tcl_patchLevel
    8.5.13
    % glob *.tcl
    x.tcl
    % file exists *.tcl
    0

    $ tclsh8.6
    % puts $tcl_patchLevel
    8.6.0
    % glob *.tcl
    x.tcl
    % file exists *.tcl
    0
    % exit

    Conclusion: the fact that using the '*' character this way
    in [file exists] worked at all in win32 was an undocumented
    feature and is non-portable. On UNIX, the '*' character can be
    used in filenames like any other character, only it must be
    escaped if the filename goes through a shell.

     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18
    • status: open --> pending-invalid
     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18
    • status: pending-invalid --> closed-invalid
     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18
    • milestone: 4024681 --> 3997836
     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18

    Apparently, the fix is already backported to 8.4 and 8.5 as well:

    >tclsh84
    % puts $tcl_patchLevel
    8.4.19
    % glob *.tcl
    x.tcl
    % file exists *.tcl
    0
    % exit
    >tclsh85
    % puts $tcl_patchLevel
    8.5.13
    % glob *.tcl
    x.tcl
    % file exists *.tcl
    0
    % exit

     
  • Donal K. Fellows

    I think you're misreading. This is (supposedly) a regression in 8.6 whereby, if 'x.tcl' exists in the current directory (but '*.tcl' does not), [file exists *.tcl] returns 1.

    If there's a problem, it exists in that NativeAccess function. (I cannot confirm or deny; I don't have the right platform. I also don't know if the Cygwin build uses the same code; if not, it doesn't count.) I suspect that if there is a problem, it is inside there and related to the use of an API that does expansion. FindFirstFile is such a function, and it is definitely used in the NativeAccess code; the commit I indicated appears to be the one that introduced it. FindFirstFile is documented here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa364418\(v=vs.85).aspx

    Moreover, I've confirmed that * is an illegal character in a filename on Windows. Search for "reserved characters" in http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247\(v=vs.85).aspx

    At the very least, we could have a test for this! In fact, I'll write one. [file exists] definitely shouldn't ever do wildcard expansion behind the scenes on any platform.

     
  • Donal K. Fellows

    • milestone: 3997836 --> current: 8.6.0
    • status: closed-invalid --> open-invalid
     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18

    > I think you're misreading. This is (supposedly) a regression in 8.6
    > whereby, if 'x.tcl' exists in the current directory (but '*.tcl' does not),
    > [file exists *.tcl] returns 1.
    If it's a regression, then it's a regression in 8.5.13 as well, because
    I cannot reproduce it any more. But I think we fully agree.
    > At the very least, we could have a test for this! In fact, I'll write one.
    > [file exists] definitely shouldn't ever do wildcard expansion behind the
    > scenes on any platform.
    I fully agree with that: [file exists] shouldn't do wildcard expansion
    on any platform! That means: this change was not a regression,
    it was a bug-fix. I welcome your testcase!

     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18
    • status: open-invalid --> closed-invalid
     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-18

    Thanks, Donal, for your test-case! I tried it on win32 too, and it works fine.

    Closing now.

     
  • Donal K. Fellows

    Test committed. If that fails on Windows with a non-Cygwin build, we know there's a serious problem. (I suspect this is all caused by the fact that you can't do an access()-equivalent check on a file easily without running into problems with locking. Which seems supremely stupid.)

     
  • Donal K. Fellows

    • status: closed-invalid --> open-invalid
     
  • Donal K. Fellows

    • status: open-invalid --> pending-invalid
     
  • Donal K. Fellows

    Uwe reports on comp.lang.tcl in message <51479973$0$6640$9b4e6d93@newsspool2.arcor-online.net> that:

    here my result with cmdAH.test from trunk
    (ActiveState tcl8.6 binaries under win7_x64 as before):

    >tclsh cmdAH.test

    ==== cmdAH-19.12 Bug 3608360: [file exists] mustn't do globbing FAILED
    ==== Contents of test case:

    list [file exists foo.bar] [file exists *.bar]

    ---- Result was:
    1 1
    ---- Result should have been (exact matching):
    1 0
    ==== cmdAH-19.12 FAILED

    cmdAH.test: Total 308 Passed 132 Skipped 175 Failed 1
    Number of tests skipped for each constraint:
    4 nonPortable
    1 tempNotWin
    3 testchmod
    146 testsetplatform
    1 testvolumetype
    20 unix

    -----
    Summary: it's a real bug in that configuration.

     
  • Donal K. Fellows

    • status: pending-invalid --> open
     
  • Donal K. Fellows

    See bug-3608360 for a possible fix. (Warning: untested!)

     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-19
    • status: open --> closed-fixed
     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-19

    Fixed now in all branches.

     
  • Jan Nijtmans

    Jan Nijtmans - 2013-03-19

    >Fixed now in all branches.
    I mean on trunk, of course. core-8-5-branch and earlier never had the bug.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks