#4064 duplicate inodes from file stat on windows

obsolete: 8.5.3
closed-fixed
9
2012-03-27
2008-07-11
No

On windows, the inodes as reported in the ino field of the array produced by 'file stat' seem to be limited to a 16bit value.

The result is that many files end up sharing an inode as far as Tcl is concerned.

This can result in 'file rename -force' silently failing to perform the operation.
Such a clash is in general highly intermittent, but can in certain circumstances occur repeatedly in a particular folder.

(Renaming a file into place over an existing file is a mechanism that is supposed to increase data safety by not having the final data file being directly opened for writing. The effect of the duplicate inodes is that this problem may go undetected for a long time, and silently corrupt data)

I suspect (but haven't been able to verify) that this may occasionally have other strange effects during file creation using Open. I have noticed a situation in a folder where Tcl is writing files where 2 files appear to have not only the same inode, but the same name (including capitalisation).

Discussion

  • Julian Noble

    Julian Noble - 2008-07-11
    • priority: 5 --> 9
     
  • Donal K. Fellows

    • labels: --> 37. File System
    • assigned_to: nobody --> vincentdarley
     
  • Julian Noble

    Julian Noble - 2008-07-11

    Logged In: YES
    user_id=598324
    Originator: YES

    verified that 'file copy -force' will also fail silently if the inodes match.

     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21

    Logged In: YES
    user_id=202636
    Originator: NO

    Looks like this is probably the problem (tclWinFile.c:2077):
    /*
    * Unfortunately our stat definition's inode field (unsigned short)
    * will throw away most of the precision we have here, which means we
    * can't rely on inode as a unique identifier of a file. We'd really
    * like to do something like how we handle 'st_size'.
    */

    inode = data.nFileIndexHigh | data.nFileIndexLow;
    Compiling with MSVC6, Tcl_StatBuf is a _stati64 and the st_ino member is a _ino_t which is declared in TYPES.H to be unsigned short.

    I guess this comes under a general heading of Tcl_Stat crappiness - but why are we depending upon an inode value in 'rename' anyway? That would seem to be the bug.

     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21

    Logged In: YES
    user_id=202636
    Originator: NO

    This problem is only in 8.5 and above. 8.4.19 is OK because there the inodes for windows are always 0. See the attached test script.
    File Added: bug2015723.test

     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21
    • milestone: --> obsolete: 8.5.3
     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21

    Sample tcl script to demo this bug.

     
  • Donal K. Fellows

    Logged In: YES
    user_id=79902
    Originator: NO

    The cheapest workaround would be to remove the test for zero st_ino fields from CopyRenameOneFile and to instead make that check not work at all on Windows with a little #ifdef.

     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21

    Logged In: YES
    user_id=202636
    Originator: NO

    #ifdef will work around this specific bug in file rename/copy however it is possible we should move back to reporting 0 for the inode field. At the moment a script author may decide to use this, expeting that it will always be unique. It turns out that within a couple of thousand files I always get a collision. So I suspect if Tcl_StatBuf isn't going to have a inode type that is large enough we should not make it available.
    See #1325803 for the original dicussion that got this feature put in (commit date was 2005-10-23)

     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21
    • status: open --> pending-fixed
     
  • Pat Thoyts

    Pat Thoyts - 2008-07-21

    Logged In: YES
    user_id=202636
    Originator: NO

    Committed fix to HEAD and 8.5 that avoids the inode test on Windows.

     
  • SourceForge Robot

    Logged In: YES
    user_id=1312539
    Originator: NO

    This Tracker item was closed automatically by the system. It was
    previously set to a Pending status, and the original submitter
    did not respond within 14 days (the time period specified by
    the administrator of this Tracker).

     
  • SourceForge Robot

    • status: pending-fixed --> closed-fixed
     
  • Jan Nijtmans

    Jan Nijtmans - 2012-03-27
    • assigned_to: vincentdarley --> nijtmans
     
  • Jan Nijtmans

    Jan Nijtmans - 2012-03-27

    Now backported to Tcl 8.4. For cygwin the
    same could happen as well, so modified
    the #ifdef for that.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks