[Tcl-bugs] [ tcl-Bugs-931847 ] TclFSNormalizeToUniquePath - incomplete?

[SourceForge.net <no...@so...>]

Bugs item #931847, was opened at 2004-04-08 12:57
Message generated for change (Comment added) made by dgp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=931847&group_id=10894

Category: 36. File System
Group: development: 8.5a2
Status: Open
Resolution: None
Priority: 5
Submitted By: Don Porter (dgp)
Assigned to: Vince Darley (vincentdarley)
Summary: TclFSNormalizeToUniquePath - incomplete?

Initial Comment:

As I understand
TclFSNormalizeToUniquePath()
it's meant to force paths into
one canonical form -- getting
rid of symlinks, establishing
the case in the filesystem, etc.
To do this, it asks the filesystems
to do the work, giving the native
filesystems first try, then giving all
the others a try, calling the
Tcl_FSNormalizePathProc of
each, if implemented.

Each Tcl_FSNormalizePathProc(),
though, might not be able to finish
the job completely.  The "startAt"
value returned by TclFSNormalizeToUniquePath()
might not point to the end of the
returned string value of the path.

This is fine, as long as the callers
of TclFSNTUP() repeatedly call
it until startAt is at the end of the string,
or until nothing changes.  It appears
the existing callers of TclFSNTUP
are calling only once, not checking
the return value, and expecting
TclFSNTUP to have done all the
normalization possible.

The alternative to changing the callers,
is to rewrite TclFSNTUP itself to do
what the callers appear to expect,
keep looping over normalization
attempts until they don't do any more,
so the path is as normalized as possible.

This bug is difficult to demostrate
because it requires a non-native VFS
that actually implements a
Tcl_FSNormalizePathProc() that
would produce a changed path that,
say, the native filesystem could
normalize further.  For example,
imagine chaining symlinks into
and out of a non-native VFS.

----------------------------------------------------------------------

>Comment By: Don Porter (dgp)
Date: 2004-04-08 13:34

Message:
Logged In: YES 
user_id=80530


FWIW, an arbitrarily nestable
filesystem is exactly what I'm
working on.

I think you've got the right basic
idea, on how to remove this limitation,
but there may be trouble with
PathInFilesystem().

Examining PathInFilesystem() is
what got me started.  This routine
gets passed an arbitrary path, and
must decide whether or not to claim
it.  Generally this can be a hard
problem, so it's natural to call
Tcl_FSGetNormalizedPath() on
the path passed in to have something
more predictable to work with.  That's
fine as long as a working Tcl_FSGNP
doesn't depend on a working PIF().
If so, we'll just pass the buck forever.

----------------------------------------------------------------------

Comment By: Vince Darley (vincentdarley)
Date: 2004-04-08 13:24

Message:
Logged In: YES 
user_id=32170

Wow, you've been looking very closely!

I didn't design the code to handle symlinks between
filesystems, although I suppose I didn't actually document
that anywhere.  I certainly agree that that would
demonstrate bugs in the existing code.

In particular, however, the basic code assumption is that
the normalization can be achieved by (i) asking the native
filesystem to normalize, then (ii) asking each other
filesystem to normalize exactly once each.

Clearly if filesystems can be nested arbitrarily (e.g.
through your symlink approach), or more simply:

C:/foo/bar.zip/ZIP/foo.kit/KIT/blah.zip/ZIP

Assuming we have a separate .zip and .kit filesystem, each
of which have decided they are case-insensitive and that all
normalized paths should be lower case.  Then the above
should be normalized first by the OS (C:/foo/bar.zip) then
by the zipfs (/ZIP/foo.kit) then by the kitfs (KIT/blah.zip)
then by the zipfs again (ZIP).

In the current code structure this would not work.

I guess the correct solution is to first ask the nativefs,
and then, for the filesystem which accepts the path which
has been normalized so far, ask it to normalize some more. 
When no filesystem accepts the path, then just stop.  So,
we'd have:

Overall path accepted by native fs.
C:/foo/bar.zip   normalized by native, leaving 'startAt' at
following '/'

C:/foo/bar.zip/ recognised by zipfs
C:/foo/bar.zip/zip/foo.kit normalized by zipfs.

C:/foo/bar.zip/zip/foo.kit/ recognized by kitfs
etc.

So (pseudo-code):

while (1) {
    fs = PathInFilesystem(path, up to 'startAt');
    if (fs == null) break;
    startAt = fs->normalizePath(path, startAt);
}

with a bit of checking to avoid infinite loops.

This would even solve your symlink issue, I think, although
I haven't thought what it means for 'dir/..' handling.


----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2004-04-08 13:04

Message:
Logged In: YES 
user_id=80530


Ack!  Expanding a symlink
might well introduce new
../ and ./ components to the path,
so perhaps the callers must
deal with this after all?

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2004-04-08 13:00

Message:
Logged In: YES 
user_id=80530


A related issue:  The documentation
for new filesystems should more
explicitly warn that any 
Tcl_FSNormalizePathProc implementation
might be called on any path whatsoever,
not only the ones the filesystem thinks
belong to it.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=931847&group_id=10894