Re: streams & truename

[Bruno H. <br...@cl...>]

Hi Sam,

> We have a problem, yet another case where the CL universe is different
> from UNIX (and, probably, Windows, but I don't have access to that, so,
> thankfully, I can excuse myself):
> 
> * CL assumes that (truename open-file-stream) works

Yes and no.

CL says that it is OK to call (truename open-file-stream).
[See http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/fun_truename.html
 The argument is a 'pathname designator'. The glossary says that this
 includes 'stream associated with a file', and 'stream associated with a file'
 includes 'file stream'.]

CL also says that the call may fail.
[Exceptional Situations:
 An error of type file-error is signaled if an appropriate file cannot be located
 within the file system for the given filespec, or if the file system cannot
 perform the requested operation.]

Cases where TRUENAME fails, on Unix, are when you have read and search access to the
current directory but not to one of the ancestor directories.

> * on unix, "/dev/fd/0" resolves to "/proc/<pid>/fd/0" which is a symlink
>   to "pipe:[123456789]" when the process is run like "clisp | cat".
>   (and the "/proc/<pid>/fd/pipe:[...]" file does not exist).
>   IOW, on unix one can do i/o on a non-existent path (sometimes).

Yes, I can reproduce this:
$ ./clisp -norc -q -x '(truename "/dev/fd/1")'
#P"/dev/pts/9"
$ ./clisp -norc -q -x '(truename "/dev/fd/1")' | cat
*** - TRUENAME: File #P"/proc/872/fd/pipe:[3163863]" does not exist

> What to do about it?
> 
> *1* We can detect that "/proc/<pid>/fd/0" is on "/proc" and refuse to
>   follow symlinks there.
>   This is fairly arbitrary, but we already use this sort of magic when
>   deciding the default value for the :buffered argument.
>   This would also prevent users from detecting that the stdout is a
>   path (not good).
>   [btw, shouldn't we treat /proc and /dev identically? neither is a real
>   file system].
> 
> *2* We can defer computing strm_file_truename until necessary, leading to
>   various special cases for (open (make-stream :input)) (which will call
>   truename) &c.
> 
> I don't particularly like either approach; but I think I dislike the
> first one less.

You need both.

You need *1* because the two commands I gave above show that the problem
already occurs when streams are not involved.

You need *2* because no one has asked to compute the truename ahead of time,
when the stream is being created.

More details about *1*:

When the symlink chain is /dev/fd/1 -> /proc/self/fd/1 -> /dev/pts/9
I want to get "/dev/pts/9" as result. Whereas when the symlink chain is
/dev/fd/1 -> /proc/self/fd/1 -> pipe:[3163863]  or
/dev/fd/1 -> /proc/self/fd/1 -> socket:[2881830]
I want to get "/proc/self/fd/1" as result, because you can't do anything with
the internal inode number of the pipe or socket.
Therefore it's only some of the symlinks in /proc that you need to refuse to
follow.

/dev is, like autofs, binfmt_misc, cgroup, debugfs, devtmpfs, efivarfs,
fusectl, hugetlbfs, mqueue, proc, pstore, securityfs, sysfs, tmpfs, tracefs
(and sure some others) a synthetic file system, that is, a file system not
backed by a disk. But its symlinks are OK, as far as I have seen. So I would
leave /dev alone until we see that it has problems.

More details about *2*:

Following these links in the hyperspec
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/fun_truename.html
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/glo_p.html#pathname_designator
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/glo_s.html#stream_associated_with_a_file
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/glo_f.html#file_stream
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/syscla_file-stream.html
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/fun_open.html
I cannot see a wording that indicates that the truename needs to be stored
in the stream. Rather it looks like the stream is supposed to store the
(not canonicalized) pathname to which it was associated when it was created,
and the TRUENAME function will canonicalize it when TRUENAME is called.

It it harmful to call TRUENAME ahead of time because the user has not asked for it
when he creates a stream, and:
  - TRUENAME may fail (see above),
  - File systems nowadays are optimized so that very few operations need to
    canonicalize a file name. This canonicalization is expensive, even when
    done through a kernel API [1].

In particular

$ ./clisp -norc -q -x '(open "/dev/fd/1")' | cat
*** - OPEN: File #P"/proc/1040/fd/pipe:[3172140]" does not exist

is wrong. The correct output would be

$ ./clisp -norc -q -x '(open "/dev/fd/1")' | cat
#<INPUT UNBUFFERED FILE-STREAM CHARACTER #P"/dev/fd/1" @1>

Bruno

[1] http://stackoverflow.com/questions/1188757/getting-filename-from-file-descriptor-in-c