Marco Antoniotti writes:
> On Feb 20, 2008, at 21:33 , Richard M Kreuter wrote:
> > "Nikodemus Siivola" writes:
> > (1) Some implementations signal errors in PROBE-FILE. Therefore,
> > your program will not do what you want on those implementations
> > anyway, but will stop when it encounters the error.
> Again: which ones and how?
Clisp signals an error in PROBE-FILE whenever stat() fails and sets
ERRNO to anything other than ENOENT on Unix. I haven't checked it on
PROBE-FILE was added to CLtL1 from an early LispM API whose documented
interface was to signal an error in all cases except the one where the
directory exists and the file does not, so systems derived from the MIT
LispM would probably signal an error here too.
The CLHS issue writeup that got the exceptional language
An error of type ‘file-error’ is signaled if the file system cannot
perform the requested operation.
added to the standard explicitly mentions cases like permission denied
errors and the directory part of a pathname naming a nondirectory file
as reasons to signal an error. So future implementors who try to get an
idea of what these routines are supposed to do by reading the CLHS and
its issues may have it signal an error here as well.
(As for why PROBE-FILE returns NIL in most modern implementations, I
think it's an accident of history and politics: CLtL1 doesn't mention
that PROBE-FILE might error, even though that was its interface on the
LispM, and CLtL1 didn't have any way to handle errors, not even
IGNORE-ERRORS. So probably non-LispM implementors of CLtL1 got the idea
that PROBE-FILE was supposed to return a pathname or NIL, and never
signal an error. In fact PROBE-FILE was originally a convenience
wrapper around TRUENAME that returned NIL only when the directory
existed and the file didn't (i.e., in the common case where the program
doesn't need to take special measures, such as creating a directory,
before trying to create the file). Otherwise, the LispM's PROBE-FILE
exposed the exact file-system-specific semantics of trying to resolve a
truename, with precise errors, etc.; and so it wasn't the vague,
high-level thing we have today, but an interface not dissimilar to
realpath(2) on Unix or GetFileAttributes() and ResolvePath() on Windows.
So in fact, the popular implementation of PROBE-FILE on today's Lisps is
a degraded descendant of the original. There are many other such
details around the file system interface, where CLtL1 was a vaguely
specified subset of a LispM API, with the consequence that by the time
of X3J13 it was too late to get other implementors to adopt more
precision from the LispM API, because those other implementors didn't
want to go back and redo their file system stuff. (Note: this isn't an
/argument/, just an interpretation of the apparent history of things.
ISTM that the LispM basically got these details right 25 years ago, in
that the high-level operators managed to also have comprehensive
file-system-specific semantics and failure modes, and so you could use
PROBE-FILE with precision; an essential part of that API was
fine-grained subclasses of FILE-ERROR, of course. The implementations
we have today have mostly retained compatibility with their CLtL1ish
ancestry for many aspects of chapters 19 and 20 of the standard, and so
their implementations pathnames and file system interfaces are somewhat
clumsier at practical tasks than the LispM's versions were, decades ago.
In any case, since some future implementation might possibly adopt more
of a LispM-like file system API, programs that can't deal with errors
from PROBE-FILE won't port to such implementations, and so aren't
portable in the CLHS sense.))
> > (2) Other implementations return NIL from PROBE-FILE for all ways
> > that stat() fails, including EACCES. Therefore, your program
> > cannot even reliably distinguish EACCES from ENOTDIR, even if
> > if it wants to.
> If you want to distinguish these, you are in non-portable land,
> unless you make up a portable library. I have no problem having this
> signalled by the use of a qualified name in the code. I have
> problems special casing SBCL in code I try to write for more than one
As has come up several times in this thread,
(1) Using PROBE-FILE is usually the wrong thing in a program anyway:
the file system can change between PROBE-FILE and any subsequent
attempt to use a file, and OPEN, LOAD both take IF-DOES-NOT-EXIST
arguments that can cause those routines to fail in controlled
(2) Most of the ways that I argue PROBE-FILE should signal an error
are ways that any immediately subsequent attempt to open or create
a file will fail (the directory part of the pathname names a
nondirectory file, permission denied on the directory, symlink
loops). So not very many programs will avoid these errors anyway.
(3) Other implementations can, do, and have had PROBE-FILE signal
errors in various cases, so if you should ever find yourself
porting your program to such an implementation, you'll have to
deal with this variation in what implementations may conformingly
> > That is, the only thing that resembles portabiltiy is to use PROBE-
> > FILE in different, implementation-specific ways on each
> > implementation you mean to target, or indeed not to use it at all.
> That is your opinion.
Unless otherwise qualified, I'm talking about portability in the CLHS
required to produce equivalent results and observable side effects
in all conforming implementations.
If you grant that implementations may conformingly signal an error or
conformingly return NIL for various outcomes, it follows
straightforwardly that a program that relies on only one or the other
behavior from an implementation will not produce equivalent results and
observable side effects on conforming implementations that express the
other behavior. Therefore, such a program is not portable in the CLHS
sense. (It may port among the implementations you care about today,
> I program in Common Lisp, not at the POSIX level, and, up to now,
> PROBE-FILE behaves nicely on most implementations I checked. I.e. it
> implements the all-or-nothing semantics.
The problem with forming a model of what the language is supposed to be
by looking at what some implementations do is that implementations can
converge by accident (or some can emulate others, I suppose). In the
case of PROBE-FILE, I think there's reasonable evidence that it was
intended by the standardizers to signal errors in various cases I'm
arguing for, and certainly to be permitted to signal errors in these
> >>> (2) Users can want compatibility with other implementations.
> Let me rephrase this. Users *do want* compatibility with other
> implementations. At least I do.
> > I don't want to get into a game of ranking other implementations for
> > the purpose of making SBCL more like some than like others.
> But this is the crux of the problem. You want to make SBCL more
> unlike the others (AFAIS). This is, IMHO, no good.
You cut some of the rest of what I said
As I've pointed out, some implementations do PROBE-FILE one way,
while others do it the other way. SBCL can't be compatible with all
of them, and I don't strive for precise compatibility with other
Now, what can it possibly mean to be compatible with the details of
other implementations? Implementations already differ on the semantics
* Clisp signals errors in various cases.
* Allegro doesn't seem to resolve symlinks in PROBE-FILE.
* SBCL says that a dangling or self-referring symlink is its own
truename, but OpenMCL and Allegro return NIL from PROBE-FILE in this
* Until 1.0.14, SBCL's truename resolution was a rather haphazard thing,
which didn't always resolve all symlinks (it was quite difficult to
describe in english, in fact). But presumably it was conforming, and
other implementations (CMUCL?) might have similarly confusing notions
of truename resolution. Consequently, programs that use PROBE-FILE or
TRUENAME to get a truename and then use some pathname manipulations to
produce another pathname (e.g., merging with a pathname having :UP in
the directory), can end up naming different files across
implementations that vary on what a truename is.
And there are probably other details I'm not thinking of.
So ISTM that what other implementations conformingly do this week is no
good guideline for what SBCL should do; though I think that these other
implementations do serve as a reasonable proof-by-example of the
variation that is permissible among conforming implementations, and so
the constraints on what a truly portable program may do.
> > If we're agreed that Clisp's behavior is conforming, then ISTM to
> > be admissible as reasonable evidence of what an implementation might
> > do -- which also means some future implementation that people may
> > care about will do things that way too.
> At a certain point CLisp came (still does to some extent) with a
> chapter in its manual: "How CLisp implements the ANSI standard". You
> can still dig out the flames that The Naggum threw Bruno Haible on
> C.L.L. on the issue (rightly so).
Do you dispute that Clisp's current PROBE-FILE behavior is conforming?
If you don't, then the space of conforming implementations includes
implementations that signal errors in PROBE-FILE. IMO, that's all there
is to the question "What can a CLHS-portable program rely on from
> You may be as disgusted as sometimes I am about the strict separation
> of language and underlying OS that is enforced pretty much everywhere
> in CL, but that does not allow you to be incompatible only because
> POSIX evolved somewhat differently.
No, you're wrong on all counts. I'm willing to collaborate with other
SBCL developers on details, but I do not recognize any other requirement
to be compatible with other implementations on any detail not in the
standard. In my opinion, it is a /virtue/ of the fact that we've a
multiple implementation language that different implementations can go
> Let me be clear on this. If you convince ACL, LW, CMUCL, ECL, CLisp,
> ABCL, and the various implementors of other CL (I am sure I forgot
> someone; oh yes; Roger Corman, of course) to implement the PROBE-
> FILE and :APPEND semantics as you like them, then by all means go
> ahead and do it. But - right now - SBCL would become incompatible
> with other implementations and this is not a good thing (IMHO).
I think we disagree about the nature of the CL implementation market.
Implementations differ on details; that's what it means to be a
multiply-implemented language. Implementors are not required to emulate
each other on unspecified details. Programs that must port among
implementations need to be careful about things.
In any case, I've already agreed to go along with whatever Nikodemus
wants PROBE-FILE to do. (Note that he and I both think PROBE-FILE
should error in "permission denied" cases, so SBCL will still be unlike
other implementations in that detail.)
(As for :APPEND, I think you're overlooking some of the details in that
thread. SBCL has done :APPEND in the unusual way since approximately
forever, so I've tried to explain how SBCL's behavior can be defended as
conforming, in case other SBCL developers wanted to retain it. I also
explained in my writeup that there are various reasons not to have
:APPEND be O_APPEND, and compatibility is among those reasons. I have a
preference, but I don't care terribly much about :APPEND, so I've tried
not to push my preference on that detail, which I consider to be quite