"Nikodemus Siivola" writes:
> C.1: I'm fine with :TRUNCATE. I would also suggest
> SB-EXT:TRUNCATE-FILE (that would truncate an open stream), but having
> it is certainly not a requisite for :TRUNCATE.
I've no strong feelings about a TRUNCATE-FILE, except that I suspect
that pinning down the semantics is a dogfight I don't want to get into
right now. So I'll just dodge the issue and point out that you can do
that already with SB-POSIX:TRUNCATE and SB-POSIX:FTRUNCATE, and wave my
hands about what to do in a Windows file handle universe.
> In addition to :TRUNCATE I would propose :APPEND-ONLY, which would
> provide O_APPEND semantics on both unixoid and Windows platforms
> (assuming Windows supports such.)
I recognize the need for O_APPEND semantics somewhere, but I'm not sure
if doing it this way is the right thing.
(1) First, users can already set O_APPEND with SB-POSIX:FCNTL
<sound of Richard waving his hands about changing the flags on
win32 file handles>
(2) As Rudi alluded to, we're not really supposed to apply the semantics
of an :IF-EXISTS action in case we're creating a new file. So in
order to not exercise the license to deviate, we'd probably have to
support something like this:
(open ... :if-does-not-exist :append-only :if-exists :append-only)
Or something similar. But then we'd probably end up supporting
orthogonal uses of :APPEND-ONLY -- only when creating a new file,
only when writing to an existing file. I'm not enthusiastic about
(3) As you've mentioned in a later post, to get O_APPEND really right,
we need to give the users features to ensure that their writes dont
get interleaved. (In the worst-case scenario, if some character's
encoded octets get split across two non-sequential write()s, SBCL
might not be able to read the file back in.) I think we can do this
in various ways, but I don't want to make any promises about how to
do it right now.
> C.4: Supporting this stuff seems like a good idea. I do find the
> requirement to pass :ALLOW-OTHER-KEYS a bit too magical, however: it
> is supposed to suppress error checking, not enable behaviour. I think
> we should either boldly extend OPEN, or put the extensions in
> SB-EXT:OPEN-FILE which would be otherwise identical to OPEN.
That's funny: I added this ALLOW-OTHER-KEYS requirement precisely
because you cautioned me against adding new keywords to OPEN! My
thoughts were that
(a) requiring :ALLOW-OTHER-KEYS preserves the desired strictness in our
implementation of ANSI operators,
(b) if users do take advantage of a :MODE argument to OPEN, they'll
either need to #+/#- it for other implementations, or use
But I'd be fine with adding those keywords to OPEN's lambda list; I'm
less comfortable with having a separate function for
ANSI-OPEN-plus-extensions, since it'll likely drift away from OPEN.
> E: I earlier expressed some reservations about doing the atomic
> renaming described here but the more I think of it the more attractive
> it sounds.
> A question: What about when SBCL crashes or exits without unwinding?
I think we already leave files behind in this case, so it's no big
change for openings like :RENAME or :RENAME-AND-DELETE (which keep the
old file around at least until CLOSE). I suppose that a program that
relies on :SUPERSEDE being truncation (and so freeing up disk space)
might use a protocol according to which SBCL may crash or exit with
files open, and the next invocation of SBCL will free up disk space by
truncating the files left behind by the previous.
> The first thought that comes to mind is that maybe we should create
> the temporary file in known location, be that /tmp/ or
> SB-EXT:*TEMPDIR*, so that if SBCL exits without closing the streams,
> the temporary files can be reaped by the system somehow instead of
> littering directories all over the place? I don't know -- and this
> doesn't sound like a terribly critical issue to me, but just thought
> to bring it up.
One tiny but important detail: it's essential to create new files beside
their eventual name, because you can't atomically rename() files across
file systems. So creating all new files in /tmp won't do.
I'll think some more about this, and get back to you.
> Section F: sounds good. Just a quick note on a vaguely related topic -
> you say: "Right now SERVE-EVENT only knows how to use file descriptors
> or file handles, and not both. I think it may be possible to get it to
> deal with both, though it'll be hairy." I've on few occasions wondered
> if SERVE-EVENT should work on streams instead of FD (at the API level,
> not internally, of course.)
In my world, APIs should always prefer Lisp objects to raw descriptors
or handles, if for no other reason than that we can attach a finalizer
on a Lisp object to act as a safety net against leaks. For SERVE-EVENT,
ISTM that the right thing is to add the following new functions:
ADD-STREAM-HANDLER stream direction function [Function]
Like ADD-FD-HANDLER, but it takes a stream and a function of a stream,
instead of a descriptor and function of a descriptor. The stream must
be such that we can get the descriptor/handle, e.g., an FD-STREAM or
an FH-STREAM (or other such things).
REMOVE-STREAM-HANDLER handler [Function]
Remove HANDLER from SERVE-EVENT's processing.
(It may also prove worthwhile to have SERVE-EVENT be responsible for
tending to RUN-PROGRAM-spawned children, too, but I'll get into that