#536 CLEAR-INPUT should clear the EOF condition

lisp error
closed-fixed
clisp (524)
5
2009-11-25
2009-11-23
No

when clisp reaches eof, it remembers it and never clears the condition even if when the file on disk changes.
sbcl and C (both FILE/fopen and int/fd/open interfaces) do not exhibit this behavior.

Discussion

  • Sam Steingold

    Sam Steingold - 2009-11-23

    lisp test file

     
  • Sam Steingold

    Sam Steingold - 2009-11-23

    C test file

     
  • Sam Steingold

    Sam Steingold - 2009-11-23

    just to clarify, this behavior can cause problems with special files ("COM1" on woe32 and "/proc/*" on unix).
    <http://article.gmane.org/gmane.lisp.clisp.general/13172>

     
  • Bruno Haible

    Bruno Haible - 2009-11-24
    • status: open --> closed-rejected
     
  • Bruno Haible

    Bruno Haible - 2009-11-24

    There are three reasons for remembering that EOF has happened on an input stream:
    1) For regular files and block devices: The normal situation
    is that a file's size does not change while it is being read
    by an application. This is the only supported situation.
    The mere fact that stat() returns an st_size information
    indicates that files are supposed to stay at the same size
    while they are being accessed. ("tail -f" being a notable
    exception.)
    2) I cannot imagine that an implementation can do the behaviour
    that you suggest without doing additional system calls. In
    other words, the price that SBCL users pay for SBCL's behaviour
    are more system calls. But Linus taught us: Optimize for the
    frequent case, not for the infrequent case. Here the frequent
    case is that files don't change in size. Can you implement
    SBCL's behaviour without extra system calls in this case?
    I don't believe so.
    3) For character devices such as ttys: It is important that
    applications don't call read() a second time after they
    EOF. Because when the an application reads from standard
    input, the user should need to press Ctrl-D *once* to
    terminate the input. If the user needs to press Ctrl-D
    twice (as is the case e.g. with some versions of the 'patch'
    program), it is a bug.

    Finally, regarding files in the Linux /proc/ file system: They
    look like regular files of size 0, but they don't behave like
    regular files. Rather they behave like pipes. My proposal to
    make them look like pipes was unfortunately rejected by Linus.
    So, you have to apply a heuristic, and treat either all "regular"
    files under /proc or all "regular" files of size 0 as pipes.
    I think clisp already does this.

    Regarding <http://article.gmane.org/gmane.lisp.clisp.general/13172>:
    > if ... there is no data ready in the com port, I receive the eof-value, as expected.
    If Windows COM ports are supposed to behave like this (i.e. if unlike pipes,
    reading does not wait) then you have to create a new stream in order to restart
    reading from COM. But honestly, that sounds more like a bug, either in Windows
    or in clisp's detection whether input is available on a stream. In any case, this
    is not a reason for changing the fundamentals of the stream design in clisp.

     
  • Sam Steingold

    Sam Steingold - 2009-11-24

    1. I don't see the implication that st_size means it's a constant, since read(2) sees the data (see attached code).
    2. the extra syscall is only necessary if we have see EOF already, which is rare (who would want to read beyond EOF?) and cheap (the alternative is signaling an error which dwarfs the cost of an extra syscall).
    3. what you are saying is that tty programs should exit on the first EOF. This is a valid point.

    how about a new CLEAR-EOF function which would remove that EOF marker from the stream and try reading again?

     
  • Bruno Haible

    Bruno Haible - 2009-11-24

    > how about a new CLEAR-EOF function which would remove that EOF marker from
    > the stream and try reading again?

    This function already exists. It is part of CLEAR-INPUT (in clisp; I don't know what SBCL does).

     
  • Sam Steingold

    Sam Steingold - 2009-11-25

    nope,
    (let ((fname "test-eof"))
    (open fname :direction :probe :if-exists :overwrite ; touch
    :if-does-not-exist :create)
    (unwind-protect
    (with-open-file (in fname :direction :input)
    (list (read-line in nil :eof)
    (progn
    (with-open-file (out fname :direction :output
    :if-exists :append)
    (write-line "foo" out))
    (clear-input in)
    (read-line in nil :eof))))
    (delete-file fname)))
    ==> (:EOF :EOF) instead of (:EOF "foo")

     
  • Sam Steingold

    Sam Steingold - 2009-11-25

    thank you for your bug report.
    the bug has been fixed in the CVS tree.
    you can either wait for the next release (recommended)
    or check out the current CVS tree (see http://clisp.cons.org\)
    and build CLISP from the sources (be advised that between
    releases the CVS tree is very unstable and may not even build
    on your platform).

     
  • Sam Steingold

    Sam Steingold - 2009-11-25
    • assigned_to: haible --> sds
    • summary: clisp does not notice when the file on disk has been changed --> CLEAR-INPUT should clear the EOF condition
    • status: closed-rejected --> closed-fixed
     

Log in to post a comment.