[Sbcl-devel] [BUG] inconsistency in moving file position between reading from file character stream and string stream

[Tobias C. R. <tc...@fr...>]

Hi all,

At first: OPEN is specified to have a default :ELEMENT-TYPE of
CHARACTER, but SBCL defaults to BASE-CHAR.

Next:

When READing from a character stream opened by WITH-OPEN-FILE, SBCL seem
to move FILE-POSITION as if the stream was a binary stream:


(sb-alien:define-alien-routine "tmpnam" sb-alien:c-string
  (dest (* sb-alien:c-string)))

(defun temp-file-name ()
  "Return a temporary file name to compile strings into."
  (concatenate 'string (tmpnam nil) ".lisp"))

(defparameter *temp-file-name* (temp-file-name))

(defparameter *string* "ÄÖÜ")

;; Write string with non ASCII characters into a temp file.
(with-open-file (f *temp-file-name* :direction :output :element-type 'character)
  (write-string (format nil "~S" *string*) f))

(defun check (stream)
  ;; READs, and returns start file position and end file position
  (let ((pos1 (file-position stream)))
    (assert (equal (read stream) *string*))
    (values pos1 (file-position stream))))

;; READ from the temp file on behalf of a character stream
(with-open-file (f *temp-file-name* :element-type 'character)
  (multiple-value-bind (pos1 pos2) (check f)
    (format t "READing directly from file; start file pos = ~D, end file pos = ~D~%"
	    pos1 pos2)))

;; Slurp the temp file first into a string
(let ((buffer (make-array '(5) :element-type 'character)))
  (with-open-file (f *temp-file-name* :element-type 'character)
    (read-sequence buffer f))
  ;; then READ from an input string stream
  (with-input-from-string (s buffer)
    (multiple-value-bind (pos1 pos2) (check s)
      (format t "READing from string stream; start file pos = ~D, end file pos = ~D~%"
	      pos1 pos2))))



Result from loading the file:

  READing directly from file; start file pos = 0, end file pos = 8
  READing from string stream; start file pos = 0, end file pos = 5

As you can see, in the first case, FILE-POSITION wasn't moved
character-wise, but byte-wise.

I find this behaviour very surprising, and it actually screws up Slime's
compiler-notes highlighting.

  -T.