#373 Compile-file: non-similarity of strings with cr/lf

closed-fixed
Sam Steingold
clisp (525)
5
2008-02-06
2006-10-16
Tomas Zellerin
No

Consider this (rather artificial) snippet:

(defmacro add-crlf (string)
(with-output-to-string (o)
(write-string string o)
(princ #\Return o)
(princ #\LineFeed o)
))

(print (length (add-crlf "a")))

It prints 3 when loaded without or with compiling, but
2 when compile-file is used and .fas loaded.

I think it is due to treatment described in impnotes
13.8., Treatment of Newline during Input and Output.

I believe this contradicts requirements of CLHS about
similarity of compile-file/loaded literals:

"The file compiler must cooperate with the loader in
order to assure that in each case where an
externalizable object is processed as a literal object,
the loader will construct a similar object." (3.2.4.1)

Discussion

  • Bruno Haible
    Bruno Haible
    2006-11-20

    Logged In: YES
    user_id=5923
    Originator: NO

    The faithful newline i/o introduces additional problems.

    What we need is a solution such that
    - .fas files are plain text file that can be edited with a text editor,
    - .fas files can be edited and resaved with a different line-terminator
    convention, without changing their semantics,
    - .fas files can be concatenated to produce an executable script;
    it should even be possible to concatenate Unix .fas files with DOS .fas
    files.

    The simplest solution is to ignore the newlines when reading .fas files.
    This requires a different string representation where all #\Return and
    #\Linefeed are explicit. I.e. instead of

    "multiline
    string"

    we write

    #0Y"multiline\n
    string"

    The escape sequences \n and \r are well-known from C and Java, \n also
    from Unix 'sed'.

    Such a solution has been implemented on 2006-11-20 for strings, by introducing a
    #0Y"..." syntax. If you care about symbol or package names that contain CR characters
    you can do the same with a #0Y|...|::|...| syntax.

     
  • Bruno Haible
    Bruno Haible
    2006-11-20

    • status: open --> closed-fixed
     
  • Sam Steingold
    Sam Steingold
    2006-11-20

    • status: closed-fixed --> open-fixed
     
  • Sam Steingold
    Sam Steingold
    2006-11-20

    Logged In: YES
    user_id=5735
    Originator: NO

    >If you care about symbol or package names
    >that contain CR characters
    >you can do the same with a #0Y|...|::|...| syntax.

    yes, please.

     
  • Sam Steingold
    Sam Steingold
    2006-12-26

    Logged In: YES
    user_id=5735
    Originator: NO

    #0Y|...|::|...| syntax is already used for stream encoding.
    how about always quoting all CR/LF in all strings/symbols/package prefixes in compiled files?

     
  • Sam Steingold
    Sam Steingold
    2008-01-28

    Logged In: YES
    user_id=5735
    Originator: NO

    I don't like the extra #0Y, especially because deciding whether to print it requires pre-scanning symbols and strings.
    maybe we could detect that the i/o is dealing with a FAS files and branch on that?

     
  • Sam Steingold
    Sam Steingold
    2008-01-28

    Logged In: YES
    user_id=5735
    Originator: NO

    another issue is that
    (let ((*print-readably* t)) (prin1-to-string (string #\return)))
    ==> "#0Y\"\\r\""
    is not ANSI-compliant.
    here we are really stuck because we use CL printer to write FAS files.
    I think we should drop #0Y for strings altogether and always use \n and \r when doing FAS i/o.
    to detect whether we are doing FAS i/o, I think we need to add a #0Y code to the beginning of fas files, similar to the encoding.

     
  • Sam Steingold
    Sam Steingold
    2008-01-30

    Logged In: YES
    user_id=5735
    Originator: NO

    we already have strmflags_reval_B which overrides *read-eval* and whose (only?) raison d'etre is for FAS files.
    I propose we rename it to strmflags_fas_B and use it instead of *compiling* and #0Y"" in io.d.

     
  • Sam Steingold
    Sam Steingold
    2008-02-06

    Logged In: YES
    user_id=5735
    Originator: NO

    thank you for your bug report.
    the bug has been fixed in the CVS tree.
    you can either wait for the next release (recommended)
    or check out the current CVS tree (see http://clisp.cons.org\)
    and build CLISP from the sources (be advised that between
    releases the CVS tree is very unstable and may not even build
    on your platform).

     
  • Sam Steingold
    Sam Steingold
    2008-02-06

    • assigned_to: haible --> sds
    • status: open-fixed --> closed-fixed