[Sbcl-bugs] Cause and possible solution for Bug #1042610

[TAKAGI M. <kam...@gm...>]

The following problem is already on launchpad as Bug #1042610 (
https://bugs.launchpad.net/sbcl/+bug/1042610), though let me report its
cause and possible solution.

Situation
------------------------------
The following operation causes AVER failed.

    $ cat loader.lisp
    (eval-when (:compile-toplevel :load-toplevel)
      (load "loadee.lisp"))

    $ cat loadee.lisp
    ;; invalid form causing READ error
    (

    $ sbcl
    * (compile-file "loader.lisp")
      failed AVER: (NULL CURRENT)

Why I do this is for using Common Lisp codes as scripts from UNIX shell via
roswell(https://github.com/snmsts/roswell). For that, some libraries are
loaded in EVAL-WHEN form with (:COMPILE-TOPLEVEL :LOAD-TOPLEVEL) situation.

    (eval-when (:compile-toplevel :load-toplevel)
      (ql:quickload :split-sequence))

Here, split-sequence's ASDF system file has implementation specific
problem, causing AVER failed on SBCL. (For the split-sequence's problem, I
have already send a pull request to it.)


Cause
------------------------------
The AVER failed happen in SB-C::FIND-ORIGINAL-SOURCE, where
SB-C::FIND-SOURCE-ROOT is called with an inconsistent *SOURCE-INFO* causing
the following AVER failed. Note that the loading is in EVAL-WHEN
(:COMPILE-TOPLEVEL :LOAD-TOPLEVEL) form.

    (when (atom form)
      (aver (null current))
      (return))

Where *SOURCE-INFO* is bound is in SB-FASL::LOAD-AS-SOURCE and its
SB-C::DO-FORMS-FROM-INFO. (*SOURCE-INFO* seems to be bound twice
redundantly.)

    (let* ((info (sb!c::make-file-source-info
                  pathname (stream-external-format stream)))
           (sb!c::*source-info* info))
      (setf (sb!c::source-info-stream info) stream)
      (sb!c::do-forms-from-info ((form current-index) info
                                 'sb!c::input-error-in-load)
        (sb!c::with-source-paths
          (sb!c::find-source-paths form current-index)
          (eval-form form current-index))))

Here, although *SOURCE-INFO* is bound to a new one, an READ error occurs in
%DO-FORMS-FROM-INFO before SB-C::WITH-SOURCE-PATHS is reached, causing
*SOURCE-INFO* inconsistency.

The READ error is handled by COMPILER-ERROR-HANDLER bound in
SB-C::WITH-COMPILATION-VALUES. Please see the call tree later in this
report.


Possible solution
------------------------------
Although I do not know if this solution is appropriate, the following diff
fixes the inconsistency. It bounds *SOURCE-INFO* after read has completed.

      (defun load-as-source (stream &key verbose print ...)
        ...
    -            (let* ((info (sb!c::make-file-source-info
    -                          pathname (stream-external- ...
    -                   (sb!c::*source-info* info))
    +            (let ((info (sb!c::make-file-source-info
    +                         pathname (stream-external- ...
                   (setf (sb!c::source-info-stream info) stream)
                   (sb!c::do-forms-from-info ((form current-index) ...
                                              'sb!c::input-error ...
...

      (defmacro do-forms-from-info (((form &rest keys) info
                                     &optional (on-error ''input- ...
                                    &body body)
        (aver (symbolp form))
        (once-only ((info info))
    -     `(let ((*source-info* ,info))
    -        (%do-forms-from-info (lambda (,form &key ,@keys ...
    -                               ,@body)
    -                             ,info ,on-error))))
    +     `(%do-forms-from-info (lambda (,form &key ,@keys ...
    +                             (let ((*source-info* ,info))
    +                               ,@body))
    +                             ,info ,on-error)))


Call tree
------------------------------
Here shows the call tree on this problem.

    swank
    - compile-file-for-emacs
    - swank-compile-file*
    - swank-compile-file
    - with-compilation-hooks
    - call-with-compilation-hooks
    - handle-notification-condition
    SBCL
    - sb-c::find-error-context
    - sb-c::find-original-source
    SBCL
    - comile-file
    - sub-compile-file
    - with-compilation-values
    - compiler-error-handler
    - sub-sub-compile-file
    - do-forms-from-info
    - process-toplevel-form
    - process-toplevel-progn
    - process-toplevel-form
    - eval-compile-toplevel
    - eval-tlf
    - eval-in-lexenv
    - simple-eval-in-lexenv
    - load
    - load-as-source
    - do-forms-from-info
    - %do-forms-from-info
    - read-preserving-whitespace


Regards,
Masayuki Takagi