Steel Bank Common Lisp

Adam Warner <lists@...> writes:

> I found this (time (api)) test code to be around twenty times slower in
> SBCL compared to CMUCL:
>
> (declaim (optimize (safety 0) (speed 3)))
> (declaim (start-block api))
>
> (defun api ()
>   (loop for i of-type (double-float 0d0)
>         from 0d0 to 100000000d0
>         sum (fn-call i) of-type double-float))
>
> (defun fn-call (float)
>   (sqrt float))
>
> (declaim (end-block))
>
> I am aware that support for this elegant form of block compilation has
> been removed from SBCL. Does SBCL provide another way to achieve this
> kind of block compilation? I have also noted that :ENTRY-POINTS for
> :BLOCK-COMPILE in COMPILE-FILE appears to have been removed.

Away from my system, but
  (declaim (inline fn-call) (optimize speed (safety 0)))
  (defun fn-call (float)
    (sqrt float))
  (defun api ...)
should have the same effect.  This isn't necessarily entirely
scalable to whatever real-world example motivates your question.

Note that this test could (though it probably isn't, but you never
know) be completely constant-folded by compilers; be sure to inspect
the compiled code to verify that it's doing what you think it's doing.

Cheers,

Christophe
-- 
http://www-jcsu.jesus.cam.ac.uk/~csr21/       +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%")    (pprint #36rJesusCollegeCambridge)

Hi Christophe Rhodes,

>> I found this (time (api)) test code to be around twenty times slower in
>> SBCL compared to CMUCL:
>>
>> (declaim (optimize (safety 0) (speed 3)))
>> (declaim (start-block api))
>>
>> (defun api ()
>>   (loop for i of-type (double-float 0d0)
>>         from 0d0 to 100000000d0
>>         sum (fn-call i) of-type double-float))
>>
>> (defun fn-call (float)
>>   (sqrt float))
>>
>> (declaim (end-block))
>>
>> I am aware that support for this elegant form of block compilation has
>> been removed from SBCL. Does SBCL provide another way to achieve this
>> kind of block compilation? I have also noted that :ENTRY-POINTS for
>> :BLOCK-COMPILE in COMPILE-FILE appears to have been removed.
> 
> Away from my system, but
>   (declaim (inline fn-call) (optimize speed (safety 0)))
>   (defun fn-call (float)
>     (sqrt float))
>   (defun api ...)
> should have the same effect.  This isn't necessarily entirely
> scalable to whatever real-world example motivates your question.

Thanks for the reply. The real world motivating example is a FN-CALL that
is arbitrarily big and called from multiple places. The cost of conversion
from a non-descriptor form and back again has to be eliminated without
destroying cache performance by duplicating the function. In other words
magically turning desired-non-globally-accessible function calls into
LABELS:

(defun api ()
  (labels ((fn-call (float) (sqrt float)))
    (loop for i of-type (double-float 0d0)
          from 0d0 to 100000000d0
          sum (fn-call i) of-type double-float)))

Removing this functionality has completely gutted SBCL's performance
when writing lispy numeric optimisation code as a series of global
function calls that can be individually tested and then simply optimised.
It's the reason SBCL now has no solution to their man page comment 
"... passing the result of a floating point calculation as a function
argument (or returning a floating point result as a function value) is
a fundamentally slow operation."

Here's the reason given for removing block compilation functionality:
<http://sourceforge.net/project/shownotes.php?release_id=5179>

   made PROCLAIM and DECLAIM conform to ANSI. PROCLAIM is now an
   ordinary function. As a consequence, START-BLOCK and END-BLOCK
   declarations are no longer supported, since their implementation
   was deeply intertwingled with the magical, non-ANSI treatment that
   PROCLAIM received in CMU CL.

The costs of this decision are clear: Extreme loss in performance for some
types of numeric optimisation problems that can only be countered by
rewriting global function calls as labels. Perhaps I could still achieve
the same degree of usability with some custom macrology. But I still
wonder at the substantive benefits to users from removing this
functionality (what were they?)

> Note that this test could (though it probably isn't, but you never
> know) be completely constant-folded by compilers; be sure to inspect
> the compiled code to verify that it's doing what you think it's doing.

Yes I believe it is. Thanks for the tip.

Regards,
Adam

> In other words magically turning desired-non-globally-accessible
> function calls into LABELS:

Here's a bare and manual version of the functionality:

(defpackage :model1
  (:use)
  #.`(:import-from :cl
      ,@(let (list)
             (do-external-symbols (symbol :cl)
               (unless (eq symbol 'defun) (push symbol list)))
             list)))

(in-package model1)

(defmacro defun (name lambda-list &body body)
  `(eval-when (:compile-toplevel)
    (cl:defun ,name ,lambda-list ,@body)
    (setf (get 'source ',name) '(,name ,lambda-list ,@body))))

(defmacro insert-labels (functions &rest args)
  `(labels ,(loop for fn in functions collect (get 'source fn))
    ,@args))

;;--------------------
;;Standard code begins
;;--------------------

(defun fn-call (float)
  (sqrt float))

(defun api ()
  (insert-labels (fn-call)
    (loop for i of-type (double-float 0d0)
          from 0d0 to 100000000d0
          sum (fn-call i) of-type double-float)))


The source code to each global function definition is seamlessly
stored using a custom DEFUN (which is why this symbol isn't imported from
:COMMON-LISP). The INSERT-LABELS macro allows each global function's
source to be easily inserted as a local function. By definition the
innermost FLET or LABELS forms override references to a global function
definition of the same name. Thus the cost of making a global function
call is avoided.

I haven't quite understood why adding this to the start of the source...

(eval-when (:compile-toplevel :load-toplevel :execute)
  (declaim (optimize (safety 0) (speed 3))))

...appears to not affect compilation. I have to type (declaim (optimize
(safety 0) (speed 3))) into the REPL before compilation for optimisation
notes to start appearing.

This exploratory work indicates that an ANSI-conforming and user-friendly
type of block compilation is feasible.

Regards,
Adam

In responding to my open question:

> The costs of this decision are clear: Extreme loss in performance for
> some types of numeric optimisation problems that can only be countered
> by rewriting global function calls as labels. Perhaps I could still
> achieve the same degree of usability with some custom macrology. But I
> still wonder at the substantive benefits to users from removing this
> functionality (what were they?)

Thanks for the detailed reply William. It would have been fascinating for
the list.

SBCL is justifiably promoted on comp.lang.lisp. It is typically stated
that the speed of resulting assembly code marginally differs from that
produced by CMUCL. Sometimes faster. Sometimes slower. This doesn't
engender an expectation that code can turn out to be tens of times slower.
Implementation of such a significant difference in performance will likely
be for better reasons than it's just not ANSI conforming. And it turns out
it is: DEFMACRO etc. can expand into EVAL-WHEN instead of weird
bogosities. I can now appreciate this.

My question was just a symptom of expectations and your project's
successes. Please forgive my unintended criticism of SBCL developers. I
genuinely wanted to understand the reasons that prompted the decision.
Thank you for setting them out.

Regards,
Adam