Steel Bank Common Lisp

Hi!

I'm using x86_64 GNU/Linux, sbcl 1.0.46.13 and latest weblocks-dev.
Week ago, I've updated sbcl from 1.0.45 to latest, and weblocks-dev to
latest (among other libs that weblocks requires) using clbuild. The
result is a bit strange. Both my app and weblocks-demo (demo weblocks
app) compile with no issues. And they also start with no issues. When
I try to connect using browser to the home page of both my app and
weblocks-demo app, I get "The value 0 is not of type SEQUENCE". As far
as I can understand from the backtrace, there is an issue with
weblocks:mark-dirty function, so I tried to trace it. But, when I
(trace weblocks:mark-dirty), suddenly everything works fine. I can
connect and use my app, and weblocks-demo, and it seems as there is no
problem. But, as soon as I (untrace weblocks:mark-dirty), app stops
and I get the same backtrace.

In my app there are no (optimize (speed x) (safetx)) setups, I'm using
default sbcl values, and as far as I can see, so does the
weblocks-demo. I've tried deleting all of the fasls in
~/.cache/common-lisp dir and recompiling, no difference.

Release sbcl 1.0.43 works fine, but every release version after that
(including latest dev version) generates the same error, in the same
backtrace place.

>From the weblocks mail list, I was instructed to contact you here. I
can provide additional information, but I'll need assistance and
pointers in doing so, since I'm not an sbcl/CL expert.

Here is the backtrace:

The value 0 is not of type SEQUENCE.
   [Condition of type TYPE-ERROR]

Restarts:
 0: [TERMINATE-THREAD] Terminate this thread (#<THREAD "Request
prelude" RUNNING {1002C4E3C1}>)

Backtrace:
  0: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGING-ENVIRONMENT) #<FUNCTION
(LAMBDA #) {10056128B9}>)
      Locals:
        SB-KERNEL:*HANDLER-CLUSTERS* = :<NOT-AVAILABLE>
        SWANK-BACKEND::*SLDB-STACK-TOP* = :<NOT-AVAILABLE>
        SWANK-BACKEND::DEBUGGER-LOOP-FN = #<FUNCTION (LAMBDA ()) {10056128B9}>
  1: (SWANK::DEBUG-IN-EMACS #<TYPE-ERROR expected-type: SEQUENCE datum: 0>)
      Locals:
        SB-DEBUG::ARG-0 = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
  2: (SWANK-BACKEND::CALL-WITH-BREAK-HOOK #<FUNCTION
SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {100887CC09}>)
      Locals:
        *INVOKE-DEBUGGER-HOOK* = :<NOT-AVAILABLE>
        SWANK-BACKEND::CONTINUATION = #<CLOSURE (LAMBDA ()) {100887CC09}>
        SWANK-BACKEND::HOOK = #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK>
  3: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK) #<FUNCTION
SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {100887CC09}>)
      Locals:
        *DEBUGGER-HOOK* = :<NOT-AVAILABLE>
        SB-KERNEL:*HANDLER-CLUSTERS* = :<NOT-AVAILABLE>
        SWANK-BACKEND::FUN = #<CLOSURE (LAMBDA ()) {100887CC09}>
        SWANK-BACKEND::HOOK = #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK>
  4: (SWANK::CALL-WITH-BINDINGS ((*STANDARD-OUTPUT* . #)
(*STANDARD-INPUT* . #) (*TRACE-OUTPUT* . #) (*ERROR-OUTPUT* . #)
(*DEBUG-IO* . #) (*QUERY-IO* . #) ...) #<CLOSURE (LAMBDA #)
{100887CC29}>)
      Locals:
        SB-DEBUG::ARG-0 = ((*STANDARD-OUTPUT* .
#<SWANK-BACKEND::SLIME-OUTPUT-STREAM {1002EEEE01}>) ...
        SB-DEBUG::ARG-1 = #<CLOSURE (LAMBDA ()) {100887CC29}>
  5: (SWANK:INVOKE-SLIME-DEBUGGER #<TYPE-ERROR expected-type: SEQUENCE
datum: 0>)
      Locals:
        SB-DEBUG::ARG-0 = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
  6: (SWANK-BACKEND::CALL-WITH-BREAK-HOOK #<FUNCTION
SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {100887CBC9}>)
      Locals:
        *INVOKE-DEBUGGER-HOOK* = :<NOT-AVAILABLE>
        SWANK-BACKEND::CONTINUATION = #<CLOSURE (LAMBDA ()) {100887CBC9}>
        SWANK-BACKEND::HOOK = #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK>
  7: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK) #<FUNCTION
SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {100887CBC9}>)
      Locals:
        *DEBUGGER-HOOK* = :<NOT-AVAILABLE>
        SB-KERNEL:*HANDLER-CLUSTERS* = :<NOT-AVAILABLE>
        SWANK-BACKEND::FUN = #<CLOSURE (LAMBDA ()) {100887CBC9}>
        SWANK-BACKEND::HOOK = #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK>
  8: (SWANK:SWANK-DEBUGGER-HOOK #<TYPE-ERROR expected-type: SEQUENCE
datum: 0> #<unavailable argument>)
      Locals:
        SB-DEBUG::ARG-0 = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
  9: (SB-DEBUG::RUN-HOOK *DEBUGGER-HOOK* #<TYPE-ERROR expected-type:
SEQUENCE datum: 0>)
      Locals:
        SB-DEBUG::ARG-0 = *DEBUGGER-HOOK*
        SB-DEBUG::ARG-1 = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
 10: (INVOKE-DEBUGGER #<TYPE-ERROR expected-type: SEQUENCE datum: 0>)
      Locals:
        SB-DEBUG::ARG-0 = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
 11: ((FLET #:LAMBDA19) #<TYPE-ERROR expected-type: SEQUENCE datum: 0>)
      Locals:
        SB-DEBUG::ARG-0 = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
 12: (SIGNAL #<TYPE-ERROR expected-type: SEQUENCE datum: 0>)
      Locals:
        SB-KERNEL::ARGUMENTS = NIL
        SB-KERNEL::DATUM = #<TYPE-ERROR expected-type: SEQUENCE datum: 0>
 13: (ERROR TYPE-ERROR :DATUM 0 :EXPECTED-TYPE SEQUENCE)
      Locals:
        SB-KERNEL::ARGUMENTS = (:DATUM 0 :EXPECTED-TYPE SEQUENCE)
        SB-KERNEL::DATUM = TYPE-ERROR
 14: (SB-KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = #.(SB-SYS:INT-SAP #X7FFFF5525A80)
        SB-DEBUG::ARG-2 = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP
#X7FFFF5525540 :TYPE (* ..)>
        SB-DEBUG::ARG-3 = (21 85)
 15: (SB-KERNEL:INTERNAL-ERROR #.(SB-SYS:INT-SAP #X7FFFF5525540)
#<unavailable argument>)
      Locals:
        SB-DEBUG::ARG-0 = #.(SB-SYS:INT-SAP #X7FFFF5525540)
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
 16: ("foreign function: call_into_lisp")
      [No Locals]
 17: ("foreign function: funcall2")
      [No Locals]
 18: ("foreign function: interrupt_internal_error")
      [No Locals]
 19: ("foreign function: #x412F2A")
      [No Locals]
 20: (REMOVE NIL 0)[:OPTIONAL]
      Locals:
        SB-DEBUG::ARG-0 = NIL
        SB-DEBUG::ARG-1 = 0
        SB-DEBUG::ARG-2 = 17592163650385
        SB-DEBUG::ARG-3 = 0
 21: (REMOVE NIL 0)[:EXTERNAL]
      Locals:
        SB-DEBUG::ARG-0 = 2
        SB-DEBUG::ARG-1 = NIL
        SB-DEBUG::ARG-2 = 0
 22: ((SB-PCL::FAST-METHOD WEBLOCKS:MARK-DIRTY (WEBLOCKS:WIDGET))
#<unused argument> #<unused argument> #<WEBLOCKS:WIDGET "dom0">
:PROPAGATE T :PUTP #<unused argument>)
      Locals:
        #:N-SUPPLIED-929 = NIL
        #:N-SUPPLIED-930 = NIL
        #:PROPAGATE-DEFAULTING-TEMP = T
        WEBLOCKS::W = #<WEBLOCKS:WIDGET "dom0">
 23: ((SB-PCL::FAST-METHOD (SETF SB-MOP:SLOT-VALUE-USING-CLASS) (T
WEBLOCKS:WIDGET-CLASS WEBLOCKS:WIDGET
WEBLOCKS::WIDGET-EFFECTIVE-SLOT-DEFINITION)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = #<CLOSURE (LAMBDA (SB-PCL::NV CLASS
SB-KERNEL:INSTANCE ...)) {1008783679}>
        SB-DEBUG::ARG-2 = NIL
        SB-DEBUG::ARG-3 = #<WEBLOCKS:WIDGET-CLASS WEBLOCKS:WIDGET>
        SB-DEBUG::ARG-4 = #<WEBLOCKS:WIDGET "dom0">
        SB-DEBUG::ARG-5 = #<WEBLOCKS::WIDGET-EFFECTIVE-SLOT-DEFINITION
WEBLOCKS:DOM-CLASS>
 24: ((LAMBDA (&REST SB-PCL::ARGS)) NIL #<WEBLOCKS:WIDGET "dom0">)[:OPTIONAL]
      Locals:
        SB-DEBUG::ARG-0 = 17592163650447
        SB-DEBUG::ARG-1 = 2
 25: ((LAMBDA ()))
      [No Locals]
 26: ((SB-PCL::FAST-METHOD WEBLOCKS:HANDLE-CLIENT-REQUEST
(WEBLOCKS:WEBLOCKS-WEBAPP)) #<unavailable argument> #<unavailable
argument> #<unavailable argument>)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = :<NOT-AVAILABLE>
 27: ((FLET #:|doit-77|))
      [No Locals]
 28: ((SB-PCL::FAST-METHOD WEBLOCKS:HANDLE-CLIENT-REQUEST :AROUND (T)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = #S(SB-PCL::FAST-METHOD-CALL ..)
        SB-DEBUG::ARG-2 = #<VTS::VTS {100CD5F4B1}>
 29: ((SB-PCL::FAST-METHOD WEBLOCKS:HANDLE-CLIENT-REQUEST :AROUND (T))
..)[:EXTERNAL]
      Locals:
        SB-DEBUG::ARG-0 = 3
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = #S(SB-PCL::FAST-METHOD-CALL ..)
        SB-DEBUG::ARG-3 = #<VTS::VTS {100CD5F4B1}>
 30: (WEBLOCKS::CALL-IN-WEBAPP #<VTS::VTS {100CD5F4B1}> #<CLOSURE
(LAMBDA #) {100848C749}>)
      Locals:
        SB-DEBUG::ARG-0 = #<VTS::VTS {100CD5F4B1}>
        SB-DEBUG::ARG-1 = #<CLOSURE (LAMBDA ()) {100848C749}>
 31: ((SB-PCL::FAST-METHOD WEBLOCKS:HANDLE-CLIENT-REQUEST :AROUND
(WEBLOCKS:WEBLOCKS-WEBAPP)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = #S(SB-PCL::FAST-METHOD-CALL ..)
        SB-DEBUG::ARG-2 = #<VTS::VTS {100CD5F4B1}>
 32: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-REQUEST
(HUNCHENTOOT:ACCEPTOR HUNCHENTOOT:REQUEST)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = #<WEBLOCKS:WEBLOCKS-ACCEPTOR (host
localhost, port 8443)>
        SB-DEBUG::ARG-3 = #<HUNCHENTOOT:REQUEST {100846F5B1}>
 33: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-REQUEST (T))
#<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:REQUEST
{100846F5B1}>)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = #<HUNCHENTOOT:REQUEST {100846F5B1}>
      Catch-tags:
        HUNCHENTOOT::HANDLER-DONE
 34: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION
(HUNCHENTOOT:ACCEPTOR T)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = #<WEBLOCKS:WEBLOCKS-ACCEPTOR (host
localhost, port 8443)>
        SB-DEBUG::ARG-3 = #<USOCKET:STREAM-USOCKET {1002C4B821}>
 35: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION
(WEBLOCKS:WEBLOCKS-ACCEPTOR T)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = #S(SB-PCL::FAST-METHOD-CALL ..)
        SB-DEBUG::ARG-2 = #<WEBLOCKS:WEBLOCKS-ACCEPTOR (host
localhost, port 8443)>
        SB-DEBUG::ARG-3 = #<USOCKET:STREAM-USOCKET {1002C4B821}>
 36: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION :AROUND
(HUNCHENTOOT:ACCEPTOR T)) ..)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = #S(SB-PCL::FAST-METHOD-CALL ..)
        SB-DEBUG::ARG-2 = #<WEBLOCKS:WEBLOCKS-ACCEPTOR (host
localhost, port 8443)>
        SB-DEBUG::ARG-3 = #<USOCKET:STREAM-USOCKET {1002C4B821}>
 37: ((LAMBDA ()))
      [No Locals]
 38: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK360]365))
      [No Locals]
 39: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
      [No Locals]
      Catch-tags:
        SB-IMPL::%END-OF-THE-WORLD
        SB-INT:TOPLEVEL-CATCHER
 40: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]300))
      [No Locals]
 41: (SB-THREAD::CALL-WITH-MUTEX ..)
      Locals:
        SB-DEBUG::ARG-0 = #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK)
{7FFFF5526CB9}>
        SB-DEBUG::ARG-1 = #S(SB-THREAD:MUTEX ..)
        SB-DEBUG::ARG-2 = #<SB-THREAD:THREAD "Request prelude" RUNNING
{1002C4E3C1}>
        SB-DEBUG::ARG-3 = T
 42: (SB-THREAD::INITIAL-THREAD-FUNCTION)
      [No Locals]
 43: ("foreign function: call_into_lisp")
      [No Locals]
 44: ("foreign function: new_thread_trampoline")
      [No Locals]

good morning;

i have noticed that a long-running sbcl accumulates "unreferenced"  
dynamic objects.
the immediate symptom is that entries remain in weak-keyed hash  
tables for which the only initial reference was a term in an  
anonymous lambda expression.

if i try something obvious like

(loop (let ((x (let ((object (copy-seq "asdf")))
                  (setf (literal-language-tag object) "EX")
                  (compile nil `(lambda (x)
                                  ,@(loop for i from 0 below 1000000  
collect (list 'incf 'x))
                                  x)))))
         (unless (typep x 'function) (return))))

after a short time, an image looks like

* (sb-ext:gc)

NIL
* (room)

Dynamic space usage is:   145,221,152 bytes.
Read-only space usage is:      3,504 bytes.
Static space usage is:         2,832 bytes.
Control stack usage is:        1,208 bytes.
Binding stack usage is:          344 bytes.
Control and binding stack usage is for the current thread only.
Garbage collection is currently enabled.

Breakdown for dynamic space:
   64,998,080 bytes for 1,383,921 instance objects.
   26,525,336 bytes for 3,315,667 cons objects.
   18,381,728 bytes for   196,759 simple-vector objects.
   17,850,176 bytes for    26,719 code objects.
   18,087,184 bytes for   437,334 other objects.
   145,842,504 bytes for 5,360,400 dynamic objects (space total.)
*

if i compile functions each with a "fingerprinted" constituent form,  
_various_ of the forms are retained:

* (let ((sym (gensym)))
   (dotimes (x 10) (compile nil `(lambda (x) (equal x '(,sym ,x)))))

   (dotimes (i 3) (sb-ext:gc))

   (let ((found nil))
     (dotimes (x 10)
       (let ((list `(,sym ,x)))
         (sb-vm::map-allocated-objects #'(lambda (object code size)
                                           (declare (ignore code size))
                                           (when (and (equal object  
list) (not (eq object list)))
                                             (push object found)))
                                       :dynamic)))
     found))

((#:G622 9) (#:G622 8) (#:G622 7) (#:G622 6) (#:G622 4) (#:G622 3)  
(#:G622 2)
  (#:G622 1) (#:G622 0))
* (let ((sym (gensym)))
   (dotimes (x 10) (compile nil `(lambda (x) (equal x '(,sym ,x)))))

   (dotimes (i 10) (sb-ext:gc))

   (let ((found nil))
     (dotimes (x 10)
       (let ((list `(,sym ,x)))
         (sb-vm::map-allocated-objects #'(lambda (object code size)
                                           (declare (ignore code size))
                                           (when (and (equal object  
list) (not (eq object list)))
                                             (push object found)))
                                       :dynamic)))
     found))

((#:G623 9) (#:G623 8) (#:G623 7) (#:G623 6) (#:G623 5) (#:G623 4)  
(#:G623 3)
  (#:G623 1) (#:G623 0))
*

if i change the report, the result differ:

* (let ((sym (gensym)))
   (dotimes (x 10) (compile nil `(lambda (x) (equal x '(,sym ,x)))))

   (dotimes (i 10) (sb-ext:gc))

   (let ((found 0))
     (dotimes (x 10)
       (let ((list `(,sym ,x)))
         (sb-vm::map-allocated-objects #'(lambda (object code size)
                                           (declare (ignore code size))
                                           (when (and (equal object  
list) (not (eq object list)))
                                             (print (cons x (incf  
found)))
                                             (sb-vm::map-referencing- 
objects #'print :dynamic object)))
                                       :dynamic)))
     found))

(0 . 1)
((#:G627 0))
#(#<HASH-TABLE :TEST EQ :COUNT 7 {AB43B89}> SB-IMPL::%EMPTY-HT-SLOT%
   (LAMBDA (X) (EQUAL X '(#:G627 0))) (SB-C::ORIGINAL-SOURCE-START 0  
0) (X)
   (SB-C::ORIGINAL-SOURCE-START 1 1 0) (EQUAL X '(#:G627 0))
   (SB-C::ORIGINAL-SOURCE-START 2 2 0) (X '(#:G627 0))
   (SB-C::ORIGINAL-SOURCE-START 3 1 2 0) '(#:G627 0)
   (SB-C::ORIGINAL-SOURCE-START 3 2 2 0) (#:G627 0)
   (SB-C::ORIGINAL-SOURCE-START 4 1 2 2 0) (0)
   (SB-C::ORIGINAL-SOURCE-START 5 1 1 2 2 0) SB-IMPL::%EMPTY-HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT%)
#((#:G627 0) #<SB-KERNEL:CONS-TYPE CONS>)
#<code object (LAMBDA (X)) {ABD6A47}>
(1 . 2)
#((#:G627 1) #<SB-KERNEL:CONS-TYPE CONS>)
(2 . 3)
#((#:G627 2) #<SB-KERNEL:CONS-TYPE CONS>)
(3 . 4)
#((#:G627 3) #<SB-KERNEL:CONS-TYPE CONS>)
(4 . 5)
#((#:G627 4) #<SB-KERNEL:CONS-TYPE CONS>)
(5 . 6)
#((#:G627 5) #<SB-KERNEL:CONS-TYPE CONS>)
(6 . 7)
#((#:G627 6) #<SB-KERNEL:CONS-TYPE CONS>)
(7 . 8)
#((#:G627 7) #<SB-KERNEL:CONS-TYPE CONS>)
(8 . 9)
#<code object (LAMBDA (X)) {ABC503F}>
#((#:G627 8) #<SB-KERNEL:CONS-TYPE CONS>)
((#:G627 8))
#<SB-KERNEL:CONSTANT :VALUE (#:G627 8) {ABF05B1}>
(9 . 10)
((#:G627 9))
#(#<HASH-TABLE :TEST EQ :COUNT 7 {ABC70A9}> SB-IMPL::%EMPTY-HT-SLOT%
   (LAMBDA (X) (EQUAL X '(#:G627 9))) (SB-C::ORIGINAL-SOURCE-START 0  
0) (X)
   (SB-C::ORIGINAL-SOURCE-START 1 1 0) (EQUAL X '(#:G627 9))
   (SB-C::ORIGINAL-SOURCE-START 2 2 0) (X '(#:G627 9))
   (SB-C::ORIGINAL-SOURCE-START 3 1 2 0) '(#:G627 9)
   (SB-C::ORIGINAL-SOURCE-START 3 2 2 0) (#:G627 9)
   (SB-C::ORIGINAL-SOURCE-START 4 1 2 2 0) (9)
   (SB-C::ORIGINAL-SOURCE-START 5 1 1 2 2 0) SB-IMPL::%EMPTY-HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY- 
HT-SLOT%
   SB-IMPL::%EMPTY-HT-SLOT% SB-IMPL::%EMPTY-HT-SLOT%)
#((#:G627 9) #<SB-KERNEL:CONS-TYPE CONS>)
#<code object (LAMBDA (X)) {ABD6BDF}>
#<SB-KERNEL:CONSTANT :VALUE (#:G627 9) {ABDB2E1}>
10
*

if i change the lambda form, the results differ again:

* (let ((sym (gensym)))
   (dotimes (x 10) (compile nil `(lambda (x) (equal x #((,sym ,x))))))

   (dotimes (i 10) (sb-ext:gc))

   (let ((found 0))
     (dotimes (x 10)
       (let ((list `(,sym ,x)))
         (sb-vm::map-allocated-objects #'(lambda (object code size)
                                           (declare (ignore code size))
                                           (when (and (equal object  
list) (not (eq object list)))
                                             (print (cons x (incf  
found)))
                                             (sb-vm::map-referencing- 
objects #'print :dynamic object)))
                                       :dynamic)))
     found))

(0 . 1)
((#:G628 0))
#((#:G628 0))
(8 . 2)
#((#:G628 8))
(9 . 3)
((#:G628 9))
#((#:G628 9))
3
*


that is, various of the ten forms have been retained.
a read through src/compiler/ didn't reveal an obvious reason for this
? am i oblivious to some feature inherent to compiled functions?
? is there some way to control this behaviour?
? is map-allocated-object imperfect, or were the missing values  
really gc'd?

* (lisp-implementation-version)

"1.0.36"
* (lisp-implementation-type)

"SBCL"
* *features*

(:ANSI-CL :COMMON-LISP :SBCL :SB-DOC :SB-TEST :SB-LDB :SB-PACKAGE-LOCKS
  :SB-UNICODE :SB-EVAL :SB-SOURCE-LOCATIONS :IEEE-FLOATING- 
POINT :X86 :UNIX :ELF
  :LINUX :SB-THREAD :LARGEFILE :GENCGC :STACK-GROWS-DOWNWARD-NOT-UPWARD
  :C-STACK-IS-CONTROL-STACK :COMPARE-AND-SWAP-VOPS :UNWIND-TO-FRAME- 
AND-CALL-VOP
  :RAW-INSTANCE-INIT-VOPS :STACK-ALLOCATABLE-CLOSURES :STACK- 
ALLOCATABLE-VECTORS
  :STACK-ALLOCATABLE-LISTS :STACK-ALLOCATABLE-FIXED-OBJECTS :ALIEN- 
CALLBACKS
  :CYCLE-COUNTER :INLINE-CONSTANTS :LINKAGE-TABLE :OS-PROVIDES-DLOPEN
  :OS-PROVIDES-PUTWC :OS-PROVIDES-SUSECONDS-T)

Hi all,

My move to Quicklisp has highlighted something that may or may not be of
interest to this list.

I have a simple script that I use to build cores and all I really had to do
make the switch was replace calls to ASDF:OPERATE with calls to QL:QUICKLOAD.

Anyway, the first time I ran the script, Quicklisp had to fetch all the
libraries from the Quicklisp repository before it could load/compile them.  The
resulting core was 53452828 bytes (52M) in size (and Quicklisp certainly lived
up to its name).

I ran the script again, and this time no fetching was necessary.  The resulting
core was 51425308 bytes (50M) in size.

That's my first observation and I've no doubt there are perfectly good reasons
for the discrepancy.

My second observation (and the reason I'm writing to this list) involves core
size across different architectures.

I ran the same Quicklisp script (without fetching) on a 64 bit box with two
(physical) cores and the resulting core was 80609328 bytes (77M) in size.

Debian GNU/Linux is installed on both machines, but the first machine is an
ordinary 32 bit box with a single (physical) core.

To summarise:

 32 bit single core with fetching: 53452828 bytes (52M)
 32 bit single core   no fetching: 51425308 bytes (50M)
 64 bit   dual core   no fetching: 80609328 bytes (77M)

SBCL version 1.0.46.

Seb 
-- 
Emacs' AlsaPlayer - Music Without Jolts
Lightweight, full-featured and mindful of your idyllic happiness.
http://home.gna.org/eap

So I've been working on a new image library called opticl that is designed to be something of a successor to my ch-image processing library. The source for opticl can be found here:

https://github.com/slyrus/opticl

But the SBCL-related issue I'm interested in is trying to wring as much performance out of SBCL's relatively-efficient array operations as possible, within reason anyway. First a little background about the way I did things in ch-image, and then on to the issue at hand with opticl. In ch-image, I used classes (and metaclasses) from my CLEM matrix package to represent arrays. This was fine enough, but required depending on CLEM and a lot of baggage folks might not want in an application or library that seeks to use opticl. So I decided to try to keep things a bit simpler this time. I decided to try using lisp arrays directly to represent images. One choice would be to use a 1D array and store the actual dimensions of the image elsewhere, but I figured I'd give using 2-d (for monochannel) and 3-d (for multichannel, RGB and RGBA) arrays to represent images. So far so good, and I've managed to get relatively efficient non-consing access to the image data with aref and setf aref (and friends).

I was curious as to how quickly I could perform simple operations on images, so I came up with this simple benchmark:

(defun foo-rgba/pixel ()
  (declare (optimize (speed 3) (safety 0)))
  (let ((height 1080) (width 1920))
    (let ((img (make-8-bit-rgba-image height width)))
      (declare (type 8-bit-rgba-image img))
      (time (loop for z fixnum below 10
               do (loop for i fixnum below height
                     do (loop for j fixnum below width 
                           do (multiple-value-bind (r g b a)
                                  (pixel img i j)
                                (declare (type (unsigned-byte 8) r g b a))
                                (setf (pixel img i j)
                                      (values (incf r)
                                              (incf g)
                                              (incf b)
                                              (incf a))))))))
      img)))

which just loops over the image 10 times, adding 1 to each channel of each pixel, using my pixel function and pixel setf expander. Performance is pretty good:

OPTICL-BENCHMARK> (defparameter q (foo-rgba/pixel))
Evaluation took:
  0.383 seconds of real time
  0.378170 seconds of total run time (0.370593 user, 0.007577 system)
  98.69% CPU
  814,054,184 processor cycles
  0 bytes consed
  

But I wanted to see how much better I could do by playing with the array accesses a bit. One idea was to use row-major-aref:

(defun foo-rgba/row-major ()
  (declare (optimize (speed 3) (safety 0)))
  (let ((height 1080) (width 1920))
    (let ((img (make-8-bit-rgba-image height width)))
      (declare (type 8-bit-rgba-image img))
      (time (loop for z fixnum below 10
               do (loop for i fixnum below height
                     do (loop for j fixnum from (* i width 4) below (* (1+ i) width 4) by 4
                           do
                           (incf (row-major-aref img j))
                           (incf (row-major-aref img (+ j 1)))
                           (incf (row-major-aref img (+ j 2)))
                           (incf (row-major-aref img (+ j 3)))))))
      img)))


This yields results such as the following:

OPTICL-BENCHMARK> (defparameter q (foo-rgba/row-major))
Evaluation took:
  0.156 seconds of real time
  0.152853 seconds of total run time (0.145918 user, 0.006935 system)
  98.08% CPU
  330,708,072 processor cycles
  0 bytes consed
  
This is clearly a pretty big win but I thought maybe I could keep going. this led me to:

(defun foo-rgba/1d-array ()
  (declare (optimize (speed 3) (safety 0)))
  (let ((height 1080) (width 1920))
    (let ((img (make-8-bit-rgba-image height width)))
      (declare (type 8-bit-rgba-image img))
      (let ((img2 (make-array (* height width 4)
                              :element-type (array-element-type img)
                              :initial-contents
                              (make-array (* height width 4)
                                          :element-type (array-element-type img)
                                          :displaced-to img))))
        (declare (type (simple-array (unsigned-byte 8) *) img2))
        (time (loop for z fixnum below 10
                 do (loop for i fixnum below height
                       do (loop for j fixnum from (* i width 4) below (* (1+ i) width 4) by 4
                             do
                               (incf (aref img2 j))
                               (incf (aref img2 (+ j 1)))
                               (incf (aref img2 (+ j 2)))
                               (incf (aref img2 (+ j 3))))))))
      img)))

which is even better:

OPTICL-BENCHMARK> (defparameter q (foo-rgba/1d-array))
Evaluation took:
  0.127 seconds of real time
  0.126291 seconds of total run time (0.126233 user, 0.000058 system)
  99.21% CPU
  268,891,560 processor cycles
  0 bytes consed
  

However, this requires copying the array, which I'd really rather not do. Nikodemus was nice enough to point out that if I'm willing to live with unportable code there is the sb-c::with-array-data macro, using which I get:

(defun foo-rgba/with-array-data ()
   (declare (optimize (speed 3) (safety 0)))
  (let ((height 1080) (width 1920))
    (let ((img (make-8-bit-rgba-image height width)))
      (declare (type 8-bit-rgba-image img))
      (sb-c::with-array-data ((img-data img)
                              (start)
                              (end))
        (declare (type (simple-array (unsigned-byte 8) *) img-data))
        (time (loop for z fixnum below 10
                 do (loop for i fixnum below height
                       do (loop for j fixnum from (* i width 4) below (* (1+ i) width 4) by 4
                             do
                             (incf (aref img-data j))
                             (incf (aref img-data (+ j 1)))
                             (incf (aref img-data (+ j 2)))
                             (incf (aref img-data (+ j 3))))))))
      img)))

and time then says:

OPTICL-BENCHMARK> (defparameter q (foo-rgba/with-array-data))
Evaluation took:
  0.136 seconds of real time
  0.136172 seconds of total run time (0.129588 user, 0.006584 system)
  100.00% CPU
  289,283,488 processor cycles
  0 bytes consed

So this is almost, but not quite, as good as the copied-to-1d-array case above. Clearly, the row-major-aref case gives a nice, big, portable win and doesn't require copying the array, but it might be nice to wring a little more performance out of the with-array-data case. Any ideas as to why with-array-data is almost, but not quite, as fast as the 1d case?

thanks,

Cyrus

On 11 June 2010 22:48, Matt Kaufmann <kaufmann@...> wrote:

> Below is the message I got before adding the --dynamic-space-size
> argument.  I wonder if some SBCL expert could figure out how to print
> a useful message suggesting an appropriate value to try for
> --dynamic-space-size.  I suppose just suggesting the use of
> --dynamic-space-size in this message could be quite useful.

Working through my backlog -- added

  https://bugs.launchpad.net/sbcl/+bug/720612

as a reminder of this. Thanks for the suggestion!

Cheers,

 -- Nikodemus

Hi All,

I'm teaching myself Lisp using SBCL (1.0.40.0.debian).  I have an
application (myfitnessdata), which uses the following code to process
command line arguments:

(:use :common-lisp)
(:export #:main))

(in-package :myfitnessdata)

(require :sb-posix)

;; snip

(defun main ()
  (if (= (length sb-ext:*posix-argv*) 3)
    (let ((username (nth 0 sb-ext:*posix-argv*))
      (password (nth 1 sb-ext:*posix-argv*))
      (path (nth 2 sb-ext:*posix-argv*)))
      (scrape (username password path)))
    (show-usage)))

I am compiling this application using the following Lisp code:

(load "myfitnessdata.lisp")
(save-lisp-and-die "myfitnessdata.bin" :executable t :toplevel
'myfitnessdata:main)

This produces an executable that I can run. However, the
sb-ext:*posix-argv* list is always empty. If I run the following:

./myfitnessdata.bin myusername mypassword /home/me/data

... then all I get is the usage instructions produced by (show-usage).

I think I must be missing something simple - could someone please let
me know how to (save-lisp-and-die) such that the resultant executable
takes command line arguments?  I suspect that the saved core contains
the empty command line argument list, but I'm not sure how to instruct
save-lisp-or-die to produce a core that replaces that empty list with
whatever is passed from the CLI.

TIA for any help :-)

-- 
{ :phone => '(+61) 420 817 082',  :skype => 'duncan_bayne',
  :email => 'dhgbayne@...', :web => 'http://www.fluidscape.com/' }

"If you're in a fair fight, you didn't plan it properly."
 - Nick Lappos, Chief R&D Pilot, Sikorsky Aircraft

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

		     4th European Lisp Symposium
	      Special Focus on Parallelism & Efficiency

		      March 31 - April 1st, 2011
		TUHH, Hamburg University of Technology
			   Hamburg, Germany

	       http://www.european-lisp-symposium.org/

Sponsored by EPITA, TUHH, Lispworks, Franz Inc., NovaSparks and Freiheit

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Registration is now open!
See http://www.european-lisp-symposium.org/content-registration-full.html
for details.

The deadline for early registration is March 12.
There is a reduced fee for students and accompanying persons.
You may also subscribe to mailing lists for this year's occurrence
on the registration page.



Invited Speakers:
~~~~~~~~~~~~~~~~~
Craig Zilles -- Compiling for the common case
Marc Battyani -- Reconfigurable computing on steroids
Apostolos Syropoulos -- Scala: an OO surprise

The complete program will be available shortly.



Scope
~~~~~~
The purpose of the European Lisp Symposium is to provide a forum for
the discussion and dissemination of all aspects of design,
implementation and application of any of the Lisp dialects, including
Common Lisp, Scheme, Emacs Lisp, AutoLisp, ISLISP, Dylan, Clojure,
ACL2, ECMAScript, Racket, SKILL and so on. We encourage everyone
interested in Lisp to participate.



Programme Chair
~~~~~~~~~~~~~~~~
Didier Verna - EPITA Research and Development Laboratory, France

Local Chair
~~~~~~~~~~~~
Ralf Moeller - Hamburg University of Technology, Germany

Programme Committee
~~~~~~~~~~~~~~~~~~~~
Antonio Leitao - Instituto Superior Tecnico/INESC-ID, Portugal
Christophe Rhodes - Goldsmiths College, University of London, UK
David Edgar Liebke - Relevance Inc., USA
Didier Verna - EPITA Research and Development Laboratory, France
Henry Lieberman - MIT Media Laboratory, USA
Jay McCarthy - Brigham Young University, USA
Jose Luis Ruiz Reina - Universidad de Sevilla, Spain
Marco Antoniotti - Universita Milano Bicocca, Italy
Manuel Serrano - INRIA, France
Michael Sperber - DeinProgramm, Germany
Pascal Costanza - Vrije Universiteit of Brussel, Belgium
Scott McKay - ITA Software, USA

-- 
Resistance is futile. You will be jazzimilated.

Scientific site:   http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com

On 10 January 2011 21:50, Blake McBride <blake@...> wrote:

> I have a need to be able to delete a package (and presumably free its
> memory).  I ran some tests and found that both CLISP and ABCL return
> free'd package space (with perhaps some leaks) but SBCL does not.  It
> seems that if you repeatably load a package, use it, and then delete
> it, you continue to use more and more space.  The test files I am
> using are as follows:  (memtest2 must be compiled to run all of the
> tests)

Yes. This is a known issue -- the crux is that SBCL stores information
about variables, functions, etc. in non-weak global data-structures.

Currently there is no supported (or even unsupported but usable) way
to completely purge all information associated with a package.

The plan (absent time to implement it) is to associate the information
now stored in the global data-structures with the relevant symbols
instead.

Cheers,

 -- Nikodemus

Roman Marynchak <roman.marynchak@...> writes:

> Hello,
>
>  I want to debug the code which is generated and compiled during the program execution. While there is a workaround (I can dump the code into a file and load it on the fly), I would like to ask for a more
> elegant solution (if any).
>
>   The next very simplified code demonstrates the real-life problem:
>
> (declaim (optimize (debug 3) (speed 0) (safety 3)))
>
> (defun generate-lambda (x)
>   `(lambda ()
>      (let ((y ,(+ x 7))
>             (z ,(* x x)))
>        (break)
>        (print y)
>        (print z))))
>
> (defun test ()
>   (let ((lm (generate-lambda 8)))
>     (funcall (compile nil lm))))
>
> After this code is compiled and loaded, I evaluate (test) and SLIME
> shows that there is no information about the locals in the generated
> lambda:
>
> Backtrace:
>   0: (BREAK "break")
>   1: ((LAMBDA ()))
>       [No Locals]
>   2: (TEST)
>       Locals:
>         LM = (LAMBDA () ..)
>   3: (SB-INT:SIMPLE-EVAL-IN-LEXENV (TEST) #<NULL-LEXENV>)
>   4: ((LAMBDA ()))
>
> Is there any way to make these local variables visible?

My guess is that despite (debug 3), the compiler optimize out the
useless variables. 

My advice would be to make use of the variable, ie. to mutate them:

 (defun generate-lambda (x)
   `(lambda ()
      (let ((y ,(+ x 7))
            (z ,(* x x)))
        (break)
        (print y)
        (print z)
        (setf y (+ y z)
              z (- y z)
              y (- y z)))))



-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.