I'm trying to track down a possible memory leak associated with SAPs, and I'm hoping that someone else has already seen this and I've just stupidly missed an obvious solution in the list archives.

First, I'm using both 1.0.20 and 1.0.23 on a Macbook Pro (Darwin 10.5.6), 32 bit version.  My features are:

(:MYSQL-CLIENT-V5 :DYN-EXT :ASDF :ANSI-CL :COMMON-LISP :SBCL :SB-DOC :SB-TEST
 :SB-LDB :SB-PACKAGE-LOCKS :SB-UNICODE :SB-EVAL :SB-SOURCE-LOCATIONS
 :IEEE-FLOATING-POINT :X86 :UNIX :MACH-O :BSD :DARWIN :MACH-EXCEPTION-HANDLER
 :SB-LUTEX :RESTORE-FS-SEGMENT-REGISTER-FROM-TLS :GENCGC
 :STACK-GROWS-DOWNWARD-NOT-UPWARD :C-STACK-IS-CONTROL-STACK
 :COMPARE-AND-SWAP-VOPS :UNWIND-TO-FRAME-AND-CALL-VOP :RAW-INSTANCE-INIT-VOPS
 :STACK-ALLOCATABLE-CLOSURES :ALIEN-CALLBACKS :CYCLE-COUNTER :LINKAGE-TABLE
 :OS-PROVIDES-DLOPEN :OS-PROVIDES-DLADDR :OS-PROVIDES-PUTWC
 :OS-PROVIDES-BLKSIZE-T :OS-PROVIDES-SUSECONDS-T)

  I usually use slime, but I'm deliberately NOT using it for these tests so as to keep things simple.  First, I have code that simply walks thru the heap and counts up SAPs (mem-test.lisp):

(in-package :cl-user)
(defun primitive-obj-alloc-details (obj-type)
  "returns two values, #objs and #bytes"
  (let ((count 0) (total-bytes 0))
    (sb-vm::map-allocated-objects
     #'(lambda (obj type bytes)
     (declare (ignore obj))
     (when (eql type obj-type)
       (incf count)
       (incf total-bytes bytes)))
     :dynamic)
    (values count total-bytes)))
(defun report-sap ()
  (multiple-value-bind (count bytes) (primitive-obj-alloc-details 70)
    (format t "~&sap: ~15:D objs ~15:D bytes~%" count bytes)))

If, from the shell, I fire up sbcl & then just run (report-sap) a bunch of times then all is OK:

* bash-3.2$ sbcl
This is SBCL 1.0.23, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (load #p"/Users/simon/mem-test.lisp")

T
* (loop repeat 100000 do (report-sap))

sap:           5,322 objs          42,576 bytes
sap:          71,265 objs         570,120 bytes
sap:         139,016 objs       1,112,128 bytes
sap:         209,069 objs       1,672,552 bytes
sap:         281,170 objs       2,249,360 bytes
sap:         355,575 objs       2,844,600 bytes
sap:         432,284 objs       3,458,272 bytes
sap:          49,442 objs         395,536 bytes
sap:         126,919 objs       1,015,352 bytes
sap:         204,384 objs       1,635,072 bytes
sap:         281,861 objs       2,254,888 bytes
sap:         359,322 objs       2,874,576 bytes
sap:         436,799 objs       3,494,392 bytes
[.... ad infinitum...]

However, if I load one of my modules, then all hell breaks loose:

bash-3.2$ sbcl
This is SBCL 1.0.23, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.

* (l utils)

; loading system definition from /Users/simon/view/asdf/utils.asd into
; #<PACKAGE "ASDF0">
; registering #<SYSTEM UTILS {11D7E3D1}> as UTILS
; loading system definition from /Users/simon/view/asdf/sbcl/acl-compat.asd
; into #<PACKAGE "ASDF0">
; registering #<SYSTEM ACL-COMPAT {11925359}> as ACL-COMPAT
; loading system definition from /usr/local/lib/sbcl/sb-posix/sb-posix.asd into
; #<PACKAGE "ASDF0">
; loading system definition from /usr/local/lib/sbcl/sb-grovel/sb-grovel.asd
; into #<PACKAGE "ASDF1">
; registering #<SYSTEM SB-GROVEL {11ADDF59}> as SB-GROVEL
; registering #<SYSTEM SB-POSIX {11E7E099}> as SB-POSIX
; registering #<SYSTEM SB-POSIX-TESTS {11F998A1}> as SB-POSIX-TESTS
; loading system definition from
; /usr/local/lib/sbcl/sb-bsd-sockets/sb-bsd-sockets.asd into #<PACKAGE "ASDF0">
; registering #<SYSTEM SB-BSD-SOCKETS {11A89249}> as SB-BSD-SOCKETS
; registering #<SYSTEM SB-BSD-SOCKETS-TESTS {11C074B1}> as SB-BSD-SOCKETS-TESTS
; loading system definition from /usr/local/lib/sbcl/sb-md5/sb-md5.asd into
; #<PACKAGE "ASDF0">
; registering #<SYSTEM SB-MD5 {11E87619}> as SB-MD5
; registering #<SYSTEM SB-MD5-TESTS {11FF97E9}> as SB-MD5-TESTS
; loading system definition from
; /usr/local/lib/sbcl/sb-rotate-byte/sb-rotate-byte.asd into #<PACKAGE "ASDF0">
; registering #<SYSTEM SB-ROTATE-BYTE {12285769}> as SB-ROTATE-BYTE
; loading system definition from /Users/simon/view/asdf/cl-ppcre.asd into
; #<PACKAGE "ASDF0">
; registering #<SYSTEM :CL-PPCRE {12452BB9}> as CL-PPCRE
; registering #<SYSTEM :CL-PPCRE-TEST {118DE499}> as CL-PPCRE-TEST
; loading system definition from /Users/simon/view/asdf/sbcl/puri.asd into
; #<PACKAGE "ASDF0">
; registering #<SYSTEM PURI {11A8A789}> as PURI
; registering #<SYSTEM PURI-TESTS {11BD8CA1}> as PURI-TESTS
; loading system definition from /Users/simon/view/asdf/sbcl/htmlgen.asd into
; #<PACKAGE "ASDF0">
; registering #<SYSTEM HTMLGEN {11F39629}> as HTMLGEN

* (loop repeat 100000 do (report-sap))

sap:       1,272,912 objs      10,183,296 bytes
sap:       3,437,863 objs      27,502,904 bytes
sap:       4,708,015 objs      37,664,120 bytes
[....]
sap:      36,858,692 objs     294,869,536 bytes
sap:      38,540,963 objs     308,327,704 bytes
sap:      40,245,997 objs     321,967,976 bytes
sap:      41,975,444 objs     335,803,552 bytes
sap:      43,727,992 objs     349,823,936 bytes
sap:      45,505,604 objs     364,044,832 bytes
sap:      47,307,094 objs     378,456,752 bytes
[....]
sap:      62,649,828 objs     501,198,624 bytes
sap:      64,689,354 objs     517,514,832 bytes
sap:      66,756,430 objs     534,051,440 bytes
sap:      68,852,878 objs     550,823,024 bytes
sap:      70,978,218 objs     567,825,744 bytes
sap:      78,426,581 objs     627,412,648 bytes
Heap exhausted during allocation: 0 bytes available, 8 requested.
 Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB   LUB  !move  Alloc  Waste   Trig    WP  GCs Mem-age
   0: 393214     0     0     0 384579     0     0     0     3 1574830568 405016  2010664    0   1  0.0000
   1:     0     0     0     0    47     0     0     0    47   143104 49408  2000000    0   0  9.2529
   2:  9691  9688  9346     0  1572   556   107   115    17  8964480 661120  2000000 1662   0  0.0000
   3:     0     0     0     0     0     0     0     0     0        0     0  2000000    0   0  0.0000
   4:     0     0     0     0     0     0     0     0     0        0     0  2000000    0   0  0.0000
   5:     0     0     0     0     0     0     0     0     0        0     0  2000000    0   0  0.0000
   6:     0     0     0     0  6239     0     0     0     0 25554944     0  2000000 6010   0  0.0000
   Total bytes allocated=1609493096
GC control variables:
          *GC-INHIBIT* = true
          *GC-PENDING* = true
fatal error encountered in SBCL pid 2563:
Heap exhausted, game over.
Welcome to LDB, a low-level debugger for the Lisp runtime environment.
ldb>

(Both the above are 100% reproducible.)

So it seems that loading this "utils" module causes something to break.  Alas, this is a large+complicated module so I'm about to do some dividing+conquering to try to track down exactly what's causing this problem.  However, I'm also hoping to avoid this many hours of work if someone else has already seen this problem and solved it.

Many thanks!

Simon Handley
Palo Alto, CA