From: Simon H. <sha...@ya...> - 2008-12-31 20:22:25
|
I'm trying to track down a possible memory leak associated with SAPs, and I'm hoping that someone else has already seen this and I've just stupidly missed an obvious solution in the list archives. First, I'm using both 1.0.20 and 1.0.23 on a Macbook Pro (Darwin 10.5.6), 32 bit version. My features are: (:MYSQL-CLIENT-V5 :DYN-EXT :ASDF :ANSI-CL :COMMON-LISP :SBCL :SB-DOC :SB-TEST :SB-LDB :SB-PACKAGE-LOCKS :SB-UNICODE :SB-EVAL :SB-SOURCE-LOCATIONS :IEEE-FLOATING-POINT :X86 :UNIX :MACH-O :BSD :DARWIN :MACH-EXCEPTION-HANDLER :SB-LUTEX :RESTORE-FS-SEGMENT-REGISTER-FROM-TLS :GENCGC :STACK-GROWS-DOWNWARD-NOT-UPWARD :C-STACK-IS-CONTROL-STACK :COMPARE-AND-SWAP-VOPS :UNWIND-TO-FRAME-AND-CALL-VOP :RAW-INSTANCE-INIT-VOPS :STACK-ALLOCATABLE-CLOSURES :ALIEN-CALLBACKS :CYCLE-COUNTER :LINKAGE-TABLE :OS-PROVIDES-DLOPEN :OS-PROVIDES-DLADDR :OS-PROVIDES-PUTWC :OS-PROVIDES-BLKSIZE-T :OS-PROVIDES-SUSECONDS-T) I usually use slime, but I'm deliberately NOT using it for these tests so as to keep things simple. First, I have code that simply walks thru the heap and counts up SAPs (mem-test.lisp): (in-package :cl-user) (defun primitive-obj-alloc-details (obj-type) "returns two values, #objs and #bytes" (let ((count 0) (total-bytes 0)) (sb-vm::map-allocated-objects #'(lambda (obj type bytes) (declare (ignore obj)) (when (eql type obj-type) (incf count) (incf total-bytes bytes))) :dynamic) (values count total-bytes))) (defun report-sap () (multiple-value-bind (count bytes) (primitive-obj-alloc-details 70) (format t "~&sap: ~15:D objs ~15:D bytes~%" count bytes))) If, from the shell, I fire up sbcl & then just run (report-sap) a bunch of times then all is OK: * bash-3.2$ sbcl This is SBCL 1.0.23, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. * (load #p"/Users/simon/mem-test.lisp") T * (loop repeat 100000 do (report-sap)) sap: 5,322 objs 42,576 bytes sap: 71,265 objs 570,120 bytes sap: 139,016 objs 1,112,128 bytes sap: 209,069 objs 1,672,552 bytes sap: 281,170 objs 2,249,360 bytes sap: 355,575 objs 2,844,600 bytes sap: 432,284 objs 3,458,272 bytes sap: 49,442 objs 395,536 bytes sap: 126,919 objs 1,015,352 bytes sap: 204,384 objs 1,635,072 bytes sap: 281,861 objs 2,254,888 bytes sap: 359,322 objs 2,874,576 bytes sap: 436,799 objs 3,494,392 bytes [.... ad infinitum...] However, if I load one of my modules, then all hell breaks loose: bash-3.2$ sbcl This is SBCL 1.0.23, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. * (l utils) ; loading system definition from /Users/simon/view/asdf/utils.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM UTILS {11D7E3D1}> as UTILS ; loading system definition from /Users/simon/view/asdf/sbcl/acl-compat.asd ; into #<PACKAGE "ASDF0"> ; registering #<SYSTEM ACL-COMPAT {11925359}> as ACL-COMPAT ; loading system definition from /usr/local/lib/sbcl/sb-posix/sb-posix.asd into ; #<PACKAGE "ASDF0"> ; loading system definition from /usr/local/lib/sbcl/sb-grovel/sb-grovel.asd ; into #<PACKAGE "ASDF1"> ; registering #<SYSTEM SB-GROVEL {11ADDF59}> as SB-GROVEL ; registering #<SYSTEM SB-POSIX {11E7E099}> as SB-POSIX ; registering #<SYSTEM SB-POSIX-TESTS {11F998A1}> as SB-POSIX-TESTS ; loading system definition from ; /usr/local/lib/sbcl/sb-bsd-sockets/sb-bsd-sockets.asd into #<PACKAGE "ASDF0"> ; registering #<SYSTEM SB-BSD-SOCKETS {11A89249}> as SB-BSD-SOCKETS ; registering #<SYSTEM SB-BSD-SOCKETS-TESTS {11C074B1}> as SB-BSD-SOCKETS-TESTS ; loading system definition from /usr/local/lib/sbcl/sb-md5/sb-md5.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM SB-MD5 {11E87619}> as SB-MD5 ; registering #<SYSTEM SB-MD5-TESTS {11FF97E9}> as SB-MD5-TESTS ; loading system definition from ; /usr/local/lib/sbcl/sb-rotate-byte/sb-rotate-byte.asd into #<PACKAGE "ASDF0"> ; registering #<SYSTEM SB-ROTATE-BYTE {12285769}> as SB-ROTATE-BYTE ; loading system definition from /Users/simon/view/asdf/cl-ppcre.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM :CL-PPCRE {12452BB9}> as CL-PPCRE ; registering #<SYSTEM :CL-PPCRE-TEST {118DE499}> as CL-PPCRE-TEST ; loading system definition from /Users/simon/view/asdf/sbcl/puri.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM PURI {11A8A789}> as PURI ; registering #<SYSTEM PURI-TESTS {11BD8CA1}> as PURI-TESTS ; loading system definition from /Users/simon/view/asdf/sbcl/htmlgen.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM HTMLGEN {11F39629}> as HTMLGEN * (loop repeat 100000 do (report-sap)) sap: 1,272,912 objs 10,183,296 bytes sap: 3,437,863 objs 27,502,904 bytes sap: 4,708,015 objs 37,664,120 bytes [....] sap: 36,858,692 objs 294,869,536 bytes sap: 38,540,963 objs 308,327,704 bytes sap: 40,245,997 objs 321,967,976 bytes sap: 41,975,444 objs 335,803,552 bytes sap: 43,727,992 objs 349,823,936 bytes sap: 45,505,604 objs 364,044,832 bytes sap: 47,307,094 objs 378,456,752 bytes [....] sap: 62,649,828 objs 501,198,624 bytes sap: 64,689,354 objs 517,514,832 bytes sap: 66,756,430 objs 534,051,440 bytes sap: 68,852,878 objs 550,823,024 bytes sap: 70,978,218 objs 567,825,744 bytes sap: 78,426,581 objs 627,412,648 bytes Heap exhausted during allocation: 0 bytes available, 8 requested. Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc Waste Trig WP GCs Mem-age 0: 393214 0 0 0 384579 0 0 0 3 1574830568 405016 2010664 0 1 0.0000 1: 0 0 0 0 47 0 0 0 47 143104 49408 2000000 0 0 9.2529 2: 9691 9688 9346 0 1572 556 107 115 17 8964480 661120 2000000 1662 0 0.0000 3: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000 4: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000 5: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000 6: 0 0 0 0 6239 0 0 0 0 25554944 0 2000000 6010 0 0.0000 Total bytes allocated=1609493096 GC control variables: *GC-INHIBIT* = true *GC-PENDING* = true fatal error encountered in SBCL pid 2563: Heap exhausted, game over. Welcome to LDB, a low-level debugger for the Lisp runtime environment. ldb> (Both the above are 100% reproducible.) So it seems that loading this "utils" module causes something to break. Alas, this is a large+complicated module so I'm about to do some dividing+conquering to try to track down exactly what's causing this problem. However, I'm also hoping to avoid this many hours of work if someone else has already seen this problem and solved it. Many thanks! Simon Handley Palo Alto, CA |