From: Nicolas N. <ne...@ma...> - 2007-08-29 16:59:26
|
Hello, I would like to do some large-scale calculations on a 16 GB machine (AMD 64) using SBCL. Unfortunately, it looks as if GC is failing to reclaim memory correctly. I had this problem before (thread from March, 13th, 2006). At that time the fix was setting (def!constant gencgc-page-size 32768) in parms.lisp. However, this remedy (now in backend-parms.lisp) unfortunately fails for current SBCL. Here is a test case: --------------------------------------------------------------------------- neuss@ma-patru:~/CL-HOME/sbcl$ sbcl --dynamic-space-size 15000 This is SBCL 1.0.9.5, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. ; loading system definition from /usr/local/lib/sbcl/sb-grovel/sb-grovel.asd ; into #<PACKAGE "ASDF1"> ; registering #<SYSTEM SB-GROVEL {100255AE71}> as SB-GROVEL * (defun test (&optional (n1 10) (n2 500) (n3 1000000)) "Sets *test* n1 times to a list of n2 double-float arrays of size n3." (let ((space ())) (format t "~&Calculation should need ~D MB.~%~%" (floor (/ (* 8 n2 n3) 1000000))) (loop for i from 1 upto n1 do (format t "~&~%i=~D~%" i) (time (progn (setq space ()) (setq space (loop repeat n2 collect (make-array n3 :element-type 'double-float :initial-element 0.0d0))) (room)))))) TEST * (test 20) Calculation should need 4000 MB. i=1 Dynamic space usage is: 4,041,661,136 bytes. Read-only space usage is: 1,664 bytes. Static space usage is: 3,872 bytes. Control stack usage is: 2,496 bytes. Binding stack usage is: 688 bytes. Control and binding stack usage is for the current thread only. Garbage collection is currently enabled. Breakdown for dynamic space: 4,000,008,016 bytes for 501 simple-array-double-float objects. 41,661,776 bytes for 695,331 other objects. 4,041,669,792 bytes for 695,832 dynamic objects (space total.) Evaluation took: 1.55f0 seconds of real time 1.508094f0 seconds of user run time 0.028002f0 seconds of system run time [Run times include 1.436f0 seconds GC run time.] 0 calls to %EVAL 0 page faults and 4,000,061,760 bytes consed. i=2 Dynamic space usage is: 3,746,763,744 bytes. ; bug (probably 32bit-integer overflow) Read-only space usage is: 1,664 bytes. Static space usage is: 3,872 bytes. Control stack usage is: 2,496 bytes. Binding stack usage is: 688 bytes. Control and binding stack usage is for the current thread only. Garbage collection is currently enabled. Breakdown for dynamic space: 8,000,016,016 bytes for 1,001 simple-array-double-float objects. 41,731,744 bytes for 697,302 other objects. 8,041,747,760 bytes for 698,303 dynamic objects (space total.) Evaluation took: 2.711f0 seconds of real time 2.652166f0 seconds of user run time 0.048003f0 seconds of system run time [Run times include 2.488f0 seconds GC run time.] 0 calls to %EVAL 0 page faults and 0 bytes consed. i=3 Dynamic space usage is: 3,451,866,288 bytes. Read-only space usage is: 1,664 bytes. Static space usage is: 3,872 bytes. Control stack usage is: 2,496 bytes. Binding stack usage is: 688 bytes. Control and binding stack usage is for the current thread only. Garbage collection is currently enabled. Breakdown for dynamic space: 12,000,024,016 bytes for 1,501 simple-array-double-float objects. 41,801,600 bytes for 699,269 other objects. 12,041,825,616 bytes for 700,770 dynamic objects (space total.) Evaluation took: 3.857f0 seconds of real time 3.84024f0 seconds of user run time 0.012001f0 seconds of system run time [Run times include 3.521f0 seconds GC run time.] 0 calls to %EVAL 0 page faults and 0 bytes consed. i=4 Heap exhausted during allocation: 393216 bytes available, 8000016 requested. debugger invoked on a SB-KERNEL::HEAP-EXHAUSTED-ERROR in thread #<THREAD "initial thread" {10023D8ED1}>: Heap exhausted: 393216 bytes available, 8000016 requested. PROCEED WITH CAUTION! Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [ABORT] Exit debugger, returning to top level. (SB-KERNEL::HEAP-EXHAUSTED-ERROR 393216 8000016) 0] 0 * (quit) --------------------------------------------------------------------------- Can somebody help me? If necessary, I could provide an account on my machine for tracking down the problem. Thank you very much, Nicolas |
From: Nicolas N. <ne...@ma...> - 2007-08-31 09:55:57
|
Hello, here is an update: the problem is not correlated with the X86-64 platform. E.g., (defun test (&optional (n1 20) (n2 500) (n3 1000000)) "Allocates n1 times a list of n2 double-float arrays of size n3." (let ((space ())) (format t "~&Calculation should need ~D MB.~%~%" (floor (/ (* 8 n2 n3) 1000000))) (loop for i from 1 upto n1 do (format t "~&~%i=~D~%" i) (time (progn (setq space ()) (setq space (loop repeat n2 collect (make-array n3 :element-type 'double-float :initial-element 0.0d0))) (room)))))) (test 500 5) breaks SBCL on a 1GB machine. Nicolas PS: Probably you have seen it already, but to be sure: ctc...@gm... has reported a GC bug (probably the same) in cll on 29.8.2007. |