From: Ilya P. <ipe...@dw...> - 2015-01-23 08:18:13
|
Hi everyone, I stumbled upon strange behavior of the GC: some objects allocated by a thread are not garbage collected till the thread exits, though the thread has no references to the objects. Ilya =================================================================== (use-package :sb-thread) (defun count-instance-of (type) (let ((counter 0)) (sb-vm::map-allocated-objects (lambda (obj typecode size) (declare (ignorable obj typecode size)) (when (typep obj type) (incf counter))) :dynamic) counter)) (defclass foo () ()) (defparameter *l* nil) (defun push-some-data-to-*l* (n-elts lock) (dotimes (k n-elts) (with-mutex (lock) (push (make-instance 'foo) *l*)))) (defun test-gc (&key (n-threads 1) (n-elts 1000)) (let* ((lock (make-mutex)) (q-exit (make-waitqueue)) (q-done (make-waitqueue)) (exit nil) (num-done 0) (threads (loop for i below n-threads collect (make-thread (lambda () (push-some-data-to-*l* n-elts lock) (with-mutex (lock) ;; Let the main thread know we are done. (incf num-done) (condition-notify q-done) ;; Wait till the main thread ask us to exit. (loop while (not exit) do (condition-wait q-exit lock)))))))) ;; Wait while the thread are pushing data to *L*. (with-mutex (lock) (loop while (< num-done n-threads) do (condition-wait q-done lock))) ;; Check what we have in memory. (setf *l* nil) (setf (gc-logfile) "/tmp/gc.log") (gc :full t) (format t "Num instances of FOO before stopping threads: ~a~%" (count-instance-of 'foo)) ;; Ask all the threads to exit. (with-mutex (lock) (setf exit t) (condition-broadcast q-exit)) (mapc #'join-thread threads) ;; Check what we have in memory. (gc :full t) (format t "Num instances of FOO after stopping threads: ~a~%" (count-instance-of 'foo)))) |
From: Stas B. <sta...@gm...> - 2015-01-23 10:01:57
|
Ilya Perminov <ipe...@dw...> writes: > Hi everyone, > > I stumbled upon strange behavior of the GC: some objects allocated by a thread > are not garbage collected till the thread exits, though the thread has no references > to the objects. The GC is conservative, so if anything in the same page as your object is determined as a potential pointer, the whole page will not be garbage collected. -- With best regards, Stas. |
From: Ilya P. <ipe...@dw...> - 2015-01-23 20:50:12
|
I know GC is conservative. I just do not understand what is going on in this case. I think the difference a thread makes to a set of roots is its stack and registers. At the point where GC runs the first time in my test, the thread is out of the function that creates/touches the data, so its stack should not have any references to the data, and values in the registers probably have short life time. In a real application I see this GC behavior with list-based queues. Sometimes the number of queue elements in memory is 100 times bigger than the maximum queue length. That is not a typical overhead of a conservative GC. |
From: Martin C. <cra...@co...> - 2015-01-23 17:08:50
|
Which version of SBCL? Does it have the patches that wipe unused parts of GC cards? I'm (still) about to provide or commit a better tracker for these situations. Also, make sure that if you issue the code below via the REPL, that you clear out *, ** and ***. The REPL will hold on to objects recently printed wrt GC. Martin Ilya Perminov wrote on Fri, Jan 23, 2015 at 08:06:02AM +0000: > Hi everyone, > > I stumbled upon strange behavior of the GC: some objects allocated by a thread > are not garbage collected till the thread exits, though the thread has no references > to the objects. > > Ilya > > =================================================================== > (use-package :sb-thread) > > (defun count-instance-of (type) > (let ((counter 0)) > (sb-vm::map-allocated-objects (lambda (obj typecode size) > (declare (ignorable obj typecode size)) > (when (typep obj type) > (incf counter))) > :dynamic) > counter)) > > (defclass foo () > ()) > > (defparameter *l* nil) > > (defun push-some-data-to-*l* (n-elts lock) > (dotimes (k n-elts) > (with-mutex (lock) > (push (make-instance 'foo) *l*)))) > > (defun test-gc (&key (n-threads 1) (n-elts 1000)) > (let* ((lock (make-mutex)) > (q-exit (make-waitqueue)) > (q-done (make-waitqueue)) > (exit nil) > (num-done 0) > (threads (loop for i below n-threads > collect (make-thread (lambda () > (push-some-data-to-*l* n-elts lock) > (with-mutex (lock) > ;; Let the main thread know we are done. > (incf num-done) > (condition-notify q-done) > ;; Wait till the main thread ask us to exit. > (loop while (not exit) do > (condition-wait q-exit lock)))))))) > ;; Wait while the thread are pushing data to *L*. > (with-mutex (lock) > (loop while (< num-done n-threads) > do (condition-wait q-done lock))) > ;; Check what we have in memory. > (setf *l* nil) > (setf (gc-logfile) "/tmp/gc.log") > (gc :full t) > (format t "Num instances of FOO before stopping threads: ~a~%" > (count-instance-of 'foo)) > ;; Ask all the threads to exit. > (with-mutex (lock) > (setf exit t) > (condition-broadcast q-exit)) > (mapc #'join-thread threads) > ;; Check what we have in memory. > (gc :full t) > (format t "Num instances of FOO after stopping threads: ~a~%" > (count-instance-of 'foo)))) > > ------------------------------------------------------------------------------ > New Year. New Location. New Benefits. New Data Center in Ashburn, VA. > GigeNET is offering a free month of service with a new server in Ashburn. > Choose from 2 high performing configs, both with 100TB of bandwidth. > Higher redundancy.Lower latency.Increased capacity.Completely compliant. > http://p.sf.net/sfu/gigenet > _______________________________________________ > Sbcl-help mailing list > Sbc...@li... > https://lists.sourceforge.net/lists/listinfo/sbcl-help -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer <cra...@co...> http://www.cons.org/cracauer/ |
From: Ilya P. <ipe...@dw...> - 2015-01-23 20:20:30
|
Martin Cracauer <cra...@co...> writes: > Which version of SBCL? Does it have the patches that wipe unused parts > of GC cards? SBCL 1.2.7, Linux x86_64. I'm not sure if it includes your patch. |
From: Ilya P. <ipe...@dw...> - 2015-02-03 21:39:21
|
It looks like WITH-MUTEX indirectly allocates a significant amount of stack space and leaves it uninitialized during execution of its body. It is completely legal and somewhat expected. Unfortunately a very typical pattern (process-data) (with-mutex (lock) (condition-wait ...) (grab-more-data)) may result in lots of false roots if the number of threads is high. I'll keep digging, but any ideas how to remedy the problem or what exactly causes allocation of uninitialized memory will be really helpful. Two examples below demonstrate the issue. The first one uses WITH-MUTEX and the second one is a stripped down version of the first one, which only uses UNWIND-PROTECT and FLET. The examples pre-fill some unused stack space with pattern 8 7 6 5 4 3 2 1 and then dump a part of allocated stack space from the body of WITH-MUTEX/UNWIND-PROTECT. As the results show in both cases the allocated stack space has many instance of 8 7 6 5 4 3 2 1. Thanks, Ilya =========================================================================== (require :alexandria) (use-package :alexandria) (defmacro dump-memory (start end) (once-only (start end) (with-gensyms (i) `(loop for ,i from 0 below (sb-sys:sap- ,end ,start) collect (sb-sys:sap-ref-8 ,start ,i))))) (defmacro fill-some-unused-stack-space (&optional sp) (with-gensyms (i p) `(loop with ,p = (or ,sp (sb-vm::current-sp)) for ,i fixnum from 8 below (* 8 1024) by 8 do (setf (sb-sys:sap-ref-64 ,p (- ,i)) #x0102030405060708)))) (defun test-stack-data1 () (declare (optimize speed (debug 0) (safety 0))) (let* ((sp (sb-vm::current-sp)) (lock (sb-thread:make-mutex))) (fill-some-unused-stack-space) (sb-thread::with-mutex (lock) (dump-memory (sb-vm::current-sp) sp)))) (defun test-stack-data2 () (declare (optimize speed (debug 0) (safety 0))) (let* ((sp (sb-vm::current-sp))) (fill-some-unused-stack-space) (flet ((func () (dump-memory (sb-vm::current-sp) sp))) (declare (notinline func)) (unwind-protect (func) nil)))) (test-stack-data1) ;; => (35 214 12 12 16 0 0 0 12 214 20 0 16 0 0 0 2 0 0 0 0 0 0 0 120 219 133 4 252 127 0 0 12 214 20 0 16 0 0 0 91 220 133 4 252 127 0 0 192 219 133 4 252 127 0 0 120 219 133 4 252 127 0 0 95 215 20 0 16 0 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 192 218 133 4 252 127 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 35 214 12 12 16 0 0 0 64 220 133 4 252 127 0 0 8 7 6 5 4 3 2 1 79 0 16 32 0 0 0 0 136 232 133 4 252 127 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 80 4 134 4 252 127 0 0 64 220 133 4 252 127 0 0 49 210 20 0 16 0 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 152 221 133 4 252 127 0 0 64 220 133 4 252 127 0 0 2 211 20 0 16 0 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 136 232 133 4 252 127 0 0 48 4 134 4 252 127 0 0 136 219 133 4 252 127 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 79 0 16 32 0 0 0 0 23 0 16 32 0 0 0 0 79 0 16 32 0 0 0 0 23 0 16 32 0 0 0 0 35 214 12 12 16 0 0 0 91 220 133 4 252 127 0 0 144 220 133 4 252 127 0 0 207 66 12 12 16 0 0 0 53 2 0 0 0 0 0 0 136 67 12 12 16 0 0 0 95 214 12 12 16 0 0 0 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1) (test-stack-data2) ;; => (8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 144 220 133 4 252 127 0 0 125 46 8 10 16 0 0 0) |
From: Ilya P. <ipe...@dw...> - 2015-02-04 01:22:53
|
I hacked WITHOUT-INTERRUPTS and WITH-MUTEX to inline helper functions (CALL-WITH-MUTEX, WITH-MUTEX-THUNK, and WITHOUT-INTERRUPTS-BODY). That solved the problem for my two small test cases. Though the original test case from my first post can't be compiled any more, it hits a compiler bug https://bugs.launchpad.net/sbcl/+bug/1417822. |