Steel Bank Common Lisp

In article <A9272832-C470-4938-9855-DCA16A8EA237@...>,
 james anderson <james.anderson@...> wrote:

> good evening, all;
> 
> it appears that memory which is allocated under "multi-threaded"  
> circumstances is not reliably released.
> once a thread which has allocated the memory has been destroyed and  
> no reference to the thread remains, the memory is released.
[...]
> 
> ---------------------------------------------------------------------
> ;;; queue memory test
> ;;;
> ;;; this demonstrates that memory which has been allocated in a new thread is 
> not released until the 
> ;;; thread itself has been released.

Nope, that's just the conservative GC in action; the bulk allocation and 
the GC logics is completely thread agnostic.  Arbitrary data on the 
stack (or in registers, more rarely) that look like pointers are treated 
as potential strong references.

That's usually not too much of an issue.  Queues built on top of conses 
are a particularly bad weakness, though: one stray reference to an 
already-dequeued cons also references all the later conses, via the cdr. 
That's why sb-queue clears all the fields of dequeued records.

(defun dq ()
  (sb-thread:wait-on-semaphore *queue-semaphore*)
  (if (eq *queue-pointer* *queue-header*)
    (error "released to an empty queue."))
  (let ((value (cadr *queue-header*)))  ;; That cons should be NILed out
    (setf *queue-header* (cdr *queue-header*))
    value))

Paul Khuong

good morning;

On 2012-02-23, at 20:09 , Paul Khuong wrote:

> In article <A9272832-C470-4938-9855-DCA16A8EA237@...>,
>  james anderson <james.anderson@...> wrote:
>
>> good evening, all;
>>
>> it appears that memory which is allocated under "multi-threaded"
>> circumstances is not reliably released.
>> once a thread which has allocated the memory has been destroyed and
>> no reference to the thread remains, the memory is released.
> [...]
>>
>> ---------------------------------------------------------------------
>> ;;; queue memory test
>> ;;;
>> ;;; this demonstrates that memory which has been allocated in a  
>> new thread is
>> not released until the
>> ;;; thread itself has been released.
>
> Nope, that's just the conservative GC in action; the bulk  
> allocation and
> the GC logics is completely thread agnostic.  Arbitrary data on the
> stack (or in registers, more rarely) that look like pointers are  
> treated
> as potential strong references.
>
> That's usually not too much of an issue.  Queues built on top of  
> conses
> are a particularly bad weakness, though: one stray reference to an
> already-dequeued cons also references all the later conses, via the  
> cdr.
> That's why sb-queue clears all the fields of dequeued records.
>
> (defun dq ()
>   (sb-thread:wait-on-semaphore *queue-semaphore*)
>   (if (eq *queue-pointer* *queue-header*)
>     (error "released to an empty queue."))
>   (let ((value (cadr *queue-header*)))  ;; That cons should be  
> NILed out
>     (setf *queue-header* (cdr *queue-header*))
>     value))
>
> Paul Khuong

thank you for the suggestion. i applied it to the queue management  
and did observe some improvement.
it was not, however, sufficient to allow sufficient memory to be  
released to avoid heap exhaustion.
i added logic to resource the (potentially numerous and large)  
objects which pass through the queues and observe that room now  
displays a count for those objects which corresponds to the contents  
in their resource cache and that the heap is no longer exhausted.

best regards, from berlin,