Re: [java-gnome-hackers] Res: The next stage in memory management

[Andrew C. <an...@op...>]

On Tue, 2010-01-12 at 17:29 +0000, Vreixo Formoso Lopes wrote: 
> Hi all,

Hey Vreixo, nice to see you.

> I hope to take a further look at this asap, but I'd like to give you
> some impressions about this issue:

Sure. Quick replies follow (and so anyone please feel free to re-reply
to the original message, or this).


> In any case, a "kill -9 whatever" will terminate the JVM without any cleanup (either in finalize() 
> or by other means) so I doubt "special circumstances" should be handled at all by java-gnome.

Sure. That said, full application termination isn't a case we're worried
about here. If the release() -> g_object_unref() -> free() doesn't run
at shutdown that doesn't really hurt because a few cycles later the
process will be destroyed by the kernel and its memory freed. But that's
GTK' & X's problem. As "users" of GTK, we need to Do The Right Thing in
all other circumstances.

> > The problem is that the full Java garbage collector [and I'm talking
> > about OpenJDK HotSpot here] does not ordinarily run until the VM thinks
> > it is out of heap. And that isn't going to happen anytime soon on most
> > systems. Which means that finalize() isn't get called until *MUCH* later
> > than we'd like, and we're holding on to tons of memory (allocated by
> > glib) unnecessarily.
> 
> I wonder if what we should do is to launch any java-gnome application with specific
> GC-related JVM options.

I hadn't thought of that. Maybe...

But -XX options aren't exactly something we can rely on. Still, if we
know we're running HotSpot VM 6 [say], then we should tune it as best we
know how.

And meanwhile there are the GLib memory related options. But I don't
think any of them (ie "always-malloc") are necessary for us [at least,
not unless we mess with the GLib allocator functions]

> > Which brings us back to the _other_ purpose of SoftReference,
> > WeakReference, and PhantomReferences: taking actions on Java objects as
> > they go through the lifecycle, including being able to do cleanup long
> > before finalize() gets called by the GC in it's second last phase.
> 
> In any case, does not the GC need to be called in order to the JVM detect an object
> has changed its reachability? In such case, using another kind of reference would be
> useless. Or am I missing something?

Good question.

My understanding is that *finalize* [-> disposal] happens very late (if
at all) and only in full GC.

So in all the work we did a few years back in ValidateMemoryManagement
etc, we were calling System.gc() and so manually triggering a full GC,
and seeing our objects release() -> removeToggleRef() all correctly.

Using a breakpoint in Eclipse's debugger the finalize() methods are
indeed getting called... but late in the game.


> In any case, does not the GC need to be called in order to the JVM
> detect an objecthas changed its reachability? In such case, using
> another kind of reference would be useless. Or am I missing something?

So my inference & observation is that

a) the GC runs fairly often, but the "full" GC does not.

b) ReferenceQueues are handled in a Thread separate from the GC, and
that thread runs like any other thread.

[see the code in java.lang.ref.Reference in OpenJDK
jdk/src/share/classes/java/lang/ref/Reference.java which is where the
ReferenceHandler thread gets started. I get that the GC treats it a bit
specially, but it's still Java code]

anyway, if (b) is true, then [say] WeakReferences being enqueued to the
queue they are registered to happens automatically & soon after they are
eligible. So we could react to that.

Whereas finalize() happens MUCH MUCH later.

[which sucks, but there you have it]

> 
> > Presumably we want another WeakReference, plus a ReferenceQueue. The
> > question is: what will poll the reference queue, and when?
> 
> What exactly are you thinking about? Do you plan that when an object changes its reachability 
> to, say, weak, a cleanup method will be invoked?

So interesting distinction here:

in the GObject ToggleRef code, we switch everything to and from
WeakReference. And later those go away.

But ReferenceQueues are for _after_ something finishes at a given
reachability.

As far as I can tell, a Reference gets enqueued to the ReferenceQueue it
is registered to after it _leaves_  (or is eligible to leave) that
reachability level.

Which would be all references to a given Java object being weak or less
[which is when the ToggleRef is the last Ref, and we don't have any
strong Java references in our Java code].

Which is when we are currently then waiting for finalize() to be
invoked, calling release(), -> g_object_remove_toggle_ref() -> free().

So, if we instead call release() in response to getting a
[Weak]Reference enqueued to a ReferenceQueue, it should be the same
point.

But maybe I'm missing something :)


> If so, why do we need another WeakReference?
> Can't we just use the weak reference we have on Plumbing? In such case, the
> 
> 
>         Plumbing.unregisterProxy(this);
> 
> we have on Proxy.finalize() can be invoked as part of such post-weak processing.

Except that is a WeakReference for Proxy only, not the Pointers (Boxeds,
etc which is what GValues and PangoLayoutLines are right?).

So I figured it would have to be a WeakReference created when Pointer
are constructed. ie call Plumbing.registerPointer() or so.

Of course, if we have to have a WeakReference for everything, then maybe
we can use it for registerProxy() too, rather than having a second one
there.

But your suggestion (?) of polling the queue when unregister*Proxy*() is
called could make sense. We just need to process the queue periodically.
Doesn't have to be in it's own thread.

++

Anyway, ReferenceQueues are just one idea. I'm not saying it's the best
way to go.

It's possible we could be more aggressive when we handle window
delete-event -> hide 'n all, or maybe special case handling of
expose-event, or...

AfC
Sydney