Hi all,
I've noticed a problem with JUNG VisualizationViewer in that it dosn't seem to make itself eligible for garbage collection. That is if I show a graph and then dispose of the dialog it dosnt get collected. If I dispose of the dialog without populating the graph the VisualizationViewer get gc'ed as expected.
Anybody shed some light on what might be locking the components? I've tried stopping the relaxer thread (stop()) and removing all vertexs and edges from the graph in the dialog exit but it's not helping.
I'm using JUNG 1.7.6 at the moment, but I've also got a partial port to JUNG 2.0 underway so ideas on both would be helpful.
Ta very much,
- Richard
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It varies.. I'm just using verbose:gc to tell me whats going on in terms general gc usage and I've simply overriden finialize() on VisualizationViewer to dump to a logfile. If it fails to appear after several displays and disposes of its parent dialog (actually the dialog gets blocks too) over x minutes I know something is holding onto references. To do it properly you'll need a profiler, but my copy of hpjmeter is broken at the moment.. probably needs an update for 5.0.
I did find setting an empty graph/layout helps free up the VisualizationViewer components for 1.7.6 at least.. but I suspect it dosnt fully collect the old layout.. hopefully calling removeAllVertices(), removeAllEdges() beforehand helps keep this pretty minimal (need my profiler working to verify). Although I don't see these two methods in JUNG2..? some cleanup methods and/or use of references would be a big help.
[code]
static class EmptyLayout extends AbstractLayout {
public EmptyLayout() {
super(new AbstractSparseGraph() {
protected void initialize() {
}
public Set getEdges() {
return Collections.EMPTY_SET;
}
public Set getVertices() {
return Collections.EMPTY_SET;
}
});
}
The JVM schedules garbage collection according to complicated and JVM implementation dependent rules; it is quite possible that an application could be running for a long while before garbage collection is run.
What's more, once an item is garbage collected, it is not guaranteed to be finalized right away.
GC behavior has (hopefully) been optimized by the implementors of whatever JVM you're using -- usually this means the garbage collector doesn't run unless heap space is running low, but like I said, there are many factors that determine GC behavior that vary from JVM to JVM.
You can force garbage collection by calling System.gc(), and you can force finalization by calling System.runFinalization(); this might give you a better idea about what references are being released.
However, calls to these methods are somewhate expensive (hence heavy JVM gc scheduling optimization), and unless you're specifically trying to debug a memory issue, I'd avoid using them. (In my experience, though, the best way to reliably debug memory issues is with a profiler). Modern JVMs do a very good job of properly managing gc scheduling.
In general, Java applications and libraries don't use memory cleanup methods because of automatic garbage collection. Hopefully listener deregistration, etc. is already happening in the library internals... are you noticing specific performance issues anywhere?
Rob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yeah I've been working with Java since '97 I know how wholly garbage collection can appear the finalize trick is very much "finger in the air" - it's the verbose:gc that's telling me I've got a leak.
Anyway I've attached a real profiler now and watching the live figures as I type. As I show and hide Graphs (both showing/hiding Graphs in the same Viewer and disposing of Viewers an re-creating new ones). I can see the live counts of Pair, DefaultUserData, UndirectedSpareEdge, UndirectedSparseVertex and Coordinates objects climb higher and higher - but they never drop. It's pretty clear they aren't being collected.. ever.. not even after 200 generations.
That was more of a surprise given I'm explicitly removing all vertex and edge objects from the graph when I hide it (previous post). I'm not attaching any of my own listeners either so it's purely down to the api. Tried a couple of different layouts just to double check it wasn't related to the FRLayout issue which was the only other gc related topic.
- Richard
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I tried profiling the project I'm currently working on (that uses Jung) to see what happened...
I have a dialog that contains an extension of VisualizationViewer. The viewer is visualizing an extension of SparseGraph. The Layout being used is my own direct implementation of the Layout interface, and the renderer being used is an extension of PluggableRenderer.
I am managing some listeners of the Viewer myself, but have double checked to make sure references are always being removed correctly when the dialog closes. All other listener registration happens inside of Jung libraries.
I tried opening and closing the dialog around 50 times at various points throughout my application; the average visualized graph contains only about 4 or 5 nodes.
No instances of the dialog were around any more at this point.
So, it looks like there are indeed quite a few DefaultUserData objects floating around. (I'm not sure why there are 1 or 2 live instances of some other things, but they'd not survived more than a couple of generations, so I'm thinking they'd be gc'd in the near future).
It's interesting, though, that after I forced gc, some of those DefaultUserData objects were collected...
It's possible that somewhere a reference to this type of object is not being properly released. I took a quick look at some of the code involved, and didn't see anything obvious, but I'll be sure to post about anything I happen to find.
Are you doing similar things with your code? (And, are your vm profiles showing similar behavior?)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
These are some of the reasons why JUNG 2.0 does not (at least currently) include a built-in user data repository that is integral to the JUNG graph/element classes: it can take up a lot of room and complicate garbage collection. I haven't spent much time looking at profiles like this and am not sufficiently familiar with the JVM garbage collection internals to guess why the DefaultUserData objects might not be getting gc'd, but it doesn't surprise me a whole lot, either.
(We may look into providing a separate user data repository at some point, but so far JUNG 2.0 seems to be doing just fine without it, and leaving it out keeps us from letting our code depend on it, which is an architectural problem that JUNG 1.x has.)
Thanks for looking into this and letting us know what you've found. If you have time, we'd definitely like to know whether there are any similar behaviors going on with JUNG 2.0.
Joshua
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry Rob, I was chasing things down my end by looking into what I put into UserData. Our UserData is a bit weighty for what JUNG actually ends up using. But as it's returned via RMI and I had a sudden fear that even though I don't make any external references to these objects it might still be cached via the proxies just in case the remote objects needed to make a callback..
Did notice a lot of our own remote objects going uncollected, but it's not easy to tell whether they were the cause or an end-effect of UserData not being fully collected. As well as a unit test to check remote object collection I also wrote a method to purge all UserData from a Graph.. Now I thought I saw some gc collection on Vertices & Edges going on after that change, but they weren't dropping back to their previous levels. But when I went back to get some figures the live & allocated counts were identical so I doubted any collection was going on and that I needed to go home and get some sleep.
I don't have much time to chase this down at the moment. Just going to have to mark a bigger block of time into the plans for profiling. As I have a port to JUNGv2 underway I'm tempted to get that up-to-spec and recheck my gc() performance then. As Joshua says as a longer term strategy this makes more sense.
- Richard
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Couldn't help myself, here's a copy of my profiles after showing & disposing a dialog with a test UndirectedGraph and VisualizationViewer. This is with my other mods to reset the layout & graph to an empty model, so the VisualizationViewer is collected and removing all UserData, Vertex and Edges from the graph before disposing of the dialog. The after shot is sometime later after several "Run gc()" operations.. expanded to include all the edu.* survivors.
Ps I noticed the allocated & live does vary very slightly so I wasn't just going crosseyed yesterday, but I did get too excited over such a tiny number being collected (better than none?).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi all,
I've noticed a problem with JUNG VisualizationViewer in that it dosn't seem to make itself eligible for garbage collection. That is if I show a graph and then dispose of the dialog it dosnt get collected. If I dispose of the dialog without populating the graph the VisualizationViewer get gc'ed as expected.
Anybody shed some light on what might be locking the components? I've tried stopping the relaxer thread (stop()) and removing all vertexs and edges from the graph in the dialog exit but it's not helping.
I'm using JUNG 1.7.6 at the moment, but I've also got a partial port to JUNG 2.0 underway so ideas on both would be helpful.
Ta very much,
- Richard
How (and when) are you determining whether or not the VisualizationViewer gets gc'ed?
It varies.. I'm just using verbose:gc to tell me whats going on in terms general gc usage and I've simply overriden finialize() on VisualizationViewer to dump to a logfile. If it fails to appear after several displays and disposes of its parent dialog (actually the dialog gets blocks too) over x minutes I know something is holding onto references. To do it properly you'll need a profiler, but my copy of hpjmeter is broken at the moment.. probably needs an update for 5.0.
I did find setting an empty graph/layout helps free up the VisualizationViewer components for 1.7.6 at least.. but I suspect it dosnt fully collect the old layout.. hopefully calling removeAllVertices(), removeAllEdges() beforehand helps keep this pretty minimal (need my profiler working to verify). Although I don't see these two methods in JUNG2..? some cleanup methods and/or use of references would be a big help.
[code]
static class EmptyLayout extends AbstractLayout {
public EmptyLayout() {
super(new AbstractSparseGraph() {
protected void initialize() {
}
public Set getEdges() {
return Collections.EMPTY_SET;
}
public Set getVertices() {
return Collections.EMPTY_SET;
}
});
}
protected void initialize_local_vertex(Vertex v) {
}
public void advancePositions() {
}
public boolean isIncremental() {
return false;
}
public boolean incrementsAreDone() {
return true;
}
}
[/code]
Related
Code: code
The JVM schedules garbage collection according to complicated and JVM implementation dependent rules; it is quite possible that an application could be running for a long while before garbage collection is run.
What's more, once an item is garbage collected, it is not guaranteed to be finalized right away.
GC behavior has (hopefully) been optimized by the implementors of whatever JVM you're using -- usually this means the garbage collector doesn't run unless heap space is running low, but like I said, there are many factors that determine GC behavior that vary from JVM to JVM.
You can force garbage collection by calling System.gc(), and you can force finalization by calling System.runFinalization(); this might give you a better idea about what references are being released.
However, calls to these methods are somewhate expensive (hence heavy JVM gc scheduling optimization), and unless you're specifically trying to debug a memory issue, I'd avoid using them. (In my experience, though, the best way to reliably debug memory issues is with a profiler). Modern JVMs do a very good job of properly managing gc scheduling.
In general, Java applications and libraries don't use memory cleanup methods because of automatic garbage collection. Hopefully listener deregistration, etc. is already happening in the library internals... are you noticing specific performance issues anywhere?
Rob
Yeah I've been working with Java since '97 I know how wholly garbage collection can appear the finalize trick is very much "finger in the air" - it's the verbose:gc that's telling me I've got a leak.
Anyway I've attached a real profiler now and watching the live figures as I type. As I show and hide Graphs (both showing/hiding Graphs in the same Viewer and disposing of Viewers an re-creating new ones). I can see the live counts of Pair, DefaultUserData, UndirectedSpareEdge, UndirectedSparseVertex and Coordinates objects climb higher and higher - but they never drop. It's pretty clear they aren't being collected.. ever.. not even after 200 generations.
That was more of a surprise given I'm explicitly removing all vertex and edge objects from the graph when I hide it (previous post). I'm not attaching any of my own listeners either so it's purely down to the api. Tried a couple of different layouts just to double check it wasn't related to the FRLayout issue which was the only other gc related topic.
- Richard
I tried profiling the project I'm currently working on (that uses Jung) to see what happened...
I have a dialog that contains an extension of VisualizationViewer. The viewer is visualizing an extension of SparseGraph. The Layout being used is my own direct implementation of the Layout interface, and the renderer being used is an extension of PluggableRenderer.
I am managing some listeners of the Viewer myself, but have double checked to make sure references are always being removed correctly when the dialog closes. All other listener registration happens inside of Jung libraries.
I tried opening and closing the dialog around 50 times at various points throughout my application; the average visualized graph contains only about 4 or 5 nodes.
Here are the live jung objects that topped the memory usage chart at that point:
http://i9.tinypic.com/2cp7rxh.jpg
And here's what it looked like after I forced garbage collection from the profiler:
http://i3.tinypic.com/44shytw.jpg
No instances of the dialog were around any more at this point.
So, it looks like there are indeed quite a few DefaultUserData objects floating around. (I'm not sure why there are 1 or 2 live instances of some other things, but they'd not survived more than a couple of generations, so I'm thinking they'd be gc'd in the near future).
It's interesting, though, that after I forced gc, some of those DefaultUserData objects were collected...
It's possible that somewhere a reference to this type of object is not being properly released. I took a quick look at some of the code involved, and didn't see anything obvious, but I'll be sure to post about anything I happen to find.
Are you doing similar things with your code? (And, are your vm profiles showing similar behavior?)
Rob, Richard:
These are some of the reasons why JUNG 2.0 does not (at least currently) include a built-in user data repository that is integral to the JUNG graph/element classes: it can take up a lot of room and complicate garbage collection. I haven't spent much time looking at profiles like this and am not sufficiently familiar with the JVM garbage collection internals to guess why the DefaultUserData objects might not be getting gc'd, but it doesn't surprise me a whole lot, either.
(We may look into providing a separate user data repository at some point, but so far JUNG 2.0 seems to be doing just fine without it, and leaving it out keeps us from letting our code depend on it, which is an architectural problem that JUNG 1.x has.)
Thanks for looking into this and letting us know what you've found. If you have time, we'd definitely like to know whether there are any similar behaviors going on with JUNG 2.0.
Joshua
Sorry Rob, I was chasing things down my end by looking into what I put into UserData. Our UserData is a bit weighty for what JUNG actually ends up using. But as it's returned via RMI and I had a sudden fear that even though I don't make any external references to these objects it might still be cached via the proxies just in case the remote objects needed to make a callback..
Did notice a lot of our own remote objects going uncollected, but it's not easy to tell whether they were the cause or an end-effect of UserData not being fully collected. As well as a unit test to check remote object collection I also wrote a method to purge all UserData from a Graph.. Now I thought I saw some gc collection on Vertices & Edges going on after that change, but they weren't dropping back to their previous levels. But when I went back to get some figures the live & allocated counts were identical so I doubted any collection was going on and that I needed to go home and get some sleep.
I don't have much time to chase this down at the moment. Just going to have to mark a bigger block of time into the plans for profiling. As I have a port to JUNGv2 underway I'm tempted to get that up-to-spec and recheck my gc() performance then. As Joshua says as a longer term strategy this makes more sense.
- Richard
Couldn't help myself, here's a copy of my profiles after showing & disposing a dialog with a test UndirectedGraph and VisualizationViewer. This is with my other mods to reset the layout & graph to an empty model, so the VisualizationViewer is collected and removing all UserData, Vertex and Edges from the graph before disposing of the dialog. The after shot is sometime later after several "Run gc()" operations.. expanded to include all the edu.* survivors.
http://www.myjavaserver.com/~osbald/temp/jung-profile1.gif
http://www.myjavaserver.com/~osbald/temp/jung-profile2.gif
Ps I noticed the allocated & live does vary very slightly so I wasn't just going crosseyed yesterday, but I did get too excited over such a tiny number being collected (better than none?).