From: Bing Li <lb...@gm...> - 2012-09-28 02:35:21
|
Dear Joshua, Before executing the method, Rank(), listed previously, edgeSet is set as follows. ... this.edgeSet.add(new Edge<String>(this.GetVertex(hostKey), this.GetVertex(neighborKey), weight)); ... The weight is calculated according to the specific scenarios in my system. All of edges are saved in the set before invoking the method of Rank(). Is it not required? It costs more memory? Your help is highly appreciated. Best regards, Bing On Fri, Sep 28, 2012 at 12:33 AM, Joshua O'Madadhain < jos...@gm...> wrote: > On Wed, Sep 26, 2012 at 10:54 PM, Bing Li <lb...@gm...> wrote: > >> Dear Joshua, >> >> Thanks so much for your reply! >> >> In my system, there are a lot of graphs. To speed the ranking, >> concurrency solution is designed. But the memory is costed a lot. >> > > So decrease the amount of Java heap available, and GC will happen more > aggressively. At some point, obviously, you'll run out of memory if you > decrease it enough, and the less memory you have the more time you'll spend > doing garbage collection, so you'll have to decide what tradeoff you're > willing to accept. > > >> >> For the edgeSet, the reason is that I need to set weights for each edge. >> The weights depend on my system's application tightly. May I have other way >> to set the weights and save the memory? >> > > That doesn't really answer the question as to why you need the edge set. > For PageRank, you need to supply (as you are) a Transformer from edges to > weights. JUNG does not care how your Transformer works, so there are > several different ways you can implement it, as discussed here: > https://sourceforge.net/apps/trac/jung/wiki/JUNGManual#UserData > > Good luck! > > Joshua > > >> >> Best regards, >> Bing >> >> On Thu, Sep 27, 2012 at 12:48 AM, Joshua O'Madadhain < >> jos...@gm...> wrote: >> >>> Bing: >>> >>> You're creating a lot of objects, so the memory usage is going to go up. >>> Since you're not actually running out of memory, it sounds like the Java >>> garbage collector is doing its job; as soon as the graph and ranker go out >>> of scope (i.e., when the rank() method returns) they are eligible for GC >>> and will be collected when the space is needed. It's not clear that you >>> have an actual problem here. >>> >>> I would point out that there's not necessarily a lot of point to >>> building an edgeSet, i.e., the Graph encodes this information itself so >>> there's no need for you to do it again. That is, just build the Graph >>> instead of the edge set and you'll save some space (and time) right there. >>> >>> Joshua >>> >>> On Wed, Sep 26, 2012 at 8:50 AM, Bing Li <lb...@gm...> wrote: >>> >>>> Dear all, >>>> >>>> It is found that a large amount of memory is taken when ranking with >>>> PageRank, from JUNG 2.0.1. When starting to rank, the memory consumption >>>> goes high and when the ranking is done, it memory usage gets low. >>>> >>>> According to jconsole, the memory usage jumps with a large range, such >>>> as 200M ~1.2G. Is it normal? >>>> >>>> I am still not sure if it is caused by PageRank. However, I notice one >>>> problem that I have to initialize an instance of PageRank and Graph each >>>> time when ranking a group. >>>> >>>> I think it must be better if the PageRank or Graph can be reused. When >>>> a new graph is available, PageRank or Graph can be reset or cleared. But no >>>> such methods are available, right? Any other ways to solve the potential >>>> problem? >>>> >>>> The code is listed as follows. The method is invoked many times because >>>> there are a lot of graphs in my systems and each of them must be ranked >>>> periodically. So the memory usage goes up and down in a large range >>>> periodically. >>>> >>>> public Map<T, Double> Rank(HashMap<T, Vertex<T>> vertexMap, >>>> Set<Edge<T>> edgeSet) >>>> { >>>> HashMap<T, Double> finalRankMap = new HashMap<T, >>>> Double>(); >>>> PageRank<Vertex<T>, Edge<T>> ranker; >>>> Graph<Vertex<T>, Edge<T>> graph = new >>>> DirectedSparseMultigraph<Vertex<T>, Edge<T>>(); >>>> for (Edge<T> edge : edgeSet) >>>> { >>>> graph.addEdge(edge, edge.GetSourceVertex(), >>>> edge.GetDestinationVertex(), EdgeType.DIRECTED); >>>> } >>>> ranker = new PageRank<Vertex<T>, Edge<T>>(graph, >>>> this.weightTransformer, RankingConfig.RANDOM_JUMP_PROPABILITY); >>>> ranker.evaluate(); >>>> >>>> for (Vertex<T> vertex : vertexMap.values()) >>>> { >>>> try >>>> { >>>> finalRankMap.put(vertex.GetKey(), >>>> ranker.getVertexScore(vertex)); >>>> } >>>> catch (IllegalArgumentException e) >>>> { >>>> e.printStackTrace(); >>>> } >>>> } >>>> return finalRankMap; >>>> } >>>> >>>> I appreciates so much for your help! >>>> >>>> Best regards, >>>> Bing >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> How fast is your code? >>>> 3 out of 4 devs don\\\'t know how their code performs in production. >>>> Find out how slow your code is with AppDynamics Lite. >>>> http://ad.doubleclick.net/clk;262219672;13503038;z? >>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html >>>> _______________________________________________ >>>> Jung-support mailing list >>>> Jun...@li... >>>> https://lists.sourceforge.net/lists/listinfo/jung-support >>>> >>>> >>> >>> >>> -- >>> jos...@gm.........sites.google.com/site/joshuaomadadhain/ >>> Joshua O'Madadhain: Information Scientist, Musician, >>> Philosopher-At-Tall >>> It's that moment of dawning comprehension that I live for. -- Bill >>> Watterson >>> My opinions are too rational and insightful to be those of any >>> organization. >>> >>> >> > > > -- > jos...@gm.........sites.google.com/site/joshuaomadadhain/ > Joshua O'Madadhain: Information Scientist, Musician, Philosopher-At-Tall > It's that moment of dawning comprehension that I live for. -- Bill > Watterson > My opinions are too rational and insightful to be those of any > organization. > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://ad.doubleclick.net/clk;258768047;13503038;j? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html > _______________________________________________ > Jung-support mailing list > Jun...@li... > https://lists.sourceforge.net/lists/listinfo/jung-support > > |