From: Bing Li <lb...@gm...> - 2012-09-28 06:10:10
|
Joshua, I see. I appreciate so much for your help! Best regards, Bing On Fri, Sep 28, 2012 at 2:07 PM, Joshua O'Madadhain < jos...@gm...> wrote: > On Thu, Sep 27, 2012 at 7:35 PM, Bing Li <lb...@gm...> wrote: > >> Dear Joshua, >> >> Before executing the method, Rank(), listed previously, edgeSet is set as >> follows. >> >> ... >> this.edgeSet.add(new Edge<String>(this.GetVertex(hostKey), this.GetVertex(neighborKey), >> weight)); >> ... >> >> The weight is calculated according to the specific scenarios in my >> system. All of edges are saved in the set before invoking the method of >> Rank(). >> >> Is it not required? It costs more memory? >> > > You don't need this. Unless there's more to an Edge than you describe > here--it looks like it's defined by its source, destination, and > weight--then you can just do this: > > graph.addEdge(new Edge(weight), this.GetVertex(hostkey), > this.GetVertex(neighborKey)); > > If edge weights are guaranteed unique, then it's even easier, just let the > edge type itself be the weight. > > The graph holds the relationship between the vertices, the Edge doesn't > need to do that. > Skipping building the edgeSet will save you both memory and time. > > Joshua > > >> >> Your help is highly appreciated. >> >> Best regards, >> Bing >> >> On Fri, Sep 28, 2012 at 12:33 AM, Joshua O'Madadhain < >> jos...@gm...> wrote: >> >>> On Wed, Sep 26, 2012 at 10:54 PM, Bing Li <lb...@gm...> wrote: >>> >>>> Dear Joshua, >>>> >>>> Thanks so much for your reply! >>>> >>>> In my system, there are a lot of graphs. To speed the ranking, >>>> concurrency solution is designed. But the memory is costed a lot. >>>> >>> >>> So decrease the amount of Java heap available, and GC will happen more >>> aggressively. At some point, obviously, you'll run out of memory if you >>> decrease it enough, and the less memory you have the more time you'll spend >>> doing garbage collection, so you'll have to decide what tradeoff you're >>> willing to accept. >>> >>> >>>> >>>> For the edgeSet, the reason is that I need to set weights for each >>>> edge. The weights depend on my system's application tightly. May I have >>>> other way to set the weights and save the memory? >>>> >>> >>> That doesn't really answer the question as to why you need the edge set. >>> For PageRank, you need to supply (as you are) a Transformer from edges to >>> weights. JUNG does not care how your Transformer works, so there are >>> several different ways you can implement it, as discussed here: >>> https://sourceforge.net/apps/trac/jung/wiki/JUNGManual#UserData >>> >>> Good luck! >>> >>> Joshua >>> >>> >>>> >>>> Best regards, >>>> Bing >>>> >>>> On Thu, Sep 27, 2012 at 12:48 AM, Joshua O'Madadhain < >>>> jos...@gm...> wrote: >>>> >>>>> Bing: >>>>> >>>>> You're creating a lot of objects, so the memory usage is going to go >>>>> up. Since you're not actually running out of memory, it sounds like the >>>>> Java garbage collector is doing its job; as soon as the graph and ranker go >>>>> out of scope (i.e., when the rank() method returns) they are eligible for >>>>> GC and will be collected when the space is needed. It's not clear that you >>>>> have an actual problem here. >>>>> >>>>> I would point out that there's not necessarily a lot of point to >>>>> building an edgeSet, i.e., the Graph encodes this information itself so >>>>> there's no need for you to do it again. That is, just build the Graph >>>>> instead of the edge set and you'll save some space (and time) right there. >>>>> >>>>> Joshua >>>>> >>>>> On Wed, Sep 26, 2012 at 8:50 AM, Bing Li <lb...@gm...> wrote: >>>>> >>>>>> Dear all, >>>>>> >>>>>> It is found that a large amount of memory is taken when ranking with >>>>>> PageRank, from JUNG 2.0.1. When starting to rank, the memory consumption >>>>>> goes high and when the ranking is done, it memory usage gets low. >>>>>> >>>>>> According to jconsole, the memory usage jumps with a large range, >>>>>> such as 200M ~1.2G. Is it normal? >>>>>> >>>>>> I am still not sure if it is caused by PageRank. However, I notice >>>>>> one problem that I have to initialize an instance of PageRank and Graph >>>>>> each time when ranking a group. >>>>>> >>>>>> I think it must be better if the PageRank or Graph can be reused. >>>>>> When a new graph is available, PageRank or Graph can be reset or cleared. >>>>>> But no such methods are available, right? Any other ways to solve the >>>>>> potential problem? >>>>>> >>>>>> The code is listed as follows. The method is invoked many times >>>>>> because there are a lot of graphs in my systems and each of them must be >>>>>> ranked periodically. So the memory usage goes up and down in a large range >>>>>> periodically. >>>>>> >>>>>> public Map<T, Double> Rank(HashMap<T, Vertex<T>> vertexMap, >>>>>> Set<Edge<T>> edgeSet) >>>>>> { >>>>>> HashMap<T, Double> finalRankMap = new HashMap<T, >>>>>> Double>(); >>>>>> PageRank<Vertex<T>, Edge<T>> ranker; >>>>>> Graph<Vertex<T>, Edge<T>> graph = new >>>>>> DirectedSparseMultigraph<Vertex<T>, Edge<T>>(); >>>>>> for (Edge<T> edge : edgeSet) >>>>>> { >>>>>> graph.addEdge(edge, edge.GetSourceVertex(), >>>>>> edge.GetDestinationVertex(), EdgeType.DIRECTED); >>>>>> } >>>>>> ranker = new PageRank<Vertex<T>, Edge<T>>(graph, >>>>>> this.weightTransformer, RankingConfig.RANDOM_JUMP_PROPABILITY); >>>>>> ranker.evaluate(); >>>>>> >>>>>> for (Vertex<T> vertex : vertexMap.values()) >>>>>> { >>>>>> try >>>>>> { >>>>>> finalRankMap.put(vertex.GetKey(), >>>>>> ranker.getVertexScore(vertex)); >>>>>> } >>>>>> catch (IllegalArgumentException e) >>>>>> { >>>>>> e.printStackTrace(); >>>>>> } >>>>>> } >>>>>> return finalRankMap; >>>>>> } >>>>>> >>>>>> I appreciates so much for your help! >>>>>> >>>>>> Best regards, >>>>>> Bing >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> How fast is your code? >>>>>> 3 out of 4 devs don\\\'t know how their code performs in production. >>>>>> Find out how slow your code is with AppDynamics Lite. >>>>>> http://ad.doubleclick.net/clk;262219672;13503038;z? >>>>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html >>>>>> _______________________________________________ >>>>>> Jung-support mailing list >>>>>> Jun...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/jung-support >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> jos...@gm......... >>>>> sites.google.com/site/joshuaomadadhain/ >>>>> Joshua O'Madadhain: Information Scientist, Musician, >>>>> Philosopher-At-Tall >>>>> It's that moment of dawning comprehension that I live for. -- Bill >>>>> Watterson >>>>> My opinions are too rational and insightful to be those of any >>>>> organization. >>>>> >>>>> >>>> >>> >>> >>> -- >>> jos...@gm.........sites.google.com/site/joshuaomadadhain/ >>> Joshua O'Madadhain: Information Scientist, Musician, >>> Philosopher-At-Tall >>> It's that moment of dawning comprehension that I live for. -- Bill >>> Watterson >>> My opinions are too rational and insightful to be those of any >>> organization. >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Everyone hates slow websites. So do we. >>> Make your web apps faster with AppDynamics >>> Download AppDynamics Lite for free today: >>> http://ad.doubleclick.net/clk;258768047;13503038;j? >>> >>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html >>> _______________________________________________ >>> Jung-support mailing list >>> Jun...@li... >>> https://lists.sourceforge.net/lists/listinfo/jung-support >>> >>> >> > > > -- > jos...@gm.........sites.google.com/site/joshuaomadadhain/ > Joshua O'Madadhain: Information Scientist, Musician, Philosopher-At-Tall > It's that moment of dawning comprehension that I live for. -- Bill > Watterson > My opinions are too rational and insightful to be those of any > organization. > > > > ------------------------------------------------------------------------------ > Got visibility? > Most devs has no idea what their production app looks like. > Find out how fast your code is with AppDynamics Lite. > http://ad.doubleclick.net/clk;262219671;13503038;y? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html > _______________________________________________ > Jung-support mailing list > Jun...@li... > https://lists.sourceforge.net/lists/listinfo/jung-support > > |