Re: [Jung-support] Does PageRank Take Much Memory?

[Joshua O'M. <jos...@gm...>]

On Wed, Sep 26, 2012 at 10:54 PM, Bing Li <lb...@gm...> wrote:

> Dear Joshua,
>
> Thanks so much for your reply!
>
> In my system, there are a lot of graphs. To speed the ranking, concurrency
> solution is designed. But the memory is costed a lot.
>

So decrease the amount of Java heap available, and GC will happen more
aggressively.  At some point, obviously, you'll run out of memory if you
decrease it enough, and the less memory you have the more time you'll spend
doing garbage collection, so you'll have to decide what tradeoff you're
willing to accept.


>
> For the edgeSet, the reason is that I need to set weights for each edge.
> The weights depend on my system's application tightly. May I have other way
> to set the weights and save the memory?
>

That doesn't really answer the question as to why you need the edge set.
 For PageRank, you need to supply (as you are) a Transformer from edges to
weights.  JUNG does not care how your Transformer works, so there are
several different ways you can implement it, as discussed here:
https://sourceforge.net/apps/trac/jung/wiki/JUNGManual#UserData

Good luck!

Joshua


>
> Best regards,
> Bing
>
> On Thu, Sep 27, 2012 at 12:48 AM, Joshua O'Madadhain <
> jos...@gm...> wrote:
>
>> Bing:
>>
>> You're creating a lot of objects, so the memory usage is going to go up.
>>  Since you're not actually running out of memory, it sounds like the Java
>> garbage collector is doing its job; as soon as the graph and ranker go out
>> of scope (i.e., when the rank() method returns) they are eligible for GC
>> and will be collected when the space is needed.  It's not clear that you
>> have an actual problem here.
>>
>> I would point out that there's not necessarily a lot of point to building
>> an edgeSet, i.e., the Graph encodes this information itself so there's no
>> need for you to do it again.  That is, just build the Graph instead of the
>> edge set and you'll save some space (and time) right there.
>>
>> Joshua
>>
>> On Wed, Sep 26, 2012 at 8:50 AM, Bing Li <lb...@gm...> wrote:
>>
>>> Dear all,
>>>
>>> It is found that a large amount of memory is taken when ranking with
>>> PageRank, from JUNG 2.0.1. When starting to rank, the memory consumption
>>> goes high and when the ranking is done, it memory usage gets low.
>>>
>>> According to jconsole, the memory usage jumps with a large range, such
>>> as 200M ~1.2G. Is it normal?
>>>
>>> I am still not sure if it is caused by PageRank. However, I notice one
>>> problem that I have to initialize an instance of PageRank and Graph each
>>> time when ranking a group.
>>>
>>> I think it must be better if the PageRank or Graph can be reused. When a
>>> new graph is available, PageRank or Graph can be reset or cleared. But no
>>> such methods are available, right? Any other ways to solve the potential
>>> problem?
>>>
>>> The code is listed as follows. The method is invoked many times because
>>> there are a lot of graphs in my systems and each of them must be ranked
>>> periodically. So the memory usage goes up and down in a large range
>>> periodically.
>>>
>>>         public Map<T, Double> Rank(HashMap<T, Vertex<T>> vertexMap,
>>> Set<Edge<T>> edgeSet)
>>>         {
>>>                 HashMap<T, Double> finalRankMap = new HashMap<T,
>>> Double>();
>>>                 PageRank<Vertex<T>, Edge<T>> ranker;
>>>                 Graph<Vertex<T>, Edge<T>> graph = new
>>> DirectedSparseMultigraph<Vertex<T>, Edge<T>>();
>>>                 for (Edge<T> edge : edgeSet)
>>>                 {
>>>                         graph.addEdge(edge, edge.GetSourceVertex(),
>>> edge.GetDestinationVertex(), EdgeType.DIRECTED);
>>>                 }
>>>                 ranker = new PageRank<Vertex<T>, Edge<T>>(graph,
>>> this.weightTransformer, RankingConfig.RANDOM_JUMP_PROPABILITY);
>>>                 ranker.evaluate();
>>>
>>>                 for (Vertex<T> vertex : vertexMap.values())
>>>                 {
>>>                         try
>>>                         {
>>>                                 finalRankMap.put(vertex.GetKey(),
>>> ranker.getVertexScore(vertex));
>>>                         }
>>>                         catch (IllegalArgumentException e)
>>>                         {
>>>                                 e.printStackTrace();
>>>                         }
>>>                 }
>>>                 return finalRankMap;
>>>         }
>>>
>>> I appreciates so much for your help!
>>>
>>> Best regards,
>>> Bing
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> How fast is your code?
>>> 3 out of 4 devs don\\\'t know how their code performs in production.
>>> Find out how slow your code is with AppDynamics Lite.
>>> http://ad.doubleclick.net/clk;262219672;13503038;z?
>>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>>> _______________________________________________
>>> Jung-support mailing list
>>> Jun...@li...
>>> https://lists.sourceforge.net/lists/listinfo/jung-support
>>>
>>>
>>
>>
>> --
>> jos...@gm.........sites.google.com/site/joshuaomadadhain/
>>    Joshua O'Madadhain: Information Scientist, Musician,
>> Philosopher-At-Tall
>> It's that moment of dawning comprehension that I live for.  -- Bill
>> Watterson
>>  My opinions are too rational and insightful to be those of any
>> organization.
>>
>>
>


-- 
jos...@gm.........sites.google.com/site/joshuaomadadhain/
   Joshua O'Madadhain: Information Scientist, Musician, Philosopher-At-Tall
It's that moment of dawning comprehension that I live for.  -- Bill
Watterson
 My opinions are too rational and insightful to be those of any
organization.