Re: [jgrapht-users] Query about scalability using JGraphT

[Aaron H. <jgr...@li...>]

Hi Lokeya,

On Apr 10, 2007, at 8:06 PM, Lokeya Venkatachalam wrote:
> I will do that testing and see. But I am just curious if anyone has  
> done this before.

I've done lots of work with about 500,000 vertices, which is a bit  
ungainly but certainly tractable.

> To be more precise, will it scale to represent 1 million document ,  
> where each document is 1KB in size which is around 1GB of data in  
> form of nodes.

Unless you're actively using the content of the documents as you  
traverse the nodes (certainly a possibility), it would seem wise to  
me to keep the node objects very lightweight, referring to a backing  
store of files or database rows as necessary. In the project I  
referred to above, we chose the latter route, with the nodes  
initially containing just id numbers, and optionally caching values  
from the database as needed.

hope that helps,
Aaron
--
Aaron Harnly
Center for Computational Learning Systems
Columbia University