Re: [VuFind-General] General Questions about VUFind

[Chris D. <ce...@ui...>]

On Tue, Jul 31, 2007 at 10:57:31AM -0400, Andrew Nagy wrote:
> > > I would love to see a large school such as yours adopt Vufind and
> > really put it to the test.  How many records do you have?
> >
> > Actually, we are not a single school; University of Illinois is our
> > host institution.  We are a consortium of at least (lost count) 65
> > schools, with the U of I being the largest.  We received our 25
> > millionth record this past week.
> 
> Wow, that would be a great test case!
> 
> >
> > I don't have the exact hardware specs with me, but in development I am
> > running a 2x 2GHz AMD-based CPU Virtual Machine (running VMWare) with
> > a Ubuntu 7.04 Server OS.  I have allocated 4GB of memory, but this
> > limit was never in danger of being a reached.  CPU usage, however, is
> > constantly being peaked out at or near 100% during SOLR searches.  Our
> > test environment has only 500,000 Bib records.
> 
> Hmm, you are seeing CPU spikes when searching 500,000 records?  I
would hate to blame someone else, but maybe VMWare is causing this
issue.  I have not heard of this as a problem yet.  Lucene/Solr should
not have any problem searching 500,000 records.  Have you tried
submitting an optimize statement to solr?

Yes, whenever I add records to SOLR, I issue a <commit/> and
<optimize/>.

Well, if this behavior is not common, then it wouldn't be unfair to
blame the one system that exhibits it :-)  I didn't set up the VMWare
instance, but my guess is that it is a very typical setup.  Also, this
system only runs VUFind.


> http://wiki.apache.org/solr/UpdateXmlMessages#head-a847de14ab548e9f3d9a5ba72aae7e5ac25cc51b
> 
> >
> > >
> > > Our library just added about 150,000 records to our collection from
> > apackage we just purchased so I think our collection is close to
> > 750,000 now.  The dataset on vufind.org is about 550,000.
> > >
> > > As to the Java vm setting, I haven't touched it yet.  I am also
> > looking at switching from Tomcat to Jetty.  Solr comes prepackaged with
> > Jetty and it would make the vufind distribution more lightweight.  Any
> > thoughts on this?
> > >
> >
> > I would be willing to try Jetty.  How difficult was it to integrate
> > SOLR with Tomcat?  Is switching to Jetty a trivial task, you think?
> 
> The switch to Jetty would be trivial.  If you would like to test it, please let me know how it works and if you notice any difference in performance.
> 


I wouldn't mind testing this out.

Chris


> Thanks!
> Andrew