[Chris Delis <cedelis@ui...>]

On Tue, Jul 31, 2007 at 11:46:46AM -0400, Wayne Graham wrote:
> You may be able to get around some of the CPU spikes by changing the GC
> variable for Java. Also, which version of Java are you running under?
> You may find a performance bump with the java-sun6-jdk.
> 

Looks like your correct.  Thanks Wayne.  It's been a while since I've
dealt with java tuning issues.  Looks like I need to profile the
garbage collector and tweak their settings appropriately.

Chris


> You can set -verbose:gc in the CATALINA_OPTS to dump to the
> solr/tomcat/logs/catalina.out file to see what's going on. I suspect
> that there is some Java tuning that needs to take place.
> 
> HTH,
> Wayne
> 
> Chris Delis wrote:
> > On Tue, Jul 31, 2007 at 10:57:31AM -0400, Andrew Nagy wrote:
> >>>> I would love to see a large school such as yours adopt Vufind and
> >>> really put it to the test.  How many records do you have?
> >>>
> >>> Actually, we are not a single school; University of Illinois is our
> >>> host institution.  We are a consortium of at least (lost count) 65
> >>> schools, with the U of I being the largest.  We received our 25
> >>> millionth record this past week.
> >> Wow, that would be a great test case!
> >>
> >>> I don't have the exact hardware specs with me, but in development I am
> >>> running a 2x 2GHz AMD-based CPU Virtual Machine (running VMWare) with
> >>> a Ubuntu 7.04 Server OS.  I have allocated 4GB of memory, but this
> >>> limit was never in danger of being a reached.  CPU usage, however, is
> >>> constantly being peaked out at or near 100% during SOLR searches.  Our
> >>> test environment has only 500,000 Bib records.
> >> Hmm, you are seeing CPU spikes when searching 500,000 records?  I
> > would hate to blame someone else, but maybe VMWare is causing this
> > issue.  I have not heard of this as a problem yet.  Lucene/Solr should
> > not have any problem searching 500,000 records.  Have you tried
> > submitting an optimize statement to solr?
> > 
> > Yes, whenever I add records to SOLR, I issue a <commit/> and
> > <optimize/>.
> > 
> > Well, if this behavior is not common, then it wouldn't be unfair to
> > blame the one system that exhibits it :-)  I didn't set up the VMWare
> > instance, but my guess is that it is a very typical setup.  Also, this
> > system only runs VUFind.
> > 
> > 
> >> http://wiki.apache.org/solr/UpdateXmlMessages#head-a847de14ab548e9f3d9a5ba72aae7e5ac25cc51b
> >>
> >>>> Our library just added about 150,000 records to our collection from
> >>> apackage we just purchased so I think our collection is close to
> >>> 750,000 now.  The dataset on vufind.org is about 550,000.
> >>>> As to the Java vm setting, I haven't touched it yet.  I am also
> >>> looking at switching from Tomcat to Jetty.  Solr comes prepackaged with
> >>> Jetty and it would make the vufind distribution more lightweight.  Any
> >>> thoughts on this?
> >>> I would be willing to try Jetty.  How difficult was it to integrate
> >>> SOLR with Tomcat?  Is switching to Jetty a trivial task, you think?
> >> The switch to Jetty would be trivial.  If you would like to test it, please let me know how it works and if you notice any difference in performance.
> >>
> > 
> > 
> > I wouldn't mind testing this out.
> > 
> > Chris
> > 
> > 
> >> Thanks!
> >> Andrew
> > 
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >>  http://get.splunk.com/
> > _______________________________________________
> > VuFind-General mailing list
> > VuFind-General@...
> > https://lists.sourceforge.net/lists/listinfo/vufind-general
> 
> 
> -- 
> /**
>  * Wayne Graham
>  * Earl Gregg Swem Library
>  * PO Box 8794
>  * Williamsburg, VA 23188
>  * 757.221.3112
>  * http://swem.wm.edu/blogs/waynegraham/
>  */
>