From: Heiko W. <h....@tu...> - 2010-12-14 12:39:07
|
Greg Pendlebury schrieb: > Coolio, > > I wouldn't worry about the output in top all that much, it looks like you'd > expect. With 8 cores you could (in theory) handle load averages of up to 8 > without slow down. and 1.84% is expected because top treats the 64 'virtual' > cpus as real, so one fully committed cpu is around 1.6%. top is more of a > linux tool then unix, but the proper unix ones always confuse me anyway :) > > You could try increasing the amount of RAM allocated as Fang Peng suggested > though. The 3.8g cap for 32 bit may give you some improvement. 2g was simply > where I noticed no extra improvements, but I did that testing almost two > years ago. > > And don't forget trying the suggestion in point 8. Solr config can make a > big difference. > > Some of the recent discussions on the Solrmarc list has led me to wonder > whether doing your submissions over HTTP would improve matters. Currently > VuFind runs Solrmarc in embedded mode, which is not thread safe, but is the > fastest single threaded method. > > For a process so heavily CPU bound, architecture like yours may find > improvements running multi-threaded marc parsing and submitted over HTTP to > a single core. Solrmarc doesn't currently support this, but if you are > interested I could look into modifying it. Indexing over http sounds a good idea, but we should have the core vufind in mind. My next step would be to test if this server can handle request good enough to play that role. Thinking of a small script with a direct (wget?) request to solr and counting the results. Or is there already a benchmark script for solr available? -- Mit freundlichen Grüßen Heiko Weier Technische Universität Hamburg-Harburg Fachreferent Bibliothek Tel.: 040-42878-3449 Denickestr. 22 Fax.: 040-42878-2527 21073 Hamburg WWW : http://www.tub.tu-harburg.de PGP Key http://www.tub.tu-harburg.de/keys/Heiko_Weier_pub.asc D8DF 3FFD 3910 AB1C 12D2 940B 65B6 2A73 524A 2C1E |