Menu
▾
▴
Re: [VuFind-General] solr frequent commit
|
From: Nilani G. <Nil...@ma...> - 2010-09-27 08:50:00
|
Hi Gerg I am planning to use solr auto commit. Way you explained I think it is safe to do so. Regards Nilani Nilani Ganeshwaran eScholar Software Developer The University of Manchester Tel: 0161 2758728 From: Greg Pendlebury [mailto:gre...@gm...] Sent: 26 September 2010 23:04 To: Nil...@ma... Cc: vuf...@li... Subject: Re: [VuFind-General] solr frequent commit I'm curious however, are you talking about forcibly committing every 5 minutes via URL? or just the Solr autocommit settings? Because the forced 5min commits would be bad. On an index of only 150k documents I'm guessing it doesn't take that long to index, so 99% of the time Solr won't be adding new documents and the autocommit interval is irrelevant. Ta, Greg On 25 September 2010 00:35, Nilani Ganeshwaran <Nil...@ma...> wrote: Hi Till Thanks you very much for detailed emailed, which gave me good understanding of what's going on behind the scene. It seems like I have to look for 5 minute commit interval. Nilani Ganeshwaran eScholar Software Developer The University of Manchester Tel: 0161 2758728 -----Original Message----- From: Till Kinstler [mailto:kin...@gm...] Sent: 24 September 2010 13:20 To: vuf...@li... Subject: Re: [VuFind-General] solr frequent commit Hi, frequent commits on Solr indexes are not a good idea in general. Solr's performance heavily relies on caching, and those caches are filled while Solr is running. After every commit these caches are empty, so they either need to be filled by real user queries (which takes time) or be "autowarmed" (a mechanism in Solr o prepopulate caches, but that takes considerable CPU load). A commit interval of 30 s is no good idea in Solr, I'd say. I heard recommendations not to commit more frequently than every 5 minutes. In the end the choice of the commit interval depends on your application (how many records, how many changes per time, is it necessary to make these changes available in short/real time (in libraries often not, I'd say), how many users per time etc.). There are some initiatives to add (near) real time search capability to Solr, and there are efforts to get it into Solr. But I don't know, how far work is and if there are usable patches out there at the moment... Till Am 24.09.2010 13:55, schrieb Nilani Ganeshwaran: > Hi > > There will be around 150,000 documents in the solr index. If solr index > is frequently committed for every 30 seconds, can this cause any > performance issues in vufind searches? Is any of you doing frequent > commits to solr index, your comments is valuable. > > Regards > > Nilani > > Nilani Ganeshwaran > > eScholar Software Developer > > Red 1.1, The John Rylands University Library > > The University of Manchester > > Oxford Road, Manchester, M13 9PP > > Tel: 0161 2758728 > > Web: http://www.escholar.manchester.ac.uk > > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps& games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > > > > _______________________________________________ > VuFind-General mailing list > VuF...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-general -- http://twitter.com/tillk ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ VuFind-General mailing list VuF...@li... https://lists.sourceforge.net/lists/listinfo/vufind-general ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ VuFind-General mailing list VuF...@li... https://lists.sourceforge.net/lists/listinfo/vufind-general |