Hi Gerg

 

I am planning to use solr auto commit. Way you explained I think it is safe to do so.

 

Regards

Nilani

 

Nilani Ganeshwaran

eScholar Software Developer

The University of Manchester

Tel: 0161 2758728

 

From: Greg Pendlebury [mailto:greg.pendlebury@gmail.com]
Sent: 26 September 2010 23:04
To: Nilani.Ganeshwaran@manchester.ac.uk
Cc: vufind-general@lists.sourceforge.net
Subject: Re: [VuFind-General] solr frequent commit

 

I'm curious however, are you talking about forcibly committing every 5 minutes via URL? or just the Solr autocommit settings? Because the forced 5min commits would be bad.

On an index of only 150k documents I'm guessing it doesn't take that long to index, so 99% of the time Solr won't be adding new documents and the autocommit interval is irrelevant.

Ta,
Greg

On 25 September 2010 00:35, Nilani Ganeshwaran <Nilani.Ganeshwaran@manchester.ac.uk> wrote:

Hi Till

Thanks you very much for detailed emailed, which gave me good understanding of what's going on behind the scene. It seems like I have to look for 5 minute commit interval.


Nilani Ganeshwaran
eScholar Software Developer

The University of Manchester
Tel: 0161 2758728


-----Original Message-----
From: Till Kinstler [mailto:kinstler@gmail.com]
Sent: 24 September 2010 13:20
To: vufind-general@lists.sourceforge.net
Subject: Re: [VuFind-General] solr frequent commit

Hi,

frequent commits on Solr indexes are not a good idea in general.
Solr's performance heavily relies on caching, and those caches are
filled while Solr is running. After every commit these caches are empty,
so they either need to be filled by real user queries (which takes time)
or be "autowarmed" (a mechanism in Solr o prepopulate caches, but that
takes considerable CPU load).
A commit interval of 30 s is no good idea in Solr, I'd say. I heard
recommendations not to commit more frequently than every 5 minutes. In
the end the choice of the commit interval depends on your application
(how many records, how many changes per time, is it necessary to make
these changes available in short/real time (in libraries often not, I'd
say), how many users per time etc.).
There are some initiatives to add (near) real time search capability to
Solr, and there are efforts to get it into Solr. But I don't know, how
far work is and if there are usable patches out there at the moment...

Till

Am 24.09.2010 13:55, schrieb Nilani Ganeshwaran:
> Hi
>
> There will be around 150,000 documents in the solr index. If solr index
> is frequently committed for every 30 seconds, can this cause any
> performance issues in vufind searches? Is any of you doing frequent
> commits to solr index, your comments is valuable.
>
> Regards
>
> Nilani
>
> Nilani Ganeshwaran
>
> eScholar Software Developer
>
> Red 1.1, The John Rylands University Library
>
> The University of Manchester
>
> Oxford Road, Manchester, M13 9PP
>
> Tel: 0161 2758728
>
> Web: http://www.escholar.manchester.ac.uk
>
>
>
> ------------------------------------------------------------------------------
> Nokia and AT&T present the 2010 Calling All Innovators-North America contest
> Create new apps&  games for the Nokia N8 for consumers in  U.S. and Canada
> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
> http://p.sf.net/sfu/nokia-dev2dev
>
>
>
> _______________________________________________
> VuFind-General mailing list
> VuFind-General@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/vufind-general


--
http://twitter.com/tillk

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
VuFind-General mailing list
VuFind-General@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/vufind-general


------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
VuFind-General mailing list
VuFind-General@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/vufind-general