We have had similar issues with the time PROAI takes to process a large number of updates,
and have settled for turning off PROAI's polling until we want to run an update. We
currently do this automatically by a cron task.
Our timed task enables polling at the appointed hour (currently once per day - late at
night) by updating the PROAI table rcAdmin (field pollingEnabled). This will enable
polling on PROAI's next fetch cycle (which we have set for 600 seconds) and will start the
We have a separate task that attempts to turn off polling once every few minutes or so.
This task will fail if PROAI is updating because the rcAdmin table is write locked at the
start of the process, but will eventually succeed after the process has completed. This
makes sure that PROAI doesn't start a batch of updates until the next time we let it.
Perhaps this is not the most elegant solution, but it has been trouble free for some time
now. The only drawback is that PROAI data is always about a day behind the repository.
HTH (and greetings from Ithaca, NY - USA)
Timothy Cornwell, Programmer/Analyst
National Science Digital Library (http://nsdl.org)
301 College Avenue
Ithaca, NY 14850
> -----Original Message-----
> From: Markus Höckner [mailto:hoeckim@...]
> Sent: Wednesday, August 12, 2009 2:27 AM
> To: fedora-commons-users@...
> Subject: [Fedora-commons-users] OAI Provider and performance
> Hi everyone,
> we use the OAI-PMH service of Fedra and have a little
> performance problem.
> When we do an batch upload to fedora (up to 600 objects) then
> the OAI dies because of the fact that OAI gets the
> new/modified objects via risearch (Mulgara) and this takes a while...
> So the OAI decides to try it later, but then nothing changed
> - it takes to long to get the result from risearch.
> A bad sideeffect is, that this endless loop takes 99% CPU ->
> so the system is overloaded.
> As a consequence we have to kill Tomcat, edit the
> lastPollDate in the MySQL DB of proai and restart it.
> So the system works but the objects of the batch process
> aren't in OAI, because the updated lastPollDate is after the import...
> Is there another way to import the objects from Fedora to PROAI?
> What can we do to get more performance for the automatic PROAI import?
> Any suggestions?
> Greetings from Austria,
> Let Crystal Reports handle the reporting - Free Crystal
> Reports 2008 30-Day
> trial. Simplify your report design, integration and
> deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> Fedora-commons-users mailing list