Re: [VuFind-Tech] oai-pmh harvester: retry on error response?

[Thomas D. <dow...@wf...>]

IME the DSpace OAI/Tomcat/Postgres interaction often dies (from a memory
leak?), generating Status 500 errors.  When that happens, it isn't coming
back until someone restarts Tomcat.  But the OAI-PMH service is not on the
sysadmin's radar, so he or she isn't aware of the problem, and so it
persists for days or weeks.

So a slick addition to this would be to grab the adminEmail value from the
Identify response and send a note after $numRetriesOnFail is exhausted.



On Thu, Dec 11, 2014 at 3:22 AM, Ere Maijala <ere...@he...>
wrote:

> I can second that experience shows this to be a very useful feature
> (we've had it in RecordManager for a long time).
>
> One thing I wanted to mention is that the retry logic should also apply
> to parsing of the response, since sometimes network or server failures
> may cause partial responses causing xml parsing errors.
>
> --Ere
>
> 10.12.2014, 15.27, Demian Katz kirjoitti:
> > Yes, I think that sounds like a reasonable feature addition. Feel free
> to open a pull request or JIRA ticket, and please let me know if I can be
> of any further assistance.
> >
> > - Demian
> >
> >> -----Original Message-----
> >> From: thomas schwaerzler [mailto:tho...@ui...]
> >> Sent: Wednesday, December 10, 2014 6:01 AM
> >> To: vufind tech
> >> Subject: [VuFind-Tech] oai-pmh harvester: retry on error response?
> >>
> >> hello,
> >>
> >> i am just facing troubles harvesting an oai-pmh server. the server
> gives me
> >> HTTP 500 responses in some cases. somehow it is not stable.
> >>
> >> now i wanted to ask if anyone can see an advantage in adding a "feature"
> >> to oai-pmh harvester:
> >>
> >> at
> >>
> >> https://github.com/vufind-
> >> org/vufind/blob/master/module/VuFind/src/VuFind/Harvester/OAI.php#L417
> >>
> >> one could add an optional argument $numRetriesOnFail to the method
> >> sendRequest() and resend the request $numRetriesOnFail times on other
> HTTP
> >> status codes than 200 or 503.
> >>
> >> this argument could be optionally defined in oai.ini
> >>
> >>
> >> would this make sense to anyone?
> >>
> >>
> >> best
> >>
> >> t.
> >> --
> >> Thomas Schwaerzler
> >> Department of Digital Services
> >> University Innsbruck Library
> >> 6020 Innsbruck - Innrain 52 - Austria
> >> Phone: ++43-(0)512-507-2489
> >> Fax: ++43-(0)512-507-94908451
> >> Email: <Tho...@ui...>
> >> URL: http://www.uibk.ac.at/ulb/ds
> >> OpenPGP key: 0x104A5476, Key fingerprint = 98D8 F849 BCD4 56C2 3E1F
> BFEC FAF7
> >> 7668 104A 5476
> >>
> >>
> ------------------------------------------------------------------------------
> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from
> >> Actuate! Instantly Supercharge Your Business Reports and Dashboards with
> >> Interactivity, Sharing, Native Excel Exports, App Integration & more Get
> >> technology previously reserved for billion-dollar corporations, FREE
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Vufind-tech mailing list
> >> Vuf...@li...
> >> https://lists.sourceforge.net/lists/listinfo/vufind-tech
> >
> >
> ------------------------------------------------------------------------------
> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> > with Interactivity, Sharing, Native Excel Exports, App Integration & more
> > Get technology previously reserved for billion-dollar corporations, FREE
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Vufind-tech mailing list
> > Vuf...@li...
> > https://lists.sourceforge.net/lists/listinfo/vufind-tech
> >
>
>
> --
> Ere Maijala
> Kansalliskirjasto / The National Library of Finland
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Vufind-tech mailing list
> Vuf...@li...
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>



-- 
Thomas Dowling
Director of Technologies, Z. Smith Reynolds Library
Wake Forest University
dow...@wf... / 336-758-5797