If you want to try to implement some of these things and share a patch, please feel free. If you’d like to but aren’t sure how, let me know and I can look at the code and offer some thoughts. I’m also happy to help with development when time permits, but since I don’t have my own Pazpar2 server set up, that’s a bit of a handicap. (I realize that in theory I could set one up – but haven’t found the time).


- Demian


From: Jay Roos [mailto:jayroos@gmail.com]
Sent: Tuesday, October 15, 2013 10:47 AM
To: Demian Katz
Cc: vufind-general@lists.sourceforge.net
Subject: Re: [VuFind-General] Pazpar2 Integration results question


It looks like the results are cached by Pazpar2 until the session expires. If there was a way to retrieve the results instead of re-executing the search, that would probably help.


As far as the time it takes to complete, maybe adding a number of execution parameters to the config file would help. Maybe after the search is 50% complete or has run for 20 seconds or the number of results is greater than 1000, then show the existing results. 


That's not a perfect result, but it would keep a lagging query from holding up the show.


Just brainstorming.


On Mon, Oct 14, 2013 at 7:30 PM, Demian Katz <demian.katz@villanova.edu> wrote:

You've pretty much captured the problem: you can either do things in a way that's easily VuFind-compatible but slow, or you can do them in a more Pazpar2-friendly way that requires significant redesign/redevelopment of the generic VuFind search framework. Since the current Pazpar2 module is a proof of concept, it follows the easier (but slower) road.

It would make sense to reuse existing result sets when possible -- may just be a matter of using a cache or session mechanism...  probably something that could happen at the backend or connector level (though having little familiarity with Pazpar2 -- I didn't write much of that code -- I could be mistaken).

- Demian

From: Jay Roos [jayroos@gmail.com]
Sent: Monday, October 14, 2013 6:27 PM
To: David Maus
Cc: Demian Katz; vufind-general@lists.sourceforge.net
Subject: Re: [VuFind-General] Pazpar2 Integration results question

I poked around a little bit and added a loop that calls the stat command until the search completes. Unfortunately, it can take a very long time for that to happen. It does work, though not very well.


A better answer might be to dynamically load additional results until the pazpar2 run is finished, but that is a messy UX issue.


The other issue I ran into is the search is re-executed when switching to a different results page. It would be good if there was some mechanism for caching the results of a pazpar2 session until they are no longer needed (however you define that).


On Mon, Oct 14, 2013 at 9:54 AM, Jay Roos <jayroos@gmail.com> wrote:

It sounds like the ajax for loading Pazpar2 results needs to refresh the results with each check. Either that or just not display any results until the 'finished' signal is returned.


On Mon, Oct 14, 2013 at 1:41 AM, David Maus <maus@hab.de> wrote:

At Fri, 11 Oct 2013 21:58:00 +0000,
Demian Katz wrote:
> [1  <multipart/alternative (7bit)>]
> [1.1  <text/plain; iso-8859-1 (quoted-printable)>]
> [1.2  <text/html; iso-8859-1 (quoted-printable)>]

> We don't have sorting implemented, but you should be getting more than 20 results. It's possible there is a timing issue involved -- Pazpar2 works in a strange way: you start the search with one API request,
> then get the results with a second API request. If you make the second request before the Pazpar2 server has finished collecting all of its data, you may not get the full result set. I've considered adding a
> configuration parameter to put a sleep in between the calls to prevent that from happening... but not sure if there's a better solution.

At the conceptual level one difference between Pazpar2 and all the
other backends is, that Pazpar2 is stateful. Thus it's not a timing
issue but a fundamentally different search model (stateful
vs. stateless).

If we follow the Pazpar2 protocol [PAZPAR2] we have to initiate a
search, read the search session identifier from the response and issue
`stat' [PAZPAR2] until the search is finished -or- a configurable
maximum wait time was reached.

  -- David

[PAZPAR2] : http://www.indexdata.com/pazpar2/doc/pazpar2_protocol.html

> - Demian
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

> From: Jay Roos [jayroos@gmail.com]
> Sent: Friday, October 11, 2013 4:12 PM
> To: vufind-general@lists.sourceforge.net
> Subject: [VuFind-General] Pazpar2 Integration results question
> I have a basic Pazpar2 setup working with VuFind. Now the questions begin.
> It seems like I can ever only get 20 results. There is no paging or sorting so if what you want is not in the first 20 you won't find it. Am I missing something or is this just part of only having the most
> basic implementation so far?

> [2  <text/plain; us-ascii (7bit)>]
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> [3  <text/plain; us-ascii (7bit)>]
> _______________________________________________
> VuFind-General mailing list
> VuFind-General@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/vufind-general
David Maus
Herzog August Bibliothek - D-38299 Wolfenbuettel
Phone: +49-5331-808-317
Email: maus@hab.de

Digital Humanities Research Collaboration

PGP Key 0x0CC2E093512F7385
Fingerprint 1AD2 EE67 224F 18C5 EA55  98AD 0CC2 E093 512F 7385