Re: [VuFind-Tech] VuFindSearch @ v1.1.2 -- Ready to test the query building

[Demian K. <dem...@vi...>]

Thanks once again for all the work.  I'll have to examine this more closely before I can comment intelligently, but it certainly sounds like good progress -- I'm glad to see the event system getting some use.  I'll try to give this a more detailed look next week, though first I want to handle your PR on VUFIND-670 before it gets too far behind master.  Hopefully I can do some review of this code in parallel with the ongoing work to finish 1.4 and port the collection branch forward.

- Demian
________________________________________
From: David Maus [ma...@ha...]
Sent: Friday, November 30, 2012 5:50 AM
To: VuFind Tech Mailinglist
Subject: [VuFind-Tech] VuFindSearch @ v1.1.2 -- Ready to test the query building

* VF2 Search Subsystem @ 1.1.2
:PROPERTIES:
:CREATED:  [2012-11-23 Fr 11:38]
:END:

git://github.com/dmj/vf2.git t/vf2-search-subsystem

The branch is a quick'n'dirty hack to hook the refactored search system in VF2 in order to check the
refactored query building/searchspecs processing. For the details of the hack see [1]. With the hack
in place VF performs an addition search using the new search system and displays a position:fixed'
<div/> container with the titles of the records found by the new search system. The SOLR request is
logged by VuFind\Logger with a severity of `debug'.

Using the hack requires a clone of git://github.com/dmj/vf2-search-subsystem.git in module/, e.g.

git clone git://github.com/dmj/vf2-search-subsystem.git module/VuFindSearch

Please note that the search system currently only supports search and retrieve (was: getRecord())
and does not support more search options than limit and offset. The task at hand is to check (a) the
refactored query building process and (b) the overall architecture.

A diagram of the SOLR backend class hierarchy and a sequence diagram of a search operation can be
found here:

https://gist.github.com/3772321

Some notes on the core classes.

** Service

VuFindSearch\Service is the central entry point for search/retrieve operations. All operations use a
backend identifier as first argument.

The search system does not provide any means of backend configuration. Instead it emits a
short-circuited resolve event in order obtain a configured backend instance and calls the respective
backend function. The system also emits pre and post events that allow other components to
transparently modify the parameters and the response before the operation is conducted and/or the
response is returned to the caller.

At the application side we implement VuFind\Search\BackendManager, listen for the resolve event and
supply a configured search backend instance based on the application configuration.

Multi-index support can be achieved in a similar way: The BackendManager listens with a low priority
for a pre search event and modifies the search specs in the backend's query builder as well as the
facet settings in the search parameters.[2]

Using the event system to implement mindex outside the search system gives the opportunity to
improve the performance by caching the pre-processed search specs in a mindex scenario. This could
be implemented by translating the selected shards into a bitfield and using the integer value as an
index in a numerical cache array.

,----
|    available shards = array('A', 'B', 'C')
|                              |    |    |
|                             bit0 bit1 bit2
|                              :    :    :
|    selected shards  = array('A'   : , 'C')
|                              |    :    |
|                              1    0    1     => Index 5
`----

@Oli: This is another use case of FieldIntersection. If you have n different indicies
in a mindex scenario then there are 2^n possible combinations and each combination can be
expressed as a FieldIntersection strategy.

** Backend (VuFindSearch\Backend\Solr)

The SOLR Backend is instantiated with an identifier of the record source and a configured SOLR
Connector instance. Optionally one can setter-inject a configured QueryBuilder that is passed to the
Connector's search function. The Backend lazy loads an empty QueryBuilder if the search function is
called and no QueryBuilder was injected.

Calls to the backend's search/retrieve functions are delegated to the Connector. The Connector
returns the body of SOLR response and the Backend passes the body to a setter-injected factory that
creates a RecordCollection instance.

By default the SOLR backend uses VuFindSearch\Backend\Solr\Response\RecordCollectionFactory which
creates a VuFindSearch\Backend\Solr\Response\Json\RecordCollection that contains
VuFindSearch\Backend\Solr\Response\Json\Record instances. Both, the collection and the records, are
simple wrappers around a deserialized JSON response; the record class provides access to SOLR
document fields via object properties.

** Connector (VuFindSearch\Backend\Solr\Connector)

The SOLR Connector is instantiated with the URL of the SOLR core to search in. The Connector handles
the HTTP transport.

Error handling (queries with invalid syntax)

The SOLR connector evaluates the HTTP status code of the response and throws a
VuFindSearch\Backend\RemoteErrorException for 5xx and a VuFindSearch\Backend\RequestErrorException
for 4xx. In both cases the exception code is set to the HTTP status code, the exception message to
the HTTP reason phrase.

To handle queries with invalid lucene syntax the caller wraps the search() in a try-catch block and
evaluates the exception code. If it is a 400 (Bad Request) it can delegate to a dedicated view for
the `You have an error in the search' screen.

TODO: Detect errors at the network layer, e.g. connect timeouts
TODO: Integrate write operations

** Footnotes

[1] https://github.com/dmj/vf2/compare/master...t;vf2-search-subsystem
[2] Current search system ships with VuFindSearch\Listener\MultiIndex just as an example.