Re: [VuFind-Tech] ILS Backend Availability and VuFind (2.x)

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Unfortunately, I believe that we have read-only access to our Voyager database, so creating a trigger is probably not an option.  But there may be a way to query the necessary information -- it's just a question of whether the database tracks status change dates (though it seems extremely likely that it does -- just a matter of finding the right field).

- Demian
________________________________________
From: Jay Roos [ja...@gm...]
Sent: Friday, January 18, 2013 10:27 AM
To: Demian Katz
Cc: vuf...@li...
Subject: Re: [VuFind-Tech] ILS Backend Availability and VuFind (2.x)

I don't know anything about Voyager, but Horizon keeps a table of
changed records for Horizon Information Portal (our opac) to use for
reindexing. When Horizon Information Portal reads those records it
also clears them from the table. My thought as I think about our
eventual implementation is to utilize the data in that table (maybe
make a copy for VuFind) to export recently updated records to
add/reindex.

Again not knowing anything about Voyager, have you considered an
insert/update trigger in the database so that when certain details are
updated it triggers the addition of the record ID to a table that
could then be used as a basis for exporting only those records? It's
just a thought, it may be completely impractical.

On Fri, Jan 18, 2013 at 8:10 AM, Demian Katz <dem...@vi...> wrote:
> marcexport can definitely export records that have changed in the last X
> days -- that is how we do nightly updates at Villanova.  The question I have
> is what counts as a change?  Is it a bib change?  A MFHD change?  A
> circulation status change?  I haven't investigated this in any detail, so
> I'm just speculating here -- but I wonder if the change detection may be
> limited to bibs...  in which case you would need to go to the SQL query/bib
> ID approach you described.
>
> Regarding suppressed MFHDs, the problem we have is that a single bib may
> have one suppressed MFHD and one non-suppressed MFHD.  When we ask
> marcexport to give us the bib with holdings, we get both MFHDs.  I'm not
> sure that the SQL query approach will help there -- though maybe I'm missing
> something.
>
> - Demian
> ________________________________
> From: Mosior, Benjamin [BEM...@sh...]
> Sent: Friday, January 18, 2013 9:00 AM
>
> To: Demian Katz; vuf...@li...;
> vuf...@li...
> Subject: RE: ILS Backend Availability and VuFind (2.x)
>
> Demian,
>
>
>
> I think the marcexport program is capable of exporting records that have
> changed in the last X days. If so, I’m hopeful that we can go that route
> instead of an additional script.
>
>
>
> As far as faceting and preventing suppressed items, in other applications
> we’ve found that we can control those sorts of details by selecting Voyager
> BIB IDs based on SQL Queries, then using marcexport to export based on the
> set of BIB IDs:
>
>
>
> my $query = qq {
>
>         SELECT Bib_Master.BIB_ID
>
>                FROM Location, Bib_Master, Bib_Mfhd, Mfhd_Master
>
>                WHERE (Bib_Mfhd.Bib_Id=Bib_Master.Bib_Id AND
> Mfhd_Master.Mfhd_Id=Bib_Mfhd.Mfhd_Id
>
>                         AND Mfhd_Master.Location_Id=Location.Location_Id)
>
>                 AND ((NOT Bib_Master.Suppress_In_Opac='Y') AND (NOT
> Mfhd_Master.Suppress_In_Opac='Y')
>
>                 AND Location.Location_Id NOT  IN ($excluded_locations))
>
> };
>
>
>
> … where $excluded_locations contains a comma-separated list of location IDs.
> Let me know if you have questions.
>
>
>
> Thank you again for the information. I’m generally encouraged that we might
> be able to do record status/location “caching” in Solr as an alternative to
> a secondary ILS DB server. I will update as we make progress. Hopefully we
> can generate some VuFind 2.x documentation for the process.
>
>
>
> Benjamin Mosior
>
> Keystone Library Network
>
>
>
> From: Demian Katz [mailto:dem...@vi...]
> Sent: Thursday, January 17, 2013 4:27 PM
> To: Mosior, Benjamin; vuf...@li...;
> vuf...@li...
> Subject: RE: ILS Backend Availability and VuFind (2.x)
>
>
>
> The question is whether Voyager’s marcexport program is smart enough to
> export records based on a change to the availability status.  If it is, then
> it’s easy.  Otherwise, as you say, some kind of scripting may be necessary.
> With 4 million records, continually reindexing everything is probably not
> entirely practical.
>
>
>
> The observation about faceting is a good point.  Here at Villanova, we
> actually just rolled out faceting based on 852 location codes last week!
> There are some issues in Voyager (we haven’t yet found a way to prevent the
> marcexport output from including suppressed holdings, and some of the
> records autogenerated by our ILL system cause problems), but it’s at least a
> start.
>
>
>
> - Demian
>
>
>
> From: Mosior, Benjamin [mailto:BEM...@sh...]
> Sent: Thursday, January 17, 2013 3:04 PM
> To: Demian Katz; vuf...@li...;
> vuf...@li...
> Subject: RE: ILS Backend Availability and VuFind (2.x)
>
>
>
> Demian,
>
>
>
> That sounds awesome, but the question then is about keeping those fields
> updated.
>
>
>
> Leaning towards the side of simplicity (vs efficiency), would it make sense
> then to have a script that runs regularly to compare each Solr record’s
> location/status against the ILS backend, triggering a re-export of records
> that have changed? I’m trying to think it through in order to reduce the
> number of queries necessary to perform an update, especially since we’re
> talking 4+ million records in our case.
>
>
>
> Jay Roos replied off-list regarding the usefulness of this feature for the
> sake of faceting. I hadn’t realized that location-based searching would be
> more feasible, since the query would be against the Solr index instead of
> the ILS. Shiny!
>
>
>
>
>
> Benjamin Mosior
>
> Keystone Library Network
>
>
>
> From: Demian Katz [mailto:dem...@vi...]
> Sent: Thursday, January 17, 2013 2:28 PM
> To: Mosior, Benjamin; vuf...@li...;
> vuf...@li...
> Subject: RE: ILS Backend Availability and VuFind (2.x)
>
>
>
> Have you looked at the “NoILS” driver yet?  You can configure VuFind to load
> NoILS if the main driver throws an exception (or you can manually make it
> the primary ILS driver), and NoILS can be configured to load location and
> status information from holdings fields in your stored MARC records in the
> Solr index.  That might prove to be a fairly simple Solr-based solution –
> just export bib + holdings from Voyager, use the SolrMarc feature that
> merges the holdings into the bibs (I can share documentation if you’re not
> familiar with it), and configure NoILS to use the 852 fields.
>
>
>
> - Demian
>
>
>
> From: Mosior, Benjamin [mailto:BEM...@sh...]
> Sent: Thursday, January 17, 2013 2:19 PM
> To: vuf...@li...; vuf...@li...
> Subject: [VuFind-General] ILS Backend Availability and VuFind (2.x)
>
>
>
> We are curious about how other institutions and consortia handle
> high-availability/redundancy of the ILS backend with respect to VuFind.
>
>
>
> While we maintain a secondary read-only ILS backend instance, we were
> pondering the pros and cons of writing a VuFind caching mechanism so that
> item statuses and locations would be stored in the Solr index and regularly
> updated in order to be available if the ILS backend was down. Does that seem
> like a reasonable approach, or is the secondary ILS backend instance
> approach better? For us, it seems like a Solr performance vs cost for Oracle
> licensing (we currently use Voyager) kind of issue.
>
>
>
> Benjamin Mosior
>
> Keystone Library Network
>
>
>
>
> ------------------------------------------------------------------------------
> Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
> much more. Get web development skills now with LearnDevNow -
> 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
> SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122812
> _______________________________________________
> Vufind-tech mailing list
> Vuf...@li...
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>

--
Jay Roos
Information Technology Coordinator
Great River Regional Library