I'd hate to try to parse the raw MARC data stored in SQL. It's ugly and pretty inefficient. Some solution that uses marcout is going to be the way to go unless you can write a complete parser for Horizon marc stored in the database.

On Thu, Sep 24, 2009 at 8:21 PM, <Peter.Neish@parliament.vic.gov.au> wrote:
Thanks Peter and Ray for sharing your ideas. Using triggers sounds like a
good idea.

I was wondering, given that we have a small collection with not many
updates, whether I could write a plugin for vufind that fetched the updated
records via php and loaded them directly into the solr index. That way I
could avoid having to coordinate scripts on two machines. Just a thought.

Cheers,

Peter
--
Peter Neish
Systems Officer
Victorian Parliamentary Library
Ph: 03 9651 8638
peter.neish@parliament.vic.gov.au




† † † † † † Peter Kennedy
† † † † † † <peter.kennedy@ca
† † † † † † nterbury.ac.nz> † † † † † † † † † † † † † † † † † † † † † †To
† † † † † † † † † † † † † † † † † † † vufind-general@lists.sourceforge.ne
† † † † † † 25/09/2009 07:40 † † † † †t
† † † † † † AM † † † † † † † † † † † † † † † † † † † † † † † † † † † † cc

† † † † † † † † † † † † † † † † † † † † † † † † † † † † † † † † † Subject
† † † † † † † † † † † † † † † † † † † Re: [VuFind-General] Horizon -
† † † † † † † † † † † † † † † † † † † updating records










This is an addenda to my earlier comment about having a
vufind_word_index_pending table. When I replied, I had not seen all the
correspondence on this topic.

Given that others are thinking of a similar route, I thought I would
provide more detail while stressing this is still experimental.

Trigger

CREATE TRIGGER dbo.word_index_pending_i_trig ON dbo.word_index_pending
†For INSERT AS

† † †BEGIN
† † † † †insert †vufind_word_index_pending
† † † † select inserted.SerialNum, inserted.Action, inserted.RecNum,
inserted.RecType
† † † † from inserted
† † †END

with an Index on the vufind_word_index_pending table

Clustered unique just on the serialnum column.

Workflow

Copy the entries into a temporary table

Truncate, then populate two tables of bib#. The first gets bib#s (i.e.
recnum) for action 1 (i.e. deletions). The second gets bib#s for actions
0,2.

Delete from the second table bib#s which also have an action 1. This is
because a deletion process may create two entries, one for update (perhaps
if an item is deleted first) and a second for deletion. We donít really
want to update a record that is going to be deleted.

Once these two bib# tables are created, we can delete selected records from
the vufind_word_index_pending table. Iím doing those based on the serialnum
field Ė Iím deleting records where the serialnum is in my temporary table,
so that I donít delete records added after the process started.

Use these two bib# tables to control marcout Ė produce one file of marc
records for deletion, one for adds/updates.

Use a ftp client with a scheduler to copy these to the Vufind server

Have scheduled tasks on VuFind to process these two files.

While this all works manually, I havenít tied it together into an automated
process. Thus far it is just a proof-of-concept process.

Regards, Peter Kennedy



This email may be confidential and subject to legal privilege, it may
not reflect the views of the University of Canterbury, and it is not
guaranteed to be virus free. If you are not an intended recipient,
please notify the sender immediately and erase all copies of the message
and any attachments.

Please refer to http://www.canterbury.ac.nz/emaildisclaimer for more
information.
------------------------------------------------------------------------------

Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf_______________________________________________
VuFind-General
mailing list
VuFind-General@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/vufind-general


------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf
_______________________________________________
VuFind-General mailing list
VuFind-General@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/vufind-general



--
Jay Roos
Computer & Information Systems Coordinator
Great River Regional Library
405 W. St. Germain
St. Cloud, MN 56301
(320) 650-2534