Re: [VuFind-General] Updating the index

[Wayne G. <ws...@wm...>]

Hi Jim,

There's a facility in the Java program to let you choose which 
field/subfield will be the ID field in Solr, so you can put the flex key 
into any field you want and just edit the import.properties file 
accordingly. It works with 001-010 as the entire field, and everything 
else requires a field and subfield entry in the properties file.

So, if you put your flex key in the 035$a field, you'd edit the 
import.properties file to read

control.field 		= 035
control.subfield 	= a

Wayne

James Farrugia wrote:
> Hi,
> 
> Chiming in from the Unicorn side with a comment and a request:
> 
> COMMENT: we can get nightly from Unicorn a list of records added,
> updated, or deleted.
> These records are identified by the internal key Unicorn uses to
> uniquely identify its items.
> This internal key bears no necessary direct (i.e., one-to-one and onto)
> relationship to
> the 001, even (as I understand things) if that 001 is what's known as a
> "flex key."
> 
> REQUEST: can you write the Java importer so that it accommodates a
> Unicorn
> unique id in order to do its updates and deletes?
> 
> I'd be happy to discuss details/issues on or off list.
> 
> THanks,
> 
> Jim
> 
>>>> On 1/17/2008 at 6:57 PM, Steven McPhillips <smc...@nl...>
> wrote:
>> Deleted records are dumped to /m1/voyager/XXXdb/rpt/deleted.bib.marc.
>  
>> The trick here is that this dump is never rolled over - its is a  
>> complete delete audit. We are planning to take a copy and diff it on 
> 
>> a weekly basis so as to not parse the entire file for deletes. Of  
>> course, there's nothing wrong with sending redundant deletes, but  
>> there's no reason to, and also our Voyager admin staff would like a 
> 
>> "recent deletes" file of more manageable size
>>
>>
>> On 18/01/2008, at 10:03 AM, Andrew Nagy wrote:
>>
>>> Mark - I have a script now that resides on my voyager server that  
>>> gets all records that have been touched in the past 24 hours using 
> 
>>> pmarcexport.  The idea from that would be to pass it to the import 
> 
>>> script (the new java importer that is).  That is pretty easy and  
>>> solves the problem of keeping the index synced with new/updated  
>>> records.
>>>
>>>
>>>
>>> The next problem to face is to remove the withdrawn records from  
>>> the index.  From what I have been told - there is a file that  
>>> Voyager creates that has the withdrawn records.  We would need to  
>>> parse this and then run it against a script that removes the  
>>> records from the index.   I am not a voyager admin - so I would  
>>> need to work with our systems librarian on this.  However if you  
>>> know your voyager system well - maybe you might be able to help me 
> 
>>> out on this?
>>>
>>>
>>>
>>> Andrew
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> From: vuf...@li... [mailto:vufind- 
>>> gen...@li...] On Behalf Of Sandford, Mark
>>> Sent: Thursday, January 17, 2008 2:31 PM
>>> To: vuf...@li... 
>>> Subject: [VuFind-General] Updating the index
>>>
>>>
>>>
>>> I’m working with an older index right now in my local  
>>> installation.  I want to sync it up again with the Voyager db, but 
> 
>>> I’d like to avoid a full re-indexing for two reasons:
>>>
>>>
>>>
>>> 1.       It takes forever on the clunker I’m running VUFind on
> now
>>> 2.       I’d like to work out a routine for this to eventually be 
> 
>>> done on production.
>>>
>>>
>>>
>>> It seems to me that I need 3 files: records to be removed from the 
> 
>>> solr index entirely, records to be updated in the index, and  
>>> records to be added to the index.  I can extract them easily enough 
> 
>>> from Voyager, but I’m not sure what to do with them.
>>>
>>>
>>>
>>> Are there scripts already created to perform the functions?  It  
>>> looks like import-night.php just adds records, though I’m not even
>  
>>> sure how to call it properly so it does anything.
>>>
>>>
>>>
>>> Advice?
>>>
>>>
>>>
>>> Mark Sandford
>>> Special Formats Cataloger
>>> David and Lorraine Cheng Library
>>> William Paterson University
>>> san...@wp... 
>>> (973) 720-2437
>>>
>>>
>>>
>>>
>>>
>>>
> ---------------------------------------------------------------------- 
>>> ---
>>> This SF.net email is sponsored by: Microsoft
>>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ 
>>> _______________________________________________
>>> VuFind-General mailing list
>>> VuF...@li... 
>>> https://lists.sourceforge.net/lists/listinfo/vufind-general 
>>
>>
>>
>> ----
>> Steven McPhillips <smc...@nl...>
>> IT Business Systems
>> National Library of Australia
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> VuFind-General mailing list
> VuF...@li...
> https://lists.sourceforge.net/lists/listinfo/vufind-general

-- 
/**
  * Wayne Graham
  * Earl Gregg Swem Library
  * PO Box 8794
  * Williamsburg, VA 23188
  * 757.221.3112
  * http://swem.wm.edu/blogs/waynegraham/
  */