Menu
▾
▴
Re: [VuFind-General] Voyager record extracts
|
From: Doran, M. D <do...@ut...> - 2010-06-30 15:57:11
|
Hi Demian, > I hope this is helpful! Yes, this *is* helpful and I will be looking at it in more detail. > I am copying this message to the solrmarc-tech list, which > may help you reach a broader audience. I am going to join that list, too. I'm not using Solr at the moment, but plan to in the (near?) future. Thanks! -- Michael # Michael Doran, Systems Librarian # University of Texas at Arlington # 817-272-5326 office # 817-688-1926 mobile # do...@ut... # http://rocky.uta.edu/doran/ > -----Original Message----- > From: Demian Katz [mailto:dem...@vi...] > Sent: Wednesday, June 30, 2010 10:12 AM > To: Doran, Michael D; vuf...@li... > Cc: Mark Sandford; sol...@go... > Subject: RE: Voyager record extracts > > VuFind ingest of MARC records makes use of the SolrMarc tool > (http://code.google.com/p/solrmarc/) shared with the Blacklight > community. I am copying this message to the solrmarc-tech list, which > may help you reach a broader audience. > > SolrMarc lets you easily map any field in a MARC record to any field in > a Solr index, applying all kinds of transformations/manipulations along > the way. Its one limitation is that MARC records are processed one at > a time, and each record becomes its own Solr index entry -- this > limitation makes it difficult to process Voyager dumps where holdings > and item records are separate from bibliographic records, and this is > why many Voyager institutions resort to remapping those separate > records to extra fields at the bibliographic level. > > I don't think there's a single standard for which fields/subfields are > used in this mapping. It's easy to customize, and individual libraries > are doing whatever works best for them. Currently, the VuFind project > has no official recommendations for this; at Villanova, we are not yet > making use of this data (though we would like to eventually). > > As far as ingests go, this again varies from place to place. At > Villanova, we do a full dump on a weekly basis (mainly for > backup/reindexing purposes) but day-to-day updates are handled by a > script that indexes all records changed in the past 24 hours. We also > regularly remove deleted/suppressed records from the index. > Villanova's procedures formed the basis for the suggestions found here: > > http://vufind.org/wiki/automation > > I hope this is helpful! > > - Demian > > > -----Original Message----- > > From: Doran, Michael D [mailto:do...@ut...] > > Sent: Tuesday, June 29, 2010 8:57 AM > > To: vuf...@li... > > Cc: Mark Sandford > > Subject: [VuFind-General] Voyager record extracts > > > > I'm currently working on a Voyager bulk record extract utility. I've > > looked at current Voyager record extract scripts, e.g. for VuFind, > for > > the eXtensible Catalog (XC) project, etc. Ours takes a different > > approach, which could potentially offer advantages in efficiency > and/or > > flexibility. We are exporting for ingest into AquaBrowser, but the > > utility (when finished) may be of wider use. > > > > Questions regarding VuFind ingests: > > > > Is there a *standard* record structure for ingest into VuFind? Roy > > Zimmer (see below) is taking holdings and item level data and > including > > it in the bib MARC record as additional fields. Is that how it is > > typically done? If so, is there any consensus as to the what data > gets > > added to what additional fields and subfields? Or is that all > > idiosyncratic to a particular institution? > > > > Are record ingests into VuFind typically full ingests, or do people > > also do incremental ingests of new and/or changed records? If > > incremental ingests are done, are they incremental since the last > full > > ingest, or incremental since the last ingest, whether full or > > incremental? Or can you do it either way? > > > > What should the ideal Voyager-to-VuFind record extract script be able > > to do? > > > > -- Michael > > > > # Michael Doran, Systems Librarian > > # University of Texas at Arlington > > # 817-272-5326 office > > # 817-688-1926 mobile > > # do...@ut... > > # http://rocky.uta.edu/doran/ > > > > > -----Original Message----- > > > From: Roy Zimmer [mailto:roy...@wm...] > > > Sent: Tuesday, June 22, 2010 9:46 AM > > > To: vuf...@li...; Mark Sandford > > > Subject: Re: [VuFind-General] Merging/appending records? > > > > > > Hi Mark: > > > > > > Here, in a nutshell, is what we extract from Voyager for VuFind: > > > > > > # output consists of bib (marc) records. > > > # additional fields are added: > > > # any 866, 867, and 868 fields from the holdings record > > > # an auxiliary field, 999 by default, which contains the > > > following: > > > # subfield data > > > # a item id > > > # b item barcode > > > # c item status > > > # d item type > > > # e call number > > > # g item enum if empty, will be two spaces > > > # h number of recalls > > > # k bib create date > > > # l location > > > # m display location > > > # p limit name > > > # x additional location > > > # y additional location > > > # TEMPORARY: > > > # t normalized title for sorting until in VuFind > > > release > > > # TEMPORARY > > > # > > > # Note: subfield k contains the bib create date value in Solr > > > format: > > > # yyyy-mm-ddTmm:dd:ssZ where characters T, Z, :, and - are > > > constants > > > # this value is required to be the equivalent UTC value > > > > > > Our extract software is available, if you're interested. > > > > > > Roy Zimmer > > > Waldo Library > > > Western Michigan University > > > > > > > > > > > > >>> "Sandford, Mark" <SAN...@wp...> 6/22/2010 10:31 AM >>> > > > Hello VUFinders, > > > > > > I'm stuck and hoping someone can help. We're a Voyager library. > > > My > > > colleagues have asked that location information be included in > > > VUFind in > > > order to limit queries by location. Here are a few things I > > > believe to > > > be true (and if they're not, this problem just got easier): > > > > > > Voyager will only export holdings information in a Bib-MFHD > > > interleaved > > > file > > > VUFind cannot handle the interleaved file > > > By default, the Solr indexer will completely replace old bib > > > records > > > with new ones > > > > > > I can think of three ways to get the information I need into > > > VUFind. > > > The first way is to generate separate MARC files, one for each > > > location, > > > then adjust the import properties file to add location information > > > into > > > a field I create. The problem with these approaches is that, if > > > my > > > assumptions about the solr indexing is correct, this doesn't work > > > for > > > titles with multiple copies in different locations. The location > > > last > > > imported will "win" and overwrite location information previously > > > loaded. The second way is to index the full MARC load, then > > > generate a > > > "fake" MARC file that only has 001s and 852s with locations. But > > > again, > > > this would require appending or merging the information in the > > > index. > > > > > > The third way is to somehow insert all the location information I > > > need > > > into a single bib record, then load the entire file at once. The > > > issue > > > here is that Voyager doesn't want to do that for me, and I can't > > > see any > > > easy way to do that after I export from Voyager. Gary Strawn's > > > vgerselect program actually will do this (it'll create multiple > > > 852s, > > > one for each MFHD in Voyager, and insert them into the MARC file as > > > it > > > extracts records) but that program is prohibitively slow. > > > > > > Thoughts? > > > > > > Mark Sandford > > > Special Formats Cataloger > > > David and Lorraine Cheng Library > > > William Paterson University > > > 973-720-2437 > > > > --------------------------------------------------------------------- > -- > > ------- > > This SF.net email is sponsored by Sprint > > What will you do first with EVO, the first 4G phone? > > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first > > _______________________________________________ > > VuFind-General mailing list > > VuF...@li... > > https://lists.sourceforge.net/lists/listinfo/vufind-general |