From: Demian K. <dem...@vi...> - 2009-10-26 18:33:25
|
Some fields in the Solr index are only able to accept a single value. If a MARC field repeats unexpectedly, you'll see this error. There are a few possible solutions: 1.) Fix the MARC records -- in the case of LCCN, I believe it's abnormal for there to be multiple values. If it's practical, you may want to try to fix the issue from the cataloging side. 2.) Accept multiple values. You can change the appropriate line in solr/biblio/conf/schema.xml like this: <field name="lccn" type="string" indexed="true" stored="true" multiValued="true"/> Note that making some fields multi-valued may require other code changes -- if the PHP code and Smarty templates assume that a field is always single-valued, you may end up seeing the word "Array" in inappropriate places when multi-valued results are encountered. 3.) Tell SolrMarc to ignore all but the first value. You can change the line in import/marc.properties like this: lccn = 010a, first This way, only the first 010a value will go into your index, and anything else in the MARC record will be ignored. Obviously, for the example of LCCN, the decision isn't that important since the value isn't used for much in VuFind. If you're seeing similar problems for other index fields, you may have to weigh your options more carefully. I hope this is a helpful start -- please let me know if you have any further questions. - Demian > -----Original Message----- > From: Philip Shafer [mailto:sh...@ro...] > Sent: Monday, October 26, 2009 2:24 PM > To: vuf...@li... > Subject: [VuFind-General] Solrmarc Import issues > > I have a few records (actually I¹m suspecting quite a few) that cannot > be > imported, so I¹m trying to pair down the errors on import. > > As I find records, I¹m exporting individual marc records (from Voyager) > and > trying to import them to see what the errors are. I¹m hoping someone > on > this mailing list can tell me what they mean. > > We are running Vufind RC1 > > 2009-10-26 14:10:04,229 [main] ERROR main > org.solrmarc.marc.MarcImporter - > Control Number 447171 > > org.apache.solr.common.SolrException: ERROR: multiple values > encountered for > non multiValued field lccn: first='80000702' second='2002213653' > > 2009-10-26 14:21:03,069 [main] ERROR main > org.solrmarc.marc.MarcImporter - > Control Number 455980 > > org.apache.solr.common.SolrException: ERROR: multiple values > encountered for > non multiValued field lccn: first='89029082' second='sn 89029082' > > > Any explanation would be very helpful. > > Thanks, > > Phil > ------------------------------ > > Philip Shafer > Library System Services > Rowan University Library > 201 Mullica Hill Rd > Glassboro, NJ 08028 > 856-256-4418 > 856-256-4924 Fax > > > > ----------------------------------------------------------------------- > ------- > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart > your > developing skills, take BlackBerry mobile applications to market and > stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > VuFind-General mailing list > VuF...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-general |
From: Philip S. <sh...@ro...> - 2009-10-26 18:38:59
|
Thanks, I'll have to do some more digging to see what the best solution. >From what I can tell multiple 010 is not preferred. With that said, I'm not sure how many records we are talking about so I might just have to change the schema.xml to accept multiple values. Thanks for the information. -Phil ------------------------------ Philip Shafer Library System Services Rowan University Library 201 Mullica Hill Rd Glassboro, NJ 08028 856-256-4418 856-256-4924 Fax > From: Demian Katz <dem...@vi...> > Date: Mon, 26 Oct 2009 14:33:08 -0400 > To: Philip Shafer <sh...@ro...>, "vuf...@li..." > <vuf...@li...> > Subject: RE: Solrmarc Import issues > > Some fields in the Solr index are only able to accept a single value. If a > MARC field repeats unexpectedly, you'll see this error. There are a few > possible solutions: > > 1.) Fix the MARC records -- in the case of LCCN, I believe it's abnormal for > there to be multiple values. If it's practical, you may want to try to fix > the issue from the cataloging side. > > 2.) Accept multiple values. You can change the appropriate line in > solr/biblio/conf/schema.xml like this: > > <field name="lccn" type="string" indexed="true" stored="true" > multiValued="true"/> |
From: Ya'aqov Z. <zi...@ro...> - 2009-10-26 18:46:00
|
010/LCCN is a not repeatable field. If more than one 010, any subsequent/repeat is a mistake and of no use. Ya¹aqov Ziso, Electronic Resource Management Librarian, Rowan University 856 256 4804 On 10/26/09 2:38 PM, "Philip Shafer" <sh...@ro...> wrote: > Thanks, I'll have to do some more digging to see what the best solution. >> >From what I can tell multiple 010 is not preferred. With that said, I'm not > sure how many records we are talking about so I might just have to change > the schema.xml to accept multiple values. > > Thanks for the information. > > -Phil > ------------------------------ > > Philip Shafer > Library System Services > Rowan University Library > 201 Mullica Hill Rd > Glassboro, NJ 08028 > 856-256-4418 > 856-256-4924 Fax > > > >> > From: Demian Katz <dem...@vi...> >> > Date: Mon, 26 Oct 2009 14:33:08 -0400 >> > To: Philip Shafer <sh...@ro...>, >> "vuf...@li..." >> > <vuf...@li...> >> > Subject: RE: Solrmarc Import issues >> > >> > Some fields in the Solr index are only able to accept a single value. If a >> > MARC field repeats unexpectedly, you'll see this error. There are a few >> > possible solutions: >> > >> > 1.) Fix the MARC records -- in the case of LCCN, I believe it's abnormal >> for >> > there to be multiple values. If it's practical, you may want to try to fix >> > the issue from the cataloging side. >> > >> > 2.) Accept multiple values. You can change the appropriate line in >> > solr/biblio/conf/schema.xml like this: >> > >> > <field name="lccn" type="string" indexed="true" stored="true" >> > multiValued="true"/> > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > VuFind-General mailing list > VuF...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-general |
From: Philip S. <sh...@ro...> - 2009-10-26 20:43:06
|
So to see if there would be any significant change in records imported, I took Demian's advice and went with option 3.) > 3.) Tell SolrMarc to ignore all but the first value. You can change the line > in import/marc.properties like this: > > lccn = 010a, first This fixed a significant number of issues: We exported: 391522 marc records We had: 386367 records indexed in vufind Now we have: 389726 records indexed in vufind An improvement of, 3,359 records. However, this still leaves us with 1,796 records not being indexed. Unfortunately this is difficult to see why the records aren't being indexed, since it seems that the solarmarc.log only has the last 1000 or so logs from the import. Is there anyway that I can dump all the errors out to a permanent error log? Thanks, Phil ------------------------------ Philip Shafer Library System Services Rowan University Library 201 Mullica Hill Rd Glassboro, NJ 08028 856-256-4418 856-256-4924 Fax > From: Demian Katz <dem...@vi...> > Date: Mon, 26 Oct 2009 14:33:08 -0400 > To: Philip Shafer <sh...@ro...>, "vuf...@li..." > <vuf...@li...> > Subject: RE: Solrmarc Import issues > > Some fields in the Solr index are only able to accept a single value. If a > MARC field repeats unexpectedly, you'll see this error. There are a few > possible solutions: > > 1.) Fix the MARC records -- in the case of LCCN, I believe it's abnormal for > there to be multiple values. If it's practical, you may want to try to fix > the issue from the cataloging side. > > 2.) Accept multiple values. You can change the appropriate line in > solr/biblio/conf/schema.xml like this: > > <field name="lccn" type="string" indexed="true" stored="true" > multiValued="true"/> > > Note that making some fields multi-valued may require other code changes -- if > the PHP code and Smarty templates assume that a field is always single-valued, > you may end up seeing the word "Array" in inappropriate places when > multi-valued results are encountered. > > 3.) Tell SolrMarc to ignore all but the first value. You can change the line > in import/marc.properties like this: > > lccn = 010a, first > > This way, only the first 010a value will go into your index, and anything else > in the MARC record will be ignored. > > Obviously, for the example of LCCN, the decision isn't that important since > the value isn't used for much in VuFind. If you're seeing similar problems > for other index fields, you may have to weigh your options more carefully. > > I hope this is a helpful start -- please let me know if you have any further > questions. > > - Demian > >> -----Original Message----- >> From: Philip Shafer [mailto:sh...@ro...] >> Sent: Monday, October 26, 2009 2:24 PM >> To: vuf...@li... >> Subject: [VuFind-General] Solrmarc Import issues >> >> I have a few records (actually I¹m suspecting quite a few) that cannot >> be >> imported, so I¹m trying to pair down the errors on import. >> >> As I find records, I¹m exporting individual marc records (from Voyager) >> and >> trying to import them to see what the errors are. I¹m hoping someone >> on >> this mailing list can tell me what they mean. >> >> We are running Vufind RC1 >> >> 2009-10-26 14:10:04,229 [main] ERROR main >> org.solrmarc.marc.MarcImporter - >> Control Number 447171 >> >> org.apache.solr.common.SolrException: ERROR: multiple values >> encountered for >> non multiValued field lccn: first='80000702' second='2002213653' >> >> 2009-10-26 14:21:03,069 [main] ERROR main >> org.solrmarc.marc.MarcImporter - >> Control Number 455980 >> >> org.apache.solr.common.SolrException: ERROR: multiple values >> encountered for >> non multiValued field lccn: first='89029082' second='sn 89029082' >> >> >> Any explanation would be very helpful. >> >> Thanks, >> >> Phil >> ------------------------------ >> >> Philip Shafer >> Library System Services >> Rowan University Library >> 201 Mullica Hill Rd >> Glassboro, NJ 08028 >> 856-256-4418 >> 856-256-4924 Fax >> >> >> >> ----------------------------------------------------------------------- >> ------- >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >> is the only developer event you need to attend this year. Jumpstart >> your >> developing skills, take BlackBerry mobile applications to market and >> stay >> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >> http://p.sf.net/sfu/devconference >> _______________________________________________ >> VuFind-General mailing list >> VuF...@li... >> https://lists.sourceforge.net/lists/listinfo/vufind-general |
From: Demian K. <dem...@vi...> - 2009-10-27 12:56:12
|
> Unfortunately this is difficult to see why the records aren't being indexed, > since it seems that the solarmarc.log only has the last 1000 or so logs from > the import. > > Is there anyway that I can dump all the errors out to a permanent error > log? Are you using the import.sh script that came with RC1? It's been a while since I used that version, but I thought it dumped all errors out to import/error-log. I don't know why it would only contain partial information. If you're just looking to troubleshoot import problems and have access to a test server where you can do this, it might also be worth pulling down the latest trunk and running an import using the newer SolrMarc to see if that gives you more helpful feedback. The new import-marc.sh script that replaces import.sh outputs all feedback to the terminal and can be easily redirected to a file for subsequent analysis. - Demian |