Menu
▾
▴
Re: [VuFind-Tech] summary holdings problem with Voyager
|
From: Jason S. <jst...@ku...> - 2014-01-10 20:28:43
|
Here’s a gist with two files: the SQL statement and its output as semicolon-delimited https://gist.github.com/jstirnaman/8361953 Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 On Jan 10, 2014, at 1:45 PM, Demian Katz <dem...@vi...<mailto:dem...@vi...>> wrote: I think so -- some of the joins cause duplicate rows, and then the code has to avoid those duplicates. It might be easier to troubleshoot if you share the full SQL and display multiple fields in your output.... - Demian ________________________________ From: Jason Stirnaman [jst...@ku...<mailto:jst...@ku...>] Sent: Friday, January 10, 2014 1:36 PM To: Demian Katz Cc: vufind-tech@lists. sourceforge. net Subject: Re: summary holdings problem with Voyager I just queried Voyager directly with PDO, executing the $sqlStmt used in getHolding and I get the same result. Is this the scenario that the snippet you cited is trying to account for? Here’s the SQL result: 33520;350282340503;259820;N;10;0;0;Not Charged;00345cx a22001213 450000100050000000400060000500500170001100800330002801400120006185200290007386600760010286600450017827283352020140110094250.09907060p 8 4001aueng00000001 a14889108 bdyk.jnlhJournaltCopy 1 080a1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002) ;v.81 (2002);Dykes Journal Stacks (Kansas City);0;Journal;;; 33520;350287416068;259820;N;10;0;0;Not Charged;00345cx a22001213 450000100050000000400060000500500170001100800330002801400120006185200290007386600760010286600450017827283352020140110094250.09907060p 8 4001aueng00000001 a14889108 bdyk.jnlhJournaltCopy 1 080a1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002) ;v.81 (2002);Dykes Journal Stacks (Kansas City);0;Journal;;; 33520;350282340503;259820;N;10;0;0;Not Charged;080a1(1922)-81(2002)zSome issues missing;v.81 (2002);Dykes Journal Stacks (Kansas City);0;Journal;;; 33520;350287416068;259820;N;10;0;0;Not Charged;080a1(1922)-81(2002)zSome issues missing;v.81 (2002);Dykes Journal Stacks (Kansas City);0;Journal;;; 33520;350281252105;85476;N;12;0;0;Not Charged;00345cx a22001213 450000100050000000400060000500500170001100800330002801400120006185200290007386600760010286600450017827283352020140110094250.09907060p 8 4001aueng00000001 a14889108 bdyk.jnlhJournaltCopy 1 080a1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002) ;v.1 (1922);Dykes Journal Stacks (Kansas City);0;Journal;;; 33520;350281252105;85476;N;12;0;0;Not Charged;080a1(1922)-81(2002)zSome issues missing;v.1 (1922);Dykes Journal Stacks (Kansas City);0;Journal;;; Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 On Jan 10, 2014, at 11:27 AM, Demian Katz <dem...@vi...<mailto:dem...@vi...>> wrote: I think the critical code is: // We don't want to concatenate the same MARC information to // itself over and over due to a record with multiple status // codes -- we should only concat wrapped rows for the FIRST // status code we encounter! $record = & $data[$row['ITEM_ID']][$number]; if ($record['STATUS_ARRAY'][0] == $row['STATUS']) { $record['RECORD_SEGMENT'] .= $row['RECORD_SEGMENT']; } Perhaps the logic for avoidign duplication is wrong. It might be helpful to add a var_dump of $record to see what's going on there... maybe some other rules are needed. Let me know if you have any questions about the existing code -- if I wrack my brain I might be able to remember exactly why this is doing what it's currently doing! :-) - Demian ________________________________ From: Jason Stirnaman [jst...@ku...<mailto:jst...@ku...>] Sent: Friday, January 10, 2014 11:27 AM To: Demian Katz Cc: vufind-tech@lists. sourceforge. net Subject: Re: summary holdings problem with Voyager Thanks for the tip, Demian! I’m still working at it, but getting closer. For the first item in the location, if MFHD is over 300 bytes (i.e. has multiple RECORD_SEGMENT) then getHoldingData() returns the segments twice. The next items return the expected result. Here are some values dumped (with added labels) from the beginning of getHoldingData() in case anyone can pinpoint the bug faster than I can: ITEM_ID: string(6) "259820" | STATUS: string(11) "Not Charged" RECORD_SEGMENT: string(300) "00345cx a22001213 4500001000500000004000600005005001700011008003300028014001200061852002900073866007600102866004500178?2728?33520?20140110094250.0?9907060p 8 4001aueng0000000?1 ?a1488910?8 ?bdyk.jnl?hJournal?tCopy 1? 0?80?a1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002)? " ITEM_ID: string(6) "259820" | STATUS: string(11) "Not Charged" RECORD_SEGMENT: string(300) "00345cx a22001213 4500001000500000004000600005005001700011008003300028014001200061852002900073866007600102866004500178?2728?33520?20140110094250.0?9907060p 8 4001aueng0000000?1 ?a1488910?8 ?bdyk.jnl?hJournal?tCopy 1? 0?80?a1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002)? " ITEM_ID: string(6) "259820" | STATUS: string(11) "Not Charged" RECORD_SEGMENT: string(45) "0?80?a1(1922)-81(2002)?zSome issues missing??" ITEM_ID: string(6) "259820" | STATUS: string(11) "Not Charged" RECORD_SEGMENT: string(45) "0?80?a1(1922)-81(2002)?zSome issues missing??" ITEM_ID: string(5) "85476" | STATUS: string(11) "Not Charged" RECORD_SEGMENT: string(300) "00345cx a22001213 4500001000500000004000600005005001700011008003300028014001200061852002900073866007600102866004500178?2728?33520?20140110094250.0?9907060p 8 4001aueng0000000?1 ?a1488910?8 ?bdyk.jnl?hJournal?tCopy 1? 0?80?a1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002)? " ITEM_ID: string(5) "85476" | STATUS: string(11) "Not Charged" RECORD_SEGMENT: string(45) "0?80?a1(1922)-81(2002)?zSome issues missing??" Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 On Jan 10, 2014, at 7:05 AM, Demian Katz <demian.katz@VILLANOVA.EDU<mailto:demian.katz@VILLANOVA.EDU>> wrote: If I'm remembering correctly, Voyager stores the MARC data split into chunks in the Oracle database. There is some logic in the Voyager driver designed to concatenate these chunks back together again. Perhaps what you are seeing is a side effect of this logic -- if so, you might start debugging in the getHoldingData() method. But before you dig into that, it's definitely worth checking if there's a problem in the raw query response, since that might rule out a problem on the VuFind side and allow you to resolve the issue from the Voyager end somehow. Let me know if you need more help! - Demian ________________________________ From: Jason Stirnaman [jst...@ku...<mailto:jst...@ku...>] Sent: Thursday, January 09, 2014 6:37 PM To: Demian Katz Cc: vufind-tech@lists. sourceforge. net Subject: Re: summary holdings problem with Voyager I finally discovered the conditions that seem to cause this. If the MFHD exceeds 301 characters (as counted in the leader) then it pads the 866$a with a leader character up to 302. After that, for each character added to the MFHD a character in the 866$a is replaced, starting at the right, by a character from the leader. e.g. mfhd char count = 301 [0] 1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002) mfhd char count = 302 [0] 1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002)0 [1] 1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002) mfhd char count = 303 [0] 1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(200200 [1] 1(1922)-4(1925),6(1927)-10(1931),13(1934)-17(1938),19(1940)-81(2002) Now I’m not sure whether to call this a Voyager problem or if I should look at the holdings query results or File:MARC results more closely. Seems like I should start with the query result. Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 On Jan 8, 2014, at 9:48 AM, Demian Katz <demian.katz@VILLANOVA.EDU<mailto:demian.katz@VILLANOVA.EDU>> wrote: Yes, I think you may be correct that the Voyager driver is doing redundant processing of the MFHD due to the way things are merged. This could probably be made more efficient, but I don’t think it should be hurting anything. The code’s a bit hard to follow, though, so it’s certainly possible there’s some kind of subtle bug in there. Let me know what you find! - Demian From: Jason Stirnaman [mailto:jst...@ku...] Sent: Wednesday, January 08, 2014 10:19 AM To: Demian Katz Cc: vufind-tech@lists. sourceforge. net Subject: Re: summary holdings problem with Voyager That’s where I have been looking, just wanted to make sure I was interpreting the code correctly since it looks like Drivers/Voyager.php retrieves the MFHD once for each item record? There’s no obvious error in the 866 field itself so I think my next step will be to export the problem Bibs+MFHDs and try parsing them with MarcEdit or Ruby MARC. Thanks, Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 On Jan 8, 2014, at 8:09 AM, Demian Katz <dem...@vi...<mailto:dem...@vi...>> wrote: The holdings summary in VuFind comes from the MFHD 866, so that would be the first place to look! Let me know if you need more help. - Demian From: Jason Stirnaman [mailto:jst...@ku...] Sent: Tuesday, January 07, 2014 2:26 PM To: Demian Katz Cc: vufind-tech@lists. sourceforge. net Subject: Re: summary holdings problem with Voyager Having done more sampling, I think it's happening for all records, but only with one location and not others. That seems to point to a data error in Voyager. I haven’t yet found differences. Would you recommend inspecting the MFHD or the Item records? Does the holdings summary in VuFind come directly from the MFHD 866 or from the item records? Dumping the value from formatHoldings() in ILS/Logic/Holds, > var_dump($retVal[$location]['summary']); array(1) { [0]=> string(39) "158(1986)-187(1993),190(1994)-265(2012)" } array(2) { [0]=> string(20) "59(1952)00313cx a22" [1]=> string(19) "59(1952)-268(2013)-" } array(1) { [0]=> string(59) "1(1923),6(1926)-9(1927),12(1929)-13(1929),18(1932)-23(1934)" } array(1) { [0]=> string(69) "2(1924)-5(1925),10(1928)-11(1928),14(1930)-17(1931),24(1935)-58(1952)” } Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 On Jan 6, 2014, at 6:07 PM, Demian Katz <dem...@vi...<mailto:dem...@vi...>> wrote: Is this happening for all records or just some? Is it possible there's a data error on the Voyager side? Where are you dumping out the raw objects? I can add the same dump to my system to compare output from my own Voyager system if that would be helpful.... (I'll think about this harder once I have a little more information -- just trying to figure out the whole context first). - Demian ________________________________ From: Jason Stirnaman [jst...@ku...<mailto:jst...@ku...>] Sent: Monday, January 06, 2014 5:38 PM To: vufind-tech@lists. sourceforge. net Subject: [VuFind-Tech] summary holdings problem with Voyager The Voyager driver is returning two items that both display for “Volume Holdings”. One is part of the holdings statement concatenated with part of the leader. One is the complete summary holdings statement that we want to display. The raw object looks like O:18:"File_MARC_Subfield":3:{s:7:"*code";s:1:"a";s:7:"*data";s:85:"3no.3-4(1951)-1100370cx a22001333 4500001000500000004000600005005001700011008003300";s:11:"*position";i:1;} O:18:"File_MARC_Subfield":3:{s:7:"*code";s:1:"a";s:7:"*data";s:84:"3no.3-4(1951)-11no.1-2pt.2,4(1959)-17no.2-4(1965)-28no.1,3-4(1976)-50(1998)-63(2011)";s:11:"*position";i:1;} This happens with Voyager 8.x and VuFind 1.2 and 2.x You can see the items are nearly identical except for the “data” value. However, the two items aren’t always in the same order. So, in VuFind 2.x, I could add a regexp to match on the leader pattern and delete that element from the array, but I’m hoping for a better, simpler solution. Thanks, Jason Jason Stirnaman Lead, Library Technology Services University of Kansas Medical Center jst...@ku...<mailto:jst...@ku...> 913-588-7319 |