From: Oliver S. <oli...@un...> - 2013-09-06 07:41:57
|
Hello VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as strings, and how can they ever match with all the munges going on in the searchspecs? Indexed as string, my callnumbers match if I enter them as they are, and if I munge only 'onephrase'. But of course I can't do wildcard searches, and I can't get the cases wrong or anything else. Can anyone clarify? Thanks and regards Oliver |
From: Christian D. <dab...@fh...> - 2013-09-06 08:11:26
|
Hi Oliver, we also had a lot of similar problems with the callnumbers: In our case we had to put the callnumber even into quotes to be able to find them - but certain chars that are used in our callnumbers (like the colon) broke the search so this approach didn't work for us. Furthermore we also had the problem that our users used only parts of the callnumbers for the query (we have two versions of each callnumber: a standard one and standard plus prefix). What finally worked for us was clearing the callnumbers from all slashes, dashes, brackets etc. before indexing. Then we made the query also escape all types of chars we cleared before. Like this we could even bind the callnumber search into allfields. Hope it helps, Christian Am 06.09.2013 09:41, schrieb Oliver Schihin: > Hello > > VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as strings, and how can > they ever match with all the munges going on in the searchspecs? > > Indexed as string, my callnumbers match if I enter them as they are, and if I munge only > 'onephrase'. But of course I can't do wildcard searches, and I can't get the cases wrong > or anything else. > > Can anyone clarify? > > Thanks and regards > Oliver > > ------------------------------------------------------------------------------ > Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! > Discover the easy way to master current and previous Microsoft technologies > and advance your career. Get an incredible 1,500+ hours of step-by-step > tutorial videos with LearnDevNow. Subscribe today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk > _______________________________________________ > VuFind-General mailing list > VuF...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-general > |
From: Oliver S. <oli...@un...> - 2013-09-06 09:27:52
|
Hi Christian But you do index callnumbers with type="string"? Or do you have a custom callnumber type? Thanks Oliver -------- Original-Nachricht -------- Betreff: Re: [VuFind-General] callnumber indexing Von: Christian Dabrowski <dab...@fh...> An: Oliver Schihin <oli...@un...>, vuf...@li... <vuf...@li...> Datum: 06.09.2013 10:11 > Hi Oliver, > > we also had a lot of similar problems with the callnumbers: In our case we had to put the > callnumber even into quotes to be able to find them - but certain chars that are used in > our callnumbers (like the colon) broke the search so this approach didn't work for us. > Furthermore we also had the problem that our users used only parts of the callnumbers for > the query (we have two versions of each callnumber: a standard one and standard plus prefix). > > What finally worked for us was clearing the callnumbers from all slashes, dashes, brackets > etc. before indexing. Then we made the query also escape all types of chars we cleared > before. Like this we could even bind the callnumber search into allfields. > > Hope it helps, > Christian > > > > Am 06.09.2013 09:41, schrieb Oliver Schihin: >> Hello >> >> VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as strings, and how can >> they ever match with all the munges going on in the searchspecs? >> >> Indexed as string, my callnumbers match if I enter them as they are, and if I munge only >> 'onephrase'. But of course I can't do wildcard searches, and I can't get the cases wrong >> or anything else. >> >> Can anyone clarify? >> >> Thanks and regards >> Oliver >> >> ------------------------------------------------------------------------------ >> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! >> Discover the easy way to master current and previous Microsoft technologies >> and advance your career. Get an incredible 1,500+ hours of step-by-step >> tutorial videos with LearnDevNow. Subscribe today and save! >> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk >> _______________________________________________ >> VuFind-General mailing list >> VuF...@li... >> https://lists.sourceforge.net/lists/listinfo/vufind-general >> > . > |
From: Christian D. <dab...@fh...> - 2013-09-06 10:44:02
|
Hi Oliver, yes it is indexed as string and it is multivalued - that was also a great deal to make it work: <field name="callnumber-swf" type="string" indexed="true" stored="true" multiValued="true"/> Christian Am 06.09.2013 11:27, schrieb Oliver Schihin: > Hi Christian > > But you do index callnumbers with type="string"? Or do you have a custom > callnumber type? > > Thanks > Oliver > > -------- Original-Nachricht -------- > Betreff: Re: [VuFind-General] callnumber indexing > Von: Christian Dabrowski <dab...@fh...> > An: Oliver Schihin <oli...@un...>, > vuf...@li... <vuf...@li...> > Datum: 06.09.2013 10:11 > >> Hi Oliver, >> >> we also had a lot of similar problems with the callnumbers: In our >> case we had to put the >> callnumber even into quotes to be able to find them - but certain >> chars that are used in >> our callnumbers (like the colon) broke the search so this approach >> didn't work for us. >> Furthermore we also had the problem that our users used only parts of >> the callnumbers for >> the query (we have two versions of each callnumber: a standard one and >> standard plus prefix). >> >> What finally worked for us was clearing the callnumbers from all >> slashes, dashes, brackets >> etc. before indexing. Then we made the query also escape all types of >> chars we cleared >> before. Like this we could even bind the callnumber search into >> allfields. >> >> Hope it helps, >> Christian >> >> >> >> Am 06.09.2013 09:41, schrieb Oliver Schihin: >>> Hello >>> >>> VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as >>> strings, and how can >>> they ever match with all the munges going on in the searchspecs? >>> >>> Indexed as string, my callnumbers match if I enter them as they are, >>> and if I munge only >>> 'onephrase'. But of course I can't do wildcard searches, and I can't >>> get the cases wrong >>> or anything else. >>> >>> Can anyone clarify? >>> >>> Thanks and regards >>> Oliver >>> >>> ------------------------------------------------------------------------------ >>> >>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! >>> Discover the easy way to master current and previous Microsoft >>> technologies >>> and advance your career. Get an incredible 1,500+ hours of step-by-step >>> tutorial videos with LearnDevNow. Subscribe today and save! >>> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk >>> >>> _______________________________________________ >>> VuFind-General mailing list >>> VuF...@li... >>> https://lists.sourceforge.net/lists/listinfo/vufind-general >>> >> . >> > > |
From: Demian K. <dem...@vi...> - 2013-09-06 11:15:07
|
Call number search was originally designed to do left-anchored searches of LC call numbers. This was achieved by indexing the call numbers as strings, then using munge rules in searchspecs.yaml to generate a basic Lucene query for the search. This was all designed very early in VuFind's development and has not been significantly revisited since then -- it's entirely possible that this can now be better achieved through more native Solr functionality. It's also true that the logic that works reasonably well for LC numbers is less suitable for other systems. I'm definitely open to proposals/patches to improve this -- but for Villanova's local needs, it has stayed since it's "good enough." - Demian ________________________________________ From: Oliver Schihin [oli...@un...] Sent: Friday, September 06, 2013 3:41 AM To: vufind-general (list) Subject: [VuFind-General] callnumber indexing Hello VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as strings, and how can they ever match with all the munges going on in the searchspecs? Indexed as string, my callnumbers match if I enter them as they are, and if I munge only 'onephrase'. But of course I can't do wildcard searches, and I can't get the cases wrong or anything else. Can anyone clarify? Thanks and regards Oliver ------------------------------------------------------------------------------ Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk _______________________________________________ VuFind-General mailing list VuF...@li... https://lists.sourceforge.net/lists/listinfo/vufind-general |
From: Oliver S. <oli...@un...> - 2013-09-06 11:58:33
|
Now I am getting closer. We are using a processing engine different from solr-marc, and I send callnumbers unchanged to the index. You seem to do some beanshell-magic on them. I looked at callnumbers.bsh, but do not really see it: What chars are you replacing and/or deleting before the fields are sent to the string-typed callnumber field? My solution works well now, I did a small textType named callnum. It does keyword tokenizing, lowercasing, replacing whitespace and trailing asterisks, trimming. With the appropriate munges in searchspecs.yaml, everything seems to be fine, exact as well as wildcard searches work I would not want to refactor a thing so proved and tested like callnumber searches in VuFind. And we do not have LC numbers. But a way to go would be to do the character replacement stuff at indexing time in a text type. It looks more generic to me, with no need to build the routine in other import paths. And if you keep them stored and want to show them somewhere, they look just the way they should. Oliver -------- Original-Nachricht -------- Betreff: Re: [VuFind-General] callnumber indexing Von: Demian Katz <dem...@vi...> An: Oliver Schihin <oli...@un...>, vufind-general (list) <vuf...@li...> Datum: 06.09.2013 13:14 > Call number search was originally designed to do left-anchored searches of LC call > numbers. This was achieved by indexing the call numbers as strings, then using munge > rules in searchspecs.yaml to generate a basic Lucene query for the search. This was > all designed very early in VuFind's development and has not been significantly > revisited since then -- it's entirely possible that this can now be better achieved > through more native Solr functionality. It's also true that the logic that works > reasonably well for LC numbers is less suitable for other systems. I'm definitely open > to proposals/patches to improve this -- but for Villanova's local needs, it has stayed > since it's "good enough." > > - Demian ________________________________________ From: Oliver Schihin > [oli...@un...] Sent: Friday, September 06, 2013 3:41 AM To: vufind-general > (list) Subject: [VuFind-General] callnumber indexing > > Hello > > VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as strings, and how > can they ever match with all the munges going on in the searchspecs? > > Indexed as string, my callnumbers match if I enter them as they are, and if I munge > only 'onephrase'. But of course I can't do wildcard searches, and I can't get the cases > wrong or anything else. > > Can anyone clarify? > > Thanks and regards Oliver > > ------------------------------------------------------------------------------ Learn > the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way > to master current and previous Microsoft technologies and advance your career. Get an > incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe > today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk > _______________________________________________ VuFind-General mailing list > VuF...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-general > |
From: Demian K. <dem...@vi...> - 2013-09-06 12:26:52
|
It looks like the only thing that's being stripped from the callnumber field is the space character... plus text is being uppercased. There's more complex logic applied to the Dewey numbers. - Demian ________________________________________ From: Oliver Schihin [oli...@un...] Sent: Friday, September 06, 2013 7:58 AM To: Demian Katz Cc: vufind-general (list) Subject: Re: [VuFind-General] callnumber indexing Now I am getting closer. We are using a processing engine different from solr-marc, and I send callnumbers unchanged to the index. You seem to do some beanshell-magic on them. I looked at callnumbers.bsh, but do not really see it: What chars are you replacing and/or deleting before the fields are sent to the string-typed callnumber field? My solution works well now, I did a small textType named callnum. It does keyword tokenizing, lowercasing, replacing whitespace and trailing asterisks, trimming. With the appropriate munges in searchspecs.yaml, everything seems to be fine, exact as well as wildcard searches work I would not want to refactor a thing so proved and tested like callnumber searches in VuFind. And we do not have LC numbers. But a way to go would be to do the character replacement stuff at indexing time in a text type. It looks more generic to me, with no need to build the routine in other import paths. And if you keep them stored and want to show them somewhere, they look just the way they should. Oliver -------- Original-Nachricht -------- Betreff: Re: [VuFind-General] callnumber indexing Von: Demian Katz <dem...@vi...> An: Oliver Schihin <oli...@un...>, vufind-general (list) <vuf...@li...> Datum: 06.09.2013 13:14 > Call number search was originally designed to do left-anchored searches of LC call > numbers. This was achieved by indexing the call numbers as strings, then using munge > rules in searchspecs.yaml to generate a basic Lucene query for the search. This was > all designed very early in VuFind's development and has not been significantly > revisited since then -- it's entirely possible that this can now be better achieved > through more native Solr functionality. It's also true that the logic that works > reasonably well for LC numbers is less suitable for other systems. I'm definitely open > to proposals/patches to improve this -- but for Villanova's local needs, it has stayed > since it's "good enough." > > - Demian ________________________________________ From: Oliver Schihin > [oli...@un...] Sent: Friday, September 06, 2013 3:41 AM To: vufind-general > (list) Subject: [VuFind-General] callnumber indexing > > Hello > > VuFinds solr-schema leaves me puzzled. Why are callnumbers indexed as strings, and how > can they ever match with all the munges going on in the searchspecs? > > Indexed as string, my callnumbers match if I enter them as they are, and if I munge > only 'onephrase'. But of course I can't do wildcard searches, and I can't get the cases > wrong or anything else. > > Can anyone clarify? > > Thanks and regards Oliver > > ------------------------------------------------------------------------------ Learn > the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way > to master current and previous Microsoft technologies and advance your career. Get an > incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe > today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk > _______________________________________________ VuFind-General mailing list > VuF...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-general > |