Re: [VuFind-Tech] MarcImporter indexing update

[Andrew N. <and...@vi...>]

Mark - I would love to include the author browse tool in vufind.  Feel free to try to munge it in with the code in SVN.

Andrew
________________________________________
From: Mark Triggs [mt...@nl...]
Sent: Tuesday, April 15, 2008 8:40 AM
To: Andrew Nagy
Cc: James Farrugia; vuf...@li...
Subject: Re: [VuFind-Tech] MarcImporter indexing update

Hi Andrew,

Andrew Nagy <and...@vi...> writes:

> Hi Mark - and thanks for jumping in here - I know you guy s have played
> around with this quite a bit.
>
> I just committed a revised query generator that uses both and'd and or'd
> searches together and weights the AND higher than the OR.  This way we
> get both results.  In my testing I think it works fairly well.

Ah, that's a clever idea.  We don't explicitly boost AND matches, but
some of the other things we do (like boosting phrases with varying
levels of slop) sort of have a similar effect.  I wonder if we'd see
improvements by doing the same thing...

> Also as an aside - I have editted the synomyms.txt file that controls
> synonyms and cleaned it up as well as added in some default synonyms
> that I think will help.  The new synonyms are for numbers and roman
> numbers.  I've created the following:
>
> II,ii,2,two,Two

Cool.  We haven't done roman numerals but we probably should.  So far
we've got a huge file of things like:

    36th,thirty sixth
    37th,thirty seventh
    38th,thirty eighth
    39th,thirty ninth

and various acronyms for the Australian states and things like that.


> So know if you do a search for "world war 2" or "world war ii" the
> results will be the same.  You could do this with your other examples
> such as: stephen,steven

Yeah, I know many of the examples I've given can be worked around with
synonyms, but I don't think they'll catch everything.  Searches like
'Where the wild things are' vs 'Where the wild things were' are the ones
I'm really worried about--where the user has misremembered one of the
words.

> Keep your eye out for 0.8.1 today!

Brilliant!  Once it's ready to go I'll get started on merging our search
code in and let you know when I'm got something to show.  This is really
well-timed as we've just released our next round of features and there's
not too much other stuff on my VuFind plate.  Our most recent notable
thing is probably browse indexes (which our staff and certain users
wouldn't let us neglect):

  http://protocat.nla.gov.au/Browse/Home?browse=author&from=knuth

  http://protocat.nla.gov.au/Browse/Home?browse=subject&from=art

If anyone's interested in this sort of thing feel free to drop me a
line.  We've implemented a new Solr request handler to support this, but
it's not too hard to get going.

Cheers,

Mark

--
Mark Triggs
<mt...@nl...>