Re: [VuFind-Tech] Problem with faceting

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

On Mon, Sep 22, 2008 at 01:03:00AM -0500, Chris Delis wrote:
> On Thu, Sep 18, 2008 at 03:04:46PM -0400, Tim Prettyman wrote:
> > Naomi, I think what I'm talking about is a little different than just long
> > facet query response time.  In my case, the facet response never returns.  I
> > can reproduce the same behavior in your system, although it takes longer
> > search terms.  For example, if I search for:
> > 
> > "Multiculturalism and the foundations of meaningful life reconciling
> > autonomy identity and community"
> 
> 
> Tim,
> 
> I notice the same thing.  And I think you are correct: the query is
> too long for the browser to handle.

I meant: server (not browser).

Chris

> 
> If I enter this search into vufind.org/demo, I get an immediate error
> response from Solr -- you need to turn on the error console (if you
> are using Firefox) in your browser to see it:
> 
> http://vufind.org/demo/Search/Home?lookfor=3DMulticulturalism+and+the+found=
> ations+of+meaningful+life+reconciling+autonomy+identity+and+community&type=
> =3Dall&submit=3DFind&search=3Dnew
> 
> 
> And here's the uber-long link to which it refers:
> 
> Error: mismatched tag. Expected: </br>.
> Source File: http://vufind.org/demo/Search/AJAX?method=3DGetNarrowOptions&q=
> uery=3D((titleStr:%22Multiculturalism%20And%20The%20Foundations%20Of%20Mean=
> ingful%20Life%20Reconciling%20Autonomy%20Identity%20And%20Community%22^1000=
> %20OR%20title_short:%22Multiculturalism%20and%20the%20foundations%20of%20me=
> aningful%20life%20reconciling%20autonomy%20identity%20and%20community%22^75=
> 0%20OR%20title:%22Multiculturalism%20and%20the%20foundations%20of%20meaning=
> ful%20life%20reconciling%20autonomy%20identity%20and%20community*%22^750%20=
> OR%20title:%22Multiculturalism%20and%20the%20foundations%20of%20meaningful%=
> 20life%20reconciling%20autonomy%20identity%20and%20community%22^500%20OR%20=
> title:(multiculturalism%20AND%20and%20AND%20the%20AND%20foundations%20AND%2=
> 0of%20AND%20meaningful%20AND%20life%20AND%20reconciling%20AND%20autonomy%20=
> AND%20identity%20AND%20and%20AND%20community)^100%20OR%20title2:(multicultu=
> ralism%20AND%20and%20AND%20the%20AND%20foundations%20AND%20of%20AND%20meani=
> ngful%20AND%20life%20AND%20reconciling%20AND%20autonomy%20AND%20identity%20=
> AND%20and%20AND%20community)^50%20OR%20oldTitle:(multiculturalism%20AND%20a=
> nd%20AND%20the%20AND%20foundations%20AND%20of%20AND%20meaningful%20AND%20li=
> fe%20AND%20reconciling%20AND%20autonomy%20AND%20identity%20AND%20and%20AND%=
> 20community)%20OR%20newTitle:(multiculturalism%20AND%20and%20AND%20the%20AN=
> D%20foundations%20AND%20of%20AND%20meaningful%20AND%20life%20AND%20reconcil=
> ing%20AND%20autonomy%20AND%20identity%20AND%20and%20AND%20community))^50%20=
> OR%20author:(%22Multiculturalism%20and%20the%20foundations%20of%20meaningfu=
> l%20life%20reconciling%20autonomy%20identity%20and%20community%22)^700%20OR=
> %20author:(multiculturalism%20AND%20and%20AND%20the%20AND%20foundations%20A=
> ND%20of%20AND%20meaningful%20AND%20life%20AND%20reconciling%20AND%20autonom=
> y%20AND%20identity%20AND%20and%20AND%20community)^100%20OR%20author2:(multi=
> culturalism%20AND%20and%20AND%20the%20AND%20foundations%20AND%20of%20AND%20=
> meaningful%20AND%20life%20AND%20reconciling%20AND%20autonomy%20AND%20identi=
> ty%20AND%20and%20AND%20community)^50%20OR%20publishDate:(Multiculturalism%2=
> 0and%20the%20foundations%20of%20meaningful%20life%20reconciling%20autonomy%=
> 20identity%20and%20community)%20OR%20physical:(multiculturalism%20AND%20and=
> %20AND%20the%20AND%20foundations%20AND%20of%20AND%20meaningful%20AND%20life=
> %20AND%20reconciling%20AND%20autonomy%20AND%20identity%20AND%20and%20AND%20=
> community)^10%20OR%20contents:(multiculturalism%20AND%20and%20AND%20the%20A=
> ND%20foundations%20AND%20of%20AND%20meaningful%20AND%20life%20AND%20reconci=
> ling%20AND%20autonomy%20AND%20identity%20AND%20and%20AND%20community)^10%20=
> OR%20series:(multiculturalism%20AND%20and%20AND%20the%20AND%20foundations%2=
> 0AND%20of%20AND%20meaningful%20AND%20life%20AND%20reconciling%20AND%20auton=
> omy%20AND%20identity%20AND%20and%20AND%20community)^50%20OR%20series2:(mult=
> iculturalism%20AND%20and%20AND%20the%20AND%20foundations%20AND%20of%20AND%2=
> 0meaningful%20AND%20life%20AND%20reconciling%20AND%20autonomy%20AND%20ident=
> ity%20AND%20and%20AND%20community)^30%20OR%20topic:(%22Multiculturalism%20a=
> nd%20the%20foundations%20of%20meaningful%20life%20reconciling%20autonomy%20=
> identity%20and%20community%22)^750%20OR%20fulltopic:(multiculturalism%20AND=
> %20and%20AND%20the%20AND%20foundations%20AND%20of%20AND%20meaningful%20AND%=
> 20life%20AND%20reconciling%20AND%20autonomy%20AND%20identity%20AND%20and%20=
> AND%20community)^30%20OR%20geographic:(%22Multiculturalism%20and%20the%20fo=
> undations%20of%20meaningful%20life%20reconciling%20autonomy%20identity%20an=
> d%20community%22)^300%20OR%20fullgeographic:(multiculturalism%20AND%20and%2=
> 0AND%20the%20AND%20foundations%20AND%20of%20AND%20meaningful%20AND%20life%2=
> 0AND%20reconciling%20AND%20autonomy%20AND%20identity%20AND%20and%20AND%20co=
> mmunity)^30%20OR%20genre:(%22Multiculturalism%20and%20the%20foundations%20o=
> f%20meaningful%20life%20reconciling%20autonomy%20identity%20and%20community=
> %22)^300%20OR%20fullgenre:(multiculturalism%20AND%20and%20AND%20the%20AND%2=
> 0foundations%20AND%20of%20AND%20meaningful%20AND%20life%20AND%20reconciling=
> %20AND%20autonomy%20AND%20identity%20AND%20and%20AND%20community)^30%20OR%2=
> 0era:(multiculturalism%20AND%20and%20AND%20the%20AND%20foundations%20AND%20=
> of%20AND%20meaningful%20AND%20life%20AND%20reconciling%20AND%20autonomy%20A=
> ND%20identity%20AND%20and%20AND%20community)^30%20OR%20allfields:(multicult=
> uralism%20OR%20and%20OR%20the%20OR%20foundations%20OR%20of%20OR%20meaningfu=
> l%20OR%20life%20OR%20reconciling%20OR%20autonomy%20O
> 
> 
> 
> If I modify the search and remove the final word, community, then I
> get a successful result, facets and all:
> 
> 
> http://vufind.org/demo/Search/Home?lookfor=3DMulticulturalism+and+the+found=
> ations+of+meaningful+life+reconciling+autonomy+identity+and+community&type=
> =3Dall&submit=3DFind&search=3Dnew
> 
> I think the problem is due to the getNarrowOptions function in
> web/services/Search/ajax.js performing an HTTP GET with the huge URL
> (which apparently gets truncated) above, which is too large for the
> server to process correctly.
> 
> I wonder if solving this problem is as easy as performing an HTTP POST
> instead of GET in getNarrowOptions.
> 
> Chris
> 
> 
> 
> 
> 
> > 
> > (without the quotes), I get the single matched record, but nothing in the
> > "Refine your search" box.
> > 
> > -Tim
> > 
> > 
> > On 9/18/08 1:42 PM, "Naomi Dushay" <nd...@st...> wrote:
> > 
> > > Tim,
> > > 
> > > Facet query response times for large collections with lots of values
> > > (think "topic" or "author" fields, in particular) ... suck.  At least
> > > ours does.   We have different facets for our home page than we do for
> > > search results pages.  None of the home page facets have zillions of
> > > values ... and they load very quickly.  But the search result facets
> > > take a  looooong time.
> > > 
> > > http://searchworks.stanford.edu
> > > 
> > > We are looking into the NLA plugin, but we are currently SOLR 1.2.2
> > > and we want to upgrade to SOLR 1.3, and it currently doesn't work with
> > > either of those versions.
> > > 
> > > The benchmark testing I've been trying with JMeter so far has been
> > > less than illuminating, because warming up the filterCache seems to
> > > take a LONG time.
> > > 
> > > machine: very fast.
> > > JVM:  20G heap space (max and initial)
> > > filterCache size:  50 million   (index has just over 38 million unique
> > > terms).   75% hit rate and up.
> > > hashDocSetMaxSize:  300000
> > > 
> > > I diddled with setting facet.enum.cache.minDf  for the topic facet
> > > field;  couldn't see a difference (see benchmarking difficulties
> > > mentioned above).  The documentation suggests that setting this
> > > *reduces* query speed.
> > >    http://wiki.apache.org/solr/SimpleFacetParameters
> > > 
> > > Suggestions welcome.
> > > 
> > > - Naomi
> > > 
> > > 
> > > On Sep 18, 2008, at 6:21 AM, Tim Prettyman wrote:
> > > 
> > 
> > 
> > 
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> > Build the coolest Linux based applications with Moblin SDK & win great prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Vufind-tech mailing list
> > Vuf...@li...
> > https://lists.sourceforge.net/lists/listinfo/vufind-tech