Re: [Bigdata-developers] stop words in text search

[Bryan T. <br...@sy...>]

Are you rebuilding the index?  The tokenization of the words in the
literals and their entry into the index occurs during data loading.

There is some manual code to drop/add the index.  If you change this
property for an existing namespace, you need to cause the property to
become durable (BigdataSailHelper.setProperty()), and then drop/add the
index.

We are adding more support for text index rebuilds now.  I am not sure if
it will be in the 2.0 release or not yet.

Bryan

----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.blazegraph.com

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  Blazegraph is now available with GPU acceleration using our disruptive
technology to accelerate data-parallel graph analytics and graph query.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Thu, Dec 10, 2015 at 4:38 PM, Jim Balhoff <ba...@gm...> wrote:

> I had tried some similar syntax, but wasn’t sure how it should look. I
> just tried the form you suggested and it did not have an affect on
> stopwords. They seem to still be active, because if my search input is
> simply “of”, I get this message:
>
> WARN : FullTextIndex.java:1052: No terms after stopword extraction:
> query=com.bigdata.rdf.lexicon.ITextIndexer$FullTextQuery@7aef6039
>
> Thanks,
> Jim
>
>
> > On Dec 9, 2015, at 11:34 PM, Brad Bebee <be...@sy...> wrote:
> >
> > Jim,
> >
> > Thank you.  Have you tried configuring your journal with the property
> below?
> >
> > com.bigdata.search.ConfigurableAnalyzerFactory.stopwords=none
> >
> >
> >
> > Thanks, --Brad
> >
> >
> > On Wed, Dec 9, 2015 at 9:36 AM, Jim Balhoff <ba...@gm...> wrote:
> > Hi Brad,
> >
> > I see, I can look into providing my own implementation. I got the
> impression from the JavaDoc that I could provide config options to modify
> the behavior of some of the analyzers. I have been looking at these pages:
> >
> >
> https://www.blazegraph.com/docs/api/com/bigdata/search/ConfigurableAnalyzerFactory.AnalyzerOptions.html#STOPWORDS_VALUE_NONE
> > https://www.blazegraph.com/docs/api/constant-values.html
> >
> > I also tried to switch from the default analyzer to the
> TermCompletionAnalyzer, but haven’t been able to get the property value set
> correctly for “wordBoundary” in the config file.
> >
> > Understanding how to translate option constants from the JavaDoc into
> correctly written config file properties has been a challenge. It would be
> really helpful to have more of those spelled out on the wiki.
> >
> > Thanks!
> > Jim
> >
> > > On Dec 8, 2015, at 9:54 PM, Brad Bebee <be...@sy...> wrote:
> > >
> > > Jim,
> > >
> > > I believe you could do this by overriding the Analyzer Factory Class
> [1] with your own implementation that does not filter stopwords [2].  Other
> may have more specific suggestions.
> > >
> > > [1]
> https://www.blazegraph.com/docs/api/com/bigdata/search/FullTextIndex.Options.html#ANALYZER_FACTORY_CLASS
> > >
> > > [2]
> https://www.blazegraph.com/docs/api/com/bigdata/search/IAnalyzerFactory.html
> > >
> > > Thanks, --Brad
> > >
> > > On Tue, Dec 8, 2015 at 9:12 PM, Jim Balhoff <ba...@gm...> wrote:
> > > Hi,
> > >
> > > I was wondering if anyone could provide examples for how to set up the
> Blazegraph properties file to configure options for the full text search. I
> have looked through the various options in the JavaDoc, but I can’t quite
> figure out the right properties file syntax for ‘stopwords=none’. Here is
> what I am trying to do:
> > >
> > > I have a term in the database with rdfs:label "skeletal element of eye
> region”. When users search for terms, I append a “*” to their input text by
> default. However this is failing when the label contains a stopword like
> “of”. So, searching with “skeletal element of” and “skeletal element*” do
> find the term as a match, but “skeletal element of*” does not. Can I
> disable stopwords entirely?
> > >
> > > Thanks,
> > > Jim
> > >
> > >
> > >
> ------------------------------------------------------------------------------
> > > _______________________________________________
> > > Bigdata-developers mailing list
> > > Big...@li...
> > > https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> > >
> > >
> > >
> > > --
> > > _______________
> > > Brad Bebee
> > > CEO, Managing Partner
> > > SYSTAP, LLC
> > > e:  be...@sy...
> > > m: 202.642.7961
> > > f:  571.367.5000
> > > w:  www.blazegraph.com
> > >
> > > Blazegraph™ is our ultra high-performance graph database that supports
> both RDF/SPARQL and Tinkerpop/Blueprints APIs.  Mapgraph™ is our disruptive
> new technology to use GPUs to accelerate data-parallel graph analytics.
> > >
> > > CONFIDENTIALITY NOTICE:  This email and its contents and attachments
> are for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments. > >
> > >
> >
> >
> >
> >
> > --
> > _______________
> > Brad Bebee
> > CEO, Managing Partner
> > SYSTAP, LLC
> > e:  be...@sy...
> > m: 202.642.7961
> > f:  571.367.5000
> > w:  www.blazegraph.com
> >
> > Blazegraph™ is our ultra high-performance graph database that supports
> both RDF/SPARQL and Tinkerpop/Blueprints APIs.  Mapgraph™ is our disruptive
> new technology to use GPUs to accelerate data-parallel graph analytics.
> >
> > CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments. >
> >
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>