bigdata-developers

[Bigdata-developers] filter not exists lookups?

[Stas M. <sma...@wi...>]

Hi!

I am trying to run a query which does a lookup for non-existing links,
specifically:

prefix wdt: <http://www.wikidata.org/prop/direct/>
prefix entity: <http://www.wikidata.org/entity/>
SELECT ?item WHERE {
  ?item wdt:P31 entity:Q5 .
  FILTER NOT EXISTS { ?item wdt:P18 ?dummy0 }
} limit 5

This tries to find all items with link to wdt:P31->entity:Q5 that lack
wdt:P18 predicate. The first line has a lot of matches (about 2.7
millions) and a lot of them do have wdt:P18 predicate. So the query is
slow. My question is - can it be imporved? The reverse query - i.e.
items having both wdt:P31 and  wdt:P18 - is very fast, since I assume it
uses indexes. But for the negative one looks like it doesn't. Can
anything be done to improve it?

-- 
Stas Malyshev
sma...@wi...

Re: [Bigdata-developers] filter not exists lookups?

[Michael S. <ms...@me...>]

Stas,

thanks for reporting, I’ll open a ticket. The problem is that FILTER NOT EXISTS is translated using a blocking (hash join) operator, so the query plan cannot pipeline through the first five results early on. There are a couple of optimisation approaches we’ve recently discussed that could help rewriting this into a more efficient plan. 

One thing you can do as a workaround is the “old-fashioned” SPARQL OPTIONAL + FILTER + !bound construct for expressing negation:

prefix wdt: <http://www.wikidata.org/prop/direct/>
prefix entity: <http://www.wikidata.org/entity/>
SELECT ?item WHERE {
 ?item wdt:P31 entity:Q5 .
 OPTIONAL { ?item wdt:P18 ?dummy0 }
 FILTER(!bound(?dummy0))
} limit 5

This gives you a fully pipelined plan and runs amazingly fast :).

Best,
Michael


> On 29 May 2015, at 21:27, Stas Malyshev <sma...@wi...> wrote:
> 
> Hi!
> 
> I am trying to run a query which does a lookup for non-existing links,
> specifically:
> 
> prefix wdt: <http://www.wikidata.org/prop/direct/>
> prefix entity: <http://www.wikidata.org/entity/>
> SELECT ?item WHERE {
>  ?item wdt:P31 entity:Q5 .
>  FILTER NOT EXISTS { ?item wdt:P18 ?dummy0 }
> } limit 5
> 
> This tries to find all items with link to wdt:P31->entity:Q5 that lack
> wdt:P18 predicate. The first line has a lot of matches (about 2.7
> millions) and a lot of them do have wdt:P18 predicate. So the query is
> slow. My question is - can it be imporved? The reverse query - i.e.
> items having both wdt:P31 and  wdt:P18 - is very fast, since I assume it
> uses indexes. But for the negative one looks like it doesn't. Can
> anything be done to improve it?
> 
> -- 
> Stas Malyshev
> sma...@wi...
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] filter not exists lookups?

[Stas M. <sma...@wi...>]

Hi!

> prefix wdt: <http://www.wikidata.org/prop/direct/>
> prefix entity: <http://www.wikidata.org/entity/>
> SELECT ?item WHERE {
>  ?item wdt:P31 entity:Q5 .
>  OPTIONAL { ?item wdt:P18 ?dummy0 }
>  FILTER(!bound(?dummy0))
> } limit 5

Works excellent! Thanks a lot, I didn't know that way of doing it.
-- 
Stas Malyshev
sma...@wi...

[Bigdata-developers] Usage of Models

[Maximilian B. <br...@su...>]

Dear all,

since I'm very new to BigData I have a maybe very simple question
regarding the java api.

Is there any way to use Models similiar to Jena?

best regards,

Max

Re: [Bigdata-developers] Usage of Models

[Bryan T. <br...@sy...>]

There is no built-in bridge for Jena.  However if you use SPARQL and SPARQL
UPDATE, then you can obviously use Jena to process the result sets within
the application.

Thanks,
Bryan

----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Tue, Jun 16, 2015 at 11:22 AM, Maximilian Brodhun <
br...@su...> wrote:

>
> Dear all,
>
> since I'm very new to BigData I have a maybe very simple question
> regarding the java api.
>
> Is there any way to use Models similiar to Jena?
>
> best regards,
>
> Max
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>