Menu
▾
▴
Re: [Bigdata-developers] Subquery wildcard projections not rewritten
|
From: Lee K. <le...@sw...> - 2015-04-22 10:04:19
|
Hi Bryan,
Yes the AST in the test is supposed to be for the query
select (count(*) as ?c) where {
select * where {
select * where { ?s ?p ?o }
} limit 21 offset 0
}
Thanks
On Tue, Apr 21, 2015 at 7:53 PM, Bryan Thompson <br...@sy...> wrote:
> Lee,
>
> I can replicate the problem with your query (as given above) against the
> sparql end point.
>
> Can you state the SPARQL that you are trying to model with this unit
> test? It appears to be not query the same as your SPARQL query above. I
> would like to make sure that it is being translated correctly into the
> AST. I can then look at the expected AST and work backwards and see if I
> believe that the test shows the problem.
>
> Thanks,
> Bryan
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Tue, Apr 21, 2015 at 11:07 AM, Lee Kitching <le...@sw...> wrote:
>
>> Hi Bryan,
>>
>> We allow users to enter their own SPARQL queries and wrap them to do
>> things like pagination so unfortunately we cannot just re-write our queries
>> to do the expansion manually.
>> I applied the fix detailed in the ticket and it fixes the for the query I
>> provided, however it fails to rewrite the following query:
>>
>> SELECT (COUNT(*) as ?c) {
>> SELECT * {
>> SELECT * WHERE { ?s ?p ?o }
>> } LIMIT 21 OFFSET 0
>> }
>>
>> I attempted to debug the issue, and it seems to re-write the * projection
>> in the inner-most subquery but not the subquery with the limit and offset.
>> I created a test based on the
>> existing tests:
>>
>> public void test_wildcardProjectionOptimizer03() {
>>
>> /*
>> * Note: DO NOT share structures in this test!!!!
>> */
>> final IBindingSet[] bsets = new IBindingSet[] {};
>>
>> // The source AST.
>> final QueryRoot given = new QueryRoot(QueryType.SELECT);
>> {
>> final SubqueryRoot selectQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final JoinGroupNode whereClause1 = new JoinGroupNode();
>> final StatementPatternNode spoPattern = new
>> StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"),
>> null, Scope.DEFAULT_CONTEXTS);
>> whereClause1.addChild(spoPattern);
>>
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("*"));
>> selectQuery.setProjection(p);
>> selectQuery.setWhereClause(whereClause1);
>> }
>>
>> final SubqueryRoot sliceQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("*"));
>> sliceQuery.setProjection(p);
>>
>> final JoinGroupNode whereClause = new JoinGroupNode();
>> whereClause.addChild(selectQuery);
>>
>> sliceQuery.setSlice(new SliceNode(0, 21));
>> }
>>
>> final FunctionNode countNode = new FunctionNode(
>> FunctionRegistry.COUNT,
>> Collections.EMPTY_MAP,
>> new VarNode("*"));
>>
>> final ProjectionNode countProjection = new ProjectionNode();
>> countProjection.addProjectionExpression(new
>> AssignmentNode(new VarNode("c"), countNode));
>>
>> JoinGroupNode countWhere = new JoinGroupNode();
>> countWhere.addChild(sliceQuery);
>>
>> given.setProjection(countProjection);
>> given.setWhereClause(countWhere);
>> }
>>
>> final QueryRoot expected = new QueryRoot(QueryType.SELECT);
>> {
>> final SubqueryRoot selectQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final JoinGroupNode whereClause1 = new JoinGroupNode();
>> final StatementPatternNode spoPattern = new
>> StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"),
>> null, Scope.DEFAULT_CONTEXTS);
>> whereClause1.addChild(spoPattern);
>>
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("s"));
>> p.addProjectionVar(new VarNode("p"));
>> p.addProjectionVar(new VarNode("o"));
>> selectQuery.setProjection(p);
>> selectQuery.setWhereClause(whereClause1);
>> }
>>
>> final SubqueryRoot sliceQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("s"));
>> p.addProjectionVar(new VarNode("p"));
>> p.addProjectionVar(new VarNode("o"));
>>
>> sliceQuery.setProjection(p);
>>
>> final JoinGroupNode whereClause = new JoinGroupNode();
>> whereClause.addChild(selectQuery);
>>
>> sliceQuery.setSlice(new SliceNode(0, 21));
>> }
>>
>> final FunctionNode countNode = new FunctionNode(
>> FunctionRegistry.COUNT,
>> Collections.EMPTY_MAP,
>> new VarNode("*"));
>>
>> final ProjectionNode countProjection = new ProjectionNode();
>> countProjection.addProjectionExpression(new
>> AssignmentNode(new VarNode("c"), countNode));
>>
>> JoinGroupNode countWhere = new JoinGroupNode();
>> countWhere.addChild(sliceQuery);
>>
>> expected.setProjection(countProjection);
>> expected.setWhereClause(countWhere);
>> }
>>
>> final IASTOptimizer rewriter = new
>> ASTWildcardProjectionOptimizer();
>>
>> final IQueryNode actual = rewriter.optimize(null/* AST2BOpContext
>> */,
>> given/* queryNode */, bsets);
>>
>> assertSameAST(expected, actual);
>>
>> }
>>
>> however I am having some problems running the tests locally so I don't
>> know if it accurately models the situation.
>>
>> Thanks
>>
>>
>>
>> On Mon, Apr 20, 2015 at 9:05 PM, Bryan Thompson <br...@sy...> wrote:
>>
>>> Lee,
>>>
>>> I've updated the ticket with the code changes and the test changes.
>>> Please try this out and let me know if you have any problems.
>>>
>>> Thanks,
>>> Bryan
>>>
>>> ----
>>> Bryan Thompson
>>> Chief Scientist & Founder
>>> SYSTAP, LLC
>>> 4501 Tower Road
>>> Greensboro, NC 27410
>>> br...@sy...
>>> http://blazegraph.com
>>> http://blog.bigdata.com <http://bigdata.com>
>>> http://mapgraph.io
>>>
>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>>> technology to use GPUs to accelerate data-parallel graph analytics.
>>>
>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments
>>> are for the sole use of the intended recipient(s) and are confidential or
>>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>>> dissemination or copying of this email or its contents or attachments is
>>> prohibited. If you have received this communication in error, please notify
>>> the sender by reply email and permanently delete all copies of the email
>>> and its contents and attachments.
>>>
>>> On Mon, Apr 20, 2015 at 1:20 PM, Lee Kitching <le...@sw...> wrote:
>>>
>>>> Hi,
>>>>
>>>> We are currently evaluating using Blazegraph as our rdf database and
>>>> have run in the issue described at http://trac.bigdata.com/ticket/757.
>>>> The below query causes the AssertionError to be thrown:
>>>>
>>>> SELECT (COUNT(*) as ?c) {
>>>> SELECT ?uri ?graph where {
>>>> {
>>>> SELECT * WHERE {
>>>> GRAPH ?graph {
>>>> ?uri a <http://object> .
>>>> ?uri <http://purl.org/dc/terms/title> ?title .
>>>> }
>>>> MINUS {
>>>> ?uri a <http://other>
>>>> }
>>>> }
>>>> ORDER BY ?title
>>>> }
>>>> }
>>>> }
>>>>
>>>> Some debugging shows that the error is caused by the
>>>> ASTWildcardProjectionOptimizer failing to recurse into the subqueries to
>>>> rewrite the * projection. However this recursion is implemented in the
>>>> BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator
>>>> to
>>>> find child operators and therefore only visits children for nodes with
>>>> an arity > 0.
>>>>
>>>> The root query node for the above query has an empty 'args' collection
>>>> and all the associated components of the top-level query are stored in the
>>>> annotations map. It looks like the iterator should search through the
>>>> annotations rather than the args for query nodes.
>>>>
>>>> As there are a lot of implementations of the BOp interface, it seems
>>>> that changing the postOrderIterator2(BOp) method is unlikely to be the
>>>> correct fix. It seems that either the AST query nodes should override the
>>>> arity() function to return the count of the annotations map, or the
>>>> ASTWildcardProjectionOptimizer should use its own iterator for the nodes of
>>>> the query. The latter option would be the least impactful change but I am
>>>> not familiar with the codebase to understand the correct fix.
>>>>
>>>> Any help in resolving the issue would be appreciated.
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>>>> Develop your own process in accordance with the BPMN 2 standard
>>>> Learn Process modeling best practices with Bonita BPM through live
>>>> exercises
>>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>>>> event?utm_
>>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>>>> _______________________________________________
>>>> Bigdata-developers mailing list
>>>> Big...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>
>>>>
>>>
>>
>
|
