bigdata-developers

[Bigdata-developers] interaction between #699 and #684

[Jeremy J C. <jj...@sy...>]

Hi Mike

Bryan pointed out that you are working on
http://sourceforge.net/apps/trac/bigdata/ticket/699

and he wondered about the interaction with

http://sourceforge.net/apps/trac/bigdata/ticket/684

since both are dealing with optimizing queries involving property paths.

My judgment is that the interaction is likely to be little - since the 684 code deals with unions, whereas the 699 fix is to do with arbitrary length paths

They may interact in cases like


[A]
  ?sub ( rdfs:subPropertyOf | my:subPropertyOf ) * ?super

===

On the topic of 684, Bryan suggested that I should:
- omit the LUBM tests since they do not involve UNION
- execute the Berlin tests

If these tests are satisfactory then to merge in with the RELEASE_1_2_0 branch

Maybe one of us should write a test case for [A] as well.

I have not yet understood how/whether the code does dynamic as well as static optimization.
On #699 Bryan seems to be suggesting detecting the fully bound case of ALP and treating it differently from other ALPs; I guess that can be done before query execution as part of planning in most cases.

===

My timeline is that I am not working on #684 today, but will find some time tomorrow to understand how to set up Berlin Benchmark



Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] interaction between #699 and #684

[Mike P. <mi...@sy...>]

Jeremy,

They are related.  The static optimizer needs to get the ordering for
property paths right, right now it does not consider them.

Do you mind if I take over ticket 684?  Can you check in a test case for
it so that I can run it?

Mike


On 7/11/13 2:31 PM, "Jeremy J Carroll" <jj...@sy...> wrote:

>
>
>Hi Mike
>
>Bryan pointed out that you are working on
>http://sourceforge.net/apps/trac/bigdata/ticket/699
>
>and he wondered about the interaction with
>
>http://sourceforge.net/apps/trac/bigdata/ticket/684
>
>since both are dealing with optimizing queries involving property paths.
>
>My judgment is that the interaction is likely to be little - since the
>684 code deals with unions, whereas the 699 fix is to do with arbitrary
>length paths
>
>They may interact in cases like
>
>
>[A]
>  ?sub ( rdfs:subPropertyOf | my:subPropertyOf ) * ?super
>
>===
>
>On the topic of 684, Bryan suggested that I should:
>- omit the LUBM tests since they do not involve UNION
>- execute the Berlin tests
>
>If these tests are satisfactory then to merge in with the RELEASE_1_2_0
>branch
>
>Maybe one of us should write a test case for [A] as well.
>
>I have not yet understood how/whether the code does dynamic as well as
>static optimization.
>On #699 Bryan seems to be suggesting detecting the fully bound case of
>ALP and treating it differently from other ALPs; I guess that can be done
>before query execution as part of planning in most cases.
>
>===
>
>My timeline is that I am not working on #684 today, but will find some
>time tomorrow to understand how to set up Berlin Benchmark
>
>
>
>Jeremy J Carroll
>Principal Architect
>Syapse, Inc.
>
>
>
>
>--------------------------------------------------------------------------
>----
>See everything from the browser to the database with AppDynamics
>Get end-to-end visibility with application monitoring from AppDynamics
>Isolate bottlenecks and diagnose root cause in seconds.
>Start your free trial of AppDynamics Pro today!
>http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktr
>k
>_______________________________________________
>Bigdata-developers mailing list
>Big...@li...
>https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] interaction between #699 and #684

[Jeremy J C. <jj...@sy...>]

This is now done.
Sorry it took me a while to get my infrastructure in place.

The tests are in

bigdata-rdf/src/test/com/bigdata/rdf/sparql/ast/optimizers/TestASTStaticJoinOptimizer.java

and are named

xtest_union_trac684_A
test_union_trac684_B
xtest_union_trac684_C


The first and last currently fail, and I renamed them as not to suggest that things had got worse.
To run the tests you will need to rename them back again

(I am used to just adding an @Ignore annotation … for this usage - what is the preference in this project?)

I am now working on producing a new patch file of my solution to these failing tests. I am sorry we had miscommunicated, since I was waiting for feedback from you on a previous patch …. :(




Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 12, 2013, at 7:46 AM, Mike Personick <mi...@sy...> wrote:

> Can you check in a test case for
> it so that I can run it?

Re: [Bigdata-developers] interaction between #699 and #684

[Jeremy J C. <jj...@sy...>]

Patch is now attached to the trace item at
https://sourceforge.net/apps/trac/bigdata/attachment/ticket/684/trac684B.patch
 $ patch -p1 -i /tmp/trac684B.patch


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 15, 2013, at 9:50 AM, Jeremy J Carroll <jj...@sy...> wrote:

> This is now done.
> Sorry it took me a while to get my infrastructure in place.
> 
> The tests are in
> 
> bigdata-rdf/src/test/com/bigdata/rdf/sparql/ast/optimizers/TestASTStaticJoinOptimizer.java
> 
> and are named
> 
> xtest_union_trac684_A
> test_union_trac684_B
> xtest_union_trac684_C
> 
> 
> The first and last currently fail, and I renamed them as not to suggest that things had got worse.
> To run the tests you will need to rename them back again
> 
> (I am used to just adding an @Ignore annotation … for this usage - what is the preference in this project?)
> 
> I am now working on producing a new patch file of my solution to these failing tests. I am sorry we had miscommunicated, since I was waiting for feedback from you on a previous patch …. :(
> 
> 
> 
> 
> Jeremy J Carroll
> Principal Architect
> Syapse, Inc.
> 
> 
> 
> On Jul 12, 2013, at 7:46 AM, Mike Personick <mi...@sy...> wrote:
> 
>> Can you check in a test case for
>> it so that I can run it?
> 

Re: [Bigdata-developers] interaction between #699 and #684

[Bryan T. <br...@sy...>]

Morning all. What is the status on this patch and the related issues around UNION optimizations as they pertain to property paths?  Also, are the changes applied in the 1.2.x maintenance branch and do they need to be migrated to the READ_CACHE branch?
Bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Monday, July 15, 2013 7:27 PM
To: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684


Patch is now attached to the trace item at
https://sourceforge.net/apps/trac/bigdata/attachment/ticket/684/trac684B.patch
 $ patch -p1 -i /tmp/trac684B.patch


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 15, 2013, at 9:50 AM, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>> wrote:

This is now done.
Sorry it took me a while to get my infrastructure in place.

The tests are in

bigdata-rdf/src/test/com/bigdata/rdf/sparql/ast/optimizers/TestASTStaticJoinOptimizer.java

and are named

xtest_union_trac684_A
test_union_trac684_B
xtest_union_trac684_C


The first and last currently fail, and I renamed them as not to suggest that things had got worse.
To run the tests you will need to rename them back again

(I am used to just adding an @Ignore annotation … for this usage - what is the preference in this project?)

I am now working on producing a new patch file of my solution to these failing tests. I am sorry we had miscommunicated, since I was waiting for feedback from you on a previous patch …. :(




Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 12, 2013, at 7:46 AM, Mike Personick <mi...@sy...<mailto:mi...@sy...>> wrote:

Can you check in a test case for
it so that I can run it?

Re: [Bigdata-developers] interaction between #699 and #684

[Mike P. <mi...@sy...>]

I have the changes applied to my local version but not committed as I was working on some other property path issues last week.  Not ready to commit right now.


From: Bryan Thompson <br...@sy...<mailto:br...@sy...>>
Date: Wednesday, July 24, 2013 2:56 AM
To: Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>>, Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684

Morning all. What is the status on this patch and the related issues around UNION optimizations as they pertain to property paths?  Also, are the changes applied in the 1.2.x maintenance branch and do they need to be migrated to the READ_CACHE branch?
Bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Monday, July 15, 2013 7:27 PM
To: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684


Patch is now attached to the trace item at
https://sourceforge.net/apps/trac/bigdata/attachment/ticket/684/trac684B.patch
 $ patch -p1 -i /tmp/trac684B.patch


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 15, 2013, at 9:50 AM, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>> wrote:

This is now done.
Sorry it took me a while to get my infrastructure in place.

The tests are in

bigdata-rdf/src/test/com/bigdata/rdf/sparql/ast/optimizers/TestASTStaticJoinOptimizer.java

and are named

xtest_union_trac684_A
test_union_trac684_B
xtest_union_trac684_C


The first and last currently fail, and I renamed them as not to suggest that things had got worse.
To run the tests you will need to rename them back again

(I am used to just adding an @Ignore annotation … for this usage - what is the preference in this project?)

I am now working on producing a new patch file of my solution to these failing tests. I am sorry we had miscommunicated, since I was waiting for feedback from you on a previous patch …. :(




Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 12, 2013, at 7:46 AM, Mike Personick <mi...@sy...<mailto:mi...@sy...>> wrote:

Can you check in a test case for
it so that I can run it?

Re: [Bigdata-developers] interaction between #699 and #684

[Bryan T. <br...@sy...>]

Ok. Is your working version the 1.2.x maintenance branch or the READ_CACHE branch (HA)?

B

From: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Date: Wednesday, July 24, 2013 1:47 PM
To: Bryan Thompson <br...@sy...<mailto:br...@sy...>>, Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684

I have the changes applied to my local version but not committed as I was working on some other property path issues last week.  Not ready to commit right now.


From: Bryan Thompson <br...@sy...<mailto:br...@sy...>>
Date: Wednesday, July 24, 2013 2:56 AM
To: Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>>, Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684

Morning all. What is the status on this patch and the related issues around UNION optimizations as they pertain to property paths?  Also, are the changes applied in the 1.2.x maintenance branch and do they need to be migrated to the READ_CACHE branch?
Bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Monday, July 15, 2013 7:27 PM
To: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684


Patch is now attached to the trace item at
https://sourceforge.net/apps/trac/bigdata/attachment/ticket/684/trac684B.patch
 $ patch -p1 -i /tmp/trac684B.patch


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 15, 2013, at 9:50 AM, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>> wrote:

This is now done.
Sorry it took me a while to get my infrastructure in place.

The tests are in

bigdata-rdf/src/test/com/bigdata/rdf/sparql/ast/optimizers/TestASTStaticJoinOptimizer.java

and are named

xtest_union_trac684_A
test_union_trac684_B
xtest_union_trac684_C


The first and last currently fail, and I renamed them as not to suggest that things had got worse.
To run the tests you will need to rename them back again

(I am used to just adding an @Ignore annotation … for this usage - what is the preference in this project?)

I am now working on producing a new patch file of my solution to these failing tests. I am sorry we had miscommunicated, since I was waiting for feedback from you on a previous patch …. :(




Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 12, 2013, at 7:46 AM, Mike Personick <mi...@sy...<mailto:mi...@sy...>> wrote:

Can you check in a test case for
it so that I can run it?

Re: [Bigdata-developers] interaction between #699 and #684

[Mike P. <mi...@sy...>]

1.2 maintenance branch.  I am pretty sure there are no commits in there that need to be migrated.


From: Bryan Thompson <br...@sy...<mailto:br...@sy...>>
Date: Wednesday, July 24, 2013 7:50 AM
To: Mike Personick <mi...@sy...<mailto:mi...@sy...>>, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684

Ok. Is your working version the 1.2.x maintenance branch or the READ_CACHE branch (HA)?

B

From: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Date: Wednesday, July 24, 2013 1:47 PM
To: Bryan Thompson <br...@sy...<mailto:br...@sy...>>, Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684

I have the changes applied to my local version but not committed as I was working on some other property path issues last week.  Not ready to commit right now.


From: Bryan Thompson <br...@sy...<mailto:br...@sy...>>
Date: Wednesday, July 24, 2013 2:56 AM
To: Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>>, Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684

Morning all. What is the status on this patch and the related issues around UNION optimizations as they pertain to property paths?  Also, are the changes applied in the 1.2.x maintenance branch and do they need to be migrated to the READ_CACHE branch?
Bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Monday, July 15, 2013 7:27 PM
To: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] interaction between #699 and #684


Patch is now attached to the trace item at
https://sourceforge.net/apps/trac/bigdata/attachment/ticket/684/trac684B.patch
 $ patch -p1 -i /tmp/trac684B.patch


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 15, 2013, at 9:50 AM, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>> wrote:

This is now done.
Sorry it took me a while to get my infrastructure in place.

The tests are in

bigdata-rdf/src/test/com/bigdata/rdf/sparql/ast/optimizers/TestASTStaticJoinOptimizer.java

and are named

xtest_union_trac684_A
test_union_trac684_B
xtest_union_trac684_C


The first and last currently fail, and I renamed them as not to suggest that things had got worse.
To run the tests you will need to rename them back again

(I am used to just adding an @Ignore annotation … for this usage - what is the preference in this project?)

I am now working on producing a new patch file of my solution to these failing tests. I am sorry we had miscommunicated, since I was waiting for feedback from you on a previous patch …. :(




Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Jul 12, 2013, at 7:46 AM, Mike Personick <mi...@sy...<mailto:mi...@sy...>> wrote:

Can you check in a test case for
it so that I can run it?