bigdata-developers — List for bigdata developers

Re: [Bigdata-developers] trac748 fix ...

[Bryan T. <br...@sy...>]

Jeremy,

Just looked at the code.  Seems like a no brainer.  Go ahead.

Bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Thursday, October 3, 2013 11:20 AM
To: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] trac748 fix ...

I have committed the test cases and the fix to the ProjectedConstraint, I will make the IMO better fix of deleting ProjectConstraint as below if you would like me to.
r7432

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 6:33 PM, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>> wrote:


748 concerned subselects.
https://sourceforge.net/apps/trac/bigdata/ticket/748

The fix is minor fixes to ProjectedConstraint which crucially ignored the potentially null return from ConditionalBind .get(). It should be exactly the same as BindingConstraint.

I have tests and code ready to commit - however, in my view a better fix is to delete the class ProjectedConstraint and change its one usage at:
lin 3148 of AST2BopUtility

i.e. from
        IConstraint c = projection ? new ProjectedConstraint(b)
                : new BindingConstraint(b);

to

        IConstraint c = new BindingConstraint(b);


Please advise.

Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] trac748 fix ...

[Jeremy J C. <jj...@sy...>]

I have committed the test cases and the fix to the ProjectedConstraint, I will make the IMO better fix of deleting ProjectConstraint as below if you would like me to.
r7432

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 6:33 PM, Jeremy J Carroll <jj...@sy...> wrote:

> 
> 748 concerned subselects.
> https://sourceforge.net/apps/trac/bigdata/ticket/748
> 
> The fix is minor fixes to ProjectedConstraint which crucially ignored the potentially null return from ConditionalBind .get(). It should be exactly the same as BindingConstraint.
> 
> I have tests and code ready to commit - however, in my view a better fix is to delete the class ProjectedConstraint and change its one usage at:
> lin 3148 of AST2BopUtility
> 
> i.e. from
>         IConstraint c = projection ? new ProjectedConstraint(b)
>                 : new BindingConstraint(b);
> 
> to
> 
>         IConstraint c = new BindingConstraint(b);
> 
> 
> Please advise.
> 
> Jeremy J Carroll
> Principal Architect
> Syapse, Inc.
> 
> 
> 

[Bigdata-developers] trac748 fix ...

[Jeremy J C. <jj...@sy...>]

748 concerned subselects.
https://sourceforge.net/apps/trac/bigdata/ticket/748

The fix is minor fixes to ProjectedConstraint which crucially ignored the potentially null return from ConditionalBind .get(). It should be exactly the same as BindingConstraint.

I have tests and code ready to commit - however, in my view a better fix is to delete the class ProjectedConstraint and change its one usage at:
lin 3148 of AST2BopUtility

i.e. from
        IConstraint c = projection ? new ProjectedConstraint(b)
                : new BindingConstraint(b);

to

        IConstraint c = new BindingConstraint(b);


Please advise.

Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] defensive code?

[Jeremy J C. <jj...@sy...>]

I modified this to use asserts and committed as r7431 please let me know if this was inappropriate.


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 9:59 AM, Jeremy J Carroll <jj...@sy...> wrote:

> 
> Hmm
> 
> I wonder if github only shows you the code if you log in …
> 
> 
> It is, e.g., this in AssignmentNode, which overrides several methods and calls super to verify that the first arg is always a VarNode, which is the class invariant that failed.
> Sounds like maybe I should I rework to turn checkFirstArg unto an assertion and then inline checkFirstArg
> 
> 
> +  
> +
> +
> +  private void checkFirstArg(final BOp newVal) {
> +    if (!(newVal instanceof VarNode)) {
> +        throw new ClassCastException("First op in AssignmentNode cannot be "+newVal.getClass().getName());
> +    }
> +  }
> +
> +    @Override
> +    public ModifiableBOpBase setArgs(final BOp[] args) {
> +      checkFirstArg(args[0]);
> +      return super.setArgs(args);
> +    }
> +    
> +
> +    @Override
> +    public ModifiableBOpBase setArg(final int index, final BOp newArg) {
> +      if (index == 0) {
> +        checkFirstArg(newArg);
> +      }
> +      return super.setArg(index, newArg);
> +    }
> 
> Jeremy J Carroll
> Principal Architect
> Syapse, Inc.
> 
> 
> 
> On Oct 2, 2013, at 9:17 AM, Bryan Thompson <br...@sy...> wrote:
> 
>> Jeremy,
>> 
>> The link to the uncommitted code is no longer valid.  I am not quite sure
>> what you meant by defensive code in this instance.  As a general policy,
>> we do use asserts to catch things that should not be violated.  More
>> defensive code is used when the situation warrants it (more risk
>> associated with incorrect code or the code is particularly tricky or
>> problem prone). 
>> 
>> Bryan
>> 
>> On 10/1/13 6:11 PM, "Jeremy J Carroll" <jj...@sy...> wrote:
>> 
>>> 
>>> Hi 
>>> I have just committed straight forward fixes to:
>>> https://sourceforge.net/apps/trac/bigdata/ticket/737
>>> 
>>> (I do not believe review is needed, but review would never be unwelcome)
>>> 
>>> I did not commit the code that is found here:
>>> 
>>> https://github.com/jeremycarroll/bigdata/commit/c9dfa430d68c03c4246ebd5025
>>> 415d58cb1ce230
>>> 
>>> which is how I worked on the failing tests in
>>> com.bigdata.rdf.sail.TestNoExceptions
>>> which exercised the ClassCastExceptions in the defect report.
>>> (I have forgotten to include that test class in the test suite and will
>>> do so now)
>>> 
>>> These allowed me to find the root cause of the incorrect typing, and
>>> hence to fix it.
>>> 
>>> My question is:
>>> - is the preference to *not* include such defensive code in the main-line
>>> source,
>>> or
>>> - should I also commit this defensive coding
>>> 
>>> I have no particularly beef either way: I believe it to be a stylistic
>>> preference.
>>> 
>>> Jeremy J Carroll
>>> Principal Architect
>>> Syapse, Inc.
>>> 
>>> 
>>> 
>>> 
>>> --------------------------------------------------------------------------
>>> ----
>>> October Webinars: Code for Performance
>>> Free Intel webinars can help you accelerate application performance.
>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>>> from 
>>> the latest Intel processors and coprocessors. See abstracts and register >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktr
>>> k
>>> _______________________________________________
>>> Bigdata-developers mailing list
>>> Big...@li...
>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>> 
> 

Re: [Bigdata-developers] defensive code?

[Bryan T. <br...@sy...>]

We generally run with asserts enabled for development and in CI.  But this sort of thing could go either way.  Lightweight.  Preference is based on how critical it is to catch the problem at that point.
bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Wednesday, October 2, 2013 1:17 PM
To: Bryan Thompson <br...@sy...<mailto:br...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] defensive code?

I modified this to use asserts and committed as r7431 please let me know if this was inappropriate.


Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 9:59 AM, Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>> wrote:


Hmm

I wonder if github only shows you the code if you log in …


It is, e.g., this in AssignmentNode, which overrides several methods and calls super to verify that the first arg is always a VarNode, which is the class invariant that failed.
Sounds like maybe I should I rework to turn checkFirstArg unto an assertion and then inline checkFirstArg


+
                +
                +
                +  private void checkFirstArg(final BOp newVal) {
                +    if (!(newVal instanceof VarNode)) {
                +        throw new ClassCastException("First op in AssignmentNode cannot be "+newVal.getClass().getName());
                +    }
                +  }
                +
                +    @Override
                +    public ModifiableBOpBase setArgs(final BOp[] args) {
                +      checkFirstArg(args[0]);
                +      return super.setArgs(args);
                +    }
                +
                +
                +    @Override
                +    public ModifiableBOpBase setArg(final int index, final BOp newArg) {
                +      if (index == 0) {
                +        checkFirstArg(newArg);
                +      }
                +      return super.setArg(index, newArg);
                +    }

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 9:17 AM, Bryan Thompson <br...@sy...<mailto:br...@sy...>> wrote:

Jeremy,

The link to the uncommitted code is no longer valid.  I am not quite sure
what you meant by defensive code in this instance.  As a general policy,
we do use asserts to catch things that should not be violated.  More
defensive code is used when the situation warrants it (more risk
associated with incorrect code or the code is particularly tricky or
problem prone).

Bryan

On 10/1/13 6:11 PM, "Jeremy J Carroll" <jj...@sy...<mailto:jj...@sy...>> wrote:


Hi
I have just committed straight forward fixes to:
https://sourceforge.net/apps/trac/bigdata/ticket/737

(I do not believe review is needed, but review would never be unwelcome)

I did not commit the code that is found here:

https://github.com/jeremycarroll/bigdata/commit/c9dfa430d68c03c4246ebd5025
415d58cb1ce230

which is how I worked on the failing tests in
com.bigdata.rdf.sail.TestNoExceptions
which exercised the ClassCastExceptions in the defect report.
(I have forgotten to include that test class in the test suite and will
do so now)

These allowed me to find the root cause of the incorrect typing, and
hence to fix it.

My question is:
- is the preference to *not* include such defensive code in the main-line
source,
or
- should I also commit this defensive coding

I have no particularly beef either way: I believe it to be a stylistic
preference.

Jeremy J Carroll
Principal Architect
Syapse, Inc.




--------------------------------------------------------------------------
----
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktr
k
_______________________________________________
Bigdata-developers mailing list
Big...@li...<mailto:Big...@li...>
https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] defensive code?

[Bryan T. <br...@sy...>]

That code as you have it below is fine.  This sort of thing is extremely light weight.
Bryan

From: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Wednesday, October 2, 2013 12:59 PM
To: Bryan Thompson <br...@sy...<mailto:br...@sy...>>
Cc: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] defensive code?


Hmm

I wonder if github only shows you the code if you log in …


It is, e.g., this in AssignmentNode, which overrides several methods and calls super to verify that the first arg is always a VarNode, which is the class invariant that failed.
Sounds like maybe I should I rework to turn checkFirstArg unto an assertion and then inline checkFirstArg


+
                +
                +
                +  private void checkFirstArg(final BOp newVal) {
                +    if (!(newVal instanceof VarNode)) {
                +        throw new ClassCastException("First op in AssignmentNode cannot be "+newVal.getClass().getName());
                +    }
                +  }
                +
                +    @Override
                +    public ModifiableBOpBase setArgs(final BOp[] args) {
                +      checkFirstArg(args[0]);
                +      return super.setArgs(args);
                +    }
                +
                +
                +    @Override
                +    public ModifiableBOpBase setArg(final int index, final BOp newArg) {
                +      if (index == 0) {
                +        checkFirstArg(newArg);
                +      }
                +      return super.setArg(index, newArg);
                +    }

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 9:17 AM, Bryan Thompson <br...@sy...<mailto:br...@sy...>> wrote:

Jeremy,

The link to the uncommitted code is no longer valid.  I am not quite sure
what you meant by defensive code in this instance.  As a general policy,
we do use asserts to catch things that should not be violated.  More
defensive code is used when the situation warrants it (more risk
associated with incorrect code or the code is particularly tricky or
problem prone).

Bryan

On 10/1/13 6:11 PM, "Jeremy J Carroll" <jj...@sy...<mailto:jj...@sy...>> wrote:


Hi
I have just committed straight forward fixes to:
https://sourceforge.net/apps/trac/bigdata/ticket/737

(I do not believe review is needed, but review would never be unwelcome)

I did not commit the code that is found here:

https://github.com/jeremycarroll/bigdata/commit/c9dfa430d68c03c4246ebd5025
415d58cb1ce230

which is how I worked on the failing tests in
com.bigdata.rdf.sail.TestNoExceptions
which exercised the ClassCastExceptions in the defect report.
(I have forgotten to include that test class in the test suite and will
do so now)

These allowed me to find the root cause of the incorrect typing, and
hence to fix it.

My question is:
- is the preference to *not* include such defensive code in the main-line
source,
or
- should I also commit this defensive coding

I have no particularly beef either way: I believe it to be a stylistic
preference.

Jeremy J Carroll
Principal Architect
Syapse, Inc.




--------------------------------------------------------------------------
----
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktr
k
_______________________________________________
Bigdata-developers mailing list
Big...@li...<mailto:Big...@li...>
https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] defensive code?

[Jeremy J C. <jj...@sy...>]

Hmm

I wonder if github only shows you the code if you log in …


It is, e.g., this in AssignmentNode, which overrides several methods and calls super to verify that the first arg is always a VarNode, which is the class invariant that failed.
Sounds like maybe I should I rework to turn checkFirstArg unto an assertion and then inline checkFirstArg


+  
+
+
+  private void checkFirstArg(final BOp newVal) {
+    if (!(newVal instanceof VarNode)) {
+        throw new ClassCastException("First op in AssignmentNode cannot be "+newVal.getClass().getName());
+    }
+  }
+
+    @Override
+    public ModifiableBOpBase setArgs(final BOp[] args) {
+      checkFirstArg(args[0]);
+      return super.setArgs(args);
+    }
+    
+
+    @Override
+    public ModifiableBOpBase setArg(final int index, final BOp newArg) {
+      if (index == 0) {
+        checkFirstArg(newArg);
+      }
+      return super.setArg(index, newArg);
+    }

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 9:17 AM, Bryan Thompson <br...@sy...> wrote:

> Jeremy,
> 
> The link to the uncommitted code is no longer valid.  I am not quite sure
> what you meant by defensive code in this instance.  As a general policy,
> we do use asserts to catch things that should not be violated.  More
> defensive code is used when the situation warrants it (more risk
> associated with incorrect code or the code is particularly tricky or
> problem prone). 
> 
> Bryan
> 
> On 10/1/13 6:11 PM, "Jeremy J Carroll" <jj...@sy...> wrote:
> 
>> 
>> Hi 
>> I have just committed straight forward fixes to:
>> https://sourceforge.net/apps/trac/bigdata/ticket/737
>> 
>> (I do not believe review is needed, but review would never be unwelcome)
>> 
>> I did not commit the code that is found here:
>> 
>> https://github.com/jeremycarroll/bigdata/commit/c9dfa430d68c03c4246ebd5025
>> 415d58cb1ce230
>> 
>> which is how I worked on the failing tests in
>> com.bigdata.rdf.sail.TestNoExceptions
>> which exercised the ClassCastExceptions in the defect report.
>> (I have forgotten to include that test class in the test suite and will
>> do so now)
>> 
>> These allowed me to find the root cause of the incorrect typing, and
>> hence to fix it.
>> 
>> My question is:
>> - is the preference to *not* include such defensive code in the main-line
>> source,
>> or
>> - should I also commit this defensive coding
>> 
>> I have no particularly beef either way: I believe it to be a stylistic
>> preference.
>> 
>> Jeremy J Carroll
>> Principal Architect
>> Syapse, Inc.
>> 
>> 
>> 
>> 
>> --------------------------------------------------------------------------
>> ----
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from 
>> the latest Intel processors and coprocessors. See abstracts and register >
>> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktr
>> k
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> 

Re: [Bigdata-developers] defensive code?

[Bryan T. <br...@sy...>]

Jeremy,

The link to the uncommitted code is no longer valid.  I am not quite sure
what you meant by defensive code in this instance.  As a general policy,
we do use asserts to catch things that should not be violated.  More
defensive code is used when the situation warrants it (more risk
associated with incorrect code or the code is particularly tricky or
problem prone). 

Bryan

On 10/1/13 6:11 PM, "Jeremy J Carroll" <jj...@sy...> wrote:

>
>Hi 
>I have just committed straight forward fixes to:
>https://sourceforge.net/apps/trac/bigdata/ticket/737
>
>(I do not believe review is needed, but review would never be unwelcome)
>
>I did not commit the code that is found here:
>
>https://github.com/jeremycarroll/bigdata/commit/c9dfa430d68c03c4246ebd5025
>415d58cb1ce230
>
>which is how I worked on the failing tests in
>com.bigdata.rdf.sail.TestNoExceptions
>which exercised the ClassCastExceptions in the defect report.
>(I have forgotten to include that test class in the test suite and will
>do so now)
>
>These allowed me to find the root cause of the incorrect typing, and
>hence to fix it.
>
>My question is:
>- is the preference to *not* include such defensive code in the main-line
>source,
>or
>- should I also commit this defensive coding
>
>I have no particularly beef either way: I believe it to be a stylistic
>preference.
>
>Jeremy J Carroll
>Principal Architect
>Syapse, Inc.
>
>
>
>
>--------------------------------------------------------------------------
>----
>October Webinars: Code for Performance
>Free Intel webinars can help you accelerate application performance.
>Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>from 
>the latest Intel processors and coprocessors. See abstracts and register >
>http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktr
>k
>_______________________________________________
>Bigdata-developers mailing list
>Big...@li...
>https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] use of -DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads and QuadsTestCase

[Bryan T. <br...@sy...>]

Yes, seems reasonable.

The pattern exists because we support various persistence stores (RWStore, WORMStore, MemStore, etc) and various RDF DB semantics (triples, triples with provenance, quads). So, the pattern is necessary, but for the QuadsTestCase we always specify a specific value, so it could be defaulted to that constant.

I would just verify that the derived classes all run in "quads mode". Historically, the QuadsTestCase was put together by Mike to help people write unit tests at the SAIL layer.  Today, we tend to prefer tests written using RDF data, SPARQL query or update, and the expected results all in files.  Maybe we will eventually use manifests.

But, to the point, I think that some of the QuadsTestCase derived tests might not actually be for quads.  Check the specific subclasses to be certain.

Bryan

From: Mike Personick <mi...@sy...<mailto:mi...@sy...>>
Date: Wednesday, October 2, 2013 8:27 AM
To: Jeremy Carroll <jj...@sy...<mailto:jj...@sy...>>, "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: Re: [Bigdata-developers] use of -DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads and QuadsTestCase

It seems reasonable to me.   Bryan?

From: Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Tuesday, October 1, 2013 4:19 PM
To: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: [Bigdata-developers] use of -DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads and QuadsTestCase


A question/suggestion, primarily for Mike.

In the trac737 tests, in r7424, I subclassed from QuadsTestCase, which says:
   Specify
 * <code>-DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads</code> to
 * run this test suite.



Because it is ever so slightly irritating to set the environment flag, I decided to hard code a default in my subclass with:


    public AbstractBigdataSailTestCase getOurDelegate() {

        if (getDelegate() == null) {

            String testClass = System.getProperty("testClass");
            if (testClass != null) {
            return super.getOurDelegate();

            }
            setDelegate(new com.bigdata.rdf.sail.TestBigdataSailWithQuads());
        }
        return (AbstractBigdataSailTestCase) super.getDelegate();
    }


I am wondering whether this method would be an improvement to the QuadsTestCase class, and should be pulled up?


Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] use of -DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads and QuadsTestCase

[Mike P. <mi...@sy...>]

It seems reasonable to me.   Bryan?

From: Jeremy J Carroll <jj...@sy...<mailto:jj...@sy...>>
Date: Tuesday, October 1, 2013 4:19 PM
To: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: [Bigdata-developers] use of -DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads and QuadsTestCase


A question/suggestion, primarily for Mike.

In the trac737 tests, in r7424, I subclassed from QuadsTestCase, which says:
   Specify
 * <code>-DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads</code> to
 * run this test suite.



Because it is ever so slightly irritating to set the environment flag, I decided to hard code a default in my subclass with:


    public AbstractBigdataSailTestCase getOurDelegate() {

        if (getDelegate() == null) {

            String testClass = System.getProperty("testClass");
            if (testClass != null) {
            return super.getOurDelegate();

            }
            setDelegate(new com.bigdata.rdf.sail.TestBigdataSailWithQuads());
        }
        return (AbstractBigdataSailTestCase) super.getDelegate();
    }


I am wondering whether this method would be an improvement to the QuadsTestCase class, and should be pulled up?


Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] update for today

[Jeremy J C. <jj...@sy...>]

Yes - I have not managed to finish that piece of work.
Should I have arranged so the tests were not active, particularly since I am now not expecting to get back to it this week.



Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Oct 2, 2013, at 2:56 AM, Bryan Thompson <br...@sy...> wrote:

> Jeremy, the 747 and 747b tests are failing in CI. Is that expected?
> 
> On Oct 1, 2013, at 9:59 PM, "Jeremy J Carroll" <jj...@sy...> wrote:
> 
>> 
>> I fixed 737 to do with class cast exceptions, and then feeling that I would be touching similar code worked on a rather uninteresting bug 747
>> 
>> SELECT *
>> WHERE {
>>   BIND ( "y" as $Y ) .
>>   {
>>     BIND ( "x" as $X )
>>     FILTER( BOUND($Y) 
>> # && True 
>> # uncommenting the above line works around the bug
>>     )
>>   }
>> }
>> 
>> I made some progress, checking in three tests and a low-risk fix for one of them, but then got bogged down in what appears to be logic errors in
>> com.bigdata.rdf.sparql.ast.optimizers.ASTBottomUpOptimizer.handleFiltersWithVariablesNotInScope(AST2BOpContext, StaticAnalysis, QueryBase, IBindingSet[])
>> 
>> Unfortunately a colleague hit a more important issue late today, which I have filed as 748, and I will work on tomorrow.
>> I have updated the 747 ticket with the progress to date, for I fear I may not get back to it this week.
>> 
>> On 747, I suspect that there is a further issue which is that the work-around really should not work!
>> 
>> 
>> Jeremy J Carroll
>> Principal Architect
>> Syapse, Inc.
>> 
>> 
>> 
>> <ATT00001.c>
>> <ATT00002.c>

Re: [Bigdata-developers] update for today

[Bryan T. <br...@sy...>]

Jeremy, the 747 and 747b tests are failing in CI. Is that expected?

On Oct 1, 2013, at 9:59 PM, "Jeremy J Carroll" <jj...@sy...<mailto:jj...@sy...>> wrote:


I fixed 737 to do with class cast exceptions, and then feeling that I would be touching similar code worked on a rather uninteresting bug 747

SELECT *
WHERE {
  BIND ( "y" as $Y ) .
  {
    BIND ( "x" as $X )
    FILTER( BOUND($Y)
# && True
# uncommenting the above line works around the bug
    )
  }
}

I made some progress, checking in three tests and a low-risk fix for one of them, but then got bogged down in what appears to be logic errors in
com.bigdata.rdf.sparql.ast.optimizers.ASTBottomUpOptimizer.handleFiltersWithVariablesNotInScope(AST2BOpContext, StaticAnalysis, QueryBase, IBindingSet[])

Unfortunately a colleague hit a more important issue late today, which I have filed as 748, and I will work on tomorrow.
I have updated the 747 ticket with the progress to date, for I fear I may not get back to it this week.

On 747, I suspect that there is a further issue which is that the work-around really should not work!


Jeremy J Carroll
Principal Architect
Syapse, Inc.



<ATT00001.c>
<ATT00002.c>

[Bigdata-developers] update for today

[Jeremy J C. <jj...@sy...>]

I fixed 737 to do with class cast exceptions, and then feeling that I would be touching similar code worked on a rather uninteresting bug 747

SELECT *
WHERE {
  BIND ( "y" as $Y ) .
  {
    BIND ( "x" as $X )
    FILTER( BOUND($Y) 
# && True 
# uncommenting the above line works around the bug
    )
  }
}

I made some progress, checking in three tests and a low-risk fix for one of them, but then got bogged down in what appears to be logic errors in
com.bigdata.rdf.sparql.ast.optimizers.ASTBottomUpOptimizer.handleFiltersWithVariablesNotInScope(AST2BOpContext, StaticAnalysis, QueryBase, IBindingSet[])

Unfortunately a colleague hit a more important issue late today, which I have filed as 748, and I will work on tomorrow.
I have updated the 747 ticket with the progress to date, for I fear I may not get back to it this week.

On 747, I suspect that there is a further issue which is that the work-around really should not work!


Jeremy J Carroll
Principal Architect
Syapse, Inc.

[Bigdata-developers] use of -DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads and QuadsTestCase

[Jeremy J C. <jj...@sy...>]

A question/suggestion, primarily for Mike.

In the trac737 tests, in r7424, I subclassed from QuadsTestCase, which says:
   Specify
 * <code>-DtestClass=com.bigdata.rdf.sail.TestBigdataSailWithQuads</code> to
 * run this test suite.



Because it is ever so slightly irritating to set the environment flag, I decided to hard code a default in my subclass with:


    public AbstractBigdataSailTestCase getOurDelegate() {

        if (getDelegate() == null) {

            String testClass = System.getProperty("testClass");
            if (testClass != null) {
            	return super.getOurDelegate();

            }
            setDelegate(new com.bigdata.rdf.sail.TestBigdataSailWithQuads());
        }
        return (AbstractBigdataSailTestCase) super.getDelegate();
    }


I am wondering whether this method would be an improvement to the QuadsTestCase class, and should be pulled up?


Jeremy J Carroll
Principal Architect
Syapse, Inc.

[Bigdata-developers] defensive code?

[Jeremy J C. <jj...@sy...>]

Hi 
I have just committed straight forward fixes to:
https://sourceforge.net/apps/trac/bigdata/ticket/737

(I do not believe review is needed, but review would never be unwelcome)

I did not commit the code that is found here:

https://github.com/jeremycarroll/bigdata/commit/c9dfa430d68c03c4246ebd5025415d58cb1ce230

which is how I worked on the failing tests in
com.bigdata.rdf.sail.TestNoExceptions
which exercised the ClassCastExceptions in the defect report.
(I have forgotten to include that test class in the test suite and will do so now)

These allowed me to find the root cause of the incorrect typing, and hence to fix it.

My question is:
- is the preference to *not* include such defensive code in the main-line source,
or
- should I also commit this defensive coding

I have no particularly beef either way: I believe it to be a stylistic preference.

Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] named subquery include does not implement IBindingProducerNode

[Bryan T. <br...@sy...>]

That sounds reasonable.

From: Jeremy Carroll <jj...@gm...<mailto:jj...@gm...>>
Date: Tuesday, October 1, 2013 3:01 PM
To: "Big...@li...<mailto:Big...@li...>" <Big...@li...<mailto:Big...@li...>>
Subject: [Bigdata-developers] named subquery include does not implement IBindingProducerNode


In com.bigdata.rdf.sparql.ast.StaticAnalysis.getDefinitelyProducedBindings(IBindingProducerNode, Set<IVariable<?>>, boolean)



around line 862 I read,
       if (node instanceof NamedSubqueryInclude) {

            final NamedSubqueryInclude nsi = (NamedSubqueryInclude) node;

            final String name = nsi.getName();




etc.

however NamedSubqueryInclude does not implement IBindingProducerNode
so this code cannot execute.

I am thinking that probably NSI should implement IBPN

===

I am working on defect 737, and this does not address my bug, so I am not making any changes.

But having seen this, I thought I would suggest the change …

Jeremy

[Bigdata-developers] named subquery include does not implement IBindingProducerNode

[Jeremy C. <jj...@gm...>]

In com.bigdata.rdf.sparql.ast.StaticAnalysis.getDefinitelyProducedBindings(IBindingProducerNode, Set<IVariable<?>>, boolean)



around line 862 I read, 
       if (node instanceof NamedSubqueryInclude) {

            final NamedSubqueryInclude nsi = (NamedSubqueryInclude) node;

            final String name = nsi.getName();
            

etc.

however NamedSubqueryInclude does not implement IBindingProducerNode
so this code cannot execute.

I am thinking that probably NSI should implement IBPN

===

I am working on defect 737, and this does not address my bug, so I am not making any changes.

But having seen this, I thought I would suggest the change …

Jeremy

[Bigdata-developers] FYGI 'accept'ed status

[Jeremy J C. <jj...@sy...>]

I just noticed that the workflow is meant to include the developers accepting trac items.
I have just accepted five as a result - if I haven't accepted some other it may just be that I haven't realized that it is in my queue. Please let me know.

More than this, I am actually starting to work on this queue … right now!!

Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] trac746 - ArbitraryLengthPathNode missing case in ASTComplexOptionalOptimizer

[Mike P. <mi...@sy...>]

The fix looks right to me.  An ALPNode is definitely a required join.
 


On 9/26/13 6:09 AM, "Jeremy J Carroll" <jj...@sy...> wrote:

>My reasoning was no more than it (naively) seemed quite like a Union to
>me, and the assertion failure was there to catch that the case had not
>been thought about.
>
>After the fix, the query does give the correct result on my dev system
>(and a load of data mentioned in an earlier bug report) - but the test
>case does not show that.
>
>Jeremy J Carroll
>Principal Architect
>Syapse, Inc.
>
>
>
>On Sep 25, 2013, at 6:15 PM, Bryan Thompson <br...@sy...> wrote:
>
>> Mike, can you comment on this? The issue appears to be the handling
>>logic for complex optionals.  That code predates the introduction of
>>property paths.  The question is whether a property path needs to be
>>treated as a required join and moved into the named sub-query. This
>>sounds reasonable in principle to me, but I am not that close to the
>>property path implementation. Bryan
>> 
>>> On Sep 25, 2013, at 8:58 PM, "Jeremy J Carroll" <jj...@sy...> wrote:
>>> 
>>> 
>>> I hit a show stopper this week in my work, and I think I have fixed it
>>>- a one liner:
>>> 
>>> I currently have the test case and the fix in my copy of the repo, and
>>>thought I should get a go ahead before committing to SVN
>>> 
>>> The defect report is:
>>> https://sourceforge.net/apps/trac/bigdata/ticket/746
>>> 
>>> The test case (which is somewhat deficient - it exercised the fault
>>>but did not suggest correct behavior other than not to die)
>>> 
>>> 
>>>https://github.com/jeremycarroll/bigdata/commit/7fa5ae51acc5f49b085f0dd3
>>>cc6654f87802b2c7
>>> 
>>> The fix:
>>> 
>>>https://github.com/jeremycarroll/bigdata/commit/c4702b76f670d7bc327ed6eb
>>>49dec139259cbc72
>>> 
>>> If someone can look at that one-line fix and OK it, I will commit to
>>>SVN
>>> 
>>> 
>>> Jeremy J Carroll
>>> Principal Architect
>>> Syapse, Inc.
>>> 
>>> 
>>> 
>>> 
>>> 
>>>------------------------------------------------------------------------
>>>------
>>> October Webinars: Code for Performance
>>> Free Intel webinars can help you accelerate application performance.
>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
>>>most from 
>>> the latest Intel processors and coprocessors. See abstracts and
>>>register >
>>> 
>>>http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clk
>>>trk
>>> _______________________________________________
>>> Bigdata-developers mailing list
>>> Big...@li...
>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>> 
>> 
>>-------------------------------------------------------------------------
>>-----
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
>>most from 
>> the latest Intel processors and coprocessors. See abstracts and
>>register >
>> 
>>http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clkt
>>rk
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>

Re: [Bigdata-developers] trac746 - ArbitraryLengthPathNode missing case in ASTComplexOptionalOptimizer

[Jeremy J C. <jj...@sy...>]

My reasoning was no more than it (naively) seemed quite like a Union to me, and the assertion failure was there to catch that the case had not been thought about.

After the fix, the query does give the correct result on my dev system (and a load of data mentioned in an earlier bug report) - but the test case does not show that.

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Sep 25, 2013, at 6:15 PM, Bryan Thompson <br...@sy...> wrote:

> Mike, can you comment on this? The issue appears to be the handling logic for complex optionals.  That code predates the introduction of property paths.  The question is whether a property path needs to be treated as a required join and moved into the named sub-query. This sounds reasonable in principle to me, but I am not that close to the property path implementation. Bryan 
> 
>> On Sep 25, 2013, at 8:58 PM, "Jeremy J Carroll" <jj...@sy...> wrote:
>> 
>> 
>> I hit a show stopper this week in my work, and I think I have fixed it - a one liner:
>> 
>> I currently have the test case and the fix in my copy of the repo, and thought I should get a go ahead before committing to SVN
>> 
>> The defect report is:
>> https://sourceforge.net/apps/trac/bigdata/ticket/746
>> 
>> The test case (which is somewhat deficient - it exercised the fault but did not suggest correct behavior other than not to die)
>> 
>> https://github.com/jeremycarroll/bigdata/commit/7fa5ae51acc5f49b085f0dd3cc6654f87802b2c7
>> 
>> The fix:
>> https://github.com/jeremycarroll/bigdata/commit/c4702b76f670d7bc327ed6eb49dec139259cbc72
>> 
>> If someone can look at that one-line fix and OK it, I will commit to SVN
>> 
>> 
>> Jeremy J Carroll
>> Principal Architect
>> Syapse, Inc.
>> 
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
>> the latest Intel processors and coprocessors. See abstracts and register >
>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> 
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] trac746 - ArbitraryLengthPathNode missing case in ASTComplexOptionalOptimizer

[Bryan T. <br...@sy...>]

Mike, can you comment on this? The issue appears to be the handling logic for complex optionals.  That code predates the introduction of property paths.  The question is whether a property path needs to be treated as a required join and moved into the named sub-query. This sounds reasonable in principle to me, but I am not that close to the property path implementation. Bryan 

> On Sep 25, 2013, at 8:58 PM, "Jeremy J Carroll" <jj...@sy...> wrote:
> 
> 
> I hit a show stopper this week in my work, and I think I have fixed it - a one liner:
> 
> I currently have the test case and the fix in my copy of the repo, and thought I should get a go ahead before committing to SVN
> 
> The defect report is:
> https://sourceforge.net/apps/trac/bigdata/ticket/746
> 
> The test case (which is somewhat deficient - it exercised the fault but did not suggest correct behavior other than not to die)
> 
> https://github.com/jeremycarroll/bigdata/commit/7fa5ae51acc5f49b085f0dd3cc6654f87802b2c7
> 
> The fix:
> https://github.com/jeremycarroll/bigdata/commit/c4702b76f670d7bc327ed6eb49dec139259cbc72
> 
> If someone can look at that one-line fix and OK it, I will commit to SVN
> 
> 
> Jeremy J Carroll
> Principal Architect
> Syapse, Inc.
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers

[Bigdata-developers] trac746 - ArbitraryLengthPathNode missing case in ASTComplexOptionalOptimizer

[Jeremy J C. <jj...@sy...>]

I hit a show stopper this week in my work, and I think I have fixed it - a one liner:

I currently have the test case and the fix in my copy of the repo, and thought I should get a go ahead before committing to SVN

The defect report is:
https://sourceforge.net/apps/trac/bigdata/ticket/746

The test case (which is somewhat deficient - it exercised the fault but did not suggest correct behavior other than not to die)

https://github.com/jeremycarroll/bigdata/commit/7fa5ae51acc5f49b085f0dd3cc6654f87802b2c7

The fix:
https://github.com/jeremycarroll/bigdata/commit/c4702b76f670d7bc327ed6eb49dec139259cbc72

If someone can look at that one-line fix and OK it, I will commit to SVN


Jeremy J Carroll
Principal Architect
Syapse, Inc.

[Bigdata-developers] FYGI: not working on bigdata this week

[Jeremy J C. <jj...@sy...>]

It looks unlikely that I will clear any time this week for bigdata work. I am hopeful that I will manage to put in two or three days next week.

Jeremy J Carroll
Principal Architect
Syapse, Inc.

Re: [Bigdata-developers] Bug with a query that contains BIND and UNION

[Antoni M. <ant...@ba...>]

Hi Bryan,

I did the same test with a bigdata.war built from rev 7414. It works! Yippie!

We're using rev 6933 now. We intended to update bigdata when it switches to Sesame 2.7, with a full-featured SPARQLRepository. Now it seems we may have to update earlier. 

Anyway, It's great this has already been fixed. Thanks very much.

--
Antoni Myłka
Software Engineer

basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22
http://www.basis06.ch - source of smart business 

----- Ursprüngliche Mail -----
Von: "Bryan Thompson" <br...@sy...>
An: "Antoni Mylka" <ant...@ba...>, big...@li...
Gesendet: Donnerstag, 19. September 2013 17:50:55
Betreff: Re: [Bigdata-developers] Bug with a query that contains BIND and UNION

Antoni,

This is probably [1].  Fixed in r7375.  The current development branch is
branches/BIGDATA_RELEASE_1_3_0.  You can switch to that branch for this
fix.  Please let me know if that fixes the problem for you.

Thanks,
Bryan

[1] http://sourceforge.net/apps/trac/bigdata/ticket/708 (BIND heisenbug)




On 9/19/13 11:46 AM, "Antoni Mylka" <ant...@ba...> wrote:

>Hi,
>
>I think I found a bug. I downloaded the latest bigdata.war release from
>sourceforge, deployed in on the latest tomcat release, with out-of-the
>box configuration. Then I went to localhost:8080/bigdata and did this
>INSERT query:
>
>PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
>PREFIX umbel: <http://umbel.org/umbel/>
>INSERT DATA
>{ 
>  <http://example/book1> skos:narrower <http://example/chapter1> .
>  <http://example/book1> umbel:isRelatedToClass <http://example/book> .
>}
>
>And then I tried to do this SELECT:
>
>PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
>PREFIX umbel: <http://umbel.org/umbel/>
>SELECT ?p ?x  
>WHERE {
> {?concept skos:narrower ?x BIND (skos:narrower as ?p) . }
>  UNION
> {?concept umbel:isRelatedToClass ?x BIND (umbel:isRelatedToClass as ?p)
>. }
> FILTER (?concept in (<http://example/book1))
>}
>
>I would expect this:
>
>?p                     ?x
>skos:narrower          <http://example/chapter1>
>umbel:isRelatedToClass <http://example/book>
>
>But when I do this query many times I always get either the first or the
>second, but never both. This looks to me like a race condition somewhere
>in the code that handles BIND or UNION.
>
>Two questions:
>1. Is this a bug? What should be the behavior?
>2. I can rephrase this query to say {?concept ?p ?x FILTER(?p in
>(skos:narrower, umbel:isRelatedToClass)) .}. This works OK. Is the
>variant with BIND likely to perform better (when it works)?
>
>Could anyone confirm?
>
>
>--
>Antoni Myłka
>Software Engineer
>
>basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22
>http://www.basis06.ch - source of smart business
>
>--------------------------------------------------------------------------
>----
>LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
>1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
>SharePoint
>2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack
>includes
>Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
>http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktr
>k
>_______________________________________________
>Bigdata-developers mailing list
>Big...@li...
>https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] Bug with a query that contains BIND and UNION

[Bryan T. <br...@sy...>]

Antoni,

This is probably [1].  Fixed in r7375.  The current development branch is
branches/BIGDATA_RELEASE_1_3_0.  You can switch to that branch for this
fix.  Please let me know if that fixes the problem for you.

Thanks,
Bryan

[1] http://sourceforge.net/apps/trac/bigdata/ticket/708 (BIND heisenbug)




On 9/19/13 11:46 AM, "Antoni Mylka" <ant...@ba...> wrote:

>Hi,
>
>I think I found a bug. I downloaded the latest bigdata.war release from
>sourceforge, deployed in on the latest tomcat release, with out-of-the
>box configuration. Then I went to localhost:8080/bigdata and did this
>INSERT query:
>
>PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
>PREFIX umbel: <http://umbel.org/umbel/>
>INSERT DATA
>{ 
>  <http://example/book1> skos:narrower <http://example/chapter1> .
>  <http://example/book1> umbel:isRelatedToClass <http://example/book> .
>}
>
>And then I tried to do this SELECT:
>
>PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
>PREFIX umbel: <http://umbel.org/umbel/>
>SELECT ?p ?x  
>WHERE {
> {?concept skos:narrower ?x BIND (skos:narrower as ?p) . }
>  UNION
> {?concept umbel:isRelatedToClass ?x BIND (umbel:isRelatedToClass as ?p)
>. }
> FILTER (?concept in (<http://example/book1))
>}
>
>I would expect this:
>
>?p                     ?x
>skos:narrower          <http://example/chapter1>
>umbel:isRelatedToClass <http://example/book>
>
>But when I do this query many times I always get either the first or the
>second, but never both. This looks to me like a race condition somewhere
>in the code that handles BIND or UNION.
>
>Two questions:
>1. Is this a bug? What should be the behavior?
>2. I can rephrase this query to say {?concept ?p ?x FILTER(?p in
>(skos:narrower, umbel:isRelatedToClass)) .}. This works OK. Is the
>variant with BIND likely to perform better (when it works)?
>
>Could anyone confirm?
>
>
>--
>Antoni Myłka
>Software Engineer
>
>basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22
>http://www.basis06.ch - source of smart business
>
>--------------------------------------------------------------------------
>----
>LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
>1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
>SharePoint
>2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack
>includes
>Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
>http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktr
>k
>_______________________________________________
>Bigdata-developers mailing list
>Big...@li...
>https://lists.sourceforge.net/lists/listinfo/bigdata-developers