Menu
▾
▴
[Bigdata-commit] SF.net SVN: bigdata:[6400] branches/BIGDATA_RELEASE_1_2_0
|
From: <tho...@us...> - 2012-08-01 13:50:30
|
Revision: 6400
http://bigdata.svn.sourceforge.net/bigdata/?rev=6400&view=rev
Author: thompsonbry
Date: 2012-08-01 13:50:21 +0000 (Wed, 01 Aug 2012)
Log Message:
-----------
Simplified the AccessPath class by moving AccessPath#get(int index) into the test suite. It was only used by two of the backchaining expander test suites.
Removed IAccessPath#tupleIterator(). It is still implemented internally by AccessPath, but it is no longer exposed and it is not present on the other IAccessPath implementation classes.
Lifted out an ISimpleIndexAccess interface from ICheckpointProtocol with rangeCount(), scan(), and removeAll() methods.
Isolated a getAccessPath() method returning an IBindingSetAccessPath in the HTreeHashJoinOp and JVMHashJoinOp classes. This currently uses BOpContext.getAccessPath(Relation,Predicate). However, this should be the only method that we need to change in order to execute a hash join against a SolutionSetStream.
The next step is to test a change from modeling named subquery outputs as hash indices to modeling those outputs as streams. I will have to change the named solution set resolution mechanism (to be simpler) and the query plans (to build the hash index from the pipeline). I will also need to do a performance assessment for this change, probably for BSBM and the govtrack data sets.
@see https://sourceforge.net/apps/trac/bigdata/ticket/531 (SPARQL UPDATE for SOLUTION SETS)
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/BOpContextBase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/ap/Predicate.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/HTreeHashJoinOp.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/JVMHashJoinOp.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/ISolutionSet.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/SolutionSetStream.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ICheckpointProtocol.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/AbstractResource.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPathFusedView.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ArrayAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/BlockingBuffer.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ChunkConsumerIterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/EmptyAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAbstractAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IBindingSetAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/rule/eval/IJoinNexus.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/OwlSameAsPropertiesExpandingIterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/TruthMaintenance.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/rules/BackchainAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/sparql/ast/eval/DGExpander.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/spo/SPOAccessPath.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/spo/SPOArrayIterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/spo/SPOPredicate.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/rules/AbstractInferenceEngineTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestBackchainOwlSameAsPropertiesIterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestOwlSameAsPropertiesExpandingIterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestRuleExpansion.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/NanoSparqlServer.java
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ISimpleIndexAccess.java
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/BOpContextBase.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/BOpContextBase.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/BOpContextBase.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -213,8 +213,7 @@
// }
/**
- * Obtain an access path reading from relation for the specified predicate
- * (from the tail of some rule).
+ * Obtain an access path reading from relation for the specified predicate.
* <p>
* Note: Passing in the {@link IRelation} is important since it otherwise
* must be discovered using the {@link IResourceLocator}. By requiring the
@@ -250,6 +249,9 @@
*
* @todo Support mutable relation views (no - just fix truth maintenance).
*/
+// * <h2>Non-Relation Access Paths</h2>
+// * <p>
+// *
public <E> IAccessPath<E> getAccessPath(final IRelation<E> relation,
final IPredicate<E> predicate) {
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/ap/Predicate.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/ap/Predicate.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/ap/Predicate.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -226,7 +226,7 @@
return new Constant(((IElement) e).get(index));
}
-
+
final public boolean isOptional() {
return (Boolean) getProperty(Annotations.OPTIONAL,Annotations.DEFAULT_OPTIONAL);
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/HTreeHashJoinOp.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/HTreeHashJoinOp.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/HTreeHashJoinOp.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -369,16 +369,27 @@
}
/**
+ * Return the access path that to be scanned. Solutions read from this
+ * access path will be used to probe the hash index to identify
+ * solutions that can join.
+ */
+ private IBindingSetAccessPath<?> getAccessPath() {
+
+ return (IBindingSetAccessPath<?>) context.getAccessPath(relation,
+ pred);
+
+ }
+
+ /**
* Do a hash join of the buffered solutions with the access path.
*/
private void doHashJoin() {
if (state.isEmpty())
return;
-
- final IAccessPath<?> accessPath = context.getAccessPath(relation,
- pred);
+ final IBindingSetAccessPath<?> accessPath = getAccessPath();
+
if (log.isInfoEnabled())
log.info("accessPath=" + accessPath);
@@ -395,7 +406,7 @@
IPredicate.Annotations.DEFAULT_CUTOFF_LIMIT);
// Obtain the iterator for the current join dimension.
- final ICloseableIterator<IBindingSet> itr = ((IBindingSetAccessPath<?>) accessPath)
+ final ICloseableIterator<IBindingSet> itr = accessPath
.solutions(cutoffLimit, stats);
state.hashJoin(
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/JVMHashJoinOp.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/JVMHashJoinOp.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/join/JVMHashJoinOp.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -305,6 +305,17 @@
}
/**
+ * Return the access path that to be scanned. Solutions read from this
+ * access path will be used to probe the hash index to identify
+ * solutions that can join.
+ */
+ private IBindingSetAccessPath<?> getAccessPath() {
+
+ return (IBindingSetAccessPath<?>) context.getAccessPath(relation,
+ pred);
+
+ }
+ /**
* Do a hash join of the buffered solutions with the access path.
*/
private void doHashJoin() {
@@ -312,8 +323,7 @@
if (state.isEmpty())
return;
- final IAccessPath<?> accessPath = context.getAccessPath(relation,
- pred);
+ final IBindingSetAccessPath<?> accessPath = getAccessPath();
if (log.isDebugEnabled())
log.debug("accessPath=" + accessPath);
@@ -331,8 +341,8 @@
IPredicate.Annotations.DEFAULT_CUTOFF_LIMIT);
// Obtain the iterator for the current join dimension.
- final ICloseableIterator<IBindingSet> itr = ((IBindingSetAccessPath<?>) accessPath)
- .solutions(cutoffLimit, stats);
+ final ICloseableIterator<IBindingSet> itr = accessPath.solutions(
+ cutoffLimit, stats);
state.hashJoin(
itr,// left
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/ISolutionSet.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/ISolutionSet.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/ISolutionSet.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -42,15 +42,19 @@
*
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
*
- * FIXME Provide {@link ISolutionSet} implementations derived from {@link HTree}
- * and {@link BTree}.
- * <p>
- * Reconcile the {@link HTree} version with the hash join code and
- * {@link NamedSolutionSetRef}.
- * <p>
- * Add an {@link IAccessPath} for HTree and {@link Stream} backed solution sets
- * so they can be played out through a simple iterator model. The
- * {@link AccessPath} can be used for the {@link BTree} backed solution sets.
+ * FIXME Provide {@link ISolutionSet} implementations derived from
+ * {@link HTree} and {@link BTree}.
+ * <P>
+ * I am not sure that we will require this. We can probably use the
+ * Stream for named solution sets for quite a while without supporting
+ * declaration of named solution sets with specific join variables.
+ * However, if we do add support for this, then we need to reconcile the
+ * {@link HTree} version with the hash join code and
+ * {@link NamedSolutionSetRef} and also add an {@link IAccessPath} for
+ * {@link HTree} and {@link Stream} backed solution sets so they can be
+ * played out through a simple iterator model. The {@link AccessPath} is
+ * relation specific, but it could be relayered to provide support for
+ * the {@link BTree} backed solution sets.
*/
public interface ISolutionSet extends ICheckpointProtocol {
@@ -91,18 +95,18 @@
*/
public void put(final ICloseableIterator<IBindingSet[]> src);
- /**
- * The {@link IVariable[]} specifying the join variables (required). The
- * order of the entries is used when forming the as-bound keys for the hash
- * table. Duplicate elements and null elements are not permitted.
- *
- * TODO We will also need a method to report the JOIN_VARS if the backing
- * data structure is an HTree or BTree. [Really, what we need is a method to
- * describe how the index key will be generated, or just a method to
- * generate the key. Also, it is possible that
- * {@link ISolutionSetStats#getAlwaysBound} will report variables which are
- * always bound but which are not part of the key for the index.
- */
+// /**
+// * The {@link IVariable[]} specifying the join variables (required). The
+// * order of the entries is used when forming the as-bound keys for the hash
+// * table. Duplicate elements and null elements are not permitted.
+// *
+// * TODO We will also need a method to report the JOIN_VARS if the backing
+// * data structure is an HTree or BTree. [Really, what we need is a method to
+// * describe how the index key will be generated, or just a method to
+// * generate the key. Also, it is possible that
+// * {@link ISolutionSetStats#getAlwaysBound} will report variables which are
+// * always bound but which are not part of the key for the index.
+// */
// public String[] getJoinVars();
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/SolutionSetStream.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/SolutionSetStream.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/bop/solutions/SolutionSetStream.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -37,24 +37,25 @@
import org.apache.log4j.Logger;
+import com.bigdata.bop.BOpContext;
import com.bigdata.bop.IBindingSet;
import com.bigdata.bop.IConstant;
+import com.bigdata.bop.IPredicate;
import com.bigdata.bop.IVariable;
-import com.bigdata.bop.controller.NamedSolutionSetRef;
-import com.bigdata.btree.BTree;
+import com.bigdata.bop.join.BaseJoinStats;
import com.bigdata.btree.Checkpoint;
import com.bigdata.btree.IndexMetadata;
-import com.bigdata.htree.HTree;
import com.bigdata.io.SerializerUtil;
import com.bigdata.rawstore.IRawStore;
import com.bigdata.rdf.internal.encoder.SolutionSetStreamDecoder;
import com.bigdata.rdf.internal.encoder.SolutionSetStreamEncoder;
import com.bigdata.rdf.sparql.ast.ISolutionSetStats;
-import com.bigdata.relation.accesspath.AccessPath;
-import com.bigdata.relation.accesspath.IAccessPath;
+import com.bigdata.relation.accesspath.ChunkConsumerIterator;
+import com.bigdata.relation.accesspath.IBindingSetAccessPath;
import com.bigdata.rwstore.IPSOutputStream;
import com.bigdata.stream.Stream;
import com.bigdata.striterator.Chunkerator;
+import com.bigdata.striterator.IChunkedIterator;
import com.bigdata.striterator.ICloseableIterator;
import cutthecrap.utils.striterators.ArrayIterator;
@@ -428,5 +429,146 @@
}
}
-
+
+ /**
+ * Return an access path that can be used to scan the solutions.
+ *
+ * @param pred
+ * ignored.
+ *
+ * @return The access path.
+ *
+ * TODO FILTERS: (A) Any filter attached to the predicate must be
+ * applied by the AP to remove solutions which do not satisfy the
+ * filter;
+ * <p>
+ * (B) When the AP has a filter, then an exact range count must scan
+ * the solutions to decide how many will match.
+ */
+ public IBindingSetAccessPath<IBindingSet> getAccessPath(
+ final IPredicate<IBindingSet> pred) {
+
+ return new SolutionSetAP(this, pred);
+
+ }
+
+ /**
+ * Class provides basic access path suitable for full scans.
+ *
+ * @author <a href="mailto:tho...@us...">Bryan
+ * Thompson</a>
+ */
+ private static class SolutionSetAP implements
+ IBindingSetAccessPath<IBindingSet> {
+
+ private final SolutionSetStream stream;
+ private final IPredicate<IBindingSet> predicate;
+
+ public SolutionSetAP(final SolutionSetStream stream,
+ final IPredicate<IBindingSet> pred) {
+
+ this.stream = stream;
+
+ this.predicate = pred;
+
+ }
+
+ @Override
+ public IPredicate<IBindingSet> getPredicate() {
+
+ return predicate;
+
+ }
+
+ @Override
+ public boolean isEmpty() {
+
+ /*
+ * Note: If we develop streams which can have non-exact range counts
+ * (due to deleted tuples) then this would have to be revisited.
+ */
+
+ return stream.entryCount == 0;
+
+ }
+
+ @Override
+ public long rangeCount(boolean exact) {
+
+ return stream.entryCount;
+
+ }
+
+ @Override
+ public ICloseableIterator<IBindingSet> solutions(final long limit,
+ final BaseJoinStats stats) {
+
+ if (limit != 0L && limit != Long.MAX_VALUE)
+ throw new UnsupportedOperationException();
+
+ final IChunkedIterator<IBindingSet> itr = new ChunkConsumerIterator<IBindingSet>(
+ stream.get());
+
+ return BOpContext.solutions(itr, predicate, stats);
+
+ }
+
+ @Override
+ public long removeAll() {
+
+ final long n = stream.rangeCount();
+
+ stream.removeAll();
+
+ return n;
+
+ }
+
+ }
+
+// /**
+// * A predicate that can be used with an {@link ISolutionSet} without having
+// * to resolve the {@link ISolutionSet} as an {@link IRelation}.
+// *
+// * @author <a href="mailto:tho...@us...">Bryan
+// * Thompson</a>
+// * @param <E>
+// */
+// public static class SolutionSetPredicate<E extends IBindingSet> extends
+// Predicate<E> {
+//
+// public SolutionSetPredicate(BOp[] args, Map<String, Object> annotations) {
+// super(args, annotations);
+// }
+//
+// public SolutionSetPredicate(BOp[] args, NV... annotations) {
+// super(args, annotations);
+// }
+//
+// public SolutionSetPredicate(IVariableOrConstant<?>[] values,
+// String relationName, int partitionId, boolean optional,
+// IElementFilter<E> constraint, IAccessPathExpander<E> expander,
+// long timestamp) {
+// super(values, relationName, partitionId, optional, constraint,
+// expander, timestamp);
+// }
+//
+// public SolutionSetPredicate(IVariableOrConstant<?>[] values,
+// String relationName, long timestamp) {
+// super(values, relationName, timestamp);
+// // TODO Auto-generated constructor stub
+// }
+//
+// public SolutionSetPredicate(Predicate<E> op) {
+// super(op);
+// // TODO Auto-generated constructor stub
+// }
+//
+// /**
+// *
+// */
+// private static final long serialVersionUID = 1L;
+//
+// }
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ICheckpointProtocol.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ICheckpointProtocol.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ICheckpointProtocol.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -1,3 +1,26 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2011. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
package com.bigdata.btree;
import com.bigdata.counters.ICounterSetAccess;
@@ -6,23 +29,19 @@
import com.bigdata.journal.ICommitter;
import com.bigdata.journal.Name2Addr;
import com.bigdata.journal.Name2Addr.Entry;
-import com.bigdata.striterator.ICloseableIterator;
/**
* Interface in support of the {@link Checkpoint} record protocol.
*
* @author tho...@us...
*
- * TODO There should be a high level method to insert objects into the
- * index (index "entries" not tuples - the index will need to compute
- * the appropriate key, etc. in an implementation dependent manner).
- *
* TODO Try to lift out an abstract implementation of this interface for
* HTree, BTree, and Stream. This will be another step towards GIST
* support. There are protected methods which are used on those classes
* which should be lifted into the abstract base class.
*/
-public interface ICheckpointProtocol extends ICommitter, ICounterSetAccess {
+public interface ICheckpointProtocol extends ICommitter, ICounterSetAccess,
+ ISimpleIndexAccess {
/**
* The value of the record version number that will be assigned to the next
@@ -134,34 +153,36 @@
*/
public IndexMetadata getIndexMetadata();
- /*
- * Generic data access methods defined for all persistence capable
- * data structures.
- */
+// /*
+// * Generic data access methods defined for all persistence capable
+// * data structures.
+// */
+//
+// /**
+// * Return the #of entries in the index.
+// * <p>
+// * Note: If the index supports deletion markers then the range count will be
+// * an upper bound and may double count tuples which have been overwritten,
+// * including the special case where the overwrite is a delete.
+// *
+// * @return The #of tuples in the index.
+// *
+// * @see IRangeQuery#rangeCount()
+// */
+// public long rangeCount();
+//
+// /**
+// * Visit all entries in the index in the natural order of the index
+// * (dereferencing visited tuples to the application objects stored within
+// * those tuples).
+// */
+// public ICloseableIterator<?> scan();
+//
+// /**
+// * Remove all entries in the index.
+// */
+// public void removeAll();
- /**
- * Return the #of entries in the index.
- * <p>
- * Note: If the index supports deletion markers then the range count will be
- * an upper bound and may double count tuples which have been overwritten,
- * including the special case where the overwrite is a delete.
- *
- * @return The #of tuples in the index.
- *
- * @see IRangeQuery#rangeCount()
- */
- public long rangeCount();
-
- /**
- * Visit all entries in the index in the natural order of the index.
- */
- public ICloseableIterator<?> scan();
-
- /**
- * Remove all entries in the index.
- */
- public void removeAll();
-
/*
* reopen() / close() protocol
*/
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ISimpleIndexAccess.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ISimpleIndexAccess.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/btree/ISimpleIndexAccess.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -0,0 +1,66 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2011. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+/*
+ * Created on Aug 1, 2012
+ */
+package com.bigdata.btree;
+
+import com.bigdata.striterator.ICloseableIterator;
+
+/**
+ * Generic data access methods defined for all persistence capable data
+ * structures.
+ *
+ * TODO There should be a high level method to insert objects into the index
+ * (index "entries" not tuples - the index will need to compute the appropriate
+ * key, etc. in an implementation dependent manner).
+ */
+public interface ISimpleIndexAccess {
+
+ /**
+ * Return the #of entries in the index.
+ * <p>
+ * Note: If the index supports deletion markers then the range count will be
+ * an upper bound and may double count tuples which have been overwritten,
+ * including the special case where the overwrite is a delete.
+ *
+ * @return The #of tuples in the index.
+ *
+ * @see IRangeQuery#rangeCount()
+ */
+ public long rangeCount();
+
+ /**
+ * Visit all entries in the index in the natural order of the index
+ * (dereferencing visited tuples to the application objects stored within
+ * those tuples).
+ */
+ public ICloseableIterator<?> scan();
+
+ /**
+ * Remove all entries in the index.
+ */
+ public void removeAll();
+
+}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/AbstractResource.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/AbstractResource.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/AbstractResource.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -138,9 +138,9 @@
/**
* If the estimated range count for an
- * {@link IAccessPath#iterator(int, int)} is LTE this threshold then do a
- * fully buffered (synchronous) read. Otherwise we will do an asynchronous
- * read.
+ * {@link IAccessPath#iterator(long,long, int)} is LTE this threshold then
+ * do a fully buffered (synchronous) read. Otherwise we will do an
+ * asynchronous read.
*
* @see Options#FULLY_BUFFERED_READ_THRESHOLD
*
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -40,7 +40,6 @@
import com.bigdata.bop.BufferAnnotations;
import com.bigdata.bop.IBindingSet;
import com.bigdata.bop.IPredicate;
-import com.bigdata.bop.IVariableOrConstant;
import com.bigdata.bop.ap.filter.SameVariableConstraint;
import com.bigdata.bop.cost.BTreeCostModel;
import com.bigdata.bop.cost.DiskCostModel;
@@ -693,7 +692,8 @@
}
- final IChunkedIterator<R> itr = iterator(1,1);
+ final IChunkedIterator<R> itr = iterator(0L/* offset */, 1L/* limit */,
+ 1/* capacity */);
try {
@@ -744,8 +744,9 @@
// final IVariable<?>[] vars = BOpUtility
// .getDistinctArgumentVariables(predicate);
- return BOpContext.solutions(iterator(0, limit, 0/* (int) limit */),
- predicate, /* vars, */stats);
+ return BOpContext.solutions(
+ iterator(0L/* offset */, limit, 0/* capacity */), predicate,
+ stats);
}
@@ -755,13 +756,13 @@
}
- final public IChunkedOrderedIterator<R> iterator(final int limit,
- final int capacity) {
+// final public IChunkedOrderedIterator<R> iterator(final int limit,
+// final int capacity) {
+//
+// return iterator(0L/* offset */, limit, capacity);
+//
+// }
- return iterator(0L/* offset */, limit, capacity);
-
- }
-
/**
* @throws RejectedExecutionException
* if the iterator is run asynchronously and the
@@ -1381,12 +1382,13 @@
}
- final public ITupleIterator<R> rangeIterator() {
+// @Override
+// final public ITupleIterator<R> rangeIterator() {
+//
+// return rangeIterator(0/* capacity */, flags, indexLocalFilter);
+//
+// }
- return rangeIterator(0/* capacity */, flags, indexLocalFilter);
-
- }
-
@SuppressWarnings( { "unchecked" })
protected ITupleIterator<R> rangeIterator(final int capacity,
final int flags, final IFilter filter) {
@@ -1446,25 +1448,6 @@
}
/**
- * Return the constant bound on the {@link #getPredicate() predicate} for
- * this access path at the specified index -or- {@link #NULL} iff the
- * predicate is not bound at that position.
- *
- * @param index
- * The index.
- *
- * @return Either the bound value -or- {@link #NULL} iff the index is
- * unbound for the predicate for this access path.
- */
- public Object get(final int index) {
-
- final IVariableOrConstant<?> t = predicate.get(index);
-
- return t == null || t.isVar() ? null : t.get();
-
- }
-
- /**
* Return an estimate of the cost of a scan on the predicate.
*
* @param pred
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPathFusedView.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPathFusedView.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/AccessPathFusedView.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -112,12 +112,14 @@
}
+ @Override
public IPredicate<E> getPredicate() {
return path1.getPredicate();
}
+ @Override
public boolean isEmpty() {
return path1.isEmpty() && path2.isEmpty();
@@ -125,6 +127,8 @@
}
/**
+ * {@inheritDoc}
+ * <P>
* Note: You can not get an exact range count for a view.
*
* @throws UnsupportedOperationException
@@ -135,6 +139,7 @@
* sync so that duplicates could be detected. This means a full key
* range scan for both source access paths.
*/
+ @Override
public long rangeCount(boolean exact) {
if (exact) {
@@ -152,24 +157,28 @@
// final private int flags;
/**
+ * {@inheritDoc}
+ *
* @throws UnsupportedOperationException
* always.
*
* @todo this could be implemented with a variant (or relaxed form) of
* {@link FusedView}.
*/
+ @Override
public IIndex getIndex() {
throw new UnsupportedOperationException();
}
- public ITupleIterator<E> rangeIterator() {
+// @Override
+// public ITupleIterator<E> rangeIterator() {
+//
+// return rangeIterator(0/* capacity */);
+//
+// }
- return rangeIterator(0/* capacity */);
-
- }
-
private ITupleIterator<E> rangeIterator(final int capacity) {
/*
@@ -188,17 +197,18 @@
}
+ @Override
public IChunkedOrderedIterator<E> iterator() {
return iterator(0L/* offset */, 0L/* limit */, 0/* capacity */);
}
- public IChunkedOrderedIterator<E> iterator(int limit, int capacity) {
-
- return iterator(0L/* offset */, limit, capacity);
-
- }
+// public IChunkedOrderedIterator<E> iterator(int limit, int capacity) {
+//
+// return iterator(0L/* offset */, limit, capacity);
+//
+// }
/**
* FIXME write tests for optimizations for point tests and small limits. See
@@ -206,7 +216,9 @@
*
* FIXME handle non-zero offset.
*/
- public IChunkedOrderedIterator<E> iterator(long offset, long limit, int capacity) {
+ @Override
+ public IChunkedOrderedIterator<E> iterator(final long offset, long limit,
+ int capacity) {
if (offset > 0L)
throw new UnsupportedOperationException();
@@ -283,12 +295,14 @@
}
+ @Override
public long removeAll() {
throw new UnsupportedOperationException();
}
+ @Override
public IKeyOrder<E> getKeyOrder() {
return path1.getKeyOrder();
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ArrayAccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ArrayAccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ArrayAccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -26,7 +26,6 @@
import com.bigdata.bop.IPredicate;
import com.bigdata.btree.IIndex;
-import com.bigdata.btree.ITupleIterator;
import com.bigdata.striterator.ChunkedArrayIterator;
import com.bigdata.striterator.ChunkedWrappedIterator;
import com.bigdata.striterator.IChunkedOrderedIterator;
@@ -74,10 +73,13 @@
}
/**
+ * {@inheritDoc}
+ *
* @throws UnsupportedOperationException
* unless the caller specified an {@link IPredicate} to the
* ctor.
*/
+ @Override
public IPredicate<E> getPredicate() {
if (predicate == null)
@@ -88,31 +90,40 @@
}
/**
+ * {@inheritDoc}
+ *
* @throws UnsupportedOperationException
* unless the caller specified an {@link IKeyOrder} to the ctor.
*/
+ @Override
public IKeyOrder<E> getKeyOrder() {
if (keyOrder == null)
throw new UnsupportedOperationException();
return keyOrder;
-
+
}
/**
+ * {@inheritDoc}
+ *
* @throws UnsupportedOperationException
* since no index is associated with this array
*/
+ @Override
public IIndex getIndex() {
-
+
throw new UnsupportedOperationException();
}
-
+
/**
+ * {@inheritDoc}
+ * <p>
* Returns <code>true</code> when the array of elements is empty.
*/
+ @Override
public boolean isEmpty() {
return e.length == 0;
@@ -120,47 +131,57 @@
}
/**
+ * {@inheritDoc}
+ * <p>
* Returns the size of the array of elements.
*/
+ @Override
public long rangeCount(boolean exact) {
return e.length;
}
- /**
- * @throws UnsupportedOperationException
- * since no index is associated with this array
- */
- public ITupleIterator<E> rangeIterator() {
+// /**
+// * @throws UnsupportedOperationException
+// * since no index is associated with this array
+// */
+// @Override
+// public ITupleIterator<E> rangeIterator() {
+//
+// throw new UnsupportedOperationException();
+//
+// }
- throw new UnsupportedOperationException();
-
- }
-
/**
+ * {@inheritDoc}
+ * <p>
* Visits the entire array of elements.
*/
+ @Override
public IChunkedOrderedIterator<E> iterator() {
- return iterator(0/* limit */, 0/* capacity */);
+ return iterator(0L/* offset */, 0L/* limit */, 0/* capacity */);
}
- /**
- * Visits the array of elements up to the specified limit.
- */
- public IChunkedOrderedIterator<E> iterator(final int limit,
- final int capacity) {
+// /**
+// * Visits the array of elements up to the specified limit.
+// */
+// public IChunkedOrderedIterator<E> iterator(final int limit,
+// final int capacity) {
+//
+// return iterator(0L/* offset */, limit, capacity);
+//
+// }
- return iterator(0L/* offset */, limit, capacity);
-
- }
-
/**
- * Visits the array of elements from the specified offset up to the
+ * {@inheritDoc}
+ * <p>
+ * Visits the array of elements from the specified offset up to the
* specified limit.
*/
+ @Override
@SuppressWarnings("unchecked")
public IChunkedOrderedIterator<E> iterator(final long offset,
long limit, final int capacity) {
@@ -232,8 +253,11 @@
}
/**
+ * {@inheritDoc}
+ * <P>
* Does nothing and always returns ZERO(0).
*/
+ @Override
public long removeAll() {
return 0L;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/BlockingBuffer.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/BlockingBuffer.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/BlockingBuffer.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -1172,7 +1172,7 @@
* <p>
* The most common reason to close an iterator early are that you want to
* only visit a limited #of elements. However, if you use either
- * {@link IAccessPath#iterator(int, int)} or {@link IRule} with an
+ * {@link IAccessPath#iterator(long,long, int)} or {@link IRule} with an
* {@link IQueryOptions} to impose that limit, then most processes that
* produce {@link IAsynchronousIterator}s will automatically terminate when
* they reach the desired limit, thereby avoiding issuing interrupts. Those
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ChunkConsumerIterator.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ChunkConsumerIterator.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ChunkConsumerIterator.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -10,12 +10,16 @@
import com.bigdata.striterator.IKeyOrder;
/**
- * A class that aligns a buffer of <code>E[]</code>s (a buffer of chunks)
- * with an {@link IChunkedOrderedIterator}.
+ * A class that aligns a buffer of <code>E[]</code>s (a buffer of chunks) with
+ * an {@link IChunkedOrderedIterator}.
*
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- * @version $Id$
+ * @version $Id: ChunkConsumerIterator.java 6130 2012-03-15 10:31:25Z
+ * thompsonbry $
* @param <E>
+ *
+ * @see <a href="https://sourceforge.net/apps/trac/bigdata/ticket/483">
+ * Eliminate unnecessary dechunking and rechunking </a>
*/
public class ChunkConsumerIterator<E> implements IChunkedOrderedIterator<E> {
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/EmptyAccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/EmptyAccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/EmptyAccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -4,8 +4,6 @@
import com.bigdata.bop.IPredicate;
import com.bigdata.btree.IIndex;
-import com.bigdata.btree.ITupleIterator;
-import com.bigdata.btree.filter.EmptyTupleIterator;
import com.bigdata.striterator.ChunkedWrappedIterator;
import com.bigdata.striterator.IChunkedOrderedIterator;
import com.bigdata.striterator.IKeyOrder;
@@ -56,10 +54,12 @@
}
/**
+ * {@inheritDoc}
* @throws UnsupportedOperationException
* unless the caller specified an {@link IPredicate} to the
* ctor.
*/
+ @Override
public IPredicate<E> getPredicate() {
if (predicate == null)
@@ -70,9 +70,11 @@
}
/**
+ * {@inheritDoc}
* @throws UnsupportedOperationException
* unless the caller specified an {@link IKeyOrder} to the ctor.
*/
+ @Override
public IKeyOrder<E> getKeyOrder() {
if (keyOrder == null)
@@ -83,9 +85,11 @@
}
/**
+ * {@inheritDoc}
* @throws UnsupportedOperationException
* since no index was selected.
*/
+ @Override
public IIndex getIndex() {
throw new UnsupportedOperationException();
@@ -93,8 +97,10 @@
}
/**
+ * {@inheritDoc}<p>
* Always returns <code>true</code>.
*/
+ @Override
public boolean isEmpty() {
return true;
@@ -102,48 +108,52 @@
}
/**
- * Always returns ZERO(0).
+ * {@inheritDoc}<p>Always returns ZERO(0).
*/
+ @Override
public long rangeCount(boolean exact) {
return 0;
}
- /**
- * Visits nothing.
- */
- @SuppressWarnings("unchecked")
- public ITupleIterator<E> rangeIterator() {
+// /**
+// * Visits nothing.
+// */
+// @SuppressWarnings("unchecked")
+// @Override
+// public ITupleIterator<E> rangeIterator() {
+//
+// return EmptyTupleIterator.INSTANCE;
+//
+// }
- return EmptyTupleIterator.INSTANCE;
-
- }
-
/**
* Visits nothing.
*/
+ @Override
public IChunkedOrderedIterator<E> iterator() {
return iterator(0L/* offset */, 0L/* limit */, 0/* capacity */);
}
- /**
- * Visits nothing.
- */
- public IChunkedOrderedIterator<E> iterator(int limit, int capacity) {
+// /**
+// * Visits nothing.
+// */
+// public IChunkedOrderedIterator<E> iterator(int limit, int capacity) {
+//
+// return iterator(0L/* offset */, limit, capacity);
+//
+// }
- return iterator(0L/* offset */, limit, capacity);
-
- }
-
/**
* Visits nothing.
*/
+ @Override
@SuppressWarnings("unchecked")
- public IChunkedOrderedIterator<E> iterator(long offset, long limit,
- int capacity) {
+ public IChunkedOrderedIterator<E> iterator(final long offset,
+ final long limit, final int capacity) {
return new ChunkedWrappedIterator<E>(Collections.EMPTY_LIST.iterator());
@@ -152,6 +162,7 @@
/**
* Does nothing and always returns ZERO(0).
*/
+ @Override
public long removeAll() {
return 0L;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAbstractAccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAbstractAccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAbstractAccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -78,4 +78,12 @@
*/
long rangeCount(boolean exact);
+ /**
+ * Remove all elements selected by the {@link IPredicate} (optional
+ * operation).
+ *
+ * @return The #of elements that were removed.
+ */
+ public long removeAll();
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IAccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -28,9 +28,9 @@
package com.bigdata.relation.accesspath;
+import com.bigdata.bop.IElement;
import com.bigdata.bop.IPredicate;
import com.bigdata.btree.IIndex;
-import com.bigdata.btree.ITupleIterator;
import com.bigdata.relation.IRelation;
import com.bigdata.relation.rule.IQueryOptions;
import com.bigdata.relation.rule.IRule;
@@ -68,15 +68,15 @@
*/
public IIndex getIndex();
- /**
- * The raw iterator for traversing the selected index within the key range
- * implied by {@link IPredicate}.
- * <p>
- * Note: The access path may incorporate additional constraints from the
- * specified {@link IPredicate} that are not present on the raw
- * {@link ITupleIterator} returned by this method.
- */
- public ITupleIterator<R> rangeIterator();
+// /**
+// * The raw iterator for traversing the selected index within the key range
+// * implied by {@link IPredicate}.
+// * <p>
+// * Note: The access path may incorporate additional constraints from the
+// * specified {@link IPredicate} that are not present on the raw
+// * {@link ITupleIterator} returned by this method.
+// */
+// public ITupleIterator<R> rangeIterator();
/**
* An iterator visiting elements using the natural order of the index
@@ -102,25 +102,25 @@
*/
public IChunkedOrderedIterator<R> iterator();
- /**
- * An iterator visiting elements using the natural order of the index
- * selected for the {@link IPredicate}.
- *
- * @param limit
- * The maximum #of elements that will be visited -or- ZERO (0) if
- * there is no limit.
- *
- * @param capacity
- * The maximum capacity for the buffer used by the iterator. When
- * ZERO(0), a default capacity will be used. When a <i>limit</i>
- * is specified, the capacity will never exceed the <i>limit</i>.
- *
- * @return The iterator.
- *
- * @deprecated by {@link #iterator(long, long, int)}. Also, [limit] should
- * have been a long, not an int.
- */
- public IChunkedOrderedIterator<R> iterator(int limit, int capacity);
+// /**
+// * An iterator visiting elements using the natural order of the index
+// * selected for the {@link IPredicate}.
+// *
+// * @param limit
+// * The maximum #of elements that will be visited -or- ZERO (0) if
+// * there is no limit.
+// *
+// * @param capacity
+// * The maximum capacity for the buffer used by the iterator. When
+// * ZERO(0), a default capacity will be used. When a <i>limit</i>
+// * is specified, the capacity will never exceed the <i>limit</i>.
+// *
+// * @return The iterator.
+// *
+// * @deprecated by {@link #iterator(long, long, int)}. Also, [limit] should
+// * have been a long, not an int.
+// */
+// public IChunkedOrderedIterator<R> iterator(int limit, int capacity);
/**
* An iterator visiting elements using the natural order of the index
@@ -166,12 +166,12 @@
public IChunkedOrderedIterator<R> iterator(long offset, long limit,
int capacity);
- /**
- * Remove all elements selected by the {@link IPredicate} (optional
- * operation).
- *
- * @return The #of elements that were removed.
- */
- public long removeAll();
+// /**
+// * Remove all elements selected by the {@link IPredicate} (optional
+// * operation).
+// *
+// * @return The #of elements that were removed.
+// */
+// public long removeAll();
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IBindingSetAccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IBindingSetAccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/IBindingSetAccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -50,8 +50,16 @@
* Return an iterator which will visit the solutions drawn from the access
* path with a limit on the number of solutions drawn.
*
- * TODO Should this be visiting IBindingSet[]s?
+ * @param limit
+ * The maximum #of solutions to visit.
+ * @param stats
+ * Some statistics are updated as solutions are visited.
*
+ * @return The iterator.
+ *
+ * TODO Should this be visiting IBindingSet[]s or be an
+ * IChunkedIterator<IBindingSet>?
+ *
* @see https://sourceforge.net/apps/trac/bigdata/ticket/209 (Access path
* should visit solutions for high level query).
*/
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/rule/eval/IJoinNexus.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/rule/eval/IJoinNexus.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/rule/eval/IJoinNexus.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -156,7 +156,7 @@
* {@link IAccessPath} will use an {@link IAsynchronousIterator} instead.
* This value should on the order of {@link #getChunkCapacity()}.
*
- * @see IAccessPath#iterator(int, int)
+ * @see IAccessPath#iterator(long,long, int)
* @see AbstractResource.Options#FULLY_BUFFERED_READ_THRESHOLD
* @deprecated by bop annotations.
*/
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/OwlSameAsPropertiesExpandingIterator.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/OwlSameAsPropertiesExpandingIterator.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/OwlSameAsPropertiesExpandingIterator.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -409,22 +409,27 @@
}
}
+ @Override
public IIndex getIndex() {
return accessPath.getIndex();
}
+ @Override
public IKeyOrder<ISPO> getKeyOrder() {
return accessPath.getKeyOrder();
}
+ @Override
public IPredicate<ISPO> getPredicate() {
return accessPath.getPredicate();
}
+ @Override
public boolean isEmpty() {
return false;
}
+ @Override
public IChunkedOrderedIterator<ISPO> iterator() {
final IChunkedOrderedIterator<ISPO> appender = getAppender();
final IChunkedOrderedIterator<ISPO> delegate =
@@ -476,22 +481,26 @@
};
}
- public IChunkedOrderedIterator<ISPO> iterator(int limit, int capacity) {
- throw new UnsupportedOperationException();
- }
+// public IChunkedOrderedIterator<ISPO> iterator(int limit, int capacity) {
+// throw new UnsupportedOperationException();
+// }
+ @Override
public IChunkedOrderedIterator<ISPO> iterator(long offset, long limit, int capacity) {
throw new UnsupportedOperationException();
}
+ @Override
public long rangeCount(boolean exact) {
return accessPath.rangeCount(exact) + 1;
}
- public ITupleIterator<ISPO> rangeIterator() {
- throw new UnsupportedOperationException();
- }
+// @Override
+// public ITupleIterator<ISPO> rangeIterator() {
+// throw new UnsupportedOperationException();
+// }
+ @Override
public long removeAll() {
throw new UnsupportedOperationException();
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/TruthMaintenance.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/TruthMaintenance.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/inf/TruthMaintenance.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -1001,6 +1001,19 @@
/*
* Suck everything in the focusStore into an SPO[].
+ *
+ * TODO There are some scalability and "old code" issues with this.
+ * It is reading everything into an SPO[], which is a scalability
+ * problem. This is also the only code which uses the
+ * SPOArrayIterator and the IAccessPath#rangeIterator(). If we
+ * change this, we can get rid of both of those things.
+ *
+ * There are other alternatives to pulling it all into an ISPO[]. We
+ * are doing this so we can remove all these ISPOs later (see
+ * below). The backing temp store does not support multiple commit
+ * points, but we could write the new data onto a different temp
+ * triple store and then read from the old temp triple store when
+ * deleting the data.
*/
final SPOArrayIterator tmp = new SPOArrayIterator(focusStore, focusStore
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/rules/BackchainAccessPath.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/rules/BackchainAccessPath.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/rules/BackchainAccessPath.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -34,7 +34,6 @@
import com.bigdata.bop.IPredicate;
import com.bigdata.btree.IIndex;
-import com.bigdata.btree.ITupleIterator;
import com.bigdata.rdf.axioms.Axioms;
import com.bigdata.rdf.inf.BackchainTypeResourceIterator;
import com.bigdata.rdf.inf.OwlSameAsPropertiesExpandingIterator;
@@ -51,9 +50,6 @@
import com.bigdata.striterator.IChunkedOrderedIterator;
import com.bigdata.striterator.IKeyOrder;
-import cutthecrap.utils.striterators.IFilter;
-import cutthecrap.utils.striterators.Striterator;
-
/**
* A read-only {@link IAccessPath} that backchains certain inferences.
* <p>
@@ -160,30 +156,32 @@
}
- /**
- * The {@link IIndex} for the source {@link IAccessPath}.
- */
+ @Override
final public IIndex getIndex() {
return accessPath.getIndex();
}
+ @Override
final public IKeyOrder<ISPO> getKeyOrder() {
return accessPath.getKeyOrder();
}
+ @Override
final public IPredicate<ISPO> getPredicate() {
return accessPath.getPredicate();
}
+ @Override
public boolean isEmpty() {
- final IChunkedIterator<ISPO> itr = iterator(1,1);
+ final IChunkedIterator<ISPO> itr = iterator(0L/* offset */,
+ 1L/* limit */, 1/* capacity */);
try {
@@ -198,20 +196,36 @@
}
/**
+ * {@inheritDoc}
+ * <p>
* Visits elements in the source {@link IAccessPath} plus all entailments
* licensed by the {@link InferenceEngine}.
*/
+ @Override
public IChunkedOrderedIterator<ISPO> iterator() {
return iterator(0L/* offset */, 0L/* limit */, 0/* capacity */);
}
+// /**
+// * Visits elements in the source {@link IAccessPath} plus all entailments
+// * licensed by the {@link InferenceEngine} as configured.
+// */
+// public IChunkedOrderedIterator<ISPO> iterator(final int limit,
+// final int capacity) {
+//
+// return iterator(0L/*offset*/,limit,capacity);
+//
+// }
+
/**
+ * {@inheritDoc}
* @todo handle non-zero offset and larger limits?
*/
- public IChunkedOrderedIterator<ISPO> iterator(long offset, long limit,
- int capacity) {
+ @Override
+ public IChunkedOrderedIterator<ISPO> iterator(final long offset,
+ long limit, int capacity) {
if (offset > 0L)
throw new UnsupportedOperationException();
@@ -222,17 +236,17 @@
if (limit > Integer.MAX_VALUE)
throw new UnsupportedOperationException();
- return iterator((int) limit, capacity);
+// return iterator((int) limit, capacity);
+//
+// }
+//
+// /**
+// * Visits elements in the source {@link IAccessPath} plus all entailments
+// * licensed by the {@link InferenceEngine} as configured.
+// */
+// public IChunkedOrderedIterator<ISPO> iterator(final int limit,
+// final int capacity) {
- }
-
- /**
- * Visits elements in the source {@link IAccessPath} plus all entailments
- * licensed by the {@link InferenceEngine} as configured.
- */
- public IChunkedOrderedIterator<ISPO> iterator(final int limit,
- final int capacity) {
-
if (INFO) {
log.info(accessPath.getPredicate().toString());
@@ -345,7 +359,7 @@
// tmp.addFilter(accessPathFilter);
IChunkedOrderedIterator<ISPO> itr = (owlSameAsItr == null//
- ? accessPath.iterator(limit, capacity) //
+ ? accessPath.iterator(offset, limit, capacity) //
: new ChunkedWrappedIterator<ISPO>(owlSameAsItr,
capacity == 0 ? inf.database.getChunkCapacity()
: capacity, null/* keyOrder */, filter)//
@@ -477,12 +491,15 @@
/**
+ * {@inheritDoc}
+ * <p>
* When <code>exact == false</code> this does not count the backchained
* entailments. When <code>exact == true</code> traverses the
* {@link #iterator()} so as to produce an exact count of the #of elements
* that would in fact be visited, which combines those from the database
* with those generated dynamically (NOT efficient).
*/
+ @Override
public long rangeCount(boolean exact) {
if (!exact)
@@ -514,19 +531,18 @@
}
- /**
- * Delegated to the source {@link IAccessPath} (does not visit any
- * entailments).
- */
- public ITupleIterator<ISPO> rangeIterator() {
-
- return accessPath.rangeIterator();
-
- }
+// /**
+// * Delegated to the source {@link IAccessPath} (does not visit any
+// * entailments).
+// */
+// @Override
+// public ITupleIterator<ISPO> rangeIterator() {
+//
+// return accessPath.rangeIterator();
+//
+// }
- /**
- * Delegated to the source {@link IAccessPath}.
- */
+ @Override
public long removeAll() {
return accessPath.removeAll();
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/sparql/ast/eval/DGExpander.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/sparql/ast/eval/DGExpander.java 2012-07-31 13:27:43 UTC (rev 6399)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/sparql/ast/eval/DGExpander.java 2012-08-01 13:50:21 UTC (rev 6400)
@@ -144,24 +144,28 @@
}
+ @Override
public IIndex getIndex() {
return sourceAccessPath.getIndex();
}
+ @Override
public IKeyOrder<ISPO> getKeyOrder() {
return sourceAccessPath.getKeyOrder();
}
+ @Override
pub...
[truncated message content] |
