Menu
▾
▴
bigdata-commit
|
From: <tho...@us...> - 2013-08-22 21:45:35
|
Revision: 7320
http://bigdata.svn.sourceforge.net/bigdata/?rev=7320&view=rev
Author: thompsonbry
Date: 2013-08-22 21:45:26 +0000 (Thu, 22 Aug 2013)
Log Message:
-----------
Updated the GAS performance testing harness in preparation to some trial runs on other hardware.
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/README.txt
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/RWStore.properties
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.properties
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.xml
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
Removed Paths:
-------------
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/PerformanceTest.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/RWStore.properties
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/README.txt
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/README.txt (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/README.txt 2013-08-22 21:45:26 UTC (rev 7320)
@@ -0,0 +1,22 @@
+This directory contains a setup for some Gather Apply Scatter (GAS) based
+graph mining algorithms against bigdata.
+
+The files in this directory include:
+
+- build.properties - configuration properties for the ant script.
+
+- build.xml - an ant script which may be used to load a generated data set
+ a local bigdata database instance and start a SPARQL
+ end point for that database instance. You will then run the
+ benchmark against that SPARQL end point.
+
+- RWStore.properties - configuration properties for a bigdata database instance
+ suitable for BSBM and backed by the RW persistence engine
+ (single machine read/write bigdata database). This is the
+ recommended database mode for BSBM as it has significantly
+ better performance.
+
+- WORMStore.properties - configuration properties for a bigdata database instance
+ suitable for BSBM and backed by the WORM persistence
+ engine (single machine write once, read many bigdata
+ database).
Copied: branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/RWStore.properties (from rev 7305, branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/RWStore.properties)
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/RWStore.properties (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/RWStore.properties 2013-08-22 21:45:26 UTC (rev 7320)
@@ -0,0 +1,35 @@
+#
+# Note: These options are applied when the journal and the triple store are
+# first created.
+
+##
+## Journal options.
+##
+
+# The backing file. This contains all your data. You want to put this someplace
+# safe. The default locator will wind up in the directory from which you start
+# your servlet container.
+com.bigdata.journal.AbstractJournal.file=bigdata.jnl
+
+# The persistence engine. Use 'Disk' for the WORM or 'DiskRW' for the RWStore.
+com.bigdata.journal.AbstractJournal.bufferMode=DiskRW
+#com.bigdata.journal.AbstractJournal.bufferMode=MemStore
+
+com.bigdata.btree.writeRetentionQueue.capacity=4000
+com.bigdata.btree.BTree.branchingFactor=128
+
+# 200M initial extent.
+com.bigdata.journal.AbstractJournal.initialExtent=209715200
+com.bigdata.journal.AbstractJournal.maximumExtent=209715200
+
+##
+## Setup for QUADS mode without the full text index.
+##
+com.bigdata.rdf.sail.truthMaintenance=false
+com.bigdata.rdf.store.AbstractTripleStore.quads=true
+com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false
+com.bigdata.rdf.store.AbstractTripleStore.textIndex=false
+com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms
+#com.bigdata.rdf.store.AbstractTripleStore.inlineDateTimes=true
+
+com.bigdata.rdf.rio.RDFParserOptions.stopAtFirstError=false
\ No newline at end of file
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.properties
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.properties (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.properties 2013-08-22 21:45:26 UTC (rev 7320)
@@ -0,0 +1,106 @@
+# ant build properties.
+
+# the bigdata base directory
+bigdata.dir=../..
+
+# Where to find the pre-build bigdata classes.
+bigdata.build.dir=${bigdata.dir}/ant-build
+
+bigdata.install.lib.dir=${bigdata.dir}/
+
+# TODO This is ignored. Parameterize the test to use the specified data file.
+#rdf.data.dir=${bigdata.dir}/bigdata-rdf/src/resources/data/foaf
+
+# TODO Parameterize the test to use the specified parallelism/strategy.
+nparallel=8
+
+##
+# javac options
+##
+
+# debug=on|off
+javac.debug=off
+# debuglevel=lines,vars,source (or any combination thereof).
+javac.debuglevel=lines,vars,source
+javac.verbose=off
+#javac.target=1.6
+#javac.source=1.6
+javac.encoding=Cp1252
+
+# GAS properties.
+#
+# Note: By default, the files will wind up in ./ant-build/bin
+#
+# Note: By degault, the server jvm will optimize for throughput and can have
+# high variation in throughput due to long GC pauses for larger heaps. You can
+# use the CMS-I GC mode to minimize latency at the expense of some throughput.
+
+# The port at which the NanoSparqlServer will respond (if started).
+nanoServerPort=80
+
+# The maximum size of the java heap.
+maxMem=4g
+
+# The namespace of the KB instance (multiple KBs can be in the same database).
+namespace=kb
+
+# The name of the file used to configure the Journal.
+journalPropertyFile=RWStore.properties
+
+# The name of the file used for the journal.
+bufferMode=DiskRW
+#bufferMode=MemStore
+
+# files to load.
+#load=-load xxx -load yyy
+load=
+
+# The #of threads to use for GATHER and SCATTER
+nthreads=4
+
+# The #of random starting vertices to select.
+nsamples=1000
+
+# The seed for the random number generator.
+seed=217
+
+# BFS, SSSP, etc. Will run corresponding XXX class.
+analytic=BFS
+
+#
+# Profiler parameters.
+#
+
+# No profiler.
+profilerAgent=
+# linux-64
+#profilerAgent=-agentpath:/usr/java/yjp-9.0.3/bin/linux-x86-64/libyjpagent.so
+# Windows
+#profilerAgent="-agentpath:C:/Program Files/YourKit Java Profiler 9.0.2/bin/win32/yjpagent.dll"
+# Windows Server 2008
+#profilerAgent="-agentpath:C:/Program Files (x86)/YourKit Java Profiler 9.0.4/bin/win64/yjpagent.dll"
+
+# No profiler.
+profilerAgentOptions=
+# all profiling initially disabled.
+#profilerAgentOptions=-agentlib:yjpagent=disableexceptiontelemetry,disablestacktelemetry
+
+profiler=${profilerAgent} ${profilerAgentOptions}
+
+# Configure GC.
+gcopts=
+#gcopts=-verbose:gc
+#gcopts=-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
+#gcopts=-XX:+UseParallelOldGC
+#gcopts=-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC
+
+# Generates detailed logging on the JVM GC behavior. The service will
+# start in the configured service directory, so the log file will be in
+# that directory as well. The service directory is typically on local
+# disk, so that is where you need to look for this file.
+gcdebug=
+#gcdebug=-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:jvm_gc.log
+
+# all jvm args for query.
+jvmArgs=-server -Xmx${maxMem} -showversion ${gcopts} ${gcdebug} ${profiler} -Dlog4j.configuration=file:log4j.properties
+# -Dlog4j.debug
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.xml
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.xml (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/build.xml 2013-08-22 21:45:26 UTC (rev 7320)
@@ -0,0 +1,44 @@
+<!-- $Id: build.xml 5779 2011-12-14 18:17:54Z thompsonbry $ -->
+<!-- -->
+<!-- do "ant bundle-jar" in the parent directory first. -->
+<!-- -->
+<project name="gas" basedir=".">
+
+ <property file="build.properties" />
+
+ <path id="runtime.classpath">
+ <!-- The bigdata dependencies (for the nano-server). -->
+ <fileset dir="${bigdata.build.dir}/lib">
+ <include name="**/*.jar" />
+ </fileset>
+ </path>
+
+ <target name="start-sparql-server"
+ description="Start a small http server fronting for a bigdata database instance.">
+ <java classname="com.bigdata.rdf.sail.webapp.NanoSparqlServer"
+ fork="true" failonerror="true"
+ >
+ <arg line="${nanoServerPort} ${namespace} ${journalPropertyFile}" />
+ <!-- specify/override the journal file name. -->
+ <jvmarg line="${jvmArgs}" />
+ <classpath>
+ <path refid="runtime.classpath" />
+ </classpath>
+ </java>
+ </target>
+
+ <target name="run-gas-engine"
+ description="Run a GAS Engine performance test.">
+ <java classname="com.bigdata.rdf.graph.analytics.${analytic}"
+ fork="true" failonerror="true"
+ >
+ <arg line="-bufferMode ${bufferMode} -namespace ${namespace} -seed ${seed} -nsamples ${nsamples} -nthreads ${nthreads} ${journalPropertyFile}" />
+ <!-- specify/override the journal file name. -->
+ <jvmarg line="${jvmArgs}" />
+ <classpath>
+ <path refid="runtime.classpath" />
+ </classpath>
+ </java>
+ </target>
+
+</project>
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties 2013-08-22 21:45:26 UTC (rev 7320)
@@ -0,0 +1,76 @@
+# Default log4j configuration. See the individual classes for the
+# specific loggers, but generally they are named for the class in
+# which they are defined.
+
+# Default log4j configuration for testing purposes.
+#
+# You probably want to set the default log level to ERROR.
+#
+log4j.rootCategory=WARN, dest1
+#log4j.rootCategory=WARN, dest2
+
+# Loggers.
+# Note: logging here at INFO or DEBUG will significantly impact throughput!
+#log4j.logger.com.bigdata=INFO
+log4j.logger.com.bigdata.rdf.graph.impl.GASEngine=INFO
+
+#log4j.logger.com.bigdata.LRUNexus=INFO
+#log4j.logger.com.bigdata.rdf.sail.BigdataSail=INFO
+#log4j.logger.com.bigdata.rdf.sail.BigdataEvaluationStrategyImpl3=INFO
+#log4j.logger.com.bigdata.rdf.sail.webapp.NanoSparqlServer=INFO
+#log4j.logger.com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener=INFO
+#log4j.logger.com.bigdata.relation.accesspath.BlockingBuffer=ERROR
+#log4j.logger.com.bigdata.rdf.store.DataLoader=INFO
+
+# dest1
+log4j.appender.dest1=org.apache.log4j.ConsoleAppender
+log4j.appender.dest1.layout=org.apache.log4j.PatternLayout
+log4j.appender.dest1.layout.ConversionPattern=%-5p: %F:%L: %m%n
+#log4j.appender.dest1.layout.ConversionPattern=%-5p: %r %l: %m%n
+#log4j.appender.dest1.layout.ConversionPattern=%-5p: %m%n
+#log4j.appender.dest1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
+#log4j.appender.dest1.layout.ConversionPattern=%-4r(%d) [%t] %-5p %c(%l:%M) %x - %m%n
+
+# dest2 includes the thread name and elapsed milliseconds.
+# Note: %r is elapsed milliseconds.
+# Note: %t is the thread name.
+# See http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html
+log4j.appender.dest2=org.apache.log4j.ConsoleAppender
+log4j.appender.dest2.layout=org.apache.log4j.PatternLayout
+log4j.appender.dest2.layout.ConversionPattern=%-5p: %r %X{hostname} %X{serviceUUID} %X{taskname} %X{timestamp} %X{resources} %t %l: %m%n
+
+##
+# Rule execution log. This is a formatted log file (comma delimited).
+#log4j.logger.com.bigdata.relation.rule.eval.RuleLog=INFO,ruleLog
+log4j.additivity.com.bigdata.relation.rule.eval.RuleLog=false
+log4j.appender.ruleLog=org.apache.log4j.FileAppender
+log4j.appender.ruleLog.Threshold=ALL
+log4j.appender.ruleLog.File=rules.log
+log4j.appender.ruleLog.Append=true
+log4j.appender.ruleLog.BufferedIO=true
+log4j.appender.ruleLog.layout=org.apache.log4j.PatternLayout
+log4j.appender.ruleLog.layout.ConversionPattern=%m
+
+##
+# Summary query evaluation log (tab delimited file). Uncomment the next line to enable.
+#log4j.logger.com.bigdata.bop.engine.QueryLog=INFO,queryLog
+log4j.additivity.com.bigdata.bop.engine.QueryLog=false
+log4j.appender.queryLog=org.apache.log4j.FileAppender
+log4j.appender.queryLog.Threshold=ALL
+log4j.appender.queryLog.File=queryLog.csv
+log4j.appender.queryLog.Append=true
+log4j.appender.queryLog.BufferedIO=true
+log4j.appender.queryLog.layout=org.apache.log4j.PatternLayout
+log4j.appender.queryLog.layout.ConversionPattern=%m
+
+##
+# BOp run state trace (tab delimited file). Uncomment the next line to enable.
+#log4j.logger.com.bigdata.bop.engine.RunState$TableLog=INFO,queryRunStateLog
+log4j.additivity.com.bigdata.bop.engine.RunState$TableLog=false
+log4j.appender.queryRunStateLog=org.apache.log4j.FileAppender
+log4j.appender.queryRunStateLog.Threshold=ALL
+log4j.appender.queryRunStateLog.File=queryRunState.log
+log4j.appender.queryRunStateLog.Append=true
+log4j.appender.queryRunStateLog.BufferedIO=true
+log4j.appender.queryRunStateLog.layout=org.apache.log4j.PatternLayout
+log4j.appender.queryRunStateLog.layout.ConversionPattern=%m
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -4,6 +4,7 @@
import com.bigdata.rdf.graph.Factory;
import com.bigdata.rdf.graph.IGASContext;
import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.graph.impl.GASRunner;
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
@@ -185,4 +186,22 @@
}
+ /**
+ * Performance testing harness.
+ */
+ public static void main(final String[] args) throws Exception {
+
+ new GASRunner<BFS.VS, BFS.ES, Void>(args) {
+
+ @Override
+ protected IGASProgram<BFS.VS, BFS.ES, Void> newGASProgram() {
+
+ return new BFS();
+
+ }
+
+ }.call();
+
+ }
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -7,6 +7,7 @@
import com.bigdata.rdf.graph.GASUtil;
import com.bigdata.rdf.graph.IGASContext;
import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.graph.impl.GASRunner;
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
@@ -256,6 +257,24 @@
* {@link ISPO#s()}. The remote vertex is {@link ISPO#o()}.
* <p>
* {@inheritDoc}
+ *
+ * FIXME Test both variations on a variety of data sets and see which is
+ * better:
+ *
+ * <p>
+ * Zhisong wrote: In the original GASengine, the scatter operator only need
+ * to access the status of the source: src.changes.
+ *
+ * To check the status of destination, it needs to load destination data:
+ * dst.dist and edge data: e. And then check if new dist is different from
+ * the old value.
+ *
+ * Bryan wrote: I will have to think about this more. It sounds like it
+ * depends on the fan-out of the scatter at time t versus the fan-in of the
+ * gather at time t+1. The optimization might only benefit if a reasonable
+ * fraction of the destination vertices wind up NOT being retriggered. I
+ * will try on these variations in the Java code as well. *
+ * </p>
*/
@Override
public void scatter(final IGASContext<SSSP.VS, SSSP.ES, Integer> ctx,
@@ -291,4 +310,22 @@
}
+ /**
+ * Performance test harness.
+ */
+ public static void main(final String[] args) throws Exception {
+
+ new GASRunner<SSSP.VS, SSSP.ES, Integer>(args) {
+
+ @Override
+ protected IGASProgram<SSSP.VS, SSSP.ES, Integer> newGASProgram() {
+
+ return new SSSP();
+
+ }
+
+ }.call();
+
+ }
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -304,7 +304,7 @@
* </pre>
* Parameterize execution runs againt these runtime options!
*/
- private final int nparallel = 1;
+ private final int nthreads;
/**
*
@@ -333,7 +333,8 @@
* through the history index).
*/
public GASEngine(final IIndexManager indexManager, final String namespace,
- final long timestamp, final IGASProgram<VS, ES, ST> program) {
+ final long timestamp, final IGASProgram<VS, ES, ST> program,
+ final int nthreads) {
if (indexManager == null)
throw new IllegalArgumentException();
@@ -341,6 +342,9 @@
if (program == null)
throw new IllegalArgumentException();
+ if (nthreads <= 0)
+ throw new IllegalArgumentException();
+
this.indexManager = indexManager;
this.namespace = namespace;
@@ -349,6 +353,8 @@
this.program = program;
+ this.nthreads = nthreads;
+
this.executorService = indexManager.getExecutorService();
this.vsf = program.getVertexStateFactory();
@@ -926,11 +932,11 @@
private Callable<Long> newFrontierStrategy(
final VertexTaskFactory<Long> taskFactory) {
- if (nparallel == 1)
+ if (nthreads == 1)
return new RunInCallersThreadFrontierStrategy(taskFactory);
return new LatchedExecutorFrontierStrategy(taskFactory,
- executorService, nparallel);
+ executorService, nthreads);
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -81,22 +81,23 @@
@SuppressWarnings("rawtypes")
public static IV getRandomVertex(final Random r, final AbstractTripleStore kb) {
+ /*
+ * TODO This assumes a local, non-sharded index. The specific
+ * approach to identifying a starting vertex relies on the
+ * ILinearList API. If the caller is specifying the starting vertex
+ * then we do not need to do this.
+ *
+ * TODO The bias here is towards vertices having more out-edges
+ * and/or attributes since the sample is uniform over the triples in
+ * the index and a triple may be either an edge or an attribute
+ * value (or a link attribute using RDR).
+ */
+ final BTree ndx = (BTree) kb.getSPORelation().getPrimaryIndex();
+
// Select a random starting vertex.
IV startingVertex = null;
{
- /*
- * TODO This assumes a local, non-sharded index. The specific
- * approach to identifying a starting vertex relies on the
- * ILinearList API. If the caller is specifying the starting vertex
- * then we do not need to do this.
- *
- * TODO The bias here is towards vertices having more out-edges
- * and/or attributes since the sample is uniform over the triples in
- * the index and a triple may be either an edge or an attribute
- * value (or a link attribute using RDR).
- */
- final BTree ndx = (BTree) kb.getSPORelation().getPrimaryIndex();
-
+
// Truncate at MAX_INT.
final int size = (int) Math
.min(ndx.rangeCount(), Integer.MAX_VALUE);
@@ -143,8 +144,9 @@
}
if (startingVertex == null)
- throw new RuntimeException("No starting vertex");
-
+ throw new RuntimeException("No starting vertex: nedges="
+ + ndx.rangeCount());
+
return startingVertex;
}
Copied: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java (from rev 7305, branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/PerformanceTest.java)
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -0,0 +1,536 @@
+package com.bigdata.rdf.graph.impl;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.util.LinkedHashSet;
+import java.util.Properties;
+import java.util.Random;
+import java.util.concurrent.Callable;
+
+import org.apache.log4j.Logger;
+import org.openrdf.rio.RDFFormat;
+
+import com.bigdata.Banner;
+import com.bigdata.journal.BufferMode;
+import com.bigdata.journal.ITx;
+import com.bigdata.journal.Journal;
+import com.bigdata.rdf.graph.IGASEngine;
+import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.internal.IV;
+import com.bigdata.rdf.rio.LoadStats;
+import com.bigdata.rdf.sail.BigdataSail;
+import com.bigdata.rdf.store.AbstractTripleStore;
+
+/**
+ * Base class for running performance tests.
+ *
+ * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
+ *
+ * TODO Need a different driver if the algorithm always visits all vertices.
+ */
+abstract public class GASRunner<VS, ES, ST> implements Callable<GASStats> {
+
+ private static final Logger log = Logger.getLogger(GASRunner.class);
+
+ /**
+ * Random number generated used for sampling the starting vertices.
+ */
+ private final Random r;
+
+ /**
+ * The #of random starting vertices to use.
+ */
+ private final int nsamples;
+
+ /**
+ * The #of threads to use for GATHER and SCATTER operators.
+ */
+ private final int nthreads;
+
+ /**
+ * The property file
+ */
+ private final String propertyFile;
+
+ /**
+ * When non-<code>null</code>, this overrides the current buffer mode.
+ */
+ private final BufferMode bufferModeOverride;
+
+ /**
+ * When non-<code>null</code>, this overrides the KB namespace.
+ */
+ private final String namespaceOverride;
+
+ /**
+ * When non-<code>null</code>, a list of zero or more resources to be
+ * loaded. The resources will be searched for as URLs, on the CLASSPATH, and
+ * in the file system.
+ */
+ private final String[] loadSet;
+
+ /**
+ * Print the optional message on stderr, print the usage information on
+ * stderr, and then force the program to exit with the given status code.
+ *
+ * @param status
+ * The status code.
+ * @param msg
+ * The optional message
+ */
+ private static void usage(final int status, final String msg) {
+
+ if (msg != null) {
+
+ System.err.println(msg);
+
+ }
+
+ System.err.println("[options] propertyFile");
+
+ System.exit(status);
+
+ }
+
+ /**
+ * Run a GAS analytic against some data set.
+ *
+ * @param args
+ * USAGE:<br/>
+ * <code>(options) propertyFile</code>
+ * <p>
+ * <i>Where:</i>
+ * <dl>
+ * <dt>propertyFile</dt>
+ * <dd>A java properties file for a standalone {@link Journal}.</dd>
+ * </dl>
+ * and <i>options</i> are any of:
+ * <dl>
+ * <dt>-nthreads</dt>
+ * <dd>The #of threads which will be used for GATHER and SCATTER
+ * operations.</dd>
+ * <dt>-nsamples</dt>
+ * <dd>The #of random sample starting vertices that will be
+ * selected. The algorithm will be run ONCE for EACH sampled
+ * vertex.</dd>
+ * <dt>-seed</dt>
+ * <dd>The seed for the random number generator (default is
+ * <code>217L</code>).</dd>
+ * <dt>-namespace</dt>
+ * <dd>The namespace of the default SPARQL endpoint (the
+ * namespace will be <code>kb</code> if none was specified when
+ * the triple/quad store was created).</dd>
+ * <dt>-load</dt>
+ * <dd>Loads the named resource. This option may appear multiple
+ * times. The resources will be searched for as URLs, on the
+ * CLASSPATH, and in the file system.</dd>
+ * <dt>-bufferMode</dt>
+ * <dd>Overrides the {@link BufferMode} (if any) specified in the
+ * <code>propertyFile</code>.</dd>
+ * </p>
+ */
+ public GASRunner(final String[] args) {
+
+ Banner.banner();
+
+ long seed = 217L;
+ int nsamples = 100;
+ int nthreads = 4;
+ BufferMode bufferMode = null; // override only.
+ String namespace = "kb";
+ // Set of files to load (optional).
+ LinkedHashSet<String> loadSet = new LinkedHashSet<String>();
+
+ /*
+ * Handle all arguments starting with "-". These should appear before
+ * any non-option arguments to the program.
+ */
+ int i = 0;
+ while (i < args.length) {
+ final String arg = args[i];
+ if (arg.startsWith("-")) {
+ if (arg.equals("-seed")) {
+ seed = Long.valueOf(args[++i]);
+ } else if (arg.equals("-nsamples")) {
+ final String s = args[++i];
+ nsamples = Integer.valueOf(s);
+ if (nsamples <= 0) {
+ usage(1/* status */,
+ "-nsamples must be positive, not: " + s);
+ }
+ } else if (arg.equals("-nthreads")) {
+ final String s = args[++i];
+ nthreads = Integer.valueOf(s);
+ if (nthreads < 0) {
+ usage(1/* status */,
+ "-nthreads must be non-negative, not: " + s);
+ }
+ } else if (arg.equals("-bufferMode")) {
+ final String s = args[++i];
+ bufferMode = BufferMode.valueOf(s);
+ } else if (arg.equals("-namespace")) {
+ final String s = args[++i];
+ namespace = s;
+ } else if (arg.equals("-load")) {
+ final String s = args[++i];
+ loadSet.add(s);
+ } else {
+ usage(1/* status */, "Unknown argument: " + arg);
+ }
+ } else {
+ break;
+ }
+ i++;
+ }
+
+ /*
+ * Check for the remaining (required) argument(s).
+ */
+ final int nremaining = args.length - i;
+ if (nremaining != 1) {
+ /*
+ * There are either too many or too few arguments remaining.
+ */
+ usage(1/* status */, nremaining < 1 ? "Too few arguments."
+ : "Too many arguments");
+ }
+
+ /*
+ * Property file.
+ */
+ this.propertyFile = args[i++];
+
+ /*
+ * Assign parsed values.
+ */
+ this.nsamples = nsamples;
+ this.nthreads = nthreads;
+ this.r = new Random(seed);
+ this.namespaceOverride = namespace;
+ this.bufferModeOverride = bufferMode;
+ this.loadSet = loadSet.isEmpty() ? null : loadSet
+ .toArray(new String[loadSet.size()]);
+
+ }
+
+ /**
+ * Return a sample of random vertices.
+ *
+ * @param kb
+ */
+ @SuppressWarnings("rawtypes")
+ protected IV[] getRandomSamples(final AbstractTripleStore kb) {
+
+ return GASGraphUtil.getRandomSample(r, kb, nsamples);
+
+ }
+
+ /**
+ * Return the {@link IGASProgram} to be evaluated.
+ */
+ abstract protected IGASProgram<VS, ES, ST> newGASProgram();
+
+ private Properties getProperties(final String resource) throws IOException {
+
+ if (log.isInfoEnabled())
+ log.info("Reading properties: " + resource);
+
+ InputStream is = null;
+ try {
+
+ // try the classpath
+ is = getClass().getResourceAsStream(resource);
+
+ if (is != null) {
+
+ } else {
+
+ // try file system.
+ final File file = new File(resource);
+
+ if (file.exists()) {
+
+ is = new FileInputStream(file);
+
+ } else {
+
+ throw new IOException("Could not locate resource: "
+ + resource);
+
+ }
+
+ }
+
+ /*
+ * Obtain a buffered reader on the input stream.
+ */
+
+ final Properties properties = new Properties();
+
+ final Reader reader = new BufferedReader(new InputStreamReader(is));
+
+ try {
+
+ properties.load(reader);
+
+ } finally {
+
+ try {
+
+ reader.close();
+
+ } catch (Throwable t) {
+
+ log.error(t);
+
+ }
+
+ }
+
+ return properties;
+
+ } finally {
+
+ if (is != null) {
+
+ try {
+
+ is.close();
+
+ } catch (Throwable t) {
+
+ log.error(t);
+
+ }
+
+ }
+
+ }
+
+ }
+
+ /**
+ * Run the test.
+ * <p>
+ * This provides a safe pattern for either loading data into a temporary
+ * journal, which is then destroyed, or using an exiting journal and
+ * optionally loading in some data set. When we load the data the journal is
+ * destroyed afterwards and when the journal is pre-existing and we neither
+ * load the data nor destroy the journal. This has to do with the effective
+ * BufferMode (if transient) and whether the file is specified and whether a
+ * temporary file is created (CREATE_TEMP_FILE). If we do our own file
+ * create if the effective buffer mode is non-transient, then we can get all
+ * this information.
+ */
+ public GASStats call() throws Exception {
+
+ final Properties properties = getProperties(propertyFile);
+
+ /*
+ * Note: Allows override through the command line argument. The default
+ * is otherwise the default and the value in the properties file (if
+ * any) will be used unless it is overridden.
+ */
+ final BufferMode bufferMode = this.bufferModeOverride == null ? BufferMode
+ .valueOf(properties.getProperty(Journal.Options.BUFFER_MODE,
+ Journal.Options.DEFAULT_BUFFER_MODE)) : this.bufferModeOverride;
+
+ final boolean isTransient = !bufferMode.isStable();
+
+ final boolean isTemporary;
+ if (isTransient) {
+
+ isTemporary = true;
+
+ } else {
+
+ final String fileStr = properties.getProperty(Journal.Options.FILE);
+
+ if (fileStr == null) {
+
+ /*
+ * We will use a temporary file that we create here. The journal
+ * will be destroyed below.
+ */
+ isTemporary = true;
+
+ final File tmpFile = File.createTempFile(
+ GASRunner.class.getSimpleName(), Journal.Options.JNL);
+
+ // Set this on the Properties so it will be used by the jnl.
+ properties.setProperty(Journal.Options.FILE,
+ tmpFile.getAbsolutePath());
+
+ } else {
+
+ // real file is named.
+ isTemporary = false;
+
+ }
+
+ }
+
+ // The effective KB name.
+ final String namespace = this.namespaceOverride == null ? properties
+ .getProperty(BigdataSail.Options.NAMESPACE,
+ BigdataSail.Options.DEFAULT_NAMESPACE) : this.namespaceOverride;
+
+ /*
+ * TODO Could start NSS and use SPARQL UPDATE "LOAD" to load the data.
+ * That exposes the SPARQL end point for other purposes during the test.
+ * Is this useful? It could also let us run the GASEngine on a remote
+ * service (submit a callable to an HA server or define a REST API for
+ * submitting these GAS algorithms).
+ */
+ final Journal jnl = new Journal(properties);
+
+ try {
+
+ // Locate/create KB.
+ {
+ final AbstractTripleStore kb;
+ if (isTemporary) {
+
+ kb = BigdataSail.createLTS(jnl, properties);
+
+ } else {
+
+ final AbstractTripleStore tmp = (AbstractTripleStore) jnl
+ .getResourceLocator().locate(namespace,
+ ITx.UNISOLATED);
+
+ if (tmp == null) {
+
+ // create.
+ kb = BigdataSail.createLTS(jnl, properties);
+
+ } else {
+
+ kb = tmp;
+
+ }
+
+ }
+ }
+
+ /*
+ * Load data sets.
+ */
+ if (isTemporary || (loadSet != null && loadSet.length > 0)) {
+
+ loadFiles(jnl, namespace, loadSet);
+
+ }
+
+ return runAnalytic(jnl, namespace);
+
+ } finally {
+
+ jnl.close();
+
+ }
+
+ }
+
+ /**
+ * Load files into the journal.
+ *
+ * @param jnl
+ * The journal.
+ * @param namespace
+ * The KB namespace.
+ * @param loadSet
+ * The files.
+ * @throws IOException
+ */
+ private LoadStats loadFiles(final Journal jnl, final String namespace,
+ final String[] loadSet) throws IOException {
+
+ // final String path = "bigdata-rdf/src/resources/data/foaf";
+ // final String dataFile[] = new String[] {//
+ // path + "/data-0.nq.gz",//
+ // path + "/data-1.nq.gz",//
+ // path + "/data-2.nq.gz",//
+ // path + "/data-3.nq.gz",//
+ // };
+ final String baseUrl[] = new String[loadSet.length];
+ for (int i = 0; i < loadSet.length; i++) {
+ baseUrl[i] = "file:" + loadSet[i];
+ }
+ // fall back RDFFormat.
+ final RDFFormat[] rdfFormat = new RDFFormat[loadSet.length];
+ for (int i = 0; i < loadSet.length; i++) {
+ rdfFormat[i] = RDFFormat.RDFXML;
+ }
+
+ // Load data using the unisolated view.
+ final AbstractTripleStore kb = (AbstractTripleStore) jnl
+ .getResourceLocator().locate(namespace, ITx.UNISOLATED);
+
+ final LoadStats stats = kb.getDataLoader().loadData(loadSet, baseUrl,
+ rdfFormat);
+
+ System.out.println(stats.toString());
+
+ return stats;
+
+ }
+
+ /**
+ * Run the analytic.
+ *
+ * @param jnl
+ * The journal.
+ * @param namespace
+ * The KB namespace.
+ * @return
+ * @throws Exception
+ */
+ private GASStats runAnalytic(final Journal jnl, final String namespace)
+ throws Exception {
+
+ @SuppressWarnings("rawtypes")
+ final IV[] samples;
+ {
+ /*
+ * Use a read-only view (sampling depends on access to the BTree
+ * rather than the ReadCommittedIndex).
+ */
+ final AbstractTripleStore kb = (AbstractTripleStore) jnl
+ .getResourceLocator().locate(namespace,
+ jnl.getLastCommitTime());
+
+ samples = getRandomSamples(kb);
+
+ }
+
+ final IGASEngine<VS, ES, ST> gasEngine = new GASEngine<VS, ES, ST>(jnl,
+ namespace, ITx.READ_COMMITTED, newGASProgram(), nthreads);
+
+ final GASStats total = new GASStats();
+
+ for (int i = 0; i < samples.length; i++) {
+
+ @SuppressWarnings("rawtypes")
+ final IV startingVertex = samples[i];
+
+ gasEngine.init(startingVertex);
+
+ // TODO Pure interface for this.
+ total.add((GASStats) gasEngine.call());
+
+ }
+
+ // Total over all sampled vertices.
+ System.out.println("TOTAL: " + total);
+
+ return total;
+
+ }
+
+}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -24,12 +24,9 @@
package com.bigdata.rdf.graph.analytics;
import com.bigdata.journal.ITx;
-import com.bigdata.journal.Journal;
import com.bigdata.rdf.graph.AbstractGraphTestCase;
import com.bigdata.rdf.graph.IGASEngine;
-import com.bigdata.rdf.graph.IGASProgram;
import com.bigdata.rdf.graph.impl.GASEngine;
-import com.bigdata.rdf.graph.impl.PerformanceTest;
/**
* Test class for Breadth First Search (BFS) traversal.
@@ -54,7 +51,7 @@
final IGASEngine<BFS.VS, BFS.ES, Void> gasEngine = new GASEngine<BFS.VS, BFS.ES, Void>(
sail.getDatabase().getIndexManager(), sail.getDatabase()
- .getNamespace(), ITx.READ_COMMITTED, new BFS());
+ .getNamespace(), ITx.READ_COMMITTED, new BFS(), 1/* nthreads */);
// Initialize the froniter.
gasEngine.init(p.mike.getIV());
@@ -76,23 +73,4 @@
}
- /**
- * Test routine to running against a {@link Journal} in which some data set
- * has already been loaded.
- */
- public static void main(final String[] args) throws Exception {
-
- new PerformanceTest<BFS.VS, BFS.ES, Void>(args) {
-
- @Override
- protected IGASProgram<BFS.VS, BFS.ES, Void> newGASProgram() {
-
- return new BFS();
-
- }
-
- }.call();
-
- }
-
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -24,12 +24,9 @@
package com.bigdata.rdf.graph.analytics;
import com.bigdata.journal.ITx;
-import com.bigdata.journal.Journal;
import com.bigdata.rdf.graph.AbstractGraphTestCase;
import com.bigdata.rdf.graph.IGASEngine;
-import com.bigdata.rdf.graph.IGASProgram;
import com.bigdata.rdf.graph.impl.GASEngine;
-import com.bigdata.rdf.graph.impl.PerformanceTest;
/**
* Test class for SSP traversal.
@@ -52,7 +49,7 @@
final IGASEngine<SSSP.VS, SSSP.ES, Integer> gasEngine = new GASEngine<SSSP.VS, SSSP.ES, Integer>(
sail.getDatabase().getIndexManager(), sail.getDatabase()
- .getNamespace(), ITx.READ_COMMITTED, new SSSP());
+ .getNamespace(), ITx.READ_COMMITTED, new SSSP(), 1/* nthreads */);
// Initialize the froniter.
gasEngine.init(p.mike.getIV());
@@ -74,23 +71,4 @@
}
- /**
- * Test routine to running against a {@link Journal} in which some data set
- * has already been loaded.
- */
- public static void main(final String[] args) throws Exception {
-
- new PerformanceTest<SSSP.VS, SSSP.ES, Integer>(args) {
-
- @Override
- protected IGASProgram<SSSP.VS, SSSP.ES, Integer> newGASProgram() {
-
- return new SSSP();
-
- }
-
- }.call();
-
- }
-
}
Deleted: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/PerformanceTest.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/PerformanceTest.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/PerformanceTest.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -1,342 +0,0 @@
-package com.bigdata.rdf.graph.impl;
-
-import java.io.BufferedReader;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.InputStreamReader;
-import java.io.Reader;
-import java.util.Properties;
-import java.util.Random;
-import java.util.concurrent.Callable;
-
-import org.apache.log4j.Logger;
-import org.openrdf.rio.RDFFormat;
-
-import com.bigdata.journal.BufferMode;
-import com.bigdata.journal.ITx;
-import com.bigdata.journal.Journal;
-import com.bigdata.rdf.graph.IGASEngine;
-import com.bigdata.rdf.graph.IGASProgram;
-import com.bigdata.rdf.internal.IV;
-import com.bigdata.rdf.sail.BigdataSail;
-import com.bigdata.rdf.store.AbstractTripleStore;
-
-/**
- * Base class for running performance tests.
- *
- * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- *
- * TODO Need a different driver if the algorithm always visits all vertices.
- *
- * TODO Parameterize for sampling of vertices.
- */
-abstract public class PerformanceTest<VS, ES, ST> implements Callable<Void> {
-
- private static final Logger log = Logger.getLogger(PerformanceTest.class);
-
- private final String[] args;
- private final Random r;
-
- /*
- * TODO args[] are ignored. Should parse for [nsamples], [seed], the data
- * file (to load when run), memstore (must load when run) verus RWStore (for
- * preexisting data).
- */
- public PerformanceTest(final String[] args) {
-
- this.args = args;
-
- this.r = new Random(seed());
-
- }
-
- protected long seed() {
-
- return 217L;
-
- }
-
- protected int sampleSize() {
-
- return 100;
-
- }
-
- /**
- *
- * @param kb
- */
- protected IV[] getRandomSamples(final AbstractTripleStore kb) {
-
- return GASGraphUtil.getRandomSample(r, kb, sampleSize());
-
- }
-
- /**
- * Return the {@link IGASProgram} to be evaluated.
- */
- abstract protected IGASProgram<VS, ES, ST> newGASProgram();
-
- private Properties getProperties(final String resource) throws IOException {
-
- if (log.isInfoEnabled())
- log.info("Reading properties: " + resource);
-
- InputStream is = null;
- try {
-
- // try the classpath
- is = getClass().getResourceAsStream(resource);
-
- if (is != null) {
-
- } else {
-
- // try file system.
- final File file = new File(resource);
-
- if (file.exists()) {
-
- is = new FileInputStream(file);
-
- } else {
-
- throw new IOException("Could not locate resource: " + resource);
-
- }
-
- }
-
- /*
- * Obtain a buffered reader on the input stream.
- */
-
- final Properties properties = new Properties();
-
- final Reader reader = new BufferedReader(new InputStreamReader(is));
-
- try {
-
- properties.load(reader);
-
- } finally {
-
- try {
-
- reader.close();
-
- } catch (Throwable t) {
-
- log.error(t);
-
- }
-
- }
-
- return properties;
-
- } finally {
-
- if (is != null) {
-
- try {
-
- is.close();
-
- } catch (Throwable t) {
-
- log.error(t);
-
- }
-
- }
-
- }
-
- }
-
- /**
- * Run the test.
- */
- public Void call() throws Exception {
-
- /*
- * The property file
- *
- * TODO Use different files for MemStore and RWStore (command line arg)
- */
- final String propertyFile = "bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/RWStore.properties";
-
- final Properties properties = getProperties(propertyFile);
-
- final BufferMode bufferMode = BufferMode.valueOf(properties
- .getProperty(Journal.Options.BUFFER_MODE,
- Journal.Options.DEFAULT_BUFFER_MODE));
-
- final boolean isTransient = !bufferMode.isStable();
-
- final boolean isTemporary;
- if (isTransient) {
-
- isTemporary = true;
-
- } else {
-
- final String fileStr = properties.getProperty(Journal.Options.FILE);
-
- if (fileStr == null) {
-
- /*
- * We will use a temporary file that we create here. The journal
- * will be destroyed below.
- */
- isTemporary = true;
-
- final File tmpFile = File.createTempFile(
- PerformanceTest.class.getSimpleName(),
- Journal.Options.JNL);
-
- // Set this on the Properties so it will be used by the jnl.
- properties.setProperty(Journal.Options.FILE,
- tmpFile.getAbsolutePath());
-
- } else {
-
- // real file is named.
- isTemporary = false;
-
- }
-
- }
-
- // The effective KB name.
- final String namespace = properties.getProperty(
- BigdataSail.Options.NAMESPACE,
- BigdataSail.Options.DEFAULT_NAMESPACE);
-
- /*
- * TODO Could start NSS and use SPARQL UPDATE "LOAD" to load the data.
- * That exposes the SPARQL end point for other purposes during the test.
- * Is this useful? It could also let us run the GASEngine on a remote
- * service (submit a callable to an HA server or define a REST API for
- * submitting these GAS algorithms).
- *
- * TODO we need a safe pattern for when we load the data the journal is
- * destroyed afterwards and when the journal is pre-existing and we
- * neither load the data nor destroy the journal. This has to do with
- * the effective BufferMode (if transient) and whether the file is
- * specified and whether a temporary file is created (CREATE_TEMP_FILE).
- * If we do our own file create if the effective buffer mode is
- * non-transient, then we can get all this information.
- */
- final Journal jnl = new Journal(properties);
-
- try {
-
- // Locate/create KB.
- {
- final AbstractTripleStore kb;
- if (isTemporary) {
-
- kb = BigdataSail.createLTS(jnl, properties);
-
- } else {
-
- final AbstractTripleStore tmp = (AbstractTripleStore) jnl
- .getResourceLocator().locate(namespace,
- ITx.UNISOLATED);
-
- if (tmp == null) {
-
- // create.
- kb = BigdataSail.createLTS(jnl, properties);
-
- } else {
-
- kb = tmp;
-
- }
-
- }
- }
-
- /*
- * Load data sets.
- */
- if (isTemporary) {
-
- final String path = "bigdata-rdf/src/resources/data/foaf";
- final String dataFile[] = new String[] {//
- path + "/data-0.nq.gz",//
- path + "/data-1.nq.gz",//
- path + "/data-2.nq.gz",//
- path + "/data-3.nq.gz",//
- };
- final String baseUrl[] = new String[dataFile.length];
- for (int i = 0; i < dataFile.length; i++) {
- baseUrl[i] = "file:" + dataFile[i];
- }
- final RDFFormat[] rdfFormat = new RDFFormat[] {//
- RDFFormat.NQUADS,//
- RDFFormat.NQUADS,//
- RDFFormat.NQUADS,//
- RDFFormat.NQUADS,//
- };
-
- // Load data using the unisolated view.
- final AbstractTripleStore kb = (AbstractTripleStore) jnl
- .getResourceLocator().locate(namespace, ITx.UNISOLATED);
-
- kb.getDataLoader().loadData(dataFile, baseUrl, rdfFormat);
-
- }
-
- final IGASEngine<VS, ES, ST> gasEngine = new GASEngine<VS, ES, ST>(
- jnl, namespace, ITx.READ_COMMITTED, newGASProgram());
-
- @SuppressWarnings("rawtypes")
- final IV[] samples;
- {
- /*
- * Use a read-only view (sampling depends on access to the BTree
- * rather than the ReadCommittedIndex).
- */
- final AbstractTripleStore kb = (AbstractTripleStore) jnl
- .getResourceLocator().locate(namespace,
- jnl.getLastCommitTime());
-
- samples = getRandomSamples(kb);
-
- }
-
- final GASStats total = new GASStats();
-
- for (int i = 0; i < samples.length; i++) {
-
- @SuppressWarnings("rawtypes")
- final IV startingVertex = samples[i];
-
- gasEngine.init(startingVertex);
-
- // TODO Pure interface for this.
- total.add((GASStats)gasEngine.call());
-
- }
-
- // Total over all sampled vertices.
- System.out.println("TOTAL: " + total);
-
- // Done.
- return null;
-
- } finally {
-
- jnl.close();
-
- }
-
- }
-
-}
Deleted: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/RWStore.properties
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/RWStore.properties 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/RWStore.properties 2013-08-22 21:45:26 UTC (rev 7320)
@@ -1,35 +0,0 @@
-#
-# Note: These options are applied when the journal and the triple store are
-# first created.
-
-##
-## Journal options.
-##
-
-# The backing file. This contains all your data. You want to put this someplace
-# safe. The default locator will wind up in the directory from which you start
-# your servlet container.
-com.bigdata.journal.AbstractJournal.file=bigdata.jnl
-
-# The persistence engine. Use 'Disk' for the WORM or 'DiskRW' for the RWStore.
-com.bigdata.journal.AbstractJournal.bufferMode=DiskRW
-#com.bigdata.journal.AbstractJournal.bufferMode=MemStore
-
-com.bigdata.btree.writeRetentionQueue.capacity=4000
-com.bigdata.btree.BTree.branchingFactor=128
-
-# 200M initial extent.
-com.bigdata.journal.AbstractJournal.initialExtent=209715200
-com.bigdata.journal.AbstractJournal.maximumExtent=209715200
-
-##
-## Setup for QUADS mode without the full text index.
-##
-com.bigdata.rdf.sail.truthMaintenance=false
-com.bigdata.rdf.store.AbstractTripleStore.quads=true
-com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false
-com.bigdata.rdf.store.AbstractTripleStore.textIndex=false
-com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms
-#com.bigdata.rdf.store.AbstractTripleStore.inlineDateTimes=true
-
-com.bigdata.rdf.rio.RDFParserOptions.stopAtFirstError=false
\ No newline at end of file
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java 2013-08-22 19:21:47 UTC (rev 7319)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java 2013-08-22 21:45:26 UTC (rev 7320)
@@ -244,7 +244,7 @@
final IGASEngine<Set<ISPO>, Set<ISPO>, Set<ISPO>> gasEngine = new GASEngine<Set<ISPO>, Set<ISPO>, Set<ISPO>>(
sail.getDatabase().getIndexManager(), sail.getDatabase()
.getNamespace(), ITx.READ_COMMITTED,
- new MockGASProgram(gatherEdges));
+ new MockGASProgram(gatherEdges), 1/* nthreads */);
// Initialize the froniter.
gasEngine.init(startingVertex);
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <tho...@us...> - 2013-08-23 12:56:18
|
Revision: 7322
http://bigdata.svn.sourceforge.net/bigdata/?rev=7322&view=rev
Author: thompsonbry
Date: 2013-08-23 12:56:10 +0000 (Fri, 23 Aug 2013)
Log Message:
-----------
improving the GASRunner environment.
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties 2013-08-23 12:05:50 UTC (rev 7321)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-perf/gas/log4j.properties 2013-08-23 12:56:10 UTC (rev 7322)
@@ -12,7 +12,8 @@
# Loggers.
# Note: logging here at INFO or DEBUG will significantly impact throughput!
#log4j.logger.com.bigdata=INFO
-log4j.logger.com.bigdata.rdf.graph.impl.GASEngine=INFO
+#log4j.logger.com.bigdata.rdf.graph.impl.GASEngine=INFO
+log4j.logger.com.bigdata.rdf.graph.impl.GASRunner=INFO
#log4j.logger.com.bigdata.LRUNexus=INFO
#log4j.logger.com.bigdata.rdf.sail.BigdataSail=INFO
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java 2013-08-23 12:05:50 UTC (rev 7321)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASGraphUtil.java 2013-08-23 12:56:10 UTC (rev 7322)
@@ -27,6 +27,11 @@
private static final Logger log = Logger.getLogger(GASGraphUtil.class);
/**
+ * Return a sample (without duplicates) of vertices from the graph.
+ * <p>
+ * Note: This sampling procedure has a bias in favor of the vertices with
+ * the most edges and properties (vertices are choosen randomly in
+ * proportion to the #of edges and properties for the vertex).
*
* @param desiredSampleSize
* The desired sample size.
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-08-23 12:05:50 UTC (rev 7321)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-08-23 12:56:10 UTC (rev 7322)
@@ -38,6 +38,11 @@
private static final Logger log = Logger.getLogger(GASRunner.class);
/**
+ * The seed used for the random number generator.
+ */
+ private final long seed;
+
+ /**
* Random number generated used for sampling the starting vertices.
*/
private final Random r;
@@ -208,26 +213,17 @@
/*
* Assign parsed values.
*/
+ this.seed = seed;
this.nsamples = nsamples;
this.nthreads = nthreads;
- this.r = new Random(seed);
this.namespaceOverride = namespace;
this.bufferModeOverride = bufferMode;
this.loadSet = loadSet.isEmpty() ? null : loadSet
.toArray(new String[loadSet.size()]);
- }
+ // Setup the random number generator.
+ this.r = new Random(seed);
- /**
- * Return a sample of random vertices.
- *
- * @param kb
- */
- @SuppressWarnings("rawtypes")
- protected IV[] getRandomSamples(final AbstractTripleStore kb) {
-
- return GASGraphUtil.getRandomSample(r, kb, nsamples);
-
}
/**
@@ -490,25 +486,33 @@
* The KB namespace.
* @return
* @throws Exception
+ *
+ * TODO What happened to the predicate summary/histogram/distribution code?
+ *
+ * TODO Are we better off using sampling based on distinct vertices or with
+ * a bais based on the #of edges for those vertices.
*/
private GASStats runAnalytic(final Journal jnl, final String namespace)
throws Exception {
+ /*
+ * Use a read-only view (sampling depends on access to the BTree rather
+ * than the ReadCommittedIndex).
+ */
+ final AbstractTripleStore kb = (AbstractTripleStore) jnl
+ .getResourceLocator()
+ .locate(namespace, jnl.getLastCommitTime());
+
@SuppressWarnings("rawtypes")
- final IV[] samples;
- {
- /*
- * Use a read-only view (sampling depends on access to the BTree
- * rather than the ReadCommittedIndex).
- */
- final AbstractTripleStore kb = (AbstractTripleStore) jnl
- .getResourceLocator().locate(namespace,
- jnl.getLastCommitTime());
+ final IV[] samples = GASGraphUtil.getRandomSample(r, kb, nsamples);
- samples = getRandomSamples(kb);
+ // total #of edges in that graph.
+ final long nedges = kb.getStatementCount();
- }
-
+ /*
+ * Setup and run over those samples.
+ */
+
final IGASEngine<VS, ES, ST> gasEngine = new GASEngine<VS, ES, ST>(jnl,
namespace, ITx.READ_COMMITTED, newGASProgram(), nthreads);
@@ -522,12 +526,23 @@
gasEngine.init(startingVertex);
// TODO Pure interface for this.
- total.add((GASStats) gasEngine.call());
+ final GASStats stats = (GASStats) gasEngine.call();
+ total.add(stats);
+
+ if (log.isInfoEnabled()) {
+ log.info("Run complete: vertex[" + i + "] of " + samples.length
+ + " : startingVertex=" + startingVertex
+ + ", stats(sample)=" + stats);
+ }
+
}
-
+
// Total over all sampled vertices.
- System.out.println("TOTAL: " + total);
+ System.out.println("TOTAL: nseed=" + seed + ", nsamples=" + nsamples
+ + ", nthreads=" + nthreads + ", bufferMode="
+ + jnl.getBufferStrategy().getBufferMode() + ", edges(kb)="
+ + nedges + ", stats(total)=" + total);
return total;
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <tho...@us...> - 2013-08-24 11:15:20
|
Revision: 7330
http://bigdata.svn.sourceforge.net/bigdata/?rev=7330&view=rev
Author: thompsonbry
Date: 2013-08-24 11:15:06 +0000 (Sat, 24 Aug 2013)
Log Message:
-----------
Optimized GASEngine to use the B+Tree iterator without requiring the materialization of the ISPO tuples into an array. This should avoid range counts, allocations of the arrays, and drive the index scan directly from the calling thread (rather than potentially using a producer/consumer pattern that is built into the AccessPath class for large key range scans).
Optimized potential hot spot in AbstractTripleStore.getSPORelation() using AtomicReference and double-checked locking pattern (was potentially contended on synchronized(this)), which could have had an impact on the GAS parallelism.
Made IStriterator implement ICloseableIterator.
See #629
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/ChunkedStriterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/IStriterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/Striterator.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/store/AbstractTripleStore.java
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/ChunkedStriterator.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/ChunkedStriterator.java 2013-08-24 00:40:01 UTC (rev 7329)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/ChunkedStriterator.java 2013-08-24 11:15:06 UTC (rev 7330)
@@ -50,12 +50,12 @@
public class ChunkedStriterator<I extends IChunkedIterator<E>, E> extends
Striterator<I, E> implements IChunkedStriterator<I, E> {
- public ChunkedStriterator(I src) {
-
- super( src );
-
+ public ChunkedStriterator(final I src) {
+
+ super(src);
+
this.src = src;
-
+
}
/**
@@ -65,8 +65,7 @@
* @param src
* The source iterator.
*/
- @SuppressWarnings("unchecked")
- public ChunkedStriterator(Iterator<E> src) {
+ public ChunkedStriterator(final Iterator<E> src) {
this(IChunkedIterator.DEFAULT_CHUNK_SIZE, src);
@@ -81,27 +80,27 @@
* The source iterator.
*/
@SuppressWarnings("unchecked")
- public ChunkedStriterator(int chunkSize, Iterator<E> src) {
+ public ChunkedStriterator(final int chunkSize, final Iterator<E> src) {
this((I) new ChunkedWrappedIterator<E>(src, chunkSize,
null/* keyOrder */, null/* filter */));
-
+
}
@Override
final public E[] nextChunk() {
return src.nextChunk();
-
- }
- @Override
- final public void close() {
-
- ((ICloseableIterator<?>) src).close();
-
}
+// @Override
+// final public void close() {
+//
+// ((ICloseableIterator<?>) src).close();
+//
+// }
+
/**
* Strengthened return type.
*/
@@ -109,7 +108,7 @@
public IChunkedStriterator<I, E> addFilter(final IFilter<I, ?, E> filter) {
return (IChunkedStriterator<I, E>) super.addFilter(filter);
-
+
}
-
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/IStriterator.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/IStriterator.java 2013-08-24 00:40:01 UTC (rev 7329)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/IStriterator.java 2013-08-24 11:15:06 UTC (rev 7330)
@@ -42,8 +42,9 @@
* @todo appender and excluder patterns. These are just filters so we only
* really need a single {@link #addFilter(IFilter)} method.
*/
-public interface IStriterator<I extends Iterator<E>,E> extends Iterator<E>, Enumeration<E> {
-
+public interface IStriterator<I extends Iterator<E>, E> extends
+ ICloseableIterator<E>, Enumeration<E> {
+
/**
* Stack a filter on the source iterator.
*
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/Striterator.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/Striterator.java 2013-08-24 00:40:01 UTC (rev 7329)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/striterator/Striterator.java 2013-08-24 11:15:06 UTC (rev 7330)
@@ -166,6 +166,17 @@
}
@Override
+ final public void close() {
+
+ if (src instanceof ICloseable) {
+
+ ((ICloseable) src).close();
+
+ }
+
+ }
+
+ @Override
final public boolean hasMoreElements() {
return src.hasNext();
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java 2013-08-24 00:40:01 UTC (rev 7329)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java 2013-08-24 11:15:06 UTC (rev 7330)
@@ -3,6 +3,7 @@
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -18,6 +19,11 @@
import org.apache.log4j.Logger;
import org.eclipse.jetty.util.ConcurrentHashSet;
+import com.bigdata.btree.IIndex;
+import com.bigdata.btree.IRangeQuery;
+import com.bigdata.btree.ITuple;
+import com.bigdata.btree.keys.IKeyBuilder;
+import com.bigdata.btree.keys.SuccessorUtil;
import com.bigdata.journal.IIndexManager;
import com.bigdata.journal.ITx;
import com.bigdata.journal.TimestampUtility;
@@ -30,15 +36,19 @@
import com.bigdata.rdf.graph.IGASStats;
import com.bigdata.rdf.graph.IReducer;
import com.bigdata.rdf.internal.IV;
+import com.bigdata.rdf.internal.IVUtility;
import com.bigdata.rdf.model.BigdataValue;
import com.bigdata.rdf.spo.ISPO;
import com.bigdata.rdf.spo.SPOFilter;
+import com.bigdata.rdf.spo.SPOKeyOrder;
import com.bigdata.rdf.store.AbstractTripleStore;
+import com.bigdata.relation.accesspath.ElementFilter;
+import com.bigdata.relation.accesspath.EmptyCloseableIterator;
import com.bigdata.relation.accesspath.IElementFilter;
import com.bigdata.service.IBigdataFederation;
-import com.bigdata.striterator.ChunkedStriterator;
-import com.bigdata.striterator.EmptyChunkedIterator;
-import com.bigdata.striterator.IChunkedIterator;
+import com.bigdata.striterator.ICloseableIterator;
+import com.bigdata.striterator.Resolver;
+import com.bigdata.striterator.Striterator;
import com.bigdata.util.concurrent.LatchedExecutor;
/**
@@ -716,26 +726,49 @@
}
- private IChunkedIterator<ISPO> getInEdges(final AbstractTripleStore kb,
- final IV u) {
+ private final SPOKeyOrder getKeyOrder(final AbstractTripleStore kb,
+ final boolean inEdges) {
+ final SPOKeyOrder keyOrder;
+ if (inEdges) {
+ // in-edges: OSP / OCSP : [u] is the Object.
+ keyOrder = kb.isQuads() ? SPOKeyOrder.OCSP : SPOKeyOrder.OSP;
+ } else {
+ // out-edges: SPO / (SPOC|SOPC) : [u] is the Subject.
+ keyOrder = kb.isQuads() ? SPOKeyOrder.SPOC : SPOKeyOrder.SPO;
+ }
+ return keyOrder;
+ }
- // in-edges: OSP / OCSP : [u] is the Object.
- return kb
- .getSPORelation()
- .getAccessPath(null/* s */, null/* p */, u/* o */, null/* c */,
- edgeOnlyFilter).iterator();
+ @SuppressWarnings("unchecked")
+ private Striterator<Iterator<ISPO>,ISPO> getEdges(final AbstractTripleStore kb,
+ final boolean inEdges, final IV u) {
- }
+ final SPOKeyOrder keyOrder = getKeyOrder(kb, inEdges);
+
+ final IIndex ndx = kb.getSPORelation().getIndex(keyOrder);
- private IChunkedIterator<ISPO> getOutEdges(final AbstractTripleStore kb,
- final IV u) {
+ final IKeyBuilder keyBuilder = ndx.getIndexMetadata().getKeyBuilder();
- // out-edges: SPO / SPOC : [u] is the Subject.
- return kb
- .getSPORelation()
- .getAccessPath(u/* s */, null/* p */, null/* o */,
- null/* c */, edgeOnlyFilter).iterator();
+ keyBuilder.reset();
+ IVUtility.encode(keyBuilder, u);
+
+ final byte[] fromKey = keyBuilder.getKey();
+
+ final byte[] toKey = SuccessorUtil.successor(fromKey.clone());
+
+ return (Striterator<Iterator<ISPO>,ISPO>) new Striterator(ndx.rangeIterator(
+ fromKey, toKey, 0/* capacity */, IRangeQuery.DEFAULT,
+ ElementFilter.newInstance(edgeOnlyFilter)))
+ .addFilter(new Resolver() {
+ private static final long serialVersionUID = 1L;
+ @Override
+ protected Object resolve(final Object e) {
+ final ITuple<ISPO> t = (ITuple<ISPO>) e;
+ return t.getObject();
+ }
+ });
+
}
/**
@@ -752,34 +785,93 @@
* predicate, then that gives us S+P bound. If there are multiple
* predicates, then we have an IElementFilter on P (in addition to
* the filter that is removing the Literals from the scan).
- *
- * TODO Use the chunk parallelism? Explicit for(x : chunk)? This
- * could make it easier to collect the edges into an array (but that
- * is not required for powergraph).
*/
- @SuppressWarnings("unchecked")
- private IChunkedIterator<ISPO> getEdges(final AbstractTripleStore kb,
+ private ICloseableIterator<ISPO> getEdges(final AbstractTripleStore kb,
final IV u, final EdgesEnum edges) {
switch (edges) {
case NoEdges:
- return new EmptyChunkedIterator<ISPO>(null/* keyOrder */);
+ return new EmptyCloseableIterator<ISPO>();
case InEdges:
- return getInEdges(kb, u);
+ return (ICloseableIterator<ISPO>) getEdges(kb, true/*inEdges*/, u);
case OutEdges:
- return getOutEdges(kb, u);
+ return (ICloseableIterator<ISPO>) getEdges(kb, false/*inEdges*/, u);
case AllEdges:{
- final IChunkedIterator<ISPO> a = getInEdges(kb, u);
- final IChunkedIterator<ISPO> b = getOutEdges(kb, u);
- final IChunkedIterator<ISPO> c = (IChunkedIterator<ISPO>) new ChunkedStriterator<IChunkedIterator<ISPO>, ISPO>(
- a).append(b);
- return c;
+ final Striterator<Iterator<ISPO>,ISPO> a = getEdges(kb, true/*inEdges*/, u);
+ final Striterator<Iterator<ISPO>,ISPO> b = getEdges(kb, false/*outEdges*/, u);
+ a.append(b);
+ return (ICloseableIterator<ISPO>) a;
}
default:
throw new UnsupportedOperationException(edges.name());
}
}
+
+// private IChunkedIterator<ISPO> getInEdges(final AbstractTripleStore kb,
+// final IV u) {
+//
+// // in-edges: OSP / OCSP : [u] is the Object.
+// return kb
+// .getSPORelation()
+// .getAccessPath(null/* s */, null/* p */, u/* o */, null/* c */,
+// edgeOnlyFilter).iterator();
+//
+// }
+//
+// private IChunkedIterator<ISPO> getOutEdges(final AbstractTripleStore kb,
+// final IV u) {
+//
+// // out-edges: SPO / SPOC : [u] is the Subject.
+// return kb
+// .getSPORelation()
+// .getAccessPath(u/* s */, null/* p */, null/* o */,
+// null/* c */, edgeOnlyFilter).iterator();
+//
+// }
+//
+// /**
+// * Return the edges for the vertex.
+// *
+// * @param u
+// * The vertex.
+// * @param edges
+// * Typesafe enumeration indicating which edges should be visited.
+// * @return An iterator that will visit the edges for that vertex.
+// *
+// * TODO There should be a means to specify a filter on the possible
+// * predicates to be used for traversal. If there is a single
+// * predicate, then that gives us S+P bound. If there are multiple
+// * predicates, then we have an IElementFilter on P (in addition to
+// * the filter that is removing the Literals from the scan).
+// *
+// * TODO Use the chunk parallelism? Explicit for(x : chunk)? This
+// * could make it easier to collect the edges into an array (but that
+// * is not required for powergraph).
+// */
+// @SuppressWarnings("unchecked")
+// private IChunkedIterator<ISPO> getEdges(final AbstractTripleStore kb,
+// final IV u, final EdgesEnum edges) {
+//
+// switch (edges) {
+// case NoEdges:
+// return new EmptyChunkedIterator<ISPO>(null/* keyOrder */);
+// case InEdges:
+// return getInEdges(kb, u);
+// case OutEdges:
+// return getOutEdges(kb, u);
+// case AllEdges:{
+// final IChunkedIterator<ISPO> a = getInEdges(kb, u);
+// final IChunkedIterator<ISPO> b = getOutEdges(kb, u);
+// final IChunkedIterator<ISPO> c = (IChunkedIterator<ISPO>) new ChunkedStriterator<IChunkedIterator<ISPO>, ISPO>(
+// a).append(b);
+// return c;
+// }
+// default:
+// throw new UnsupportedOperationException(edges.name());
+// }
+//
+// }
/**
* A factory for tasks that are applied to each vertex in the frontier.
@@ -1115,7 +1207,8 @@
*/
long nedges = 0L;
- final IChunkedIterator<ISPO> eitr = getEdges(kb, u, getEdgesEnum());
+ final ICloseableIterator<ISPO> eitr = getEdges(kb, u,
+ getEdgesEnum());
try {
@@ -1164,7 +1257,8 @@
long nedges = 0;
- final IChunkedIterator<ISPO> eitr = getEdges(kb, u, getEdgesEnum());
+ final ICloseableIterator<ISPO> eitr = getEdges(kb, u,
+ getEdgesEnum());
try {
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-08-24 00:40:01 UTC (rev 7329)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-08-24 11:15:06 UTC (rev 7330)
@@ -131,9 +131,10 @@
* namespace will be <code>kb</code> if none was specified when
* the triple/quad store was created).</dd>
* <dt>-load</dt>
- * <dd>Loads the named resource. This option may appear multiple
- * times. The resources will be searched for as URLs, on the
- * CLASSPATH, and in the file system.</dd>
+ * <dd>Loads the named resource IFF the KB is empty (or does not
+ * exist) at the time this utility is executed. This option may
+ * appear multiple times. The resources will be searched for as
+ * URLs, on the CLASSPATH, and in the file system.</dd>
* <dt>-bufferMode</dt>
* <dd>Overrides the {@link BufferMode} (if any) specified in the
* <code>propertyFile</code>.</dd>
@@ -419,7 +420,7 @@
}
/*
- * Load data sets.
+ * Load data sets. TODO Document that KB load is IFF empty!!! (Or change the code.)
*/
if (newKB && (loadSet != null && loadSet.length > 0)) {
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/store/AbstractTripleStore.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/store/AbstractTripleStore.java 2013-08-24 00:40:01 UTC (rev 7329)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/store/AbstractTripleStore.java 2013-08-24 11:15:06 UTC (rev 7330)
@@ -1591,12 +1591,12 @@
}
- spoRelation = new SPORelation(this/* container */,
+ spoRelationRef.set(new SPORelation(this/* container */,
getIndexManager(), SPO_NAMESPACE, getTimestamp(),
new Properties(tmp)// Note: must wrap properties!
- );
+ ));
- spoRelation.create();//assignedSplits);
+ spoRelationRef.get().create();
/*
* The axioms require the lexicon to pre-exist. The axioms also
@@ -1775,7 +1775,7 @@
spo.destroy();
}
- spoRelation = null;
+ spoRelationRef.set(null/* clearRef */);
super.destroy();
@@ -1919,21 +1919,38 @@
/**
* The {@link SPORelation} (triples and their access paths).
*/
- final synchronized public SPORelation getSPORelation() {
-
- if (spoRelation == null) {
+ final public SPORelation getSPORelation() {
- spoRelation = (SPORelation) getIndexManager().getResourceLocator()
- .locate(getNamespace() + "." + SPORelation.NAME_SPO_RELATION,
- getTimestamp());
+ if (spoRelationRef.get() == null) {
+ /*
+ * Note: double-checked locking pattern (mostly non-blocking). Only
+ * synchronized if not yet resolved. The AtomicReference is reused
+ * as the monitor to serialize the resolution of the SPORelation in
+ * order to have that operation not contend with any other part of
+ * the API.
+ */
+ synchronized (this) {
+
+ if (spoRelationRef.get() == null) {
+
+ spoRelationRef.set((SPORelation) getIndexManager()
+ .getResourceLocator().locate(
+ getNamespace() + "."
+ + SPORelation.NAME_SPO_RELATION,
+ getTimestamp()));
+
+ }
+
+ }
+
}
- return spoRelation;
+ return spoRelationRef.get();
}
- private SPORelation spoRelation;
+ private final AtomicReference<SPORelation> spoRelationRef = new AtomicReference<SPORelation>();
/**
* The {@link LexiconRelation} handles all things related to the indices
@@ -2103,14 +2120,14 @@
}
- if (spoRelation != null) {
+ final SPORelation tmp = spoRelationRef.getAndSet(null/* clearRef */);
- locator.discard(spoRelation, false/*destroyed*/);
+ if (tmp != null) {
- spoRelation = null;
+ locator.discard(tmp, false/* destroyed */);
}
-
+
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <tho...@us...> - 2013-08-27 19:24:34
|
Revision: 7346
http://bigdata.svn.sourceforge.net/bigdata/?rev=7346&view=rev
Author: thompsonbry
Date: 2013-08-27 19:24:25 +0000 (Tue, 27 Aug 2013)
Log Message:
-----------
Refactored to add link weight support (still need the RDR parser to use this except in SIDs mode and still need to co-locate that SIDs with the ground triples in the SPO(C) and OSP(C) indices).
Refactored nearly everything to abstract out a concrete implementation that does not depend on the bigdata backend. The next step will be to replace the use of IV and ISPO throughput with openrdf Value and openrdf Statement. We can then easily implement a variant over the MemorySail for openrdf (or over some custom classes that are optimized for the GAS traversal patterns). That will give us a fully functioning implementation outside of the core bigdata code base. We can then isolate the remaining dependencies (striterator and a few other items) and create an Apache 2 package for the APIs and the non-bigdata specific implementation.
See #629
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ElementFilter.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ElementFilter.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ElementFilter.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata/src/java/com/bigdata/relation/accesspath/ElementFilter.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -33,6 +33,7 @@
import com.bigdata.btree.filter.TupleFilter;
import cutthecrap.utils.striterators.IFilter;
+import cutthecrap.utils.striterators.IFilterTest;
/**
* Align the predicate's {@link IElementFilter} constraint with
@@ -69,6 +70,26 @@
}
+ /**
+ * Helper method conditionally wraps the <i>test</i>.
+ *
+ * @param <R>
+ * @param test
+ * The test.
+ *
+ * @return The wrapper test -or- <code>null</code> iff the <i>test</i> is
+ * <code>null</code>.
+ */
+ @SuppressWarnings("rawtypes")
+ public static <R> IFilter newInstance(final IFilterTest test) {
+
+ if (test == null)
+ return null;
+
+ return new ElementFilter(test);
+
+ }
+
public ElementFilter(final IElementFilter<R> test) {
if (test == null)
@@ -78,6 +99,37 @@
}
+ @SuppressWarnings("unchecked")
+ public ElementFilter(final IFilterTest test) {
+
+ if (test == null)
+ throw new IllegalArgumentException();
+
+ this.test = new FilterTestWrapper(test);
+
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static class FilterTestWrapper implements IElementFilter {
+ private static final long serialVersionUID = 1L;
+ private final IFilterTest test;
+ private FilterTestWrapper(final IFilterTest test) {
+ this.test = test;
+ }
+
+ @Override
+ public boolean isValid(final Object e) {
+ return test.isValid(e);
+ }
+
+ @Override
+ public boolean canAccept(Object o) {
+ return true;
+ }
+
+ }
+
+ @Override
public boolean isValid(final ITuple<R> tuple) {
final R obj = (R) tuple.getObject();
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -3,6 +3,8 @@
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
+import cutthecrap.utils.striterators.IStriterator;
+
/**
* Interface for options that are understood by the {@link IGASEngine} and which
* may be declared by the {@link IGASProgram}.
@@ -11,7 +13,10 @@
* plan (like GraphChi). I believe that this reduces to computing a DAG over the
* frontier before executing the GATHER and then executing the frontier such
* that the parallel execution is constrained by arcs in the DAG that do not
- * have mutual dependencies.
+ * have mutual dependencies. This is really an option that would be implemented
+ * by the {@link IGASContext}, which would have to place a partial ordering over
+ * the vertices in the frontier and then process the frontier with limited
+ * parallelism based on that partial ordering.
*
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
*/
@@ -46,4 +51,65 @@
*/
Factory<ISPO, ES> getEdgeStateFactory();
+ /**
+ * Return non-<code>null</code> iff there is a single link type to be
+ * visited. This corresponds to a view of the graph as a sparse\xCAconnectivity
+ * matrix. The {@link IGASEngine} can optimize traversal patterns using the
+ * <code>POS</code> index.
+ * <p>
+ * Note: When this option is used, the scatter and gather will not visit the
+ * property set for the vertex. The graph is treated as if it were an
+ * unattributed graph and only mined for the connectivity data.
+ *
+ * @return The {@link IV} for the predicate that identifies the desired link
+ * type (there can be many types of links - the return value
+ * specifies which attribute is of interest).
+ *
+ * @see #getLinkAttribType()
+ */
+ @SuppressWarnings("rawtypes")
+ IV getLinkType();
+
+// /**
+// * Return non-<code>null</code> iff there is a single link type to be
+// * visited. This corresponds to a view of the graph as a sparse\xCAmatrix where
+// * the data in the matrix provides the link weights. The type of the visited
+// * link weights is specified by the return value for this method. The
+// * {@link IGASEngine} can optimize traversal patterns using the
+// * <code>POS</code> index.
+// * <p>
+// * Note: When this option is used, the scatter and gather will not visit the
+// * property set for the vertex. The graph is treated as if it were an
+// * unattributed graph and only mined for the connectivity data.
+// *
+// * @return The {@link IV} for the predicate that identifies the desired link
+// * attribute type (a link can have many attributes - the return
+// * value specifies which attribute is of interest).
+// *
+// * @see #getLinkType()
+// */
+// IV getLinkAttribType();
+//
+// /**
+// * When non-<code>null</code>, the specified {@link Filter} will be used to
+// * restrict the visited edges. For example, you can restrict the visitation
+// * to a subset of the predicates that are of interest, to only visit edges
+// * that have link edges, to visit only select property values, etc. Some
+// * useful filters are defined in an abstract implementation of this
+// * interface.
+// *
+// * @see #visitPropertySet()
+// */
+// IFilterTest getEdgeFilter();
+
+ /**
+ * Hook to impose a constraint on the visited edges and/or property values.
+ *
+ * @param itr
+ * The iterator visiting those edges and/or property values.
+ *
+ * @return Either the same iterator or a constrained iterator.
+ */
+ IStriterator constrainFilter(IStriterator eitr);
+
}
\ No newline at end of file
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -18,12 +18,6 @@
* true. The SUM type is scoped to the GATHER + SUM operation (NOT
* the computation).
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- *
- * TODO There should be a means to specify a filter on the possible
- * predicates to be used for traversal. If there is a single predicate,
- * then that gives us S+P bound. If there are multiple predicates, then
- * we have an IElementFilter on P (in addition to the filter that is
- * removing the Literals from the scan).
*/
@SuppressWarnings("rawtypes")
public interface IGASProgram<VS, ES, ST> extends IGASOptions<VS, ES> {
@@ -238,6 +232,6 @@
* @return <code>true</code> if the algorithm should continue (as long as
* the frontier is non-empty).
*/
- boolean nextRound(IGASContext ctx);
+ boolean nextRound(IGASContext<VS, ES, ST> ctx);
}
\ No newline at end of file
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -1,8 +1,9 @@
package com.bigdata.rdf.graph;
+import java.util.Iterator;
+
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
-import com.bigdata.striterator.ICloseableIterator;
/**
* Interface abstracts access to a backend graph implementation.
@@ -14,19 +15,20 @@
/**
* Return the edges for the vertex.
*
+ * @param p The {@link IGASProgram}
* @param u
* The vertex.
* @param edges
* Typesafe enumeration indicating which edges should be visited.
* @return An iterator that will visit the edges for that vertex.
*/
- ICloseableIterator<ISPO> getEdges(@SuppressWarnings("rawtypes") final IV u,
- final EdgesEnum edges);
+ @SuppressWarnings("rawtypes")
+ Iterator<ISPO> getEdges(IGASProgram<?, ?, ?> p, IV u, EdgesEnum edges);
/**
* Hook to advance the view of the graph. This is invoked at the end of each
* GAS computation round for a given {@link IGASProgram}.
*/
void advanceView();
-
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -5,12 +5,14 @@
import com.bigdata.rdf.graph.EdgesEnum;
import com.bigdata.rdf.graph.Factory;
import com.bigdata.rdf.graph.IGASContext;
-import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.graph.IGASScheduler;
import com.bigdata.rdf.graph.IGASState;
-import com.bigdata.rdf.graph.IGASScheduler;
+import com.bigdata.rdf.graph.impl.BaseGASProgram;
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
+import cutthecrap.utils.striterators.IStriterator;
+
/**
* Breadth First Search (BFS) is an iterative graph traversal primitive. The
* frontier is expanded iteratively until no new vertices are discovered. Each
@@ -20,7 +22,7 @@
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
*/
@SuppressWarnings("rawtypes")
-public class BFS implements IGASProgram<BFS.VS, BFS.ES, Void> {
+public class BFS extends BaseGASProgram<BFS.VS, BFS.ES, Void> {
static class VS {
@@ -125,10 +127,23 @@
}
/**
+ * {@inheritDoc}
+ * <p>
+ * Overridden to only visit the edges of the graph.
+ */
+ @Override
+ public IStriterator constrainFilter(IStriterator itr) {
+
+ return itr.addFilter(edgeOnlyFilter);
+
+ }
+
+ /**
* Not used.
*/
@Override
- public void init(IGASState<BFS.VS, BFS.ES, Void> state, IV u) {
+ public void init(final IGASState<BFS.VS, BFS.ES, Void> state, final IV u) {
+
state.getState(u).visit(0);
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -2,16 +2,16 @@
import org.apache.log4j.Logger;
-import com.bigdata.rdf.graph.EdgesEnum;
import com.bigdata.rdf.graph.Factory;
import com.bigdata.rdf.graph.GASUtil;
-import com.bigdata.rdf.graph.IGASContext;
-import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.graph.IGASScheduler;
import com.bigdata.rdf.graph.IGASState;
-import com.bigdata.rdf.graph.IGASScheduler;
+import com.bigdata.rdf.graph.impl.BaseGASProgram;
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
+import cutthecrap.utils.striterators.IStriterator;
+
/**
* SSSP (Single Source, Shortest Path). This analytic computes the shortest path
* to each vertex in the graph starting from the given vertex. Only connected
@@ -32,7 +32,7 @@
* undirected scatter/gather. Add unit test for undirected.
*/
@SuppressWarnings("rawtypes")
-public class SSSP implements IGASProgram<SSSP.VS, SSSP.ES, Integer/* dist */> {
+public class SSSP extends BaseGASProgram<SSSP.VS, SSSP.ES, Integer/* dist */> {
private static final Logger log = Logger.getLogger(SSSP.class);
@@ -43,10 +43,16 @@
* a pattern to get the link attributes materialized with the {@link ISPO}
* for the link. That could be done using a read-ahead filter on the
* striterator if the link weights are always clustered with the ground
- * triple.
- *
+ * triple. See {@link #decodeStatement(IV)}.
+ * <P>
* When we make this change, the distance should be of the same type as the
* link weight or generalized as <code>double</code>.
+ * <p>
+ * Maybe add a factory method or alternative constructor for the version of
+ * SSSP that uses link weights? All we need to do is filter out anything
+ * that is not a link weight. In addition, it will often be true that there
+ * is a single link attribute type that is of interest, so the caller should
+ * also be able to specify that.
*/
private final static int EDGE_LENGTH = 1;
@@ -127,27 +133,39 @@
}
- @Override
- public Factory<ISPO, SSSP.ES> getEdgeStateFactory() {
+// @Override
+// public Factory<ISPO, SSSP.ES> getEdgeStateFactory() {
+//
+// return null;
+//
+// }
+//
+// @Override
+// public EdgesEnum getGatherEdges() {
+//
+// return EdgesEnum.InEdges;
+//
+// }
+//
+// @Override
+// public EdgesEnum getScatterEdges() {
+//
+// return EdgesEnum.OutEdges;
+//
+// }
- return null;
-
- }
-
+ /**
+ * {@inheritDoc}
+ * <p>
+ * Overridden to only visit the edges of the graph.
+ */
@Override
- public EdgesEnum getGatherEdges() {
+ public IStriterator constrainFilter(IStriterator itr) {
- return EdgesEnum.InEdges;
-
+ return itr.addFilter(edgeOnlyFilter);
+
}
- @Override
- public EdgesEnum getScatterEdges() {
-
- return EdgesEnum.OutEdges;
-
- }
-
/**
* Set the {@link VS#dist()} to ZERO (0).
* <p>
@@ -259,8 +277,8 @@
* <p>
* {@inheritDoc}
*
- * FIXME Test both variations on a variety of data sets and see which is
- * better:
+ * FIXME OPTIMIZE: Test both variations on a variety of data sets and see
+ * which is better:
*
* <p>
* Zhisong wrote: In the original GASengine, the scatter operator only need
@@ -311,11 +329,11 @@
}
- @Override
- public boolean nextRound(IGASContext ctx) {
+// @Override
+// public boolean nextRound(IGASContext ctx) {
+//
+// return true;
+//
+// }
- return true;
-
- }
-
}
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -0,0 +1,225 @@
+package com.bigdata.rdf.graph.impl;
+
+import com.bigdata.rdf.graph.EdgesEnum;
+import com.bigdata.rdf.graph.Factory;
+import com.bigdata.rdf.graph.IGASContext;
+import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.graph.IGASState;
+import com.bigdata.rdf.internal.IV;
+import com.bigdata.rdf.internal.impl.bnode.SidIV;
+import com.bigdata.rdf.spo.ISPO;
+
+import cutthecrap.utils.striterators.Filter;
+import cutthecrap.utils.striterators.IFilter;
+import cutthecrap.utils.striterators.IStriterator;
+
+/**
+ * Abstract base class with some useful defaults.
+ *
+ * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
+ * @param <VS>
+ * @param <ES>
+ * @param <ST>
+ */
+@SuppressWarnings("rawtypes")
+abstract public class BaseGASProgram<VS, ES, ST> implements
+ IGASProgram<VS, ES, ST> {
+
+ /**
+ * Filter visits only edges (filters out attribute values).
+ * <p>
+ * Note: This filter is pushed down onto the AP and evaluated close to the
+ * data.
+ */
+ protected static final IFilter edgeOnlyFilter = new Filter() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public boolean isValid(final Object e) {
+ return ((ISPO) e).o().isURI();
+ }
+ };
+
+ /**
+ * Return <code>true</code> iff the visited {@link ISPO} is an instance
+ * of the specified link attribute type.
+ *
+ * @return
+ */
+ protected static final IFilter newLinkAttribFilter(final IV linkAttribType) {
+
+ return new LinkAttribFilter(linkAttribType);
+
+ }
+
+ static class LinkAttribFilter extends Filter {
+
+ private static final long serialVersionUID = 1L;
+
+ private final IV linkAttribType;
+
+ public LinkAttribFilter(final IV linkAttribType) {
+
+ if (linkAttribType == null)
+ throw new IllegalArgumentException();
+
+ this.linkAttribType = linkAttribType;
+
+ }
+
+ @Override
+ public boolean isValid(final Object e) {
+ final ISPO edge = (ISPO) e;
+ if(!edge.p().equals(linkAttribType)) {
+ // Edge does not use the specified link attribute type.
+ return false;
+ }
+ if (!(edge.s() instanceof SidIV)) {
+ // The subject of the edge is not a Statement.
+ return false;
+ }
+ return true;
+ }
+
+ }
+
+ /**
+ * If the vertex is actually an edge, then return the decoded edge.
+ * <p>
+ * Note: A vertex may be an edge. A link attribute is modeled by treating
+ * the link as a vertex and then asserting a property value about that
+ * "link vertex". For bigdata, this is handled efficiently as inline
+ * statements about statements. This approach subsumes the property graph
+ * model (property graphs do not permit recursive nesting of these
+ * relationships) and is 100% consistent with RDF reification, except that
+ * the link attributes are modeled efficiently inline with the links. This
+ * is what we call <a
+ * href="http://www.bigdata.com/whitepapers/reifSPARQL.pdf" > Reification
+ * Done Right </a>.
+ *
+ * @param v
+ * The vertex.
+ *
+ * @return The edge decoded from that vertex and <code>null</code> iff the
+ * vertex is not an edge.
+ *
+ * TODO RDR : Link to an RDR wiki page as well.
+ *
+ * TODO We can almost write the same logic at the openrdf layer
+ * using <code>v instanceof Statement</code>. However, v can not be
+ * a Statement for openrdf and there is no way to decode the vertex
+ * as a Statement in openrdf.
+ */
+ protected ISPO decodeStatement(final IV v) {
+
+ if (!v.isStatement())
+ return null;
+
+ final ISPO decodedEdge = (ISPO) v.getInlineValue();
+
+ return decodedEdge;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default implementation does not restrict the visitation to a
+ * connectivity matrix (returns <code>null</code>).
+ */
+ @Override
+ public IV getLinkType() {
+
+ return null;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default implementation returns its argument.
+ */
+ @Override
+ public IStriterator constrainFilter(IStriterator itr) {
+
+ return itr;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default gathers on the {@link EdgesEnum#InEdges}.
+ */
+ @Override
+ public EdgesEnum getGatherEdges() {
+
+ return EdgesEnum.InEdges;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default scatters on the {@link EdgesEnum#OutEdges}.
+ */
+ @Override
+ public EdgesEnum getScatterEdges() {
+
+ return EdgesEnum.OutEdges;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default is a NOP.
+ */
+ @Override
+ public void init(final IGASState<VS, ES, ST> state, final IV u) {
+
+ // NOP
+
+ }
+
+// public Factory<IV, VS> getVertexStateFactory();
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default implementation returns <code>null</code>. Override this if
+ * the algorithm uses per-edge computation state.
+ */
+ @Override
+ public Factory<ISPO, ES> getEdgeStateFactory() {
+
+ return null;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default implementation returns <code>true</code>. Override this if
+ * you know whether or not the computation state of this vertex has changed.
+ */
+ @Override
+ public boolean isChanged(IGASState<VS, ES, ST> state, IV u) {
+
+ return true;
+
+ }
+
+ /**
+ * {@inheritDoc}
+ * <p>
+ * The default returns <code>true</code>.
+ */
+ @Override
+ public boolean nextRound(IGASContext<VS, ES, ST> ctx) {
+
+ return true;
+
+ }
+
+}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -1,5 +1,6 @@
package com.bigdata.rdf.graph.impl;
+import java.util.Iterator;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
@@ -14,10 +15,10 @@
import com.bigdata.rdf.graph.IGASState;
import com.bigdata.rdf.graph.IGASStats;
import com.bigdata.rdf.graph.IGraphAccessor;
+import com.bigdata.rdf.graph.IReducer;
import com.bigdata.rdf.graph.IStaticFrontier;
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.spo.ISPO;
-import com.bigdata.striterator.ICloseableIterator;
@SuppressWarnings("rawtypes")
public class GASContext<VS, ES, ST> implements IGASContext<VS, ES, ST> {
@@ -321,6 +322,12 @@
* TODO The apply() should be parallelized. For some algorithms, there is a
* moderate amount of work per vertex in apply(). Use {@link #nthreads} to
* set the parallelism.
+ * <p>
+ * Note: This is very similar to the {@link IGASState#reduce(IReducer)}
+ * operation. This operates over the frontier. reduce() operates over the
+ * activated vertices. Both need fine grained parallelism. Both can have
+ * either light or moderately heavy operations (a dot product would be an
+ * example of a heavier operation).
*/
private void apply(final IStaticFrontier f) {
@@ -519,7 +526,7 @@
final IGASScheduler sch = scheduler();
- final ICloseableIterator<ISPO> eitr = graphAccessor.getEdges(u,
+ final Iterator<ISPO> eitr = graphAccessor.getEdges(program, u,
getEdgesEnum());
try {
@@ -540,7 +547,7 @@
} finally {
- eitr.close();
+// eitr.close();
}
@@ -569,7 +576,7 @@
long nedges = 0;
- final ICloseableIterator<ISPO> eitr = graphAccessor.getEdges(u,
+ final Iterator<ISPO> eitr = graphAccessor.getEdges(program, u,
getEdgesEnum());
try {
@@ -622,7 +629,7 @@
} finally {
- eitr.close();
+// eitr.close();
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASState.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASState.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASState.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -273,7 +273,12 @@
}
- // TODO REDUCE : parallelize with nthreads.
+ /**
+ * {@inheritDoc}
+ *
+ * TODO REDUCE : parallelize with nthreads. The reduce operations are often
+ * lightweight, so maybe a fork/join pool would work better?
+ */
@Override
public <T> T reduce(final IReducer<VS, ES, ST, T> op) {
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASEngine.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASEngine.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASEngine.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -6,6 +6,7 @@
import com.bigdata.btree.IIndex;
import com.bigdata.btree.IRangeQuery;
import com.bigdata.btree.ITuple;
+import com.bigdata.btree.ITupleIterator;
import com.bigdata.btree.keys.IKeyBuilder;
import com.bigdata.btree.keys.SuccessorUtil;
import com.bigdata.journal.IIndexManager;
@@ -23,17 +24,16 @@
import com.bigdata.rdf.internal.IV;
import com.bigdata.rdf.internal.IVUtility;
import com.bigdata.rdf.spo.ISPO;
-import com.bigdata.rdf.spo.SPOFilter;
import com.bigdata.rdf.spo.SPOKeyOrder;
import com.bigdata.rdf.store.AbstractTripleStore;
-import com.bigdata.relation.accesspath.ElementFilter;
import com.bigdata.relation.accesspath.EmptyCloseableIterator;
-import com.bigdata.relation.accesspath.IElementFilter;
import com.bigdata.service.IBigdataFederation;
-import com.bigdata.striterator.ICloseableIterator;
-import com.bigdata.striterator.Resolver;
-import com.bigdata.striterator.Striterator;
+import cutthecrap.utils.striterators.Filter;
+import cutthecrap.utils.striterators.IStriterator;
+import cutthecrap.utils.striterators.Resolver;
+import cutthecrap.utils.striterators.Striterator;
+
/**
* {@link IGASEngine} for dynamic activation of vertices. This implementation
* maintains a frontier and lazily initializes the vertex state when the vertex
@@ -104,21 +104,6 @@
// private static final Logger log = Logger.getLogger(GASEngine.class);
/**
- * Filter visits only edges (filters out attribute values).
- * <p>
- * Note: This filter is pushed down onto the AP and evaluated close to the
- * data.
- */
- static private final IElementFilter<ISPO> edgeOnlyFilter = new SPOFilter<ISPO>() {
- private static final long serialVersionUID = 1L;
-
- @Override
- public boolean isValid(final Object e) {
- return ((ISPO) e).o().isURI();
- }
- };
-
- /**
* The {@link IIndexManager} is used to access the graph.
*/
private final IIndexManager indexManager;
@@ -317,48 +302,127 @@
}
@SuppressWarnings({ "unchecked", "rawtypes" })
- private Striterator<Iterator<ISPO>, ISPO> getEdges(
- final AbstractTripleStore kb, final boolean inEdges, final IV u) {
+ private IStriterator getEdges(final AbstractTripleStore kb,
+ final boolean inEdges, final IGASProgram<?, ?, ?> program,
+ final IV u) {
- final SPOKeyOrder keyOrder = getKeyOrder(kb, inEdges);
+ final SPOKeyOrder keyOrder;
+ final IIndex ndx;
+ final IKeyBuilder keyBuilder;
+ final IV linkTypeIV = program.getLinkType();
+ /*
+ * Optimize case where P is a constant and O is known (2 bound).
+ *
+ * P is a constant.
+ *
+ * [u] gets bound on O.
+ *
+ * We use the POS(C) index. The S values give us the out-edges for
+ * that [u] and the specified link type.
+ *
+ * FIXME POS OPTIMIZATION: write unit test for this option to make
+ * sure that the right filter is imposed. write performance test to
+ * verify expected benefit. Watch out for the in-edges vs out-edges
+ * since only one is optimized.
+ */
+ final boolean posOptimization = linkTypeIV != null
+ && !inEdges;
- final IIndex ndx = kb.getSPORelation().getIndex(keyOrder);
+ if (posOptimization) {
+
+ /*
+ * POS(C)
+ */
+ keyOrder = kb.isQuads() ? SPOKeyOrder.POCS : SPOKeyOrder.POS;
- final IKeyBuilder keyBuilder = ndx.getIndexMetadata().getKeyBuilder();
+ ndx = kb.getSPORelation().getIndex(keyOrder);
- keyBuilder.reset();
+ keyBuilder = ndx.getIndexMetadata().getKeyBuilder();
- IVUtility.encode(keyBuilder, u);
+ keyBuilder.reset();
+ // Bind P as a constant.
+ IVUtility.encode(keyBuilder, linkTypeIV);
+
+ // Bind O for this key-range scan.
+ IVUtility.encode(keyBuilder, u);
+
+ } else {
+
+ /*
+ * SPO(C) or OSP(C)
+ */
+
+ keyOrder = getKeyOrder(kb, inEdges);
+
+ ndx = kb.getSPORelation().getIndex(keyOrder);
+
+ keyBuilder = ndx.getIndexMetadata().getKeyBuilder();
+
+ keyBuilder.reset();
+
+ IVUtility.encode(keyBuilder, u);
+
+ }
+
final byte[] fromKey = keyBuilder.getKey();
final byte[] toKey = SuccessorUtil.successor(fromKey.clone());
- return (Striterator<Iterator<ISPO>, ISPO>) new Striterator(
- ndx.rangeIterator(fromKey, toKey, 0/* capacity */,
- IRangeQuery.DEFAULT,
- ElementFilter.newInstance(edgeOnlyFilter)))
- .addFilter(new Resolver() {
- private static final long serialVersionUID = 1L;
- @Override
- protected Object resolve(final Object e) {
- final ITuple<ISPO> t = (ITuple<ISPO>) e;
- return t.getObject();
- }
- });
+ final ITupleIterator<ISPO> titr = ndx.rangeIterator(fromKey, toKey,
+ 0/* capacity */, IRangeQuery.DEFAULT, null/* filter */);
+ final IStriterator sitr = new Striterator(titr);
+
+ sitr.addFilter(new Resolver() {
+ private static final long serialVersionUID = 1L;
+ @Override
+ protected Object resolve(final Object e) {
+ final ITuple<ISPO> t = (ITuple<ISPO>) e;
+ return t.getObject();
+ }
+ });
+
+ if (linkTypeIV != null && !posOptimization) {
+ /*
+ * A link type constraint was specified, but we were not able to
+ * use the POS(C) index optimization. In this case we have to
+ * add a filter to impose that link type constraint.
+ */
+ sitr.addFilter(new Filter() {
+ private static final long serialVersionUID = 1L;
+ @Override
+ public boolean isValid(final Object e) {
+ return ((ISPO) e).p().equals(linkTypeIV);
+ }
+ });
+ }
+
+ /*
+ * Optionally wrap the program specified filter. This filter will be
+ * pushed down onto the index. If the index is remote, then this is
+ * much more efficient. (If the index is local, then simply stacking
+ * striterators is just as efficient.)
+ */
+ return program.constrainFilter(sitr);
+
}
/**
* Return the edges for the vertex.
*
+ * @param p
+ * The {@link IGASProgram}.
* @param u
* The vertex.
* @param edges
- * Typesafe enumeration indicating which edges should be visited.
+ * Typesafe enumeration indicating which edges should be
+ * visited.
+ *
* @return An iterator that will visit the edges for that vertex.
*/
- public ICloseableIterator<ISPO> getEdges(
+ @SuppressWarnings("unchecked")
+ public Iterator<ISPO> getEdges(final IGASProgram<?, ?, ?> p,
@SuppressWarnings("rawtypes") final IV u, final EdgesEnum edges) {
final AbstractTripleStore kb = getKB();
@@ -367,17 +431,14 @@
case NoEdges:
return new EmptyCloseableIterator<ISPO>();
case InEdges:
- return (ICloseableIterator<ISPO>) getEdges(kb, true/* inEdges */, u);
+ return getEdges(kb, true/* inEdges */, p, u);
case OutEdges:
- return (ICloseableIterator<ISPO>) getEdges(kb, false/* inEdges */,
- u);
+ return getEdges(kb, false/* inEdges */, p, u);
case AllEdges: {
- final Striterator<Iterator<ISPO>, ISPO> a = getEdges(kb,
- true/* inEdges */, u);
- final Striterator<Iterator<ISPO>, ISPO> b = getEdges(kb,
- false/* outEdges */, u);
+ final IStriterator a = getEdges(kb, true/* inEdges */, p, u);
+ final IStriterator b = getEdges(kb, false/* outEdges */, p, u);
a.append(b);
- return (ICloseableIterator<ISPO>) a;
+ return a;
}
default:
throw new UnsupportedOperationException(edges.name());
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java 2013-08-27 17:05:59 UTC (rev 7345)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java 2013-08-27 19:24:25 UTC (rev 7346)
@@ -32,9 +32,8 @@
import com.bigdata.rdf.graph.Factory;
import com.bigdata.rdf.graph.IGASContext;
import com.bigdata.rdf.graph.IGASEngine;
-import com.bigdata.rdf.graph.IGASProgram;
+import com.bigdata.rdf.graph.IGASScheduler;
import com.bigdata.rdf.graph.IGASState;
-import com.bigdata.rdf.graph.IGASScheduler;
import com.bigdata.rdf.graph.impl.bd.BigdataGASEngine;
import com.bigdata.rdf.graph.impl.bd.BigdataGASEngine.BigdataGraphAccessor;
import com.bigdata.rdf.internal.IV;
@@ -42,6 +41,8 @@
import com.bigdata.rdf.spo.ISPO;
import com.bigdata.rdf.spo.SPO;
+import cutthecrap.utils.striterators.IStriterator;
+
/**
* Test class for GATHER.
*
@@ -65,8 +66,8 @@
*
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
*/
- private static class MockGASProgram implements
- IGASProgram<Set<ISPO>, Set<ISPO>, Set<ISPO>> {
+ private static class MockGASProgram extends
+ BaseGASProgram<Set<ISPO>, Set<ISPO>, Set<ISPO>> {
private final EdgesEnum gatherEdges;
@@ -86,7 +87,19 @@
return EdgesEnum.NoEdges;
}
+ /**
+ * {@inheritDoc}
+ * <p>
+ * Overridden to only visit the edges of the graph.
+ */
@Override
+ public IStriterator constrainFilter(IStriterator itr) {
+
+ return itr.addFilter(edgeOnlyFilter);
+
+ }
+
+ @Override
public Factory<IV, Set<ISPO>> getVertexStateFactory() {
return new Factory<IV, Set<ISPO>>() {
@Override
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <tho...@us...> - 2013-09-01 14:32:24
|
Revision: 7377
http://bigdata.svn.sourceforge.net/bigdata/?rev=7377&view=rev
Author: thompsonbry
Date: 2013-09-01 14:32:09 +0000 (Sun, 01 Sep 2013)
Log Message:
-----------
- Checkpoint prior to creation of an Apache 2 GAS module. License headers have been applied, but code has not yet been relocated.
- Changed the vertex sampling procedure. It now has a known basis and is
available for both bigdata and the SAIL backends. This change means
that we need to establish a new baseline for performance
optimizations. Also, while the same distribution is used for all
SAIL implementations, and while all implementations accept and use a
random seed, there is NOT a guarantee the different implementations
will sample the same vertices from the same graph. This is because
the order in which the vertex sample is collected and the natural
order of the vertices currently depends on the implementation
objects. This COULD be fixed if we required the implementation to
put the sampled vertices into an order based on their external
representation as RDF Values, but even then implementation specific
characteristics might not provide us with the same vertices (e.g.,
bigdata imposes a canonical mapping on xsd:dateTime).
- Disabled ordering for the generic SAIL frontier compaction. (First,
order does not matter since there is no disk access. Second, Value
does not implement Comparable so the sort() throws a
ClassCastException.)
See #629
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/EdgesEnum.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/Factory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASSchedulerImpl.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASStats.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IReducer.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IStaticFrontier.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASStats.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/VertexTaskFactory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/mem/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/CHMScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/STScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler2.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/GASImplUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IIntArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/ManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/ManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/internal/constraints/BindingConstraint.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/AbstractGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java
branches/BIGDATA_RELEASE_1_2_0/ctc-striterators/src/java/cutthecrap/utils/striterators/Striterator.java
branches/BIGDATA_RELEASE_1_2_0/ctc-striterators/src/test/cutthecrap/utils/striterators/TestCloseable.java
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGraphFixture.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/GASRunner.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/frontier/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/frontier/StaticFrontier2.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/sail/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/sail/SAILGASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/sail/SailGraphFixture.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/sail/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/VertexDistribution.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/util/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/util/AbstractGraphFixture.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/util/GASUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/util/IGraphFixture.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/util/IGraphFixtureFactory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/AbstractBigdataGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/TestBFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/TestGather.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/TestSSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/AbstractSailGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/TestGather.java
Removed Paths:
-------------
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/GASUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/StaticFrontier2.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/bd/BigdataGASUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS0.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/TestGather.java
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/EdgesEnum.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/EdgesEnum.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/EdgesEnum.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
/**
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/Factory.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/Factory.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/Factory.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
/**
Deleted: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/GASUtil.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/GASUtil.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/GASUtil.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,79 +0,0 @@
-package com.bigdata.rdf.graph;
-
-
-import java.util.Iterator;
-import java.util.concurrent.TimeUnit;
-
-import org.openrdf.model.Statement;
-import org.openrdf.model.Value;
-
-import cutthecrap.utils.striterators.EmptyIterator;
-
-/**
- * Utility class for operations on the public interfaces.
- *
- * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- */
-public class GASUtil {
-
-// private static final Logger log = Logger.getLogger(GASUtil.class);
-
- /**
- * The average fan out of the frontier.
- *
- * @param frontierSize
- * The size of the frontier.
- * @param nedges
- * The number of edges visited when mapping the operation across
- * that frontier.
- *
- * @return The average fan out.
- */
- public static double fanOut(final int frontierSize, final long nedges) {
-
- return ((int) (nedges * 10d / frontierSize)) / 10d;
-
- }
-
- /**
- * The traversed edges per second.
- *
- * @param nedges
- * The number of traversed edges.
- * @param elapsedNanos
- * The elapsed time (nanoseconds).
- *
- * @return The traversed edges per second.
- */
- public static long getTEPS(final long nedges, long elapsedNanos) {
-
- // avoid division by zero.
- if (elapsedNanos == 0)
- elapsedNanos = 1;
-
- // edges/nanosecond.
- final double tepns = ((double) nedges) / elapsedNanos;
-
- // scale units to edges/second.
- final double teps = tepns * TimeUnit.SECONDS.toNanos(1);
-
- // Round off to get rid of those nasty factions.
- final long r = Math.round(teps);
-
- return r;
-
- }
-
- /**
- * An empty vertex iterator.
- */
- @SuppressWarnings({ "unchecked" })
- public static final Iterator<Value> EMPTY_VERTICES_ITERATOR = EmptyIterator.DEFAULT;
-
- /**
- * An empty edge iterator.
- */
- @SuppressWarnings({ "unchecked" })
- public static final Iterator<Statement> EMPTY_EDGES_ITERATOR = EmptyIterator.DEFAULT;
-
-}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASContext.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASContext.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASContext.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import java.util.concurrent.Callable;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASEngine.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASEngine.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASEngine.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
/**
@@ -31,6 +46,15 @@
IGraphAccessor graphAccessor, IGASProgram<VS, ES, ST> program);
/**
+ * Return <code>true</code> iff the frontier should be sorted. Backends that
+ * benefit from an ordered frontier (e.g., to vector IOs) should return
+ * <code>true</code>. Backends that do not benefit from an ordered frontier
+ * (e.g., a backend based on hash collections in memory) should return
+ * <code>false</code>.
+ */
+ boolean getSortFrontier();
+
+ /**
* Polite shutdown.
*/
void shutdown();
@@ -39,5 +63,16 @@
* Immediate shutdown.
*/
void shutdownNow();
+
+ /*
+ * TODO This is a problem since we then need to scope the SailConnection
+ * internally.
+ */
+// /**
+// * Access the default graph. In platforms that support multiple graphs, the
+// * accessed graph will be platform specific. (The sense of graph here is a
+// * triple or a quad store, not a named graph.)
+// */
+// IGraphAccessor newGraphAccessor();
}
\ No newline at end of file
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,6 +1,22 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
import org.openrdf.model.Value;
import cutthecrap.utils.striterators.IStriterator;
@@ -66,7 +82,7 @@
*
* @see #getLinkAttribType()
*/
- Value getLinkType();
+ URI getLinkType();
/**
* Hook to impose a constraint on the visited edges and/or property values.
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import org.openrdf.model.Statement;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASScheduler.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASScheduler.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASScheduler.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import org.openrdf.model.Value;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASSchedulerImpl.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASSchedulerImpl.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASSchedulerImpl.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
/**
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASState.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASState.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASState.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import org.openrdf.model.Statement;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASStats.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASStats.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASStats.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
/**
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import java.util.Iterator;
@@ -6,6 +21,8 @@
import org.openrdf.model.Statement;
import org.openrdf.model.Value;
+import com.bigdata.rdf.graph.impl.util.VertexDistribution;
+
/**
* Interface abstracts access to a backend graph implementation.
*
@@ -35,18 +52,36 @@
void advanceView();
/**
- * Return a sample (without duplicates) of vertices from the graph.
+ * Obtain a weighted distribution of the vertices in the graph from which
+ * samples may then be taken. The weight is the #of in-edges and out-edges
+ * in which a given vertex appears. Statements that model property values
+ * and statement that model link attributes SHOULD NOT be counted in this
+ * sample.
*
- * @param desiredSampleSize
- * The desired sample size.
+ * @param r
+ * A random number generator that (a) MAY be used to select the
+ * sample distribution (the distribution does not need to be
+ * exhaustive so long as it obeys the sampling requirements); and
+ * (b) WILL be used to select vertices from that distribution.
*
- * @return The distinct samples that were found.
+ * @return The distribution.
*
- * FIXME Specify whether the sample must be uniform over the
- * vertices, proportional to the #of (in/out)edges for those
- * vertices, etc. Without a clear specification, we will not
- * have the same behavior across different backends.
+ * TODO SAMPLING: While the same distribution is used for all SAIL
+ * implementations, and while all implementations accept and use a
+ * random seed, there is NOT a guarantee the different
+ * implementations will sample the same vertices from the same
+ * graph. This is because the order in which the vertex sample is
+ * collected and the natural order of the vertices currently depends
+ * on the implementation objects.
+ * <p>
+ * This COULD be fixed if we required the implementation to put the
+ * sampled vertices into an order based on their external
+ * representation as RDF Values, but still reporting their internal
+ * since that is what is used to actually specify a starting vertex
+ * of the GAS program. However, even then, implementation specific
+ * characteristics might not provide us with the same vertices
+ * (e.g., bigdata imposes a canonical mapping on xsd:dateTime).
*/
- Value[] getRandomSample(final Random r, final int desiredSampleSize);
-
+ VertexDistribution getDistribution(Random r);
+
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IReducer.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IReducer.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IReducer.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,29 +1,18 @@
/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
-Copyright (C) SYSTAP, LLC 2006-2007. All rights reserved.
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
+ http://www.apache.org/licenses/LICENSE-2.0
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
*/
-/*
- * Created on Jan 16, 2008
- */
package com.bigdata.rdf.graph;
import org.openrdf.model.Value;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IStaticFrontier.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IStaticFrontier.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IStaticFrontier.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph;
import java.util.Iterator;
@@ -49,18 +64,12 @@
* is specified since many techniques to compact the frontier can
* only estimate the required capacity.)
* @param ordered
- * <code>true</code> iff the frontier is known to be ordered.
+ * <code>true</code> iff the vertices are not known to be ordered
+ * and SHOULD be ordered (some backends do not benefit from
+ * ordering the vertices).
* @param vertices
* The vertices in the new frontier.
- *
- * FIXME The MergeSortIterator and ordered frontiers should only
- * be used when the graph is backed by indices. The cache
- * efficiency concerns for ordered frontiers apply when accessing
- * indices. However, we should not presump that order of
- * traversal matters for other backends. For example, a backend
- * based on hash collections over RDF Statement objects will not
- * benefit from an ordering based on the RDF Value objects.
*/
- void resetFrontier(int minCapacity, boolean ordered, Iterator<Value> vertices);
+ void resetFrontier(int minCapacity, boolean sort, Iterator<Value> vertices);
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph.analytics;
import java.util.concurrent.atomic.AtomicInteger;
@@ -24,7 +39,7 @@
*/
public class BFS extends BaseGASProgram<BFS.VS, BFS.ES, Void> {
- static class VS {
+ public static class VS {
/**
* <code>-1</code> until visited. When visited, set to the current round
@@ -83,7 +98,7 @@
/**
* Edge state is not used.
*/
- static class ES {
+ public static class ES {
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph.analytics;
import org.apache.log4j.Logger;
@@ -55,7 +70,7 @@
*/
private final static int EDGE_LENGTH = 1;
- static class VS {
+ public static class VS {
/**
* The minimum observed distance (in hops) from the source to this
@@ -110,7 +125,7 @@
/**
* Edge state is not used.
*/
- static class ES {
+ public static class ES {
}
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph.impl;
import org.openrdf.model.Statement;
@@ -32,7 +47,7 @@
* connectivity matrix (returns <code>null</code>).
*/
@Override
- public Value getLinkType() {
+ public URI getLinkType() {
return null;
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph.impl;
import java.util.Iterator;
@@ -10,7 +25,6 @@
import org.openrdf.model.Value;
import com.bigdata.rdf.graph.EdgesEnum;
-import com.bigdata.rdf.graph.GASUtil;
import com.bigdata.rdf.graph.IGASContext;
import com.bigdata.rdf.graph.IGASProgram;
import com.bigdata.rdf.graph.IGASScheduler;
@@ -19,6 +33,7 @@
import com.bigdata.rdf.graph.IGraphAccessor;
import com.bigdata.rdf.graph.IReducer;
import com.bigdata.rdf.graph.IStaticFrontier;
+import com.bigdata.rdf.graph.util.GASUtil;
public class GASContext<VS, ES, ST> implements IGASContext<VS, ES, ST> {
Modified: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,3 +1,18 @@
+/**
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
package com.bigdata.rdf.graph.impl;
import java.lang.reflect.Constructor;
@@ -12,6 +27,7 @@
import org.openrdf.model.Value;
+import com.bigdata.rdf.graph.IGASContext;
import com.bigdata.rdf.graph.IGASEngine;
import com.bigdata.rdf.graph.IGASProgram;
import com.bigdata.rdf.graph.IGASScheduler;
@@ -19,6 +35,7 @@
import com.bigdata.rdf.graph.IGASState;
import com.bigdata.rdf.graph.IGraphAccessor;
import com.bigdata.rdf.graph.IStaticFrontier;
+import com.bigdata.rdf.graph.impl.frontier.StaticFrontier2;
import com.bigdata.rdf.graph.impl.scheduler.CHMScheduler;
import com.bigdata.util.concurrent.DaemonThreadFactory;
@@ -402,6 +419,19 @@
}
+ @Override
+ public <VS, ES, ST> IGASContext<VS, ES, ST> newGASContext(
+ final IGraphAccessor graphAccessor,
+ final IGASProgram<VS, ES, ST> gasProgram) {
+
+ final IGASState<VS, ES, ST> gasState = newGASState(graphAccessor,
+ gasProgram);
+
+ return new GASContext<VS, ES, ST>(this/* GASEngine */, graphAccessor,
+ gasState, gasProgram);
+
+ }
+
public <VS, ES, ST> IGASState<VS, ES, ST> newGASState(
final IGraphAccessor graphAccessor,
final IGASProgram<VS, ES, ST> gasProgram) {
@@ -410,7 +440,8 @@
final IGASSchedulerImpl gasScheduler = newScheduler();
- return new GASState<VS, ES, ST>(graphAccessor, frontier, gasScheduler, gasProgram);
+ return new GASState<VS, ES, ST>(this, graphAccessor, frontier,
+ gasScheduler, gasProgram);
}
Deleted: branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-09-01 14:29:10 UTC (rev 7376)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASRunner.java 2013-09-01 14:32:09 UTC (rev 7377)
@@ -1,699 +0,0 @@
-package com.bigdata.rdf.graph.impl;
-
-import java.io.BufferedReader;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.InputStreamReader;
-import java.io.Reader;
-import java.lang.reflect.Constructor;
-import java.util.LinkedHashSet;
-import java.util.Properties;
-import java.util.Random;
-import java.util.concurrent.Callable;
-
-import org.apache.log4j.Logger;
-import org.openrdf.model.Value;
-import org.openrdf.rio.RDFFormat;
-
-import com.bigdata.Banner;
-import com.bigdata.journal.BufferMode;
-import com.bigdata.journal.ITx;
-import com.bigdata.journal.Journal;
-import com.bigdata.rdf.graph.IGASContext;
-import com.bigdata.rdf.graph.IGASEngine;
-import com.bigdata.rdf.graph.IGASProgram;
-import com.bigdata.rdf.graph.IGASScheduler;
-import com.bigdata.rdf.graph.IGASSchedulerImpl;
-import com.bigdata.rdf.graph.IGASState;
-import com.bigdata.rdf.graph.IGASStats;
-import com.bigdata.rdf.graph.impl.bd.BigdataGASEngine;
-import com.bigdata.rdf.graph.impl.bd.BigdataGASEngine.BigdataGraphAccessor;
-import com.bigdata.rdf.rio.LoadStats;
-import com.bigdata.rdf.sail.BigdataSail;
-import com.bigdata.rdf.store.AbstractTripleStore;
-
-/**
- * Base class for running performance tests.
- *
- * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- *
- * TODO Do we need a different driver if the algorithm always visits all
- * vertices? For such algorithms, we just run them once per graph
- * (unless the graph is dynamic).
- */
-public class GASRunner<VS, ES, ST> implements Callable<IGASStats> {
-
- private static final Logger log = Logger.getLogger(GASRunner.class);
-
- /**
- * The seed used for the random number generator.
- */
- private final long seed;
-
- /**
- * Random number generated used for sampling the starting vertices.
- */
- private final Random r;
-
- /**
- * The #of random starting vertices to use.
- */
- private final int nsamples;
-
- /**
- * The #of threads to use for GATHER and SCATTER operators.
- */
- private final int nthreads;
-
- /**
- * The analytic class to be executed.
- */
- private final Class<IGASProgram<VS, ES, ST>> analyticClass;
-
- /**
- * The property file
- */
- private final String propertyFile;
-
- /**
- * When non-<code>null</code>, this overrides the current buffer mode.
- */
- private final BufferMode bufferModeOverride;
-
- /**
- * When non-<code>null</code>, this overrides the KB namespace.
- */
- private final String namespaceOverride;
-
- /**
- * The {@link IGASSchedulerImpl} class to use.
- */
- private final Class<IGASSchedulerImpl> schedulerClassOverride;
-
- /**
- * When non-<code>null</code>, a list of zero or more resources to be
- * loaded. The resources will be searched for as URLs, on the CLASSPATH, and
- * in the file system.
- */
- private final String[] loadSet;
-
- /**
- * Print the optional message on stderr, print the usage information on
- * stderr, and then force the program to exit with the given status code.
- *
- * @param status
- * The status code.
- * @param msg
- * The optional message
- */
- private static void usage(final int status, final String msg) {
-
- if (msg != null) {
-
- System.err.println(msg);
-
- }
-
- System.err.println("[options] analyticClass propertyFile");
-
- System.exit(status);
-
- }
-
- /**
- * Run a GAS analytic against some data set.
- *
- * @param args
- * USAGE:<br/>
- * <code>(options) analyticClass propertyFile</code>
- * <p>
- * <i>Where:</i>
- * <dl>
- * <dt>propertyFile</dt>
- * <dd>A java properties file for a standalone {@link Journal}.</dd>
- * </dl>
- * and <i>options</i> are any of:
- * <dl>
- * <dt>-nthreads</dt>
- * <dd>The #of threads which will be used for GATHER and SCATTER
- * operations.</dd>
- * <dt>-nsamples</dt>
- * <dd>The #of random sample starting vertices that will be
- * selected. The algorithm will be run ONCE for EACH sampled
- * vertex.</dd>
- * <dt>-seed</dt>
- * <dd>The seed for the random number generator (default is
- * <code>217L</code>).</dd>
- * <dt>-bufferMode</dt>
- * <dd>Overrides the {@link BufferMode} (if any) specified in the
- * <code>propertyFile</code>.</dd>
- * <dt>-schedulerClass</dt>
- * <dd>Override the default {@link IGASScheduler}. Class must
- * implement {@link IGASSchedulerImpl}.</dd>
- * <dt>-namespace</dt>
- * <dd>The namespace of the default SPARQL endpoint (the
- * namespace will be <code>kb</code> if none was specified when
- * the triple/quad store was created).</dd>
- * <dt>-load</dt>
- * <dd>Loads the named resource IFF the KB is empty (or does not
- * exist) at the time this utility is executed. This option may
- * appear multiple times. The resources will be searched for as
- * URLs, on the CLASSPATH, and in the file system.</dd>
- * </p>
- * @throws ClassNotFoundException
- */
- public GASRunner(final String[] args) throws ClassNotFoundException {
-
- Banner.banner();
-
- long seed = 217L;
- int nsamples = 100;
- int nthreads = 4;
- BufferMode bufferMode = null; // override only.
- Class<IGASSchedulerImpl> schedulerClass = null; // override only.
- String namespace = "kb";
- // Set of files to load (optional).
- LinkedHashSet<String> loadSet = new LinkedHashSet<String>();
-
- /*
- * Handle all arguments starting with "-". These should appear before
- * any non-option arguments to the program.
- */
- int i = 0;
- while (i < args.length) {
- final String arg = args[i];
- if (arg.startsWith("-")) {
- if (arg.equals("-seed")) {
- seed = Long.valueOf(args[++i]);
- } else if (arg.equals("-nsamples")) {
- final String s = args[++i];
- nsamples = Integer.valueOf(s);
- if (nsamples <= 0) {
- usage(1/* status */,
- "-nsamples must be positive, not: " + s);
- }
- } else if (arg.equals("-nthreads")) {
- final String s = args[++i];
- nthreads = Integer.valueOf(s);
- if (nthreads < 0) {
- usage(1/* status */,
- "-nthreads must be non-negative, not: " + s);
- }
- } else if (arg.equals("-bufferMode")) {
- final String s = args[++i];
- bufferMode = BufferMode.valueOf(s);
- } else if (arg.equals("-schedulerClass")) {
- final String s = args[++i];
- schedulerClass = (Class<IGASSchedulerImpl>) Class.forName(s);
- } else if (arg.equals("-namespace")) {
- final String s = args[++i];
- namespace = s;
- } else if (arg.equals("-load")) {
- final String s = args[++i];
- loadSet.add(s);
- } else {
- usage(1/* status */, "Unknown argument: " + arg);
- }
- } else {
- break;
- }
- i++;
- }
-
- /*
- * Check for the remaining (required) argument(s).
- */
- final int nremaining = args.length - i;
- if (nremaining != 2) {
- /*
- * There are either too many or too few arguments remaining.
- */
- usage(1/* status */, nremaining < 1 ? "Too few arguments."
- : "Too many arguments");
- }
-
- /*
- * The analytic to be executed.
- */
- {
-
- final String s = args[i++];
-
- this.analyticClass = (Class<IGASProgram<VS, ES, ST>>) Class
- .forName(s);
-
- }
-
- /*
- * Property file.
- */
- this.propertyFile = args[i++];
-
- /*
- * Assign parsed values.
- */
- this.seed = seed;
- this.nsamples = nsamples;
- this.nthreads = nthreads;
- this.namespaceOverride = namespace;
- this.bufferModeOverride = bufferMode;
- this.schedulerClassOverride = schedulerClass;
- this.loadSet = loadSet.isEmpty() ? null : loadSet
- .toArray(new String[loadSet.size()]);
-
- // Setup the random number generator.
- this.r = new Random(seed);
-
- }
-
- /**
- * Return an instance of the {@link IGASProgram} to be evaluated.
- */
- protected IGASProgram<VS, ES, ST> newGASProgram() {
-
- final Class<IGASProgram<VS, ES, ST>> cls = analyticClass;
-
- try {
-
- final Constructor<IGASProgram<VS, ES, ST>> ctor = cls
- .getConstructor(new Class[] {});
-
- final IGASProgram<VS, ES, ST> gasProgram = ctor
- .newInstance(new Object[] {});
-
- return gasProgram;
-
- } catch (Exception e) {
-
- throw new RuntimeException(e);
-
- }
-
- }
-
- private Properties getProperties(final String resource) throws IOException {
-
- if (log.isInfoEnabled())
- log.info("Reading properties: " + resource);
-
- InputStream is = null;
- try {
-
- // try the classpath
- is = getClass().getResourceAsStream(resource);
-
- if (is != null) {
-
- } else {
-
- // try file system.
- final File file = new File(resource);
-
- if (file.exists()) {
-
- is = new FileInputStream(file);
-
- } else {
-
- throw new IOException("Could not locate resource: "
- + resource);
-
- }
-
- }
-
- /*
- * Obtain a buffered reader on the input stream.
- */
-
- final Properties properties = new Properties();
-
- final Reader reader = new BufferedReader(new InputStreamReader(is));
-
- try {
-
- properties.load(reader);
-
- } finally {
-
- try {
-
- reader.close();
-
- } catch (Throwable t) {
-
- log.error(t);
-
- }
-
- }
-
- /*
- * Allow override of select options from the command line.
- */
- {
- final String[] overrides = new String[] {
- // Journal options.
- com.bigdata.journal.Options.FILE,
-// // RDFParserOptions.
-// RDFParserOptions.Options.DATATYPE_HANDLING,
-// RDFParserOptions.Options.PRESERVE_BNODE_IDS,
-// RDFParserOptions.Options.STOP_AT_FIRST_ERROR,
-// RDFParserOptions.Options.VERIFY_DATA,
-// // DataLoader options.
-// DataLoader.Options.BUFFER_CAPACITY,
-// DataLoader.Options.CLOSURE,
-// DataLoader.Options.COMMIT,
-// DataLoader.Options.FLUSH,
- };
- for (String s : overrides) {
- if (System.getProperty(s) != null) {
- // Override/set from the environment.
- final String v = System.getProperty(s);
- if (log.isInfoEnabled())
- log.info("OVERRIDE:: Using: " + s + "=" + v);
- properties.setProperty(s, v);
- }
- }
- }
-
- return properties;
-
- } finally {
-
- if (is != null) {
-
- try {
-
- is.close();
-
- } catch (Throwable t) {
-
- log.error(t);
-
- }
-
- }
-
- }
-
- }
-
- /**
- * Run the test.
- * <p>
- * This provides a safe pattern for either loading data into a temporary
- * journal, which is then destroyed, or using an exiting journal and
- * optionally loading in some data set. When we load the data the journal is
- * destroyed afterwards and when the journal is pre-existing and we neither
- * load the data nor destroy the journal. This has to do with the effective
- * BufferMode (if transient) and whether the file is specified and whether a
- * temporary file is created (CREATE_TEMP_FILE). If we do our own file
- * create if the effective buffer mode is non-transient, then we can get all
- * this information.
- */
- public IGASStats call() throws Exception {
-
- final Properties properties = getProperties(propertyFile);
-
- /*
- * Note: Allows override through the command line argument. The default
- * is otherwise the default and the value in the properties file (if
- * any) will be used unless it is overridden.
- */
- final BufferMode bufferMode = this.bufferModeOverride == null ? BufferMode
- .valueOf(properties.getProperty(Journal.Options.BUFFER_MODE,
- Journal.Options.DEFAULT_BUFFER_MODE)) : this.bufferModeOverride;
-
- properties.setProperty(Journal.Options.BUFFER_MODE, bufferMode.name());
-
- final boolean isTransient = !bufferMode.isStable();
-
- final boolean isTemporary;
- if (isTransient) {
-
- isTemporary = true;
-
- } else {
-
- final String fileStr = properties.getProperty(Journal.Options.FILE);
-
- if (fileStr == null) {
-
- /*
- * We will use a temporary file that we create here. The journal
- * will be destroyed below.
- */
- isTemporary = true;
-
- final File tmpFile = File.createTempFile(
- GASRunner.class.getSimpleName(), Journal.Options.JNL);
-
- // Set this on the Properties so it will be used by the jnl.
- properties.setProperty(Journal.Options.FILE,
- tmpFile.getAbsolutePath());
-
- } else {
-
- // real file is named.
- isTemporary = false;
-
- }
-
- }
-
- // The effective KB name.
- final String namespace = this.namespaceOverride == null ? properties
- .getProperty(BigdataSail.Options.NAMESPACE,
- BigdataSail.Options.DEFAULT_NAMESPACE) : this.namespaceOverride;
-
- properties.setProperty(BigdataSail.Options.NAMESPACE, namespace);
-
- /*
- * TODO Could start NSS and use SPARQL UPDATE "LOAD" to load the data.
- * That exposes the SPARQL end point for other purposes during the test.
- * Is this useful? It could also let us run the GASEngine on a remote
- * service (submit a callable to an HA server or define a REST API for
- * submitting these GAS algorithms).
- */
- final Journal jnl = new Journal(properties);
-
- try {
-
- // Locate/create KB.
- final boolean newKB;
- {
- final AbstractTripleStore kb;
- if (isTemporary) {
-
- kb = BigdataSail.createLTS(jnl, properties);
- newKB = true;
-
- } else {
-
- final AbstractTripleStore tmp = (AbstractTripleStore) jnl
- .getResourceLocator().locate(namespace,
- ITx.UNISOLATED);
-
- if (tmp == null) {
-
- // create.
- kb = BigdataSail.createLTS(jnl, properties);
- newKB = true;
-
- } else {
-
- kb = tmp;
- newKB = kb.getStatementCount() == 0L;
-
- }
-
- }
- }
-
- /*
- * Load data sets (iff new KB).
- */
- if (newKB && (loadSet != null && loadSet.length > 0)) {
-
- loadFiles(jnl, namespace, loadSet);
-
- }
-
- return runAnalytic(jnl, namespace);
-
- } finally {
-
- if (isTemporary) {
-
- log.warn("Destroying temporary journal.");
-
- jnl.destroy();
-
- } else {
-
- jnl.close();
-
- }
-
- }
-
- }
-
- /**
- * Load files into the journal.
- *
- * @param jnl
- * The journal.
- * @param namespace
- * The KB namespace.
- * @param loadSet
- * The files.
- * @throws IOException
- */
- private LoadStats loadFiles(final Journal jnl, final String namespace,
- final String[] loadSet) throws IOException {
-
- // final String path = "bigdata-rdf/src/resources/data/foaf";
- // final String dataFile[] = new String[] {//
- // path + "/data-0.nq.gz",//
- // path + "/data-1.nq.gz",//
- // path + "/data-2.nq.gz",//
- // path + "/data-3.nq.gz",//
- // };
- final String baseUrl[] = new String[loadSet.length];
- for (int i = 0; i < loadSet.length; i++) {
- baseUrl[i] = "file:" + loadSet[i];
- }
- // fall back RDFFormat.
- final RDFFormat[] rdfFormat = new RDFFormat[loadSet.length];
- for (int i = 0; i < loadSet.length; i++) {
- rdfFormat[i] = RDFFormat.RDFXML;
- }
-
- // Load data using...
[truncated message content] |
|
From: <tho...@us...> - 2013-09-01 15:50:00
|
Revision: 7380
http://bigdata.svn.sourceforge.net/bigdata/?rev=7380&view=rev
Author: thompsonbry
Date: 2013-09-01 15:49:47 +0000 (Sun, 01 Sep 2013)
Log Message:
-----------
Moved the GAS code into a top-level Apache-2 module. The module is not yet truely independent. We need to expose a few additional classes for that (mainly CAT and DaemonThreadFactory). Also, module builds for the striterator is a pre-condition. We probably need a commons module for things like CAT.
I was able to compile and start executing the test suite using "ant junit", so this should not break CI.
See #629
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_2_0/.classpath
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/bd/AbstractBigdataGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/build.properties
branches/BIGDATA_RELEASE_1_2_0/build.xml
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/apache-license-2_0.txt
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/junit-license.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/log4j-license.txt
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/sesame2.x-license.txt
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/NOTICE
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/build.properties
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/build.xml
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/lib/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/lib/junit-3.8.1.jar
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/lib/log4j-1.2.15.jar
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/lib/openrdf-sesame-2.6.10-onejar.jar
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/EdgesEnum.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/Factory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASOptions.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASSchedulerImpl.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGASStats.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IGraphAccessor.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IReducer.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/IStaticFrontier.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/analytics/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/analytics/BFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/GASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/GASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/GASStats.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/VertexTaskFactory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/frontier/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/sail/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/CHMScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/STScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler2.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/GASImplUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IIntArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/ManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/ManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/VertexDistribution.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/util/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/resources/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/AbstractGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/data/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/AbstractSailGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/TestGather.java
Removed Paths:
-------------
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/analytics/BFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/CHMScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/CHSScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/STScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler2.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/GASImplUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IIntArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/IManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/ManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/java/com/bigdata/rdf/graph/impl/util/ManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestBFS0.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/AbstractSailGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/src/test/com/bigdata/rdf/graph/impl/sail/TestGather.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/EdgesEnum.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/Factory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASOptions.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASSchedulerImpl.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGASStats.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IGraphAccessor.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IReducer.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/IStaticFrontier.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/BFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/SSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/analytics/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/BaseGASProgram.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASContext.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASEngine.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASState.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/GASStats.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/VertexTaskFactory.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/frontier/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/sail/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/CHMScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/STScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/scheduler/TLScheduler2.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/GASImplUtil.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IIntArraySlice.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/IManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/ManagedArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/ManagedIntArray.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/impl/util/VertexDistribution.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/package.html
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/java/com/bigdata/rdf/graph/util/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/AbstractGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestBFS.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/analytics/TestSSSP.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/data/
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/AbstractSailGraphTestCase.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/TestAll.java
branches/BIGDATA_RELEASE_1_2_0/bigdata-rdf/src/test/com/bigdata/rdf/graph/impl/sail/TestGather.java
Modified: branches/BIGDATA_RELEASE_1_2_0/.classpath
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/.classpath 2013-09-01 15:21:14 UTC (rev 7379)
+++ branches/BIGDATA_RELEASE_1_2_0/.classpath 2013-09-01 15:49:47 UTC (rev 7380)
@@ -29,6 +29,8 @@
<classpathentry kind="src" path="bigdata-gom/src/test"/>
<classpathentry kind="src" path="bigdata-gom/src/resources"/>
<classpathentry kind="src" path="bigdata-gom/src/samples"/>
+ <classpathentry kind="src" path="bigdata-gas/src/java"/>
+ <classpathentry kind="src" path="bigdata-gas/src/test"/>
<classpathentry exported="true" kind="lib" path="bigdata/lib/dsi-utils-1.0.6-020610.jar"/>
<classpathentry exported="true" kind="lib" path="bigdata/lib/lgpl-utils-1.0.6-020610.jar"/>
<classpathentry exported="true" kind="lib" path="bigdata-jini/lib/apache/zookeeper-3.3.3.jar"/>
Property changes on: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas
___________________________________________________________________
Added: svn:ignore
+ ant-build
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/apache-license-2_0.txt
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/apache-license-2_0.txt (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/apache-license-2_0.txt 2013-09-01 15:49:47 UTC (rev 7380)
@@ -0,0 +1,201 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/junit-license.html
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/junit-license.html (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/junit-license.html 2013-09-01 15:49:47 UTC (rev 7380)
@@ -0,0 +1,226 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3c.org/TR/1999/REC-html401-19991224/loose.dtd">
+<!-- saved from url=(0041)http://www.eclipse.org/legal/cpl-v10.html -->
+<HTML><HEAD><TITLE>Common Public License - v 1.0</TITLE>
+<META http-equiv=Content-Type content="text/html; charset=ISO-8859-1">
+<META content="MSHTML 6.00.2900.3059" name=GENERATOR></HEAD>
+<BODY vLink=#800000 bgColor=#ffffff>
+<P align=center><B>Common Public License - v 1.0</B>
+<P><B></B><FONT size=3></FONT>
+<P><FONT size=3></FONT><FONT size=2>THE ACCOMPANYING PROGRAM IS PROVIDED UNDER
+THE TERMS OF THIS COMMON PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR
+DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS
+AGREEMENT.</FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2><B>1. DEFINITIONS</B></FONT>
+<P><FONT size=2>"Contribution" means:</FONT>
+<UL><FONT size=2>a) in the case of the initial Contributor, the initial code
+ and documentation distributed under this Agreement, and<BR clear=left>b) in
+ the case of each subsequent Contributor:</FONT></UL>
+<UL><FONT size=2>i) changes to the Program, and</FONT></UL>
+<UL><FONT size=2>ii) additions to the Program;</FONT></UL>
+<UL><FONT size=2>where such changes and/or additions to the Program originate
+ from and are distributed by that particular Contributor. </FONT><FONT size=2>A
+ Contribution 'originates' from a Contributor if it was added to the Program by
+ such Contributor itself or anyone acting on such Contributor's behalf.
+ </FONT><FONT size=2>Contributions do not include additions to the Program
+ which: (i) are separate modules of software distributed in conjunction with
+ the Program under their own license agreement, and (ii) are not derivative
+ works of the Program. </FONT></UL>
+<P><FONT size=2></FONT>
+<P><FONT size=2>"Contributor" means any person or entity that distributes the
+Program.</FONT>
+<P><FONT size=2></FONT><FONT size=2></FONT>
+<P><FONT size=2>"Licensed Patents " mean patent claims licensable by a
+Contributor which are necessarily infringed by the use or sale of its
+Contribution alone or when combined with the Program. </FONT>
+<P><FONT size=2></FONT><FONT size=2></FONT>
+<P><FONT size=2></FONT><FONT size=2>"Program" means the Contributions
+distributed in accordance with this Agreement.</FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2>"Recipient" means anyone who receives the Program under this
+Agreement, including all Contributors.</FONT>
+<P><FONT size=2><B></B></FONT>
+<P><FONT size=2><B>2. GRANT OF RIGHTS</B></FONT>
+<UL><FONT size=2></FONT><FONT size=2>a) </FONT><FONT size=2>Subject to the
+ terms of this Agreement, each Contributor hereby grants</FONT><FONT size=2>
+ Recipient a non-exclusive, worldwide, royalty-free copyright license
+ to</FONT><FONT color=#ff0000 size=2> </FONT><FONT size=2>reproduce, prepare
+ derivative works of, publicly display, publicly perform, distribute and
+ sublicense the Contribution of such Contributor, if any, and such derivative
+ works, in source code and object code form.</FONT></UL>
+<UL><FONT size=2></FONT></UL>
+<UL><FONT size=2></FONT><FONT size=2>b) Subject to the terms of this
+ Agreement, each Contributor hereby grants </FONT><FONT size=2>Recipient a
+ non-exclusive, worldwide,</FONT><FONT color=#008000 size=2> </FONT><FONT
+ size=2>royalty-free patent license under Licensed Patents to make, use, sell,
+ offer to sell, import and otherwise transfer the Contribution of such
+ Contributor, if any, in source code and object code form. This patent license
+ shall apply to the combination of the Contribution and the Program if, at the
+ time the Contribution is added by the Contributor, such addition of the
+ Contribution causes such combination to be covered by the Licensed Patents.
+ The patent license shall not apply to any other combinations which include the
+ Contribution. No hardware per se is licensed hereunder. </FONT></UL>
+<UL><FONT size=2></FONT></UL>
+<UL><FONT size=2>c) Recipient understands that although each Contributor
+ grants the licenses to its Contributions set forth herein, no assurances are
+ provided by any Contributor that the Program does not infringe the patent or
+ other intellectual property rights of any other entity. Each Contributor
+ disclaims any liability to Recipient for claims brought by any other entity
+ based on infringement of intellectual property rights or otherwise. As a
+ condition to exercising the rights and licenses granted hereunder, each
+ Recipient hereby assumes sole responsibility to secure any other intellectual
+ property rights needed, if any. For example, if a third party patent license
+ is required to allow Recipient to distribute the Program, it is Recipient's
+ responsibility to acquire that license before distributing the
+Program.</FONT></UL>
+<UL><FONT size=2></FONT></UL>
+<UL><FONT size=2>d) Each Contributor represents that to its knowledge it has
+ sufficient copyright rights in its Contribution, if any, to grant the
+ copyright license set forth in this Agreement. </FONT></UL>
+<UL><FONT size=2></FONT></UL>
+<P><FONT size=2><B>3. REQUIREMENTS</B></FONT>
+<P><FONT size=2><B></B>A Contributor may choose to distribute the Program in
+object code form under its own license agreement, provided that:</FONT>
+<UL><FONT size=2>a) it complies with the terms and conditions of this
+ Agreement; and</FONT></UL>
+<UL><FONT size=2>b) its license agreement:</FONT></UL>
+<UL><FONT size=2>i) effectively disclaims</FONT><FONT size=2> on behalf of all
+ Contributors all warranties and conditions, express and implied, including
+ warranties or conditions of title and non-infringement, and implied warranties
+ or conditions of merchantability and fitness for a particular purpose;
+</FONT></UL>
+<UL><FONT size=2>ii) effectively excludes on behalf of all Contributors all
+ liability for damages, including direct, indirect, special, incidental and
+ consequential damages, such as lost profits; </FONT></UL>
+<UL><FONT size=2>iii)</FONT><FONT size=2> states that any provisions which
+ differ from this Agreement are offered by that Contributor alone and not by
+ any other party; and</FONT></UL>
+<UL><FONT size=2>iv) states that source code for the Program is available from
+ such Contributor, and informs licensees how to obtain it in a reasonable
+ manner on or through a medium customarily used for software
+ exchange.</FONT><FONT color=#0000ff size=2> </FONT><FONT color=#ff0000
+ size=2></FONT></UL>
+<UL><FONT color=#ff0000 size=2></FONT><FONT size=2></FONT></UL>
+<P><FONT size=2>When the Program is made available in source code form:</FONT>
+<UL><FONT size=2>a) it must be made available under this Agreement; and
+</FONT></UL>
+<UL><FONT size=2>b) a copy of this Agreement must be included with each copy
+ of the Program. </FONT></UL>
+<P><FONT size=2></FONT><FONT color=#0000ff size=2><STRIKE></STRIKE></FONT>
+<P><FONT color=#0000ff size=2><STRIKE></STRIKE></FONT><FONT size=2>Contributors
+may not remove or alter any copyright notices contained within the Program.
+</FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2>Each Contributor must identify itself as the originator of its
+Contribution, if any, in a manner that reasonably allows subsequent Recipients
+to identify the originator of the Contribution. </FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2><B>4. COMMERCIAL DISTRIBUTION</B></FONT>
+<P><FONT size=2>Commercial distributors of software may accept certain
+responsibilities with respect to end users, business partners and the like.
+While this license is intended to facilitate the commercial use of the Program,
+the Contributor who includes the Program in a commercial product offering should
+do so in a manner which does not create potential liability for other
+Contributors. Therefore, if a Contributor includes the Program in a commercial
+product offering, such Contributor ("Commercial Contributor") hereby agrees to
+defend and indemnify every other Contributor ("Indemnified Contributor") against
+any losses, damages and costs (collectively "Losses") arising from claims,
+lawsuits and other legal actions brought by a third party against the
+Indemnified Contributor to the extent caused by the acts or omissions of such
+Commercial Contributor in connection with its distribution of the Program in a
+commercial product offering. The obligations in this section do not apply to any
+claims or Losses relating to any actual or alleged intellectual property
+infringement. In order to qualify, an Indemnified Contributor must: a) promptly
+notify the Commercial Contributor in writing of such claim, and b) allow the
+Commercial Contributor to control, and cooperate with the Commercial Contributor
+in, the defense and any related settlement negotiations. The Indemnified
+Contributor may participate in any such claim at its own expense.</FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2>For example, a Contributor might include the Program in a
+commercial product offering, Product X. That Contributor is then a Commercial
+Contributor. If that Commercial Contributor then makes performance claims, or
+offers warranties related to Product X, those performance claims and warranties
+are such Commercial Contributor's responsibility alone. Under this section, the
+Commercial Contributor would have to defend claims against the other
+Contributors related to those performance claims and warranties, and if a court
+requires any other Contributor to pay any damages as a result, the Commercial
+Contributor must pay those damages.</FONT>
+<P><FONT size=2></FONT><FONT color=#0000ff size=2></FONT>
+<P><FONT color=#0000ff size=2></FONT><FONT size=2><B>5. NO WARRANTY</B></FONT>
+<P><FONT size=2>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS
+PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
+EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR
+CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A
+PARTICULAR PURPOSE. Each Recipient is</FONT><FONT size=2> solely responsible for
+determining the appropriateness of using and distributing </FONT><FONT
+size=2>the Program</FONT><FONT size=2> and assumes all risks associated with its
+exercise of rights under this Agreement</FONT><FONT size=2>, including but not
+limited to the risks and costs of program errors, compliance with applicable
+laws, damage to or loss of data, </FONT><FONT size=2>programs or equipment, and
+unavailability or interruption of operations</FONT><FONT size=2>. </FONT><FONT
+size=2></FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2></FONT><FONT size=2><B>6. DISCLAIMER OF LIABILITY</B></FONT>
+<P><FONT size=2></FONT><FONT size=2>EXCEPT AS EXPRESSLY SET FORTH IN THIS
+AGREEMENT, NEITHER RECIPIENT NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+</FONT><FONT size=2>(INCLUDING WITHOUT LIMITATION LOST PROFITS),</FONT><FONT
+size=2> HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS
+GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.</FONT>
+<P><FONT size=2></FONT><FONT size=2></FONT>
+<P><FONT size=2><B>7. GENERAL</B></FONT>
+<P><FONT size=2></FONT><FONT size=2>If any provision of this Agreement is
+invalid or unenforceable under applicable law, it shall not affect the validity
+or enforceability of the remainder of the terms of this Agreement, and without
+further action by the parties hereto, such provision shall be reformed to the
+minimum extent necessary to make such provision valid and enforceable.</FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2>If Recipient institutes patent litigation against a Contributor
+with respect to a patent applicable to software (including a cross-claim or
+counterclaim in a lawsuit), then any patent licenses granted by that Contributor
+to such Recipient under this Agreement shall terminate as of the date such
+litigation is filed. In addition, if Recipient institutes patent litigation
+against any entity (including a cross-claim or counterclaim in a lawsuit)
+alleging that the Program itself (excluding combinations of the Program with
+other software or hardware) infringes such Recipient's patent(s), then such
+Recipient's rights granted under Section 2(b) shall terminate as of the date
+such litigation is filed. </FONT><FONT size=2></FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2>All Recipient's rights under this Agreement shall terminate if
+it fails to comply with any of the material terms or conditions of this
+Agreement and does not cure such failure in a reasonable period of time after
+becoming aware of such noncompliance. If all Recipient's rights under this
+Agreement terminate, Recipient agrees to cease use and distribution of the
+Program as soon as reasonably practicable. However, Recipient's obligations
+under this Agreement and any licenses granted by Recipient relating to the
+Program shall continue and survive. </FONT><FONT size=2></FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2></FONT><FONT size=2>Everyone is permitted to copy and distribute
+copies of this Agreement, but in order to avoid inconsistency the Agreement is
+copyrighted and may only be modified in the following manner. The Agreement
+Steward reserves the right to </FONT><FONT size=2>publish new versions
+(including revisions) of this Agreement from time to </FONT><FONT size=2>time.
+No one other than the Agreement Steward has the right to modify this Agreement.
+IBM is the initial Agreement Steward. IBM may assign the responsibility to serve
+as the Agreement Steward to a suitable separate entity. </FONT><FONT size=2>Each
+new version of the Agreement will be given a distinguishing version number. The
+Program (including Contributions) may always be distributed subject to the
+version of the Agreement under which it was received. In addition, after a new
+version of the Agreement is published, Contributor may elect to distribute the
+Program (including its Contributions) under the new </FONT><FONT size=2>version.
+</FONT><FONT size=2>Except as expressly stated in Sections 2(a) and 2(b) above,
+Recipient receives no rights or licenses to the intellectual property of any
+Contributor under this Agreement, whether expressly, </FONT><FONT size=2>by
+implication, estoppel or otherwise</FONT><FONT size=2>.</FONT><FONT size=2> All
+rights in the Program not expressly granted under this Agreement are
+reserved.</FONT>
+<P><FONT size=2></FONT>
+<P><FONT size=2>This Agreement is governed by the laws of the State of New York
+and the intellectual property laws of the United States of America. No party to
+this Agreement will bring a legal action under this Agreement more than one year
+after the cause of action arose. Each party waives its rights to a jury trial in
+any resulting litigation.</FONT>
+<P><FONT size=2></FONT><FONT size=2></FONT>
+<P><FONT size=2></FONT></P></BODY></HTML>
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/log4j-license.txt
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/log4j-license.txt (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/log4j-license.txt 2013-09-01 15:49:47 UTC (rev 7380)
@@ -0,0 +1,201 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/sesame2.x-license.txt
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/sesame2.x-license.txt (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/LEGAL/sesame2.x-license.txt 2013-09-01 15:49:47 UTC (rev 7380)
@@ -0,0 +1,25 @@
+Copyright Aduna (http://www.aduna-software.com/) \xA9 2001-2007
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+ * Neither the name of the copyright holder nor the names of its contributors
+ may be used to endorse or promote products derived from this software
+ without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/NOTICE
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/NOTICE (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/NOTICE 2013-09-01 15:49:47 UTC (rev 7380)
@@ -0,0 +1,18 @@
+
+ Copyright (C) SYSTAP, LLC 2006-2012. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+----
+This product includes software developed by The Apache Software Foundation (http://www.apache.org/).
+License: http://www.apache.org/licenses/LICENSE-2.0
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/build.properties
===================================================================
--- branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/build.properties (rev 0)
+++ branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/build.properties 2013-09-01 15:49:47 UTC (rev 7380)
@@ -0,0 +1,50 @@
+# The root of the checked out svn source. This assumes that you have checked
+# out the trunk so that all modules were automatically checked out and are in
+# direct subdirectories of the directory containing this properties file and
+# the ant build.xml file.
+bigdata-gas.dir=.
+# Where the generated files will be written.
+# build.dir/classes [compiled classes and misc resources for classpath]
+# build.dir/docs [generated documentation].
+# build.dir/docs/api [generated javadoc].
+# build.dir/lib [bundled libraries copied here for easier deployment]
+# build.dir/src [source code copied here for releases]
+build.dir=ant-build
+
+##
+# javac options
+##
+
+# debug=on|off
+javac.debug=on
+# debuglevel=lines,vars,source (or any combination thereof).
+javac.debuglevel=lines,vars,source
+javac.verbose=off
+#javac.target=1.6
+#javac.source=1.6
+javac.encoding=Cp1252
+
+# Where to find the library dependencies (junit)
+build.lib.dir=lib
+
+# Where the write the output from the unit tests.
+test.results.dir=${build.dir}/test-results
+
+##
+# Properties for creating a release.
+##
+
+# Where the releases will be written.
+release.dir=ant-release
+
+# The build version.
+build.ver=0.1.0
+
+# Set true to do a snapshot build. This changes the value of ${version} to
+# include the date.
+#snapshot=true
+snapshot=
+
+# Javadoc build may be disabled using this property. The javadoc target will
+# not be executed unless this property is defined (its value does not matter).
+javadoc=
Added: branches/BIGDATA_RELEASE_1_2_0/bigdata-gas/build.xml
=============================...
[truncated message content] |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X