Menu
▾
▴
bigdata-commit — Commit traffic.
This list is closed, nobody may subscribe to it.
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(139) |
Aug
(94) |
Sep
(232) |
Oct
(143) |
Nov
(138) |
Dec
(55) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
(127) |
Feb
(90) |
Mar
(101) |
Apr
(74) |
May
(148) |
Jun
(241) |
Jul
(169) |
Aug
(121) |
Sep
(157) |
Oct
(199) |
Nov
(281) |
Dec
(75) |
| 2012 |
Jan
(107) |
Feb
(122) |
Mar
(184) |
Apr
(73) |
May
(14) |
Jun
(49) |
Jul
(26) |
Aug
(103) |
Sep
(133) |
Oct
(61) |
Nov
(51) |
Dec
(55) |
| 2013 |
Jan
(59) |
Feb
(72) |
Mar
(99) |
Apr
(62) |
May
(92) |
Jun
(19) |
Jul
(31) |
Aug
(138) |
Sep
(47) |
Oct
(83) |
Nov
(95) |
Dec
(111) |
| 2014 |
Jan
(125) |
Feb
(60) |
Mar
(119) |
Apr
(136) |
May
(270) |
Jun
(83) |
Jul
(88) |
Aug
(30) |
Sep
(47) |
Oct
(27) |
Nov
(23) |
Dec
|
| 2015 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(3) |
Oct
|
Nov
|
Dec
|
| 2016 |
Jan
|
Feb
|
Mar
(4) |
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: <mrp...@us...> - 2014-05-12 15:35:35
|
Revision: 8284
http://sourceforge.net/p/bigdata/code/8284
Author: mrpersonick
Date: 2014-05-12 15:35:32 +0000 (Mon, 12 May 2014)
Log Message:
-----------
add the avro dependency:
static {
// // pull a Tinkerpop interface into the class loader
// log.info(com.tinkerpop.blueprints.Graph.class.getName());
log.info(org.apache.avro.Schema.class.getName());
}
Added Paths:
-----------
branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
Added: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
===================================================================
(Binary files differ)
Index: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar 2014-05-12 15:04:33 UTC (rev 8283)
+++ branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar 2014-05-12 15:35:32 UTC (rev 8284)
Property changes on: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/octet-stream
\ No newline at end of property
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 15:04:37
|
Revision: 8283
http://sourceforge.net/p/bigdata/code/8283
Author: mrpersonick
Date: 2014-05-12 15:04:33 +0000 (Mon, 12 May 2014)
Log Message:
-----------
add the avro dependency:
static {
// // pull a Tinkerpop interface into the class loader
// log.info(com.tinkerpop.blueprints.Graph.class.getName());
log.info(org.apache.avro.Schema.class.getName());
}
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Added Paths:
-----------
branches/BLUEPRINTS/bigdata-blueprints/lib/avro-1.7.6.jar
Added: branches/BLUEPRINTS/bigdata-blueprints/lib/avro-1.7.6.jar
===================================================================
(Binary files differ)
Index: branches/BLUEPRINTS/bigdata-blueprints/lib/avro-1.7.6.jar
===================================================================
--- branches/BLUEPRINTS/bigdata-blueprints/lib/avro-1.7.6.jar 2014-05-12 11:25:13 UTC (rev 8282)
+++ branches/BLUEPRINTS/bigdata-blueprints/lib/avro-1.7.6.jar 2014-05-12 15:04:33 UTC (rev 8283)
Property changes on: branches/BLUEPRINTS/bigdata-blueprints/lib/avro-1.7.6.jar
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/octet-stream
\ No newline at end of property
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 11:25:13 UTC (rev 8282)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 15:04:33 UTC (rev 8283)
@@ -46,12 +46,12 @@
private static final transient Logger log = Logger
.getLogger(RESTServlet.class);
-// static {
-//// // pull a Tinkerpop interface into the class loader
-//// log.info(com.tinkerpop.blueprints.Graph.class.getName());
-//
-// log.info(org.apache.avro.Schema.class.getName());
-// }
+ static {
+// // pull a Tinkerpop interface into the class loader
+// log.info(com.tinkerpop.blueprints.Graph.class.getName());
+
+ log.info(org.apache.avro.Schema.class.getName());
+ }
/**
*
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <tob...@us...> - 2014-05-12 11:25:18
|
Revision: 8282
http://sourceforge.net/p/bigdata/code/8282
Author: tobycraig
Date: 2014-05-12 11:25:13 +0000 (Mon, 12 May 2014)
Log Message:
-----------
Fixed ctrl-return for CodeMirror form submission and ctrl-right looping round from last tab
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_3_0/bigdata-war/src/html/js/workbench.js
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata-war/src/html/js/workbench.js
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata-war/src/html/js/workbench.js 2014-05-12 03:58:10 UTC (rev 8281)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata-war/src/html/js/workbench.js 2014-05-12 11:25:13 UTC (rev 8282)
@@ -56,11 +56,19 @@
}
}
+function moveTabLeft() {
+ moveTab(false);
+}
+
+function moveTabRight() {
+ moveTab(true);
+}
+
function moveTab(next) {
// get current position
var current = $('#tab-selector .active');
if(next) {
- if(current.next().length) {
+ if(current.next('a').length) {
current.next().click();
} else {
$('#tab-selector a:first').click();
@@ -75,8 +83,8 @@
}
// these should be , and . but Hotkeys views those keypresses as these characters
-$('html, textarea, select').bind('keydown', 'ctrl+¼', function() { moveTab(false); });
-$('html, textarea, select').bind('keydown', 'ctrl+¾', function() { moveTab(true); });
+$('html, textarea, select').bind('keydown', 'ctrl+¼', moveTabLeft);
+$('html, textarea, select').bind('keydown', 'ctrl+¾', moveTabRight);
/* Namespaces */
@@ -407,20 +415,23 @@
$('#update-box').on('dragover', handleDragOver)
.on('drop', handleFile)
.on('paste', handlePaste)
- .on('input propertychange', function() { $('#update-errors').hide(); })
- .bind('keydown', 'ctrl+return', submitUpdate);
+ .on('input propertychange', function() { $('#update-errors').hide(); });
$('#clear-file').click(clearFile);
$('#update-update').click(submitUpdate);
-UPDATE_EDITOR = CodeMirror.fromTextArea($('#update-box')[0], {lineNumbers: true, mode: 'sparql'});
+UPDATE_EDITOR = CodeMirror.fromTextArea($('#update-box')[0], {lineNumbers: true, mode: 'sparql',
+ extraKeys: {'Ctrl-Enter': submitUpdate, 'Ctrl-,': moveTabLeft, 'Ctrl-.': moveTabRight}
+});
function submitUpdate(e) {
// Updates are submitted as a regular form for SPARQL updates in monitor mode, and via AJAX for non-monitor SPARQL, RDF & file path updates.
// When submitted as a regular form, the output is sent to an iframe. This is to allow monitor mode to work.
// jQuery only gives us data when the request is complete, so we wouldn't see monitor results as they come in.
- e.preventDefault();
+ try {
+ e.preventDefault();
+ } catch(e) {}
$('#update-response').show();
@@ -512,8 +523,7 @@
/* Query */
-$('#query-box').bind('keydown', 'ctrl+return', function(e) { e.preventDefault(); $('#query-form').submit(); })
- .on('input propertychange', function() { $('#query-errors').hide(); });
+$('#query-box').on('input propertychange', function() { $('#query-errors').hide(); });
$('#query-form').submit(submitQuery);
$('#query-explain').change(function() {
@@ -528,17 +538,21 @@
}
});
-QUERY_EDITOR = CodeMirror.fromTextArea($('#query-box')[0], {lineNumbers: true, mode: 'sparql'});
+QUERY_EDITOR = CodeMirror.fromTextArea($('#query-box')[0], {lineNumbers: true, mode: 'sparql',
+ extraKeys: {'Ctrl-Enter': submitQuery, 'Ctrl-,': moveTabLeft, 'Ctrl-.': moveTabRight}
+});
function submitQuery(e) {
- e.preventDefault();
+ try {
+ e.preventDefault();
+ } catch(e) {}
// transfer CodeMirror content to textarea
QUERY_EDITOR.save();
var settings = {
type: 'POST',
- data: $(this).serialize(),
+ data: $('#query-form').serialize(),
headers: { 'Accept': 'application/sparql-results+json, application/rdf+xml' },
success: showQueryResults,
error: queryResultsError
@@ -553,7 +567,7 @@
if($('#query-explain').is(':checked')) {
settings = {
type: 'POST',
- data: $(this).serialize() + '&explain=' + ($('#query-details').is(':checked') ? 'details' : 'true'),
+ data: $('#query-form').serialize() + '&explain=' + ($('#query-details').is(':checked') ? 'details' : 'true'),
dataType: 'html',
success: showQueryExplanation,
error: queryResultsError
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 03:58:13
|
Revision: 8281
http://sourceforge.net/p/bigdata/code/8281
Author: mrpersonick
Date: 2014-05-12 03:58:10 +0000 (Mon, 12 May 2014)
Log Message:
-----------
once again, starting over from a clean CI run
Removed Paths:
-------------
branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
Deleted: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
===================================================================
(Binary files differ)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 03:58:02
|
Revision: 8280
http://sourceforge.net/p/bigdata/code/8280
Author: mrpersonick
Date: 2014-05-12 03:57:57 +0000 (Mon, 12 May 2014)
Log Message:
-----------
once again, starting over from a clean CI run
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSailFactory.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSailFactory.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSailFactory.java 2014-05-12 03:53:14 UTC (rev 8279)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSailFactory.java 2014-05-12 03:57:57 UTC (rev 8280)
@@ -28,12 +28,11 @@
import java.util.List;
import java.util.Properties;
-import org.openrdf.repository.Repository;
-
import com.bigdata.journal.BufferMode;
import com.bigdata.journal.Journal;
import com.bigdata.rdf.axioms.NoAxioms;
import com.bigdata.rdf.axioms.OwlAxioms;
+import com.bigdata.rdf.sail.remote.BigdataSailRemoteRepository;
import com.bigdata.rdf.vocab.RDFSVocabulary;
/**
@@ -87,6 +86,34 @@
}
/**
+ * Connect to a remote bigdata instance.
+ */
+ public static BigdataSailRemoteRepository connect(final String host, final int port) {
+
+ return connect("http://"+host+":"+port);
+
+ }
+
+ /**
+ * Connect to a remote bigdata instance.
+ */
+ public static BigdataSailRemoteRepository connect(final String serviceEndpoint) {
+
+ if (serviceEndpoint.endsWith("/bigdata/sparql")) {
+ return new BigdataSailRemoteRepository(serviceEndpoint);
+ } else if (serviceEndpoint.endsWith("/bigdata/")) {
+ return new BigdataSailRemoteRepository(serviceEndpoint + "sparql");
+ } else if (serviceEndpoint.endsWith("/bigdata")) {
+ return new BigdataSailRemoteRepository(serviceEndpoint + "/sparql");
+ } else if (serviceEndpoint.endsWith("/")) {
+ return new BigdataSailRemoteRepository(serviceEndpoint + "bigdata/sparql");
+ } else {
+ return new BigdataSailRemoteRepository(serviceEndpoint + "/bigdata/sparql");
+ }
+
+ }
+
+ /**
* Open an existing persistent bigdata instance.
*/
public static BigdataSailRepository openRepository(final String file) {
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 03:53:20
|
Revision: 8279
http://sourceforge.net/p/bigdata/code/8279
Author: mrpersonick
Date: 2014-05-12 03:53:14 +0000 (Mon, 12 May 2014)
Log Message:
-----------
once again, starting over from a clean CI run
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 02:58:59 UTC (rev 8278)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 03:53:14 UTC (rev 8279)
@@ -46,12 +46,12 @@
private static final transient Logger log = Logger
.getLogger(RESTServlet.class);
- static {
-// // pull a Tinkerpop interface into the class loader
-// log.info(com.tinkerpop.blueprints.Graph.class.getName());
-
- log.info(org.apache.avro.Schema.class.getName());
- }
+// static {
+//// // pull a Tinkerpop interface into the class loader
+//// log.info(com.tinkerpop.blueprints.Graph.class.getName());
+//
+// log.info(org.apache.avro.Schema.class.getName());
+// }
/**
*
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 02:59:04
|
Revision: 8278
http://sourceforge.net/p/bigdata/code/8278
Author: mrpersonick
Date: 2014-05-12 02:58:59 +0000 (Mon, 12 May 2014)
Log Message:
-----------
pull Avro classes back in:
static {
// // pull a Tinkerpop interface into the class loader
// log.info(com.tinkerpop.blueprints.Graph.class.getName());
log.info(org.apache.avro.Schema.class.getName());
}
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 01:47:38 UTC (rev 8277)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 02:58:59 UTC (rev 8278)
@@ -50,7 +50,7 @@
// // pull a Tinkerpop interface into the class loader
// log.info(com.tinkerpop.blueprints.Graph.class.getName());
-// log.info(org.apache.avro.Schema.class.getName());
+ log.info(org.apache.avro.Schema.class.getName());
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 01:47:41
|
Revision: 8277
http://sourceforge.net/p/bigdata/code/8277
Author: mrpersonick
Date: 2014-05-12 01:47:38 +0000 (Mon, 12 May 2014)
Log Message:
-----------
entirely removing the blueprints integration just to be sure
Removed Paths:
-------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataPredicate.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataRDFFactory.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataVertex.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BlueprintsRDFFactory.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/edge.rq
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/edges.rq
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/edgesByProperty.rq
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/vertex.rq
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,107 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import java.util.Arrays;
-import java.util.List;
-
-import org.openrdf.model.Statement;
-import org.openrdf.model.URI;
-import org.openrdf.model.vocabulary.RDFS;
-
-import com.tinkerpop.blueprints.Direction;
-import com.tinkerpop.blueprints.Edge;
-import com.tinkerpop.blueprints.Vertex;
-
-/**
- * Edge implementation that wraps an Edge statement and points to a
- * {@link BigdataGraph} instance.
- *
- * @author mikepersonick
- *
- */
-public class BigdataEdge extends BigdataElement implements Edge {
-
- private static final List<String> blacklist = Arrays.asList(new String[] {
- "id", "", "label"
- });
-
- protected final Statement stmt;
-
- public BigdataEdge(final Statement stmt, final BigdataGraph graph) {
- super(stmt.getPredicate(), graph);
-
- this.stmt = stmt;
- }
-
- @Override
- public Object getId() {
- return graph.factory.fromEdgeURI(uri);
- }
-
- @Override
- public void remove() {
- graph.removeEdge(this);
- }
-
- @Override
- public String getLabel() {
- return (String) graph.getProperty(uri, RDFS.LABEL);
- }
-
- @Override
- public Vertex getVertex(final Direction dir) throws IllegalArgumentException {
-
- if (dir == Direction.BOTH) {
- throw new IllegalArgumentException();
- }
-
- final URI uri = (URI)
- (dir == Direction.OUT ? stmt.getSubject() : stmt.getObject());
-
- final String id = graph.factory.fromVertexURI(uri);
-
- return graph.getVertex(id);
-
- }
-
- @Override
- public void setProperty(final String property, final Object val) {
-
- if (property == null || blacklist.contains(property)) {
- throw new IllegalArgumentException();
- }
-
- super.setProperty(property, val);
-
- }
-
- @Override
- public String toString() {
- final URI s = (URI) stmt.getSubject();
- final URI p = (URI) stmt.getPredicate();
- final URI o = (URI) stmt.getObject();
- return "e["+p.getLocalName()+"]["+s.getLocalName()+"->"+o.getLocalName()+"]";
- }
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,134 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import java.util.Arrays;
-import java.util.List;
-import java.util.Set;
-
-import org.openrdf.model.Literal;
-import org.openrdf.model.URI;
-
-import com.tinkerpop.blueprints.Element;
-
-/**
- * Base class for {@link BigdataVertex} and {@link BigdataEdge}. Handles
- * property-related methods.
- *
- * @author mikepersonick
- *
- */
-public abstract class BigdataElement implements Element {
-
- private static final List<String> blacklist = Arrays.asList(new String[] {
- "id", ""
- });
-
- protected final URI uri;
- protected final BigdataGraph graph;
-
- public BigdataElement(final URI uri, final BigdataGraph graph) {
- this.uri = uri;
- this.graph = graph;
- }
-
- @Override
- @SuppressWarnings("unchecked")
- public <T> T getProperty(final String property) {
-
- final URI p = graph.factory.toPropertyURI(property);
-
- return (T) graph.getProperty(uri, p);
-
- }
-
- @Override
- public Set<String> getPropertyKeys() {
-
- return graph.getPropertyKeys(uri);
-
- }
-
- @Override
- @SuppressWarnings("unchecked")
- public <T> T removeProperty(final String property) {
-
- final URI p = graph.factory.toPropertyURI(property);
-
- return (T) graph.removeProperty(uri, p);
-
- }
-
- @Override
- public void setProperty(final String property, final Object val) {
-
- if (property == null || blacklist.contains(property)) {
- throw new IllegalArgumentException();
- }
-
- final URI p = graph.factory.toPropertyURI(property);
-
- final Literal o = graph.factory.toLiteral(val);
-
- graph.setProperty(uri, p, o);
-
- }
-
- @Override
- public int hashCode() {
- final int prime = 31;
- int result = 1;
- result = prime * result + ((graph == null) ? 0 : graph.hashCode());
- result = prime * result + ((uri == null) ? 0 : uri.hashCode());
- return result;
- }
-
- @Override
- public boolean equals(Object obj) {
- if (this == obj)
- return true;
- if (obj == null)
- return false;
- if (getClass() != obj.getClass())
- return false;
- BigdataElement other = (BigdataElement) obj;
- if (graph == null) {
- if (other.graph != null)
- return false;
- } else if (!graph.equals(other.graph))
- return false;
- if (uri == null) {
- if (other.uri != null)
- return false;
- } else if (!uri.equals(other.uri))
- return false;
- return true;
- }
-
- @Override
- public String toString() {
- return uri.toString();
- }
-
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,796 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import info.aduna.iteration.CloseableIteration;
-
-import java.util.Iterator;
-import java.util.LinkedHashSet;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Set;
-import java.util.UUID;
-
-import org.apache.commons.io.IOUtils;
-import org.openrdf.OpenRDFException;
-import org.openrdf.model.Literal;
-import org.openrdf.model.Statement;
-import org.openrdf.model.URI;
-import org.openrdf.model.Value;
-import org.openrdf.model.impl.StatementImpl;
-import org.openrdf.model.impl.URIImpl;
-import org.openrdf.model.vocabulary.RDF;
-import org.openrdf.model.vocabulary.RDFS;
-import org.openrdf.query.GraphQueryResult;
-import org.openrdf.query.QueryLanguage;
-import org.openrdf.repository.RepositoryConnection;
-import org.openrdf.repository.RepositoryResult;
-
-import com.bigdata.rdf.store.BD;
-import com.tinkerpop.blueprints.Direction;
-import com.tinkerpop.blueprints.Edge;
-import com.tinkerpop.blueprints.Features;
-import com.tinkerpop.blueprints.Graph;
-import com.tinkerpop.blueprints.GraphQuery;
-import com.tinkerpop.blueprints.Vertex;
-
-/**
- * A base class for a Blueprints wrapper around a bigdata back-end.
- *
- * @author mikepersonick
- *
- */
-public abstract class BigdataGraph implements Graph {
-
- public static final URI VERTEX = new URIImpl(BD.NAMESPACE + "Vertex");
-
- public static final URI EDGE = new URIImpl(BD.NAMESPACE + "Edge");
-
- final BlueprintsRDFFactory factory;
-
- public BigdataGraph(final BlueprintsRDFFactory factory) {
-
- this.factory = factory;
-
- }
-
- public String toString() {
- return getClass().getSimpleName().toLowerCase();
- }
-
- /**
- * Post a GraphML file to the remote server. (Bulk-upload operation.)
- */
- public void loadGraphML(final String file) throws Exception {
-// GraphMLReader.inputGraph(this, file);
- }
-
- protected abstract RepositoryConnection cxn() throws Exception;
-
- /**
- * Return a property for an edge or vertex.
- */
- public Object getProperty(final URI uri, final String property) {
-
- return getProperty(uri, factory.toPropertyURI(property));
-
- }
-
- /**
- * Return a property for an edge or vertex.
- */
- public Object getProperty(final URI uri, final URI property) {
-
- try {
-
- final RepositoryResult<Statement> result =
- cxn().getStatements(uri, property, null, false);
-
- if (result.hasNext()) {
-
- final Value value = result.next().getObject();
-
- if (result.hasNext()) {
- throw new RuntimeException(uri
- + ": more than one value for p: " + property
- + ", did you mean to call getProperties()?");
- }
-
- if (!(value instanceof Literal)) {
- throw new RuntimeException("not a property: " + value);
- }
-
- final Literal lit = (Literal) value;
-
- return factory.fromLiteral(lit);
-
- }
-
- return null;
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- /**
- * Return a collection of properties for an edge or vertex.
- */
- public List<Object> getProperties(final URI uri, final URI property) {
-
- try {
-
- final RepositoryResult<Statement> result =
- cxn().getStatements(uri, property, null, false);
-
- final List<Object> props = new LinkedList<Object>();
-
- while (result.hasNext()) {
-
- final Value value = result.next().getObject();
-
- if (!(value instanceof Literal)) {
- throw new RuntimeException("not a property: " + value);
- }
-
- final Literal lit = (Literal) value;
-
- props.add(factory.fromLiteral(lit));
-
- }
-
- return props;
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- /**
- * Return the property keys used by an edge or vertex.
- */
- public Set<String> getPropertyKeys(final URI uri) {
-
- try {
-
- final RepositoryResult<Statement> result =
- cxn().getStatements(uri, null, null, false);
-
- final Set<String> properties = new LinkedHashSet<String>();
-
- while (result.hasNext()) {
-
- final Statement stmt = result.next();
-
- if (!(stmt.getObject() instanceof Literal)) {
- continue;
- }
-
- if (stmt.getPredicate().equals(RDFS.LABEL)) {
- continue;
- }
-
- final String p =
- factory.fromPropertyURI(stmt.getPredicate());
-
- properties.add(p);
-
- }
-
- return properties;
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- /**
- * Remove all values for a particular property on an edge or vertex.
- */
- public Object removeProperty(final URI uri, final String property) {
-
- return removeProperty(uri, factory.toPropertyURI(property));
-
- }
-
- /**
- * Remove all values for a particular property on an edge or vertex.
- */
- public Object removeProperty(final URI uri, final URI property) {
-
- try {
-
- final Object oldVal = getProperty(uri, property);
-
- cxn().remove(uri, property, null);
-
- return oldVal;
-
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
-
- }
-
- /**
- */
- public void setProperty(final URI s, final URI p, final Literal o) {
-
- try {
-
- cxn().remove(s, p, null);
-
- cxn().add(s, p, o);
-
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
-
- }
-
- @Override
- public Edge addEdge(final Object key, final Vertex from, final Vertex to,
- final String label) {
-
- if (label == null) {
- throw new IllegalArgumentException();
- }
-
- final String eid = key != null ? key.toString() : UUID.randomUUID().toString();
-
- final URI edgeURI = factory.toEdgeURI(eid);
-
- if (key != null) {
-
- final Edge edge = getEdge(key);
-
- if (edge != null) {
- if (!(edge.getVertex(Direction.OUT).equals(from) &&
- (edge.getVertex(Direction.OUT).equals(to)))) {
- throw new IllegalArgumentException("edge already exists: " + key);
- }
- }
-
- }
-
- try {
-
-// if (cxn().hasStatement(edgeURI, RDF.TYPE, EDGE, false)) {
-// throw new IllegalArgumentException("edge " + eid + " already exists");
-// }
-
- final URI fromURI = factory.toVertexURI(from.getId().toString());
- final URI toURI = factory.toVertexURI(to.getId().toString());
-
- cxn().add(fromURI, edgeURI, toURI);
- cxn().add(edgeURI, RDF.TYPE, EDGE);
- cxn().add(edgeURI, RDFS.LABEL, factory.toLiteral(label));
-
- return new BigdataEdge(new StatementImpl(fromURI, edgeURI, toURI), this);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- @Override
- public Vertex addVertex(final Object key) {
-
- try {
-
- final String vid = key != null ?
- key.toString() : UUID.randomUUID().toString();
-
- final URI uri = factory.toVertexURI(vid);
-
-// if (cxn().hasStatement(vertexURI, RDF.TYPE, VERTEX, false)) {
-// throw new IllegalArgumentException("vertex " + vid + " already exists");
-// }
-
- cxn().add(uri, RDF.TYPE, VERTEX);
-
- return new BigdataVertex(uri, this);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- @Override
- public Edge getEdge(final Object key) {
-
- if (key == null)
- throw new IllegalArgumentException();
-
- try {
-
- final URI edge = factory.toEdgeURI(key.toString());
-
- final RepositoryResult<Statement> result =
- cxn().getStatements(null, edge, null, false);
-
- if (result.hasNext()) {
-
- final Statement stmt = result.next();
-
- if (result.hasNext()) {
- throw new RuntimeException(
- "duplicate edge: " + key);
- }
-
- return new BigdataEdge(stmt, this);
-
- }
-
- return null;
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- @Override
- public Iterable<Edge> getEdges() {
-
- final URI wild = null;
- return getEdges(wild, wild);
-
- }
-
- public Iterable<Edge> getEdges(final URI s, final URI o, final String... labels) {
-
- final StringBuilder sb = new StringBuilder();
- sb.append("construct { ?from ?edge ?to . } where {\n");
- sb.append("?edge rdf:type bd:Edge . ?from ?edge ?to .\n");
- if (labels != null && labels.length > 0) {
- if (labels.length == 1) {
- sb.append("?edge rdfs:label \"").append(labels[0]).append("\" .\n");
- } else {
- sb.append("?edge rdfs:label ?label .\n");
- sb.append("filter(?label in (");
- for (String label : labels) {
- sb.append("\""+label+"\", ");
- }
- sb.setLength(sb.length()-2);
- sb.append(")) .\n");
- }
- }
- sb.append("}");
-
- final String queryStr = sb.toString()
- .replace("?from", s != null ? "<"+s+">" : "?from")
- .replace("?to", o != null ? "<"+o+">" : "?to");
-
- return getEdges(queryStr);
-
- }
-
- Iterable<Edge> getEdges(final String queryStr) {
-
- try {
-
- final org.openrdf.query.GraphQuery query =
- cxn().prepareGraphQuery(QueryLanguage.SPARQL, queryStr);
-
- final GraphQueryResult stmts = query.evaluate();
-
- return new EdgeIterable(stmts);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- public Iterable<Vertex> getVertices(final URI s, final URI o,
- final String... labels) {
-
- if (s != null && o != null) {
- throw new IllegalArgumentException();
- }
-
- if (s == null && o == null) {
- throw new IllegalArgumentException();
- }
-
- final StringBuilder sb = new StringBuilder();
- sb.append("construct { ?from ?edge ?to . } where {\n");
- sb.append("?edge rdf:type bd:Edge . ?from ?edge ?to .\n");
- if (labels != null && labels.length > 0) {
- if (labels.length == 1) {
- sb.append("?edge rdfs:label \"").append(labels[0]).append("\" .\n");
- } else {
- sb.append("?edge rdfs:label ?label .\n");
- sb.append("filter(?label in (");
- for (String label : labels) {
- sb.append("\""+label+"\", ");
- }
- sb.setLength(sb.length()-2);
- sb.append(")) .\n");
- }
- }
- sb.append("}");
-
- final String queryStr = sb.toString()
- .replace("?from", s != null ? "<"+s+">" : "?from")
- .replace("?to", o != null ? "<"+o+">" : "?to");
-
- return getVertices(queryStr, s == null);
-
- }
-
- Iterable<Vertex> getVertices(final String queryStr, final boolean subject) {
-
- try {
-
- final org.openrdf.query.GraphQuery query =
- cxn().prepareGraphQuery(QueryLanguage.SPARQL, queryStr);
-
- final GraphQueryResult stmts = query.evaluate();
-
- return new VertexIterable(stmts, subject);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- public final <T> Iterable<T> fuse(final Iterable<T>... args) {
-
- return new FusedIterable<T>(args);
- }
-
-
- @Override
- public Iterable<Edge> getEdges(final String prop, final Object val) {
-
- final URI p = factory.toPropertyURI(prop);
- final Literal o = factory.toLiteral(val);
-
- try {
-
- final String queryStr = IOUtils.toString(
- getClass().getResourceAsStream("edgesByProperty.rq"))
- .replace("?prop", "<"+p+">")
- .replace("?val", o.toString());
-
- final org.openrdf.query.GraphQuery query =
- cxn().prepareGraphQuery(QueryLanguage.SPARQL, queryStr);
-
- final GraphQueryResult stmts = query.evaluate();
-
- return new EdgeIterable(stmts);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- @Override
- public Features getFeatures() {
-
- return FEATURES;
-
- }
-
- @Override
- public Vertex getVertex(final Object key) {
-
- if (key == null)
- throw new IllegalArgumentException();
-
- final URI uri = factory.toVertexURI(key.toString());
- try {
- if (cxn().hasStatement(uri, RDF.TYPE, VERTEX, false)) {
- return new BigdataVertex(uri, this);
- }
- return null;
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- @Override
- public Iterable<Vertex> getVertices() {
-
- try {
- final RepositoryResult<Statement> result =
- cxn().getStatements(null, RDF.TYPE, VERTEX, false);
- return new VertexIterable(result, true);
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- @Override
- public Iterable<Vertex> getVertices(String prop, Object val) {
-
- final URI p = factory.toPropertyURI(prop);
- final Literal o = factory.toLiteral(val);
- try {
- final RepositoryResult<Statement> result =
- cxn().getStatements(null, p, o, false);
- return new VertexIterable(result, true);
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- /**
- * Providing an override implementation for our GraphQuery to avoid the
- * low-performance scan and filter paradigm. See {@link BigdataGraphQuery}.
- */
- @Override
- public GraphQuery query() {
-// return new DefaultGraphQuery(this);
- return new BigdataGraphQuery(this);
- }
-
- @Override
- public void removeEdge(final Edge edge) {
- try {
- final URI uri = factory.toURI(edge);
- if (!cxn().hasStatement(uri, RDF.TYPE, EDGE, false)) {
- throw new IllegalStateException();
- }
- final URI wild = null;
- // remove the edge statement
- cxn().remove(wild, uri, wild);
- // remove its properties
- cxn().remove(uri, wild, wild);
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void removeVertex(final Vertex vertex) {
- try {
- final URI uri = factory.toURI(vertex);
- if (!cxn().hasStatement(uri, RDF.TYPE, VERTEX, false)) {
- throw new IllegalStateException();
- }
- final URI wild = null;
- // remove outgoing links and properties
- cxn().remove(uri, wild, wild);
- // remove incoming links
- cxn().remove(wild, wild, uri);
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- /**
- * Translate a collection of Bigdata statements into an iteration of
- * Blueprints vertices.
- *
- * @author mikepersonick
- *
- */
- public class VertexIterable implements Iterable<Vertex>, Iterator<Vertex> {
-
- private final CloseableIteration<Statement, ? extends OpenRDFException> stmts;
-
- private final boolean subject;
-
- private final List<Vertex> cache;
-
- public VertexIterable(
- final CloseableIteration<Statement, ? extends OpenRDFException> stmts,
- final boolean subject) {
- this.stmts = stmts;
- this.subject = subject;
- this.cache = new LinkedList<Vertex>();
- }
-
- @Override
- public boolean hasNext() {
- try {
- return stmts.hasNext();
- } catch (OpenRDFException e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public Vertex next() {
- try {
- final Statement stmt = stmts.next();
- final URI v = (URI)
- (subject ? stmt.getSubject() : stmt.getObject());
- if (!hasNext()) {
- stmts.close();
- }
- final Vertex vertex = new BigdataVertex(v, BigdataGraph.this);
- cache.add(vertex);
- return vertex;
- } catch (OpenRDFException e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void remove() {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterator<Vertex> iterator() {
- return hasNext() ? this : cache.iterator();
- }
-
- }
-
- /**
- * Translate a collection of Bigdata statements into an iteration of
- * Blueprints edges.
- *
- * @author mikepersonick
- *
- */
- public class EdgeIterable implements Iterable<Edge>, Iterator<Edge> {
-
- private final CloseableIteration<Statement, ? extends OpenRDFException> stmts;
-
- private final List<Edge> cache;
-
- public EdgeIterable(
- final CloseableIteration<Statement, ? extends OpenRDFException> stmts) {
- this.stmts = stmts;
- this.cache = new LinkedList<Edge>();
- }
-
- @Override
- public boolean hasNext() {
- try {
- return stmts.hasNext();
- } catch (OpenRDFException e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public Edge next() {
- try {
- final Statement stmt = stmts.next();
- if (!hasNext()) {
- stmts.close();
- }
- final Edge edge = new BigdataEdge(stmt, BigdataGraph.this);
- cache.add(edge);
- return edge;
- } catch (OpenRDFException e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void remove() {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterator<Edge> iterator() {
- return hasNext() ? this : cache.iterator();
- }
-
- }
-
- /**
- * Fuse two iterables together into one. Useful for combining IN and OUT
- * edges for a vertex.
- *
- * @author mikepersonick
- */
- public class FusedIterable<T> implements Iterable<T>, Iterator<T> {
-
- private final Iterable<T>[] args;
-
- private transient int i = 0;
-
- private transient Iterator<T> curr;
-
- public FusedIterable(final Iterable<T>... args) {
- this.args = args;
- this.curr = args[0].iterator();
- }
-
- @Override
- public boolean hasNext() {
- if (curr.hasNext()) {
- return true;
- }
- while (!curr.hasNext() && i < (args.length-1)) {
- curr = args[++i].iterator();
- if (curr.hasNext()) {
- return true;
- }
- }
- return false;
- }
-
- @Override
- public T next() {
- return curr.next();
- }
-
- @Override
- public void remove() {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterator<T> iterator() {
- return this;
- }
-
- }
-
- protected static final Features FEATURES = new Features();
-
- static {
-
- FEATURES.supportsSerializableObjectProperty = false;
- FEATURES.supportsBooleanProperty = true;
- FEATURES.supportsDoubleProperty = true;
- FEATURES.supportsFloatProperty = true;
- FEATURES.supportsIntegerProperty = true;
- FEATURES.supportsPrimitiveArrayProperty = false;
- FEATURES.supportsUniformListProperty = false;
- FEATURES.supportsMixedListProperty = false;
- FEATURES.supportsLongProperty = true;
- FEATURES.supportsMapProperty = false;
- FEATURES.supportsStringProperty = true;
-
- FEATURES.supportsDuplicateEdges = true;
- FEATURES.supportsSelfLoops = true;
- FEATURES.isPersistent = true;
- FEATURES.isWrapper = false;
- FEATURES.supportsVertexIteration = true;
- FEATURES.supportsEdgeIteration = true;
- FEATURES.supportsVertexIndex = false;
- FEATURES.supportsEdgeIndex = false;
- FEATURES.ignoresSuppliedIds = true;
- FEATURES.supportsTransactions = false;
- FEATURES.supportsIndices = true;
- FEATURES.supportsKeyIndices = true;
- FEATURES.supportsVertexKeyIndex = true;
- FEATURES.supportsEdgeKeyIndex = true;
- FEATURES.supportsEdgeRetrieval = true;
- FEATURES.supportsVertexProperties = true;
- FEATURES.supportsEdgeProperties = true;
- FEATURES.supportsThreadedTransactions = false;
- }
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,325 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import java.util.UUID;
-
-import org.openrdf.model.Literal;
-import org.openrdf.model.URI;
-import org.openrdf.model.impl.StatementImpl;
-import org.openrdf.model.vocabulary.RDF;
-import org.openrdf.model.vocabulary.RDFS;
-import org.openrdf.repository.RepositoryConnection;
-
-import com.bigdata.rdf.changesets.IChangeLog;
-import com.bigdata.rdf.changesets.IChangeRecord;
-import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
-import com.tinkerpop.blueprints.Edge;
-import com.tinkerpop.blueprints.GraphQuery;
-import com.tinkerpop.blueprints.TransactionalGraph;
-import com.tinkerpop.blueprints.Vertex;
-
-/**
- * Simple bulk loader that will insert graph data without any consistency
- * checking (won't check for duplicate vertex or edge identifiers). Currently
- * does not overwrite old property values, but we may need to change this.
- * <p>
- * Implements {@link IChangeLog} so that we can report a mutation count.
- *
- * @author mikepersonick
- *
- */
-public class BigdataGraphBulkLoad extends BigdataGraph
- implements TransactionalGraph, IChangeLog {
-
- private final BigdataSailRepositoryConnection cxn;
-
- public BigdataGraphBulkLoad(final BigdataSailRepositoryConnection cxn) {
- this(cxn, BigdataRDFFactory.INSTANCE);
- }
-
- public BigdataGraphBulkLoad(final BigdataSailRepositoryConnection cxn,
- final BlueprintsRDFFactory factory) {
- super(factory);
-
- this.cxn = cxn;
- this.cxn.addChangeLog(this);
- }
-
- protected RepositoryConnection cxn() throws Exception {
- return cxn;
- }
-
- @Override
- public void commit() {
- try {
- cxn.commit();
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void rollback() {
- try {
- cxn.rollback();
- cxn.close();
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void shutdown() {
- try {
- cxn.close();
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- @Deprecated
- public void stopTransaction(Conclusion arg0) {
- }
-
-
- static {
-
-// FEATURES.supportsSerializableObjectProperty = false;
-// FEATURES.supportsBooleanProperty = true;
-// FEATURES.supportsDoubleProperty = true;
-// FEATURES.supportsFloatProperty = true;
-// FEATURES.supportsIntegerProperty = true;
-// FEATURES.supportsPrimitiveArrayProperty = false;
-// FEATURES.supportsUniformListProperty = false;
-// FEATURES.supportsMixedListProperty = false;
-// FEATURES.supportsLongProperty = true;
-// FEATURES.supportsMapProperty = false;
-// FEATURES.supportsStringProperty = true;
-//
-// FEATURES.supportsDuplicateEdges = true;
-// FEATURES.supportsSelfLoops = true;
-// FEATURES.isPersistent = true;
-// FEATURES.isWrapper = false;
-// FEATURES.supportsVertexIteration = true;
-// FEATURES.supportsEdgeIteration = true;
-// FEATURES.supportsVertexIndex = false;
-// FEATURES.supportsEdgeIndex = false;
-// FEATURES.ignoresSuppliedIds = true;
- BigdataGraph.FEATURES.supportsTransactions = true;
-// FEATURES.supportsIndices = true;
-// FEATURES.supportsKeyIndices = true;
-// FEATURES.supportsVertexKeyIndex = true;
-// FEATURES.supportsEdgeKeyIndex = true;
-// FEATURES.supportsEdgeRetrieval = true;
-// FEATURES.supportsVertexProperties = true;
-// FEATURES.supportsEdgeProperties = true;
-// FEATURES.supportsThreadedTransactions = false;
- }
-
-
- @Override
- public Edge getEdge(Object arg0) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterable<Edge> getEdges() {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterable<Edge> getEdges(String arg0, Object arg1) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Vertex getVertex(Object arg0) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterable<Vertex> getVertices() {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Iterable<Vertex> getVertices(String arg0, Object arg1) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public GraphQuery query() {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public void removeEdge(Edge arg0) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public void removeVertex(Vertex arg0) {
- throw new UnsupportedOperationException();
- }
-
- /**
- * Set a property without removing the old value first.
- */
- @Override
- public void setProperty(final URI s, final URI p, final Literal o) {
-
- try {
-
-// cxn().remove(s, p, null);
-
- cxn().add(s, p, o);
-
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
-
- }
-
- /**
- * Add a vertex without consistency checking (does not check for a duplicate
- * identifier).
- */
- @Override
- public Vertex addVertex(final Object key) {
-
- try {
-
- final String vid = key != null ?
- key.toString() : UUID.randomUUID().toString();
-
- final URI uri = factory.toVertexURI(vid);
-
-// if (cxn().hasStatement(vertexURI, RDF.TYPE, VERTEX, false)) {
-// throw new IllegalArgumentException("vertex " + vid + " already exists");
-// }
-
- cxn().add(uri, RDF.TYPE, VERTEX);
-
- return new BigdataVertex(uri, this);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- /**
- * Add an edge without consistency checking (does not check for a duplicate
- * identifier).
- */
- @Override
- public Edge addEdge(final Object key, final Vertex from, final Vertex to,
- final String label) {
-
- if (label == null) {
- throw new IllegalArgumentException();
- }
-
- final String eid = key != null ? key.toString() : UUID.randomUUID().toString();
-
- final URI edgeURI = factory.toEdgeURI(eid);
-
-// if (key != null) {
-//
-// final Edge edge = getEdge(key);
-//
-// if (edge != null) {
-// if (!(edge.getVertex(Direction.OUT).equals(from) &&
-// (edge.getVertex(Direction.OUT).equals(to)))) {
-// throw new IllegalArgumentException("edge already exists: " + key);
-// }
-// }
-//
-// }
-
- try {
-
-// if (cxn().hasStatement(edgeURI, RDF.TYPE, EDGE, false)) {
-// throw new IllegalArgumentException("edge " + eid + " already exists");
-// }
-
- final URI fromURI = factory.toVertexURI(from.getId().toString());
- final URI toURI = factory.toVertexURI(to.getId().toString());
-
- cxn().add(fromURI, edgeURI, toURI);
- cxn().add(edgeURI, RDF.TYPE, EDGE);
- cxn().add(edgeURI, RDFS.LABEL, factory.toLiteral(label));
-
- return new BigdataEdge(new StatementImpl(fromURI, edgeURI, toURI), this);
-
- } catch (Exception ex) {
- throw new RuntimeException(ex);
- }
-
- }
-
- private transient long mutationCountTotal = 0;
- private transient long mutationCountCurrentCommit = 0;
- private transient long mutationCountLastCommit = 0;
-
- @Override
- public void changeEvent(final IChangeRecord record) {
- mutationCountTotal++;
- mutationCountCurrentCommit++;
- }
-
- @Override
- public void transactionBegin() {
- }
-
- @Override
- public void transactionPrepare() {
- }
-
- @Override
- public void transactionCommited(long commitTime) {
- mutationCountLastCommit = mutationCountCurrentCommit;
- mutationCountCurrentCommit = 0;
- }
-
- @Override
- public void transactionAborted() {
- }
-
- public long getMutationCountTotal() {
- return mutationCountTotal;
- }
-
- public long getMutationCountCurrentCommit() {
- return mutationCountCurrentCommit;
- }
-
- public long getMutationCountLastCommit() {
- return mutationCountLastCommit;
- }
-
-
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,113 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import com.bigdata.rdf.sail.remote.BigdataSailRemoteRepository;
-import com.bigdata.rdf.sail.remote.BigdataSailRemoteRepositoryConnection;
-import com.bigdata.rdf.sail.webapp.client.RemoteRepository;
-
-/**
- * This is a thin-client implementation of a Blueprints wrapper around the
- * client library that interacts with the NanoSparqlServer. This is a functional
- * implementation suitable for writing POCs - it is not a high performance
- * implementation by any means (currently does not support caching, batched
- * update, or Blueprints query re-writes). Does have a single "bulk upload"
- * operation that wraps a method on RemoteRepository that will POST a graphml
- * file to the blueprints layer of the bigdata server.
- *
- * @see {@link BigdataSailRemoteRepository}
- * @see {@link BigdataSailRemoteRepositoryConnection}
- * @see {@link RemoteRepository}
- *
- * @author mikepersonick
- *
- */
-public class BigdataGraphClient extends BigdataGraph {
-
- final BigdataSailRemoteRepository repo;
-
- transient BigdataSailRemoteRepositoryConnection cxn;
-
- public BigdataGraphClient(final String bigdataEndpoint) {
- this(bigdataEndpoint, BigdataRDFFactory.INSTANCE);
- }
-
- public BigdataGraphClient(final String bigdataEndpoint,
- final BlueprintsRDFFactory factory) {
- this(new BigdataSailRemoteRepository(bigdataEndpoint), factory);
- }
-
- public BigdataGraphClient(final RemoteRepository repo) {
- this(repo, BigdataRDFFactory.INSTANCE);
- }
-
- public BigdataGraphClient(final RemoteRepository repo,
- final BlueprintsRDFFactory factory) {
- this(new BigdataSailRemoteRepository(repo), factory);
- }
-
- public BigdataGraphClient(final BigdataSailRemoteRepository repo) {
- this(repo, BigdataRDFFactory.INSTANCE);
- }
-
- public BigdataGraphClient(final BigdataSailRemoteRepository repo,
- final BlueprintsRDFFactory factory) {
- super(factory);
-
- this.repo = repo;
- }
-
- /**
- * Post a GraphML file to the remote server. (Bulk-upload operation.)
- */
- @Override
- public void loadGraphML(final String file) throws Exception {
- this.repo.getRemoteRepository().postGraphML(file);
- }
-
- /**
- * Get a {@link BigdataSailRemoteRepositoryConnection}.
- */
- protected BigdataSailRemoteRepositoryConnection cxn() throws Exception {
- if (cxn == null) {
- cxn = repo.getConnection();
- }
- return cxn;
- }
-
- /**
- * Shutdown the connection and repository (client-side, not server-side).
- */
- @Override
- public void shutdown() {
- try {
- if (cxn != null) {
- cxn.close();
- }
- repo.shutDown();
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,166 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import org.openrdf.repository.RepositoryConnection;
-
-import com.bigdata.rdf.sail.BigdataSail;
-import com.bigdata.rdf.sail.BigdataSailRepository;
-import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
-import com.tinkerpop.blueprints.TransactionalGraph;
-
-/**
- * This is the most basic possible implementation of the Blueprints Graph API.
- * It wraps an embedded {@link BigdataSailRepository} and holds open an
- * unisolated connection to the database for the lifespan of the Graph (until
- * {@link #shutdown()} is called.
- *
- * @author mikepersonick
- *
- */
-public class BigdataGraphEmbedded extends BigdataGraph implements TransactionalGraph {
-
- final BigdataSailRepository repo;
-
- transient BigdataSailRepositoryConnection cxn;
-
- /**
- * Create a Blueprints wrapper around a {@link BigdataSail} instance.
- */
- public BigdataGraphEmbedded(final BigdataSail sail) {
- this(sail, BigdataRDFFactory.INSTANCE);
- }
-
- /**
- * Create a Blueprints wrapper around a {@link BigdataSail} instance with
- * a non-standard {@link BlueprintsRDFFactory} implementation.
- */
- public BigdataGraphEmbedded(final BigdataSail sail,
- final BlueprintsRDFFactory factory) {
- this(new BigdataSailRepository(sail), factory);
- }
-
- /**
- * Create a Blueprints wrapper around a {@link BigdataSailRepository}
- * instance.
- */
- public BigdataGraphEmbedded(final BigdataSailRepository repo) {
- this(repo, BigdataRDFFactory.INSTANCE);
- }
-
- /**
- * Create a Blueprints wrapper around a {@link BigdataSailRepository}
- * instance with a non-standard {@link BlueprintsRDFFactory} implementation.
- */
- public BigdataGraphEmbedded(final BigdataSailRepository repo,
- final BlueprintsRDFFactory factory) {
- super(factory);
-
- this.repo = repo;
- }
-
- protected RepositoryConnection cxn() throws Exception {
- if (cxn == null) {
- cxn = repo.getUnisolatedConnection();
- cxn.setAutoCommit(false);
- }
- return cxn;
- }
-
- @Override
- public void commit() {
- try {
- if (cxn != null)
- cxn.commit();
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void rollback() {
- try {
- if (cxn != null) {
- cxn.rollback();
- cxn.close();
- cxn = null;
- }
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public void shutdown() {
- try {
- if (cxn != null) {
- cxn.close();
- }
- repo.shutDown();
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- @Deprecated
- public void stopTransaction(Conclusion arg0) {
- }
-
-
- static {
-
-// FEATURES.supportsSerializableObjectProperty = false;
-// FEATURES.supportsBooleanProperty = true;
-// FEATURES.supportsDoubleProperty = true;
-// FEATURES.supportsFloatProperty = true;
-// FEATURES.supportsIntegerProperty = true;
-// FEATURES.supportsPrimitiveArrayProperty = false;
-// FEATURES.supportsUniformListProperty = false;
-// FEATURES.supportsMixedListProperty = false;
-// FEATURES.supportsLongProperty = true;
-// FEATURES.supportsMapProperty = false;
-// FEATURES.supportsStringProperty = true;
-//
-// FEATURES.supportsDuplicateEdges = true;
-// FEATURES.supportsSelfLoops = true;
-// FEATURES.isPersistent = true;
-// FEATURES.isWrapper = false;
-// FEATURES.supportsVertexIteration = true;
-// FEATURES.supportsEdgeIteration = true;
-// FEATURES.supportsVertexIndex = false;
-// FEATURES.supportsEdgeIndex = false;
-// FEATURES.ignoresSuppliedIds = true;
- BigdataGraph.FEATURES.supportsTransactions = true;
-// FEATURES.supportsIndices = true;
-// FEATURES.supportsKeyIndices = true;
-// FEATURES.supportsVertexKeyIndex = true;
-// FEATURES.supportsEdgeKeyIndex = true;
-// FEATURES.supportsEdgeRetrieval = true;
-// FEATURES.supportsVertexProperties = true;
-// FEATURES.supportsEdgeProperties = true;
-// FEATURES.supportsThreadedTransactions = false;
- }
-
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,86 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import org.apache.log4j.Logger;
-
-import com.bigdata.rdf.sail.BigdataSail;
-import com.bigdata.rdf.sail.BigdataSailFactory;
-
-
-/**
- * Helper class to create BigdataGraph instances.
- *
- * @author mikepersonick
- *
- */
-public class BigdataGraphFactory {
-
- protected static final transient Logger log = Logger.getLogger(BigdataGraphFactory.class);
-
- /**
- * Connect to a remote bigdata instance.
- */
- public static BigdataGraph connect(final String serviceEndpoint) {
- if (serviceEndpoint.endsWith("/bigdata/sparql")) {
- return new BigdataGraphClient(serviceEndpoint);
- } else if (serviceEndpoint.endsWith("/bigdata/")) {
- return new BigdataGraphClient(serviceEndpoint + "sparql");
- } else if (serviceEndpoint.endsWith("/bigdata")) {
- return new BigdataGraphClient(serviceEndpoint + "/sparql");
- } else if (serviceEndpoint.endsWith("/")) {
- return new BigdataGraphClient(serviceEndpoint + "bigdata/sparql");
- } else {
- return new BigdataGraphClient(serviceEndpoint + "/bigdata/sparql");
- }
- }
-
- /**
- * Open an existing persistent local bigdata instance.
- */
- public static BigdataGraph open(final String file) throws Exception {
- final BigdataSail sail = BigdataSailFactory.openSail(file);
- sail.initialize();
- return new BigdataGraphEmbedded(sail);
- }
-
- /**
- * Create a new local in-memory bigdata instance.
- */
- public static BigdataGraph create() throws Exception {
- final BigdataSail sail = BigdataSailFactory.createSail();
- sail.initialize();
- return new BigdataGraphEmbedded(sail);
- }
-
- /**
- * Create a new persistent local bigdata instance.
- */
- public static BigdataGraph create(final String file)
- throws Exception {
- final BigdataSail sail = BigdataSailFactory.createSail(file);
- sail.initialize();
- return new BigdataGraphEmbedded(sail);
- }
-
-}
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java 2014-05-12 01:44:40 UTC (rev 8276)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java 2014-05-12 01:47:38 UTC (rev 8277)
@@ -1,489 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.LinkedList;
-import java.util.List;
-
-import org.apache.log4j.Logger;
-import org.openrdf.model.Literal;
-import org.openrdf.model.URI;
-
-import com.tinkerpop.blueprints.Edge;
-import com.tinkerpop.blueprints.Predicate;
-import com.tinkerpop.blueprints.GraphQuery;
-import com.tinkerpop.blueprints.Vertex;
-
-/**
- * Translate a low-performance Blueprints GraphQuery into a high-performance
- * SPARQL query.
- *
- * @author mikepersonick
- *
- */
-public class BigdataGraphQuery implements GraphQuery {
-
- protected final static transient Logger log = Logger.getLogger(BigdataGraphQuery.class);
-
- /**
- * These are the only Predicate implementations we handle currently.
- */
- protected static List<Class> knownPredicates = Arrays.asList(new Class[] {
- BigdataPredicate.class,
- com.tinkerpop.blueprints.Query.Compare.class,
- com.tinkerpop.blueprints.Contains.class,
- com.tinkerpop.blueprints.Compare.class
- });
-
- /**
- * The graph.
- */
- private final BigdataGraph graph;
-
- /**
- * The list of criteria. Bigdata's query optimizer will re-order the
- * criteria based on selectivity and execute for maximum performance and
- * minimum IO.
- */
- private final List<Has> criteria = new LinkedList<Has>();
-
- /**
- * Limit the number of results.
- */
- private transient int limit = Integer.MAX_VALUE;
-
- public BigdataGraphQuery(final BigdataGraph graph) {
- this.graph = graph;
- }
-
- /**
- * Filter out elements that do not have a property with provided key.
- *
- * ?s <key> ?value
- *
- * @param key the key of the property
- * @return the modified query object
- */
- @Override
- public GraphQuery has(final String key) {
- criteria.add(new Has(key));
- return this;
- }
-
- /**
- * Filter out elements that have a property with provided key.
- *
- * ?s ?p ?o .
- * filter not exists { ?s <key> ?value } .
- *
- * @param key the key of the property
- * @return the modified query object
- */
- @Override
- public GraphQuery hasNot(final String key) {
- criteria.add(new HasNot(key));
- return this;
- }
-
- /**
- * Filter out elements that do not have a property value equal to provided value.
- *
- * ?s <key> <value> .
- *
- * @param key the key of the property
- * @param value the value to check against
- * @return the modified query object
- */
- @Override
- public GraphQuery has(final String key, final Object value) {
- criteria.add(new Has(key, value));
- return this;
- }
-
- /**
- * Filter out elements that have a property value equal to provided value.
- *
- * ?s ?p ?o .
- * filter not exists { ?s <key> <value> } .
- *
- * @param key the key of the property
- * @param value the value to check against
- * @return the modified query object
- */
- @Override
- public GraphQuery hasNot(final String key, final Object value) {
- criteria.add(new HasNot(key, value));
- return this;
- }
-
- /**
- * Filter out the element if it does not have a property with a comparable value.
- *
- * @param key the key of the property
- * @param predicate the comparator to use for comparison
- * @param value the value to check against
- * @return the modified query object
- */
- @Override
- public GraphQuery has(final String key, final Predicate predicate, final Object value) {
- if (!knownPredicates.contains(predicate.getClass())) {
- throw new IllegalArgumentException();
- }
- criteria.add(new Has(key, value, BigdataPredicate.toBigdataPredicate(predicate)));
- return this;
- }
-
- /**
- * Filter out the element if it does not have a property with a comparable value.
- *
- * @param key the key of the property
- * @param value the value to check against
- * @param compare the comparator to use for comparison
- * @return the modified query object
- */
- @Override
- @Deprecated
- public <T extends Comparable<T>> GraphQuery has(final String key, final T value,
- final Compare compare) {
- return has(key, compare, value);
- }
-
- /**
- * Filter out the element of its property value is not within the provided interval.
- *
- * @param key the key of the property
- * @param startValue the inclusive start value of the interval
- * @param endValue the exclusive end value of the interval
- * @return the modified query object
- */
- @Override
- public <T extends Comparable<?>> GraphQuery interval(final String key,
- final T startValue, final T endValue) {
- return has(key, BigdataPredicate.GTE, startValue)
- .has(key, BigdataPredicate.LT, endValue);
- }
-
- /**
- * Filter out the element if the take number of incident/adjacent elements to retrieve has already been reached.
- *
- * @param limit the take number of elements to return
- * @return the modified query object
- */
- @Override
- public GraphQuery limit(final int limit) {
- this.limit = limit;
- return this;
- }
-
- /**
- * Execute the query and return the matching edges.
- *
- * @return the unfiltered incident edges
- */
- @Override
- public Iterable<Edge> edges() {
- final String queryStr = toQueryStr(BigdataGraph.EDGE);
- return graph.getEdges(queryStr);
- }
-
- /**
- * Execute the query and return the vertices on the other end of the matching edges.
- *
- * @return the unfiltered adjacent vertices
- */
- @Override
- public Iterable<Vertex> vertices() {
- final String queryStr = toQueryStr(BigdataGraph.VERTEX);
- return graph.getVertices(queryStr, true);
- }
-
- /**
- * Generate the SPARQL query.
- */
- protected String toQueryStr(final URI type) {
-
- final StringBuilder sb = new StringBuilder();
-
- if (type == BigdataGraph.VERTEX) {
- sb.append("construct { ?x rdf:type <"+type+"> . }\n");
- sb.append("{\n select distinct ?x where {\n");
- } else {
- sb.append("construct { ?from ?x ?to . }\n");
- sb.append("{\n select distinct ?from ?x ?to where {\n");
- sb.append(" ?from ?x ?to .\n");
- }
-
- final BlueprintsRDFFactory factory = graph.factory;
-
- boolean hasHas = false;
-
- int i = 1;
- for (Has has : criteria) {
-
- if (log.isTraceEnabled()) {
- log.trace(has);
- }
-
- if (has instanceof HasNot) {
-
- sb.append(" filter not exists { ");
-
- sb.append("?x <").append(factory.toPropertyURI(has.key)).append("> ");
-
- if (has.val != null) {
-
- final...
[truncated message content] |
|
From: <mrp...@us...> - 2014-05-12 01:44:42
|
Revision: 8276
http://sourceforge.net/p/bigdata/code/8276
Author: mrpersonick
Date: 2014-05-12 01:44:40 +0000 (Mon, 12 May 2014)
Log Message:
-----------
once again, starting over from a clean CI run
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 01:44:07 UTC (rev 8275)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 01:44:40 UTC (rev 8276)
@@ -50,7 +50,7 @@
// // pull a Tinkerpop interface into the class loader
// log.info(com.tinkerpop.blueprints.Graph.class.getName());
- log.info(org.apache.avro.Schema.class.getName());
+// log.info(org.apache.avro.Schema.class.getName());
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 01:44:11
|
Revision: 8275
http://sourceforge.net/p/bigdata/code/8275
Author: mrpersonick
Date: 2014-05-12 01:44:07 +0000 (Mon, 12 May 2014)
Log Message:
-----------
entirely removing the blueprints dependency just to be sure
Removed Paths:
-------------
branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar
Deleted: branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar
===================================================================
(Binary files differ)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 00:01:35
|
Revision: 8274
http://sourceforge.net/p/bigdata/code/8274
Author: mrpersonick
Date: 2014-05-12 00:01:32 +0000 (Mon, 12 May 2014)
Log Message:
-----------
let's try adding a different dependency
Added Paths:
-----------
branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
Added: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
===================================================================
(Binary files differ)
Index: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar 2014-05-12 00:00:23 UTC (rev 8273)
+++ branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar 2014-05-12 00:01:32 UTC (rev 8274)
Property changes on: branches/BLUEPRINTS/bigdata-sails/lib/avro-1.7.6.jar
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/octet-stream
\ No newline at end of property
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-12 00:00:27
|
Revision: 8273
http://sourceforge.net/p/bigdata/code/8273
Author: mrpersonick
Date: 2014-05-12 00:00:23 +0000 (Mon, 12 May 2014)
Log Message:
-----------
let's try adding a different dependency
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-11 20:52:28 UTC (rev 8272)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-12 00:00:23 UTC (rev 8273)
@@ -115,12 +115,11 @@
// graph.commit();
//
// final long nmodified = graph.getMutationCountLastCommit();
- final long nmodified = 0;
-
- final long elapsed = System.currentTimeMillis() - begin;
+//
+// final long elapsed = System.currentTimeMillis() - begin;
+//
+// reportModifiedCount(resp, nmodified, elapsed);
- reportModifiedCount(resp, nmodified, elapsed);
-
return;
} catch(Throwable t) {
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-11 20:52:28 UTC (rev 8272)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-12 00:00:23 UTC (rev 8273)
@@ -47,8 +47,10 @@
.getLogger(RESTServlet.class);
static {
- // pull a Tinkerpop interface into the class loader
- log.info(com.tinkerpop.blueprints.Graph.class.getName());
+// // pull a Tinkerpop interface into the class loader
+// log.info(com.tinkerpop.blueprints.Graph.class.getName());
+
+ log.info(org.apache.avro.Schema.class.getName());
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-11 20:52:32
|
Revision: 8272
http://sourceforge.net/p/bigdata/code/8272
Author: mrpersonick
Date: 2014-05-11 20:52:28 +0000 (Sun, 11 May 2014)
Log Message:
-----------
Watch me magically break CI just by pulling an interface into the class loader:
static {
// pull a Tinkerpop interface into the class loader
log.info(com.tinkerpop.blueprints.Graph.class.getName());
}
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-11 16:41:02 UTC (rev 8271)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-11 20:52:28 UTC (rev 8272)
@@ -47,7 +47,8 @@
.getLogger(RESTServlet.class);
static {
-// log.info(Graph.class.getName());
+ // pull a Tinkerpop interface into the class loader
+ log.info(com.tinkerpop.blueprints.Graph.class.getName());
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-11 16:41:04
|
Revision: 8271
http://sourceforge.net/p/bigdata/code/8271
Author: mrpersonick
Date: 2014-05-11 16:41:02 +0000 (Sun, 11 May 2014)
Log Message:
-----------
remove all blueprints dependencies (should be a clean CI run)
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-11 15:45:17 UTC (rev 8270)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-11 16:41:02 UTC (rev 8271)
@@ -35,7 +35,6 @@
import org.openrdf.model.impl.URIImpl;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
-import com.tinkerpop.blueprints.Graph;
/**
* Default dispatch pattern for a core REST API.
@@ -48,7 +47,7 @@
.getLogger(RESTServlet.class);
static {
- log.info(Graph.class.getName());
+// log.info(Graph.class.getName());
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <jer...@us...> - 2014-05-11 15:45:20
|
Revision: 8270
http://sourceforge.net/p/bigdata/code/8270
Author: jeremy_carroll
Date: 2014-05-11 15:45:17 +0000 (Sun, 11 May 2014)
Log Message:
-----------
Restructured ConfigurableAnalyzerFactory to allow lazy one-time initialization with weak caching by UUID of namespace, giving clearer lifecycle management
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/TermCompletionAnalyzer.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestConfigurableAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestLanguageRange.java
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfiguredAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/LanguageRange.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/NeedsConfiguringAnalyzerFactory.java
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java 2014-05-11 15:08:38 UTC (rev 8269)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java 2014-05-11 15:45:17 UTC (rev 8270)
@@ -27,17 +27,9 @@
package com.bigdata.search;
import java.io.IOException;
+import java.io.Reader;
import java.io.StringReader;
-import java.lang.reflect.Constructor;
-import java.util.Arrays;
-import java.util.Collections;
-import java.util.Enumeration;
-import java.util.HashMap;
-import java.util.Locale;
-import java.util.Map;
-import java.util.Properties;
import java.util.Set;
-import java.util.concurrent.ConcurrentHashMap;
import java.util.regex.Pattern;
import org.apache.log4j.Logger;
@@ -45,15 +37,14 @@
import org.apache.lucene.analysis.KeywordAnalyzer;
import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.analysis.StopAnalyzer;
+import org.apache.lucene.analysis.TokenFilter;
+import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.WhitespaceAnalyzer;
import org.apache.lucene.analysis.miscellaneous.PatternAnalyzer;
-import org.apache.lucene.analysis.ru.RussianAnalyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
+import org.apache.lucene.analysis.tokenattributes.TermAttribute;
import org.apache.lucene.util.Version;
-import com.bigdata.btree.keys.IKeyBuilder;
-import com.bigdata.btree.keys.KeyBuilder;
-
/**
* This class can be used with the bigdata properties file to specify
* which {@link Analyzer}s are used for which languages.
@@ -127,127 +118,6 @@
final private static transient Logger log = Logger.getLogger(ConfigurableAnalyzerFactory.class);
/**
- * This is an implementation of RFC 4647 language range,
- * targetted at the specific needs within bigdata, and only
- * supporting the extended filtering specified in section 3.3.2
- * <p>
- * Language ranges are comparable so that
- * sorting an array and then matching a language tag against each
- * member of the array in sequence will give the longest match.
- * i.e. the longer ranges come first.
- * @author jeremycarroll
- *
- */
- public static class LanguageRange implements Comparable<LanguageRange> {
-
- private final String range[];
- private final String full;
- /**
- * Note range must be in lower case, this is not verified.
- * @param range
- */
- public LanguageRange(String range) {
- this.range = range.split("-");
- full = range;
- }
-
- @Override
- public int compareTo(LanguageRange o) {
- if (equals(o)) {
- return 0;
- }
- int diff = o.range.length - range.length;
- if (diff != 0) {
- // longest first
- return diff;
- }
- if (range.length == 1) {
- // * last
- if (range[0].equals("*")) {
- return 1;
- }
- if (o.range[0].equals("*")) {
- return -1;
- }
- }
- // alphabetically
- for (int i=0; i<range.length; i++) {
- diff = range[i].compareTo(o.range[i]);
- if (diff != 0) {
- return diff;
- }
- }
- throw new RuntimeException("Impossible - supposedly");
- }
-
- @Override
- public boolean equals(Object o) {
- return (o instanceof LanguageRange) && ((LanguageRange)o).full.equals(full);
- }
- @Override
- public int hashCode() {
- return full.hashCode();
- }
-
- /**
- * This implements the algoirthm of section 3.3.2 of RFC 4647
- * as modified with the observation about private use tags
- * in <a href="http://lists.w3.org/Archives/Public/www-international/2014AprJun/0084">
- * this message</a>.
- *
- *
- * @param langTag The RFC 5646 Language tag in lower case
- * @return The result of the algorithm
- */
- public boolean extendedFilterMatch(String langTag) {
- return extendedFilterMatch(langTag.toLowerCase(Locale.ROOT).split("-"));
- }
-
- // See RFC 4647, 3.3.2
- boolean extendedFilterMatch(String[] language) {
- // RFC 4647 step 2
- if (!matchSubTag(language[0], range[0])) {
- return false;
- }
- int rPos = 1;
- int lPos = 1;
- // variant step - for private use flags
- if (language[0].equals("x") && range[0].equals("*")) {
- lPos = 0;
- }
- // RFC 4647 step 3
- while (rPos < range.length) {
- // step 3A
- if (range[rPos].equals("*")) {
- rPos ++;
- continue;
- }
- // step 3B
- if (lPos >= language.length) {
- return false;
- }
- // step 3C
- if (matchSubTag(language[lPos], range[rPos])) {
- lPos++;
- rPos++;
- continue;
- }
- if (language[lPos].length()==1) {
- return false;
- }
- lPos++;
- }
- // RFC 4647 step 4
- return true;
- }
-
- // RFC 4647, 3.3.2, step 1
- private boolean matchSubTag(String langSubTag, String rangeSubTag) {
- return langSubTag.equals(rangeSubTag) || "*".equals(rangeSubTag);
- }
-
- }
- /**
* Options understood by the {@link ConfigurableAnalyzerFactory}.
*/
public interface Options {
@@ -394,638 +264,55 @@
}
- private static final String ALL_LUCENE_NATURAL_LANGUAGES =
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.*.like=eng\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.por.analyzerClass=org.apache.lucene.analysis.br.BrazilianAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.pt.like=por\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.zho.analyzerClass=org.apache.lucene.analysis.cn.ChineseAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.chi.like=zho\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.zh.like=zho\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.jpn.analyzerClass=org.apache.lucene.analysis.cjk.CJKAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ja.like=jpn\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.kor.like=jpn\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ko.like=kor\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ces.analyzerClass=org.apache.lucene.analysis.cz.CzechAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.cze.like=ces\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.cs.like=ces\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.dut.analyzerClass=org.apache.lucene.analysis.nl.DutchAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.nld.like=dut\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.nl.like=dut\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.deu.analyzerClass=org.apache.lucene.analysis.de.GermanAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ger.like=deu\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.de.like=deu\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.gre.analyzerClass=org.apache.lucene.analysis.el.GreekAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ell.like=gre\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.el.like=gre\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.rus.analyzerClass=org.apache.lucene.analysis.ru.RussianAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ru.like=rus\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.tha.analyzerClass=org.apache.lucene.analysis.th.ThaiAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.th.like=tha\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.eng.analyzerClass=org.apache.lucene.analysis.standard.StandardAnalyzer\n" +
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.en.like=eng\n";
-
- private static final String LUCENE_STANDARD_ANALYZER =
- "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.*.analyzerClass=org.apache.lucene.analysis.standard.StandardAnalyzer\n";
-
- /**
- * This comment describes the implementation of {@link ConfigurableAnalyzerFactory}.
- * The only method in the interface is {@link ConfigurableAnalyzerFactory#getAnalyzer(String, boolean)},
- * a map is used from language tag to {@link AnalyzerPair}, where the pair contains
- * an {@link Analyzer} both with and without stopwords configured (some times these two analyzers are identical,
- * if, for example, stop words are not supported or not required).
- * <p>
- * If there is no entry for the language tag in the map {@link ConfigurableAnalyzerFactory#langTag2AnalyzerPair},
- * then one is created, by walking down the array {@link ConfigurableAnalyzerFactory#config} of AnalyzerPairs
- * until a matching one is found.
- * <p>
- * The bulk of the code in this class is invoked from the constructor in order to set up this
- * {@link ConfigurableAnalyzerFactory#config} array. For example, all of the subclasses of {@link AnalyzerPair}s,
- * are simply to call the appropriate constructor in the appropriate way: the difficulty is that many subclasses
- * of {@link Analyzer} have constructors with different signatures, and our code needs to navigate each sort.
- * @author jeremycarroll
- *
- */
- private static class AnalyzerPair implements Comparable<AnalyzerPair>{
- final LanguageRange range;
- private final Analyzer withStopWords;
- private final Analyzer withoutStopWords;
-
- public Analyzer getAnalyzer(boolean filterStopwords) {
- return filterStopwords ? withStopWords : withoutStopWords;
- }
-
- public boolean extendedFilterMatch(String[] language) {
- return range.extendedFilterMatch(language);
- }
-
- AnalyzerPair(String range, Analyzer withStopWords, Analyzer withOutStopWords) {
- this.range = new LanguageRange(range);
- this.withStopWords = withStopWords;
- this.withoutStopWords = withOutStopWords;
- }
-
- /**
- * This clone constructor implements {@link AnalyzerOptions#LIKE}.
- * @param range
- * @param copyMe
- */
- AnalyzerPair(String range, AnalyzerPair copyMe) {
- this.range = new LanguageRange(range);
- this.withStopWords = copyMe.withStopWords;
- this.withoutStopWords = copyMe.withoutStopWords;
- }
-
- /**
- * If we have a constructor, with arguments including a populated
- * stop word set, then we can use it to make both the withStopWords
- * analyzer, and the withoutStopWords analyzer.
- * @param range
- * @param cons A Constructor including a {@link java.util.Set} argument
- * for the stop words.
- * @param params The arguments to pass to the constructor including a populated stopword set.
- * @throws Exception
- */
- AnalyzerPair(String range, Constructor<? extends Analyzer> cons, Object ... params) throws Exception {
- this(range, cons.newInstance(params), cons.newInstance(useEmptyStopWordSet(params)));
- }
- AnalyzerPair(String range, Analyzer stopWordsNotSupported) {
- this(range, stopWordsNotSupported, stopWordsNotSupported);
- }
- private static Object[] useEmptyStopWordSet(Object[] params) {
- Object rslt[] = new Object[params.length];
- for (int i=0; i<params.length; i++) {
- if (params[i] instanceof Set) {
- rslt[i] = Collections.EMPTY_SET;
- } else {
- rslt[i] = params[i];
- }
- }
- return rslt;
- }
-
- @Override
- public String toString() {
- return range.full + "=(" + withStopWords.getClass().getSimpleName() +")";
- }
-
- @Override
- public int compareTo(AnalyzerPair o) {
- return range.compareTo(o.range);
- }
- }
-
-
- /**
- * Used for Analyzer classes with a constructor with signature (Version, Set).
- * @author jeremycarroll
- *
- */
- private static class VersionSetAnalyzerPair extends AnalyzerPair {
- public VersionSetAnalyzerPair(ConfigOptionsToAnalyzer lro,
- Class<? extends Analyzer> cls) throws Exception {
- super(lro.languageRange, getConstructor(cls, Version.class, Set.class), Version.LUCENE_CURRENT, lro.getStopWords());
- }
- }
-
- /**
- * Used for Analyzer classes which do not support stopwords and have a constructor with signature (Version).
- * @author jeremycarroll
- *
- */
- private static class VersionAnalyzerPair extends AnalyzerPair {
- public VersionAnalyzerPair(String range, Class<? extends Analyzer> cls) throws Exception {
- super(range, getConstructor(cls, Version.class).newInstance(Version.LUCENE_CURRENT));
- }
- }
-
- /**
- * Special case code for {@link PatternAnalyzer}
- * @author jeremycarroll
- *
- */
- private static class PatternAnalyzerPair extends AnalyzerPair {
- public PatternAnalyzerPair(ConfigOptionsToAnalyzer lro, Pattern pattern) throws Exception {
- super(lro.languageRange, getConstructor(PatternAnalyzer.class,Version.class,Pattern.class,Boolean.TYPE,Set.class),
- Version.LUCENE_CURRENT,
- pattern,
- true,
- lro.getStopWords());
- }
- }
-
-
- /**
- * This class is initialized with the config options, using the {@link #setProperty(String, String)}
- * method, for a particular language range and works out which pair of {@link Analyzer}s
- * to use for that language range.
- * <p>
- * Instances of this class are only alive during the execution of
- * {@link ConfigurableAnalyzerFactory#ConfigurableAnalyzerFactory(FullTextIndex)},
- * the life-cycle is:
- * <ol>
- * <li>The relveant config properties are applied, and are used to populate the fields.
- * <li>The fields are validated
- * <li>An {@link AnalyzerPair} is constructed
- * </ol>
- *
- * @author jeremycarroll
- *
- */
- private static class ConfigOptionsToAnalyzer {
-
- String like;
- String className;
- String stopwords;
- Pattern pattern;
- final String languageRange;
- AnalyzerPair result;
- Pattern wordBoundary;
- Pattern subWordBoundary;
- Pattern softHyphens;
- Boolean alwaysRemoveSoftHyphens;
-
- public ConfigOptionsToAnalyzer(String languageRange) {
- this.languageRange = languageRange;
- }
-
- /**
- * This is called only when we have already identified that
- * the class does support stopwords.
- * @return
- */
- public Set<?> getStopWords() {
-
- if (doNotUseStopWords())
- return Collections.EMPTY_SET;
-
- if (useDefaultStopWords()) {
- return getStopWordsForClass(className);
- }
-
- return getStopWordsForClass(stopwords);
- }
-
- boolean doNotUseStopWords() {
- return AnalyzerOptions.STOPWORDS_VALUE_NONE.equals(stopwords) || (stopwords == null && pattern != null);
- }
-
- protected Set<?> getStopWordsForClass(String clazzName) {
- Class<? extends Analyzer> analyzerClass = getAnalyzerClass(clazzName);
- try {
- return (Set<?>) analyzerClass.getMethod("getDefaultStopSet").invoke(null);
- } catch (Exception e) {
- if (StandardAnalyzer.class.equals(analyzerClass)) {
- return StandardAnalyzer.STOP_WORDS_SET;
- }
- if (StopAnalyzer.class.equals(analyzerClass)) {
- return StopAnalyzer.ENGLISH_STOP_WORDS_SET;
- }
- throw new RuntimeException("Failed to find stop words from " + clazzName + " for language range "+languageRange);
- }
- }
-
- protected boolean useDefaultStopWords() {
- return ( stopwords == null && pattern == null ) || AnalyzerOptions.STOPWORDS_VALUE_DEFAULT.equals(stopwords);
- }
-
- /**
- * The first step in the life-cycle, used to initialize the fields.
- * @return true if the property was recognized.
- */
- public boolean setProperty(String shortProperty, String value) {
- if (shortProperty.equals(AnalyzerOptions.LIKE) ) {
- like = value;
- } else if (shortProperty.equals(AnalyzerOptions.ANALYZER_CLASS) ) {
- className = value;
- } else if (shortProperty.equals(AnalyzerOptions.STOPWORDS) ) {
- stopwords = value;
- } else if (shortProperty.equals(AnalyzerOptions.PATTERN) ) {
- pattern = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
- } else if (shortProperty.equals(AnalyzerOptions.WORD_BOUNDARY) ) {
- wordBoundary = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
- } else if (shortProperty.equals(AnalyzerOptions.SUB_WORD_BOUNDARY) ) {
- subWordBoundary = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
- } else if (shortProperty.equals(AnalyzerOptions.SOFT_HYPHENS) ) {
- softHyphens = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
- } else if (shortProperty.equals(AnalyzerOptions.ALWAYS_REMOVE_SOFT_HYPHENS) ) {
- alwaysRemoveSoftHyphens = Boolean.valueOf(value);
- } else {
- return false;
- }
- return true;
- }
-
- /**
- * The second phase of the life-cycle, used for sanity checking.
- */
- public void validate() {
- if (pattern != null ) {
- if ( className != null && className != PatternAnalyzer.class.getName()) {
- throw new RuntimeException("Bad Option: Language range "+languageRange + " with pattern propety for class "+ className);
- }
- className = PatternAnalyzer.class.getName();
- }
- if (this.wordBoundary != null ) {
- if ( className != null && className != TermCompletionAnalyzer.class.getName()) {
- throw new RuntimeException("Bad Option: Language range "+languageRange + " with pattern propety for class "+ className);
- }
- className = TermCompletionAnalyzer.class.getName();
-
- if ( subWordBoundary == null ) {
- subWordBoundary = AnalyzerOptions.DEFAULT_SUB_WORD_BOUNDARY;
- }
- if ( alwaysRemoveSoftHyphens != null && softHyphens == null ) {
- throw new RuntimeException("Bad option: Language range "+languageRange + ": must specify softHypens when setting alwaysRemoveSoftHyphens");
- }
- if (softHyphens != null && alwaysRemoveSoftHyphens == null) {
- alwaysRemoveSoftHyphens = AnalyzerOptions.DEFAULT_ALWAYS_REMOVE_SOFT_HYPHENS;
- }
-
- } else if ( subWordBoundary != null || softHyphens != null || alwaysRemoveSoftHyphens != null ||
- TermCompletionAnalyzer.class.getName().equals(className) ) {
- throw new RuntimeException("Bad option: Language range "+languageRange + ": must specify wordBoundary for TermCompletionAnalyzer");
- }
-
- if (PatternAnalyzer.class.getName().equals(className) && pattern == null ) {
- throw new RuntimeException("Bad Option: Language range "+languageRange + " must specify pattern for PatternAnalyzer.");
- }
- if ( (like != null) == (className != null) ) {
- throw new RuntimeException("Bad Option: Language range "+languageRange + " must specify exactly one of implementation class or like.");
- }
- if (stopwords != null && like != null) {
- throw new RuntimeException("Bad Option: Language range "+languageRange + " must not specify stopwords with like.");
- }
-
- }
-
- /**
- * The third and final phase of the life-cyle used for identifying
- * the AnalyzerPair.
- */
- private AnalyzerPair construct() throws Exception {
- if (className == null) {
- return null;
- }
- if (pattern != null) {
- return new PatternAnalyzerPair(this, pattern);
- }
- if (softHyphens != null) {
- return new AnalyzerPair(
- languageRange,
- new TermCompletionAnalyzer(
- wordBoundary,
- subWordBoundary,
- softHyphens,
- alwaysRemoveSoftHyphens));
- }
- if (wordBoundary != null) {
- return new AnalyzerPair(
- languageRange,
- new TermCompletionAnalyzer(
- wordBoundary,
- subWordBoundary));
- }
- final Class<? extends Analyzer> cls = getAnalyzerClass();
-
- if (hasConstructor(cls, Version.class, Set.class)) {
-
- // RussianAnalyzer is missing any way to access stop words.
- if (RussianAnalyzer.class.equals(cls)) {
- if (useDefaultStopWords()) {
- return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT), new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
- }
- if (doNotUseStopWords()) {
- return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
- }
- }
- return new VersionSetAnalyzerPair(this, cls);
- }
-
- if (stopwords != null && !stopwords.equals(AnalyzerOptions.STOPWORDS_VALUE_NONE)) {
- throw new RuntimeException("Bad option: language range: " + languageRange + " stopwords are not supported by " + className);
- }
- if (hasConstructor(cls, Version.class)) {
- return new VersionAnalyzerPair(languageRange, cls);
- }
-
- if (hasConstructor(cls)) {
- return new AnalyzerPair(languageRange, cls.newInstance());
- }
- throw new RuntimeException("Bad option: cannot find constructor for class " + className + " for language range " + languageRange);
- }
-
- /**
- * Also part of the third phase of the life-cycle, following the {@link AnalyzerOptions#LIKE}
- * properties.
- * @param depth
- * @param max
- * @param analyzers
- * @return
- */
- AnalyzerPair followLikesToAnalyzerPair(int depth, int max,
- Map<String, ConfigOptionsToAnalyzer> analyzers) {
- if (result == null) {
- if (depth == max) {
- throw new RuntimeException("Bad configuration: - 'like' loop for language range " + languageRange);
- }
- ConfigOptionsToAnalyzer next = analyzers.get(like);
- if (next == null) {
- throw new RuntimeException("Bad option: - 'like' not found for language range " + languageRange+ " (not found: '"+ like +"')");
- }
- result = new AnalyzerPair(languageRange, next.followLikesToAnalyzerPair(depth+1, max, analyzers));
- }
- return result;
- }
-
- protected Class<? extends Analyzer> getAnalyzerClass() {
- return getAnalyzerClass(className);
- }
-
- @SuppressWarnings("unchecked")
- protected Class<? extends Analyzer> getAnalyzerClass(String className2) {
- final Class<? extends Analyzer> cls;
- try {
- cls = (Class<? extends Analyzer>) Class.forName(className2);
- } catch (ClassNotFoundException e) {
- throw new RuntimeException("Bad option: cannot find class " + className2 + " for language range " + languageRange, e);
- }
- return cls;
- }
-
- void setAnalyzerPair(AnalyzerPair ap) {
- result = ap;
- }
- }
-
- private final AnalyzerPair config[];
-
- private final Map<String, AnalyzerPair> langTag2AnalyzerPair = new ConcurrentHashMap<String, AnalyzerPair>();
-
/**
- * While it would be very unusual to have more than 500 different language tags in a store
- * it is possible - we use a max size to prevent a memory explosion, and a naive caching
- * strategy so the code will still work on the {@link #MAX_LANG_CACHE_SIZE}+1 th entry.
+ * Initialization is a little tricky, because on the very first
+ * call to the constructor with a new namespace or a new journal
+ * the fullTextIndex is not ready for use.
+ * Therefore we delegate to an unconfigured object
+ * which on the first call to {@link NeedsConfiguringAnalyzerFactory#getAnalyzer(String, boolean)}
+ * does the configuration and replaces itself here with a
+ * {@link ConfiguredAnalyzerFactory}
*/
- private static final int MAX_LANG_CACHE_SIZE = 500;
+ IAnalyzerFactory delegate;
-
- private String defaultLanguage;
- private final FullTextIndex<?> fullTextIndex;
-
-
/**
* Builds a new ConfigurableAnalyzerFactory.
* @param fullTextIndex
*/
public ConfigurableAnalyzerFactory(final FullTextIndex<?> fullTextIndex) {
- // A description of the operation of this method is found on AnalyzerPair and
- // ConfigOptionsToAnalyzer.
- // despite our name, we actually make all the analyzers now, and getAnalyzer method is merely a lookup.
-
- if (fullTextIndex == null)
- throw new IllegalArgumentException();
-
- this.fullTextIndex = fullTextIndex;
-
- final Properties properties = initProperties();
-
- final Map<String, ConfigOptionsToAnalyzer> analyzers = new HashMap<String, ConfigOptionsToAnalyzer>();
-
- properties2analyzers(properties, analyzers);
-
- if (!analyzers.containsKey("*")) {
- throw new RuntimeException("Bad config: must specify behavior on language range '*'");
- }
-
- for (ConfigOptionsToAnalyzer a: analyzers.values()) {
- a.validate();
- }
-
- try {
- for (ConfigOptionsToAnalyzer a: analyzers.values()) {
- a.setAnalyzerPair(a.construct());
- }
- } catch (Exception e) {
- throw new RuntimeException("Cannot construct ConfigurableAnalyzerFactory", e);
- }
- int sz = analyzers.size();
- for (ConfigOptionsToAnalyzer a: analyzers.values()) {
- a.followLikesToAnalyzerPair(0, sz, analyzers);
- }
-
- config = new AnalyzerPair[sz];
- int i = 0;
- for (ConfigOptionsToAnalyzer a: analyzers.values()) {
- config[i++] = a.result;
- }
- Arrays.sort(config);
- if (log.isInfoEnabled()) {
- StringBuilder sb = new StringBuilder();
- sb.append("Installed text Analyzer's: ");
- for (AnalyzerPair ap: config) {
- sb.append(ap.toString());
- sb.append(", ");
- }
- log.info(sb.toString());
- }
+ delegate = new NeedsConfiguringAnalyzerFactory(this, fullTextIndex);
}
- private String getDefaultLanguage(final FullTextIndex<?> fullTextIndex) {
-
- final IKeyBuilder keyBuilder = fullTextIndex.getKeyBuilder();
+ static int loggerIdCounter = 0;
+ @Override
+ public Analyzer getAnalyzer(final String languageCode, boolean filterStopwords) {
- if (keyBuilder.isUnicodeSupported()) {
-
- // The configured local for the database.
- final Locale locale = ((KeyBuilder) keyBuilder)
- .getSortKeyGenerator().getLocale();
-
- // The analyzer for that locale.
- return locale.getLanguage();
-
+ final Analyzer unlogged = delegate.getAnalyzer(languageCode, filterStopwords);
+ if (log.isDebugEnabled()) {
+ return new Analyzer() {
+ @Override
+ public TokenStream tokenStream(final String fieldName, final Reader reader) {
+ final int id = loggerIdCounter++;
+ final String term = TermCompletionAnalyzer.getStringReaderContents((StringReader)reader);
+ log.debug(id + " " + languageCode +" **"+term+"**");
+ return new TokenFilter(unlogged.tokenStream(fieldName, reader)){
+
+ TermAttribute attr = addAttribute(TermAttribute.class);
+ @Override
+ public boolean incrementToken() throws IOException {
+ if (input.incrementToken()) {
+ log.debug(id + " |"+attr.term()+"|");
+ return true;
+ }
+ return false;
+ }};
+ }
+ };
} else {
- // Rule, Britannia!
- return "en";
-
+ return unlogged;
}
- }
- private String getDefaultLanguage() {
- if (defaultLanguage == null) {
- defaultLanguage = getDefaultLanguage(fullTextIndex);
- }
- return defaultLanguage;
- }
-
- private static boolean hasConstructor(Class<? extends Analyzer> cls, Class<?> ... parameterTypes) {
- return getConstructor(cls, parameterTypes) != null;
- }
-
- protected static Constructor<? extends Analyzer> getConstructor(Class<? extends Analyzer> cls,
- Class<?>... parameterTypes) {
- try {
- return cls.getConstructor(parameterTypes);
- } catch (NoSuchMethodException | SecurityException e) {
- return null;
- }
- }
-
- private void properties2analyzers(Properties props, Map<String, ConfigOptionsToAnalyzer> analyzers) {
- Enumeration<?> en = props.propertyNames();
- while (en.hasMoreElements()) {
-
- String prop = (String)en.nextElement();
- if (prop.equals(Options.NATURAL_LANGUAGE_SUPPORT)) continue;
- if (prop.startsWith(Options.ANALYZER)) {
- String languageRangeAndProperty[] = prop.substring(Options.ANALYZER.length()).replaceAll("_","*").split("[.]");
- if (languageRangeAndProperty.length == 2) {
-
- String languageRange = languageRangeAndProperty[0].toLowerCase(Locale.US); // Turkish "I" could create a problem
- String shortProperty = languageRangeAndProperty[1];
- String value = props.getProperty(prop);
- log.info("Setting language range: " + languageRange + "/" + shortProperty + " = " + value);
- ConfigOptionsToAnalyzer cons = analyzers.get(languageRange);
- if (cons == null) {
- cons = new ConfigOptionsToAnalyzer(languageRange);
- analyzers.put(languageRange, cons);
- }
- if (cons.setProperty(shortProperty, value)) {
- continue;
- }
- }
- }
-
- log.warn("Failed to process configuration property: " + prop);
- }
-
}
- protected Properties initProperties() {
- final Properties parentProperties = fullTextIndex.getProperties();
- Properties myProps;
- if (Boolean.valueOf(parentProperties.getProperty(
- Options.NATURAL_LANGUAGE_SUPPORT,
- Options.DEFAULT_NATURAL_LANGUAGE_SUPPORT))) {
-
- myProps = loadPropertyString(ALL_LUCENE_NATURAL_LANGUAGES);
-
- } else if (hasPropertiesForStarLanguageRange(parentProperties)){
-
- myProps = new Properties();
-
- } else {
-
- myProps = loadPropertyString(LUCENE_STANDARD_ANALYZER);
- }
-
- copyRelevantProperties(fullTextIndex.getProperties(), myProps);
- return myProps;
- }
-
- Properties loadPropertyString(String props) {
- Properties rslt = new Properties();
- try {
- rslt.load(new StringReader(props));
- } catch (IOException e) {
- throw new RuntimeException("Impossible - well clearly not!", e);
- }
- return rslt;
- }
-
- private void copyRelevantProperties(Properties from, Properties to) {
- Enumeration<?> en = from.propertyNames();
- while (en.hasMoreElements()) {
- String prop = (String)en.nextElement();
- if (prop.startsWith(ConfigurableAnalyzerFactory.class.getName())) {
- to.setProperty(prop, from.getProperty(prop));
- }
- }
- }
-
- private boolean hasPropertiesForStarLanguageRange(Properties from) {
- Enumeration<?> en = from.propertyNames();
- while (en.hasMoreElements()) {
- String prop = (String)en.nextElement();
- if (prop.startsWith(Options.ANALYZER+"_.")
- || prop.startsWith(Options.ANALYZER+"*.")) {
- return true;
- }
- }
- return false;
- }
- @Override
- public Analyzer getAnalyzer(String languageCode, boolean filterStopwords) {
-
- if (languageCode == null || languageCode.equals("")) {
-
- languageCode = getDefaultLanguage();
- }
-
- AnalyzerPair pair = langTag2AnalyzerPair.get(languageCode);
-
- if (pair == null) {
- pair = lookupPair(languageCode);
-
- // naive cache - clear everything if cache is full
- if (langTag2AnalyzerPair.size() == MAX_LANG_CACHE_SIZE) {
- langTag2AnalyzerPair.clear();
- }
- // there is a race condition below, but we don't care who wins.
- langTag2AnalyzerPair.put(languageCode, pair);
- }
-
- return pair.getAnalyzer(filterStopwords);
-
- }
-
- private AnalyzerPair lookupPair(String languageCode) {
- String language[] = languageCode.split("-");
- for (AnalyzerPair p: config) {
- if (p.extendedFilterMatch(language)) {
- return p;
- }
- }
- throw new RuntimeException("Impossible - supposedly - did not match '*'");
- }
}
Added: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfiguredAnalyzerFactory.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfiguredAnalyzerFactory.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfiguredAnalyzerFactory.java 2014-05-11 15:45:17 UTC (rev 8270)
@@ -0,0 +1,161 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+/*
+ * Created on May 6, 2014 by Jeremy J. Carroll, Syapse Inc.
+ */
+package com.bigdata.search;
+
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.lucene.analysis.Analyzer;
+
+import com.bigdata.search.ConfigurableAnalyzerFactory.AnalyzerOptions;
+/**
+ * This comment describes the implementation of {@link ConfiguredAnalyzerFactory}.
+ * The only method in the interface is {@link ConfiguredAnalyzerFactory#getAnalyzer(String, boolean)},
+ * a map is used from language tag to {@link AnalyzerPair}, where the pair contains
+ * an {@link Analyzer} both with and without stopwords configured (some times these two analyzers are identical,
+ * if, for example, stop words are not supported or not required).
+ * <p>
+ * If there is no entry for the language tag in the map {@link ConfiguredAnalyzerFactory#langTag2AnalyzerPair},
+ * then one is created, by walking down the array {@link ConfiguredAnalyzerFactory#config} of AnalyzerPairs
+ * until a matching one is found.
+ * @author jeremycarroll
+ *
+ */
+class ConfiguredAnalyzerFactory implements IAnalyzerFactory {
+
+
+ /**
+ * These provide a mapping from a language range to a pair of Analyzers
+ * and sort with the best-match (i.e. longest match) first.
+ * @author jeremycarroll
+ *
+ */
+ protected static class AnalyzerPair implements Comparable<AnalyzerPair>{
+ final LanguageRange range;
+ private final Analyzer withStopWords;
+ private final Analyzer withoutStopWords;
+
+ public Analyzer getAnalyzer(boolean filterStopwords) {
+ return filterStopwords ? withStopWords : withoutStopWords;
+ }
+
+ public boolean extendedFilterMatch(String[] language) {
+ return range.extendedFilterMatch(language);
+ }
+
+ AnalyzerPair(String range, Analyzer withStopWords, Analyzer withOutStopWords) {
+ this.range = new LanguageRange(range);
+ this.withStopWords = withStopWords;
+ this.withoutStopWords = withOutStopWords;
+ }
+
+ /**
+ * This clone constructor implements {@link AnalyzerOptions#LIKE}.
+ * @param range
+ * @param copyMe
+ */
+ AnalyzerPair(String range, AnalyzerPair copyMe) {
+ this(range, copyMe.withStopWords, copyMe.withoutStopWords);
+ }
+
+ @Override
+ public String toString() {
+ return range.full + "=(" + withStopWords.getClass().getSimpleName() +")";
+ }
+
+ @Override
+ public int compareTo(AnalyzerPair o) {
+ return range.compareTo(o.range);
+ }
+ }
+
+
+ private final AnalyzerPair config[];
+
+ /**
+ * This caches the result of looking up a lang tag in the
+ * config of language ranges.
+ */
+ private final Map<String, AnalyzerPair> langTag2AnalyzerPair = new ConcurrentHashMap<String, AnalyzerPair>();;
+
+ /**
+ * While it would be very unusual to have more than 500 different language tags in a store
+ * it is possible - we use a max size to prevent a memory explosion, and a naive caching
+ * strategy so the code will still work on the {@link #MAX_LANG_CACHE_SIZE}+1 th entry.
+ */
+ private static final int MAX_LANG_CACHE_SIZE = 500;
+
+
+ private final String defaultLanguage;
+ /**
+ * Builds a new ConfigurableAnalyzerFactory.
+ * @param fullTextIndex
+ */
+ public ConfiguredAnalyzerFactory(AnalyzerPair config[], String defaultLanguage) {
+ this.config = config;
+ this.defaultLanguage = defaultLanguage;
+ }
+
+ private String getDefaultLanguage() {
+ return defaultLanguage;
+ }
+
+ @Override
+ public Analyzer getAnalyzer(String languageCode, boolean filterStopwords) {
+
+ if (languageCode == null || languageCode.equals("")) {
+
+ languageCode = getDefaultLanguage();
+ }
+
+ AnalyzerPair pair = langTag2AnalyzerPair.get(languageCode);
+
+ if (pair == null) {
+ pair = lookupPair(languageCode);
+
+ // naive cache - clear everything if cache is full
+ if (langTag2AnalyzerPair.size() == MAX_LANG_CACHE_SIZE) {
+ langTag2AnalyzerPair.clear();
+ }
+ // there is a race condition below, but we don't care who wins.
+ langTag2AnalyzerPair.put(languageCode, pair);
+ }
+
+ return pair.getAnalyzer(filterStopwords);
+
+ }
+
+ private AnalyzerPair lookupPair(String languageCode) {
+ String language[] = languageCode.split("-");
+ for (AnalyzerPair p: config) {
+ if (p.extendedFilterMatch(language)) {
+ return p;
+ }
+ }
+ throw new RuntimeException("Impossible - supposedly - did not match '*'");
+ }
+}
Added: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/LanguageRange.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/LanguageRange.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/LanguageRange.java 2014-05-11 15:45:17 UTC (rev 8270)
@@ -0,0 +1,126 @@
+package com.bigdata.search;
+
+import java.util.Locale;
+
+
+/**
+ * This is an implementation of RFC 4647 language range,
+ * targetted at the specific needs within bigdata, and only
+ * supporting the extended filtering specified in section 3.3.2
+ * <p>
+ * Language ranges are comparable so that
+ * sorting an array and then matching a language tag against each
+ * member of the array in sequence will give the longest match.
+ * i.e. the longer ranges come first.
+ * @author jeremycarroll
+ *
+ */
+public class LanguageRange implements Comparable<LanguageRange> {
+
+ private final String range[];
+ final String full;
+ /**
+ * Note range must be in lower case, this is not verified.
+ * @param range
+ */
+ public LanguageRange(String range) {
+ this.range = range.split("-");
+ full = range;
+ }
+
+ @Override
+ public int compareTo(LanguageRange o) {
+ if (equals(o)) {
+ return 0;
+ }
+ int diff = o.range.length - range.length;
+ if (diff != 0) {
+ // longest first
+ return diff;
+ }
+ if (range.length == 1) {
+ // * last
+ if (range[0].equals("*")) {
+ return 1;
+ }
+ if (o.range[0].equals("*")) {
+ return -1;
+ }
+ }
+ // alphabetically
+ for (int i=0; i<range.length; i++) {
+ diff = range[i].compareTo(o.range[i]);
+ if (diff != 0) {
+ return diff;
+ }
+ }
+ throw new RuntimeException("Impossible - supposedly");
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ return (o instanceof LanguageRange) && ((LanguageRange)o).full.equals(full);
+ }
+ @Override
+ public int hashCode() {
+ return full.hashCode();
+ }
+
+ /**
+ * This implements the algoirthm of section 3.3.2 of RFC 4647
+ * as modified with the observation about private use tags
+ * in <a href="http://lists.w3.org/Archives/Public/www-international/2014AprJun/0084">
+ * this message</a>.
+ *
+ *
+ * @param langTag The RFC 5646 Language tag in lower case
+ * @return The result of the algorithm
+ */
+ public boolean extendedFilterMatch(String langTag) {
+ return extendedFilterMatch(langTag.toLowerCase(Locale.ROOT).split("-"));
+ }
+
+ // See RFC 4647, 3.3.2
+ boolean extendedFilterMatch(String[] language) {
+ // RFC 4647 step 2
+ if (!matchSubTag(language[0], range[0])) {
+ return false;
+ }
+ int rPos = 1;
+ int lPos = 1;
+ // variant step - for private use flags
+ if (language[0].equals("x") && range[0].equals("*")) {
+ lPos = 0;
+ }
+ // RFC 4647 step 3
+ while (rPos < range.length) {
+ // step 3A
+ if (range[rPos].equals("*")) {
+ rPos ++;
+ continue;
+ }
+ // step 3B
+ if (lPos >= language.length) {
+ return false;
+ }
+ // step 3C
+ if (matchSubTag(language[lPos], range[rPos])) {
+ lPos++;
+ rPos++;
+ continue;
+ }
+ if (language[lPos].length()==1) {
+ return false;
+ }
+ lPos++;
+ }
+ // RFC 4647 step 4
+ return true;
+ }
+
+ // RFC 4647, 3.3.2, step 1
+ private boolean matchSubTag(String langSubTag, String rangeSubTag) {
+ return langSubTag.equals(rangeSubTag) || "*".equals(rangeSubTag);
+ }
+
+}
\ No newline at end of file
Copied: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/NeedsConfiguringAnalyzerFactory.java (from rev 8263, branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java)
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/NeedsConfiguringAnalyzerFactory.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/NeedsConfiguringAnalyzerFactory.java 2014-05-11 15:45:17 UTC (rev 8270)
@@ -0,0 +1,649 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+/*
+ * Created on May 6, 2014 by Jeremy J. Carroll, Syapse Inc.
+ */
+package com.bigdata.search;
+
+import java.io.IOException;
+import java.io.StringReader;
+import java.lang.reflect.Constructor;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Enumeration;
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.UUID;
+import java.util.WeakHashMap;
+import java.util.regex.Pattern;
+
+import org.apache.log4j.Logger;
+import org.apache.lucene.analysis.Analyzer;
+import org.apache.lucene.analysis.StopAnalyzer;
+import org.apache.lucene.analysis.miscellaneous.PatternAnalyzer;
+import org.apache.lucene.analysis.ru.RussianAnalyzer;
+import org.apache.lucene.analysis.standard.StandardAnalyzer;
+import org.apache.lucene.util.Version;
+
+import com.bigdata.btree.keys.IKeyBuilder;
+import com.bigdata.btree.keys.KeyBuilder;
+import com.bigdata.search.ConfigurableAnalyzerFactory.AnalyzerOptions;
+import com.bigdata.search.ConfigurableAnalyzerFactory.Options;
+
+
+/**
+ * <p>
+ * The bulk of the code in this class is invoked from {@link #init()} to set up the array of
+ * {@link ConfiguredAnalyzerFactory.AnalyzerPair}s. For example, all of the subclasses of {@link AnalyzerPair}s,
+ * are simply to call the appropriate constructor in the appropriate way: the difficulty is that many subclasses
+ * of {@link Analyzer} have constructors with different signatures, and our code needs to navigate each sort.
+ * @author jeremycarroll
+ *
+ */
+class NeedsConfiguringAnalyzerFactory implements IAnalyzerFactory {
+ final private static transient Logger log = Logger.getLogger(NeedsConfiguringAnalyzerFactory.class);
+
+ /**
+ * We create only one {@link ConfiguredAnalyzerFactory} per namespace
+ * and store it here. The UUID is stable and allows us to side-step lifecycle
+ * issues such as creation and destruction of namespaces, potentially with different properties.
+ * We use a WeakHashMap to ensure that after the destruction of a namespace we clean up.
+ * We have to synchronize this for thread safety.
+ */
+ private static final Map<UUID, ConfiguredAnalyzerFactory> allConfigs =
+ Collections.synchronizedMap(new WeakHashMap<UUID, ConfiguredAnalyzerFactory>());
+
+
+ private static final String ALL_LUCENE_NATURAL_LANGUAGES =
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.*.like=eng\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.por.analyzerClass=org.apache.lucene.analysis.br.BrazilianAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.pt.like=por\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.zho.analyzerClass=org.apache.lucene.analysis.cn.ChineseAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.chi.like=zho\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.zh.like=zho\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.jpn.analyzerClass=org.apache.lucene.analysis.cjk.CJKAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ja.like=jpn\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.kor.like=jpn\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ko.like=kor\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ces.analyzerClass=org.apache.lucene.analysis.cz.CzechAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.cze.like=ces\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.cs.like=ces\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.dut.analyzerClass=org.apache.lucene.analysis.nl.DutchAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.nld.like=dut\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.nl.like=dut\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.deu.analyzerClass=org.apache.lucene.analysis.de.GermanAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ger.like=deu\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.de.like=deu\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.gre.analyzerClass=org.apache.lucene.analysis.el.GreekAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ell.like=gre\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.el.like=gre\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.rus.analyzerClass=org.apache.lucene.analysis.ru.RussianAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.ru.like=rus\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.tha.analyzerClass=org.apache.lucene.analysis.th.ThaiAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.th.like=tha\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.eng.analyzerClass=org.apache.lucene.analysis.standard.StandardAnalyzer\n" +
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.en.like=eng\n";
+
+ private static final String LUCENE_STANDARD_ANALYZER =
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.*.analyzerClass=org.apache.lucene.analysis.standard.StandardAnalyzer\n";
+
+ static int loggerIdCounter = 0;
+
+ /**
+ * This class and all its subclasses provide a variety of patterns
+ * for mapping from the various constructor patterns of subclasses
+ * of {@link Analyzer} to {@link ConfiguredAnalyzerFactory#AnalyzerPair}.
+ * @author jeremycarroll
+ *
+ */
+ private static class AnalyzerPair extends ConfiguredAnalyzerFactory.AnalyzerPair {
+
+ AnalyzerPair(String range, Analyzer withStopWords, Analyzer withOutStopWords) {
+ super(range, withStopWords, withOutStopWords);
+ }
+
+ /**
+ * This clone constructor implements {@link AnalyzerOptions#LIKE}.
+ * @param range
+ * @param copyMe
+ */
+ AnalyzerPair(String range, AnalyzerPair copyMe) {
+ super(range, copyMe);
+ }
+
+ /**
+ * If we have a constructor, with arguments including a populated
+ * stop word set, then we can use it to make both the withStopWords
+ * analyzer, and the withoutStopWords analyzer.
+ * @param range
+ * @param cons A Constructor including a {@link java.util.Set} argument
+ * for the stop words.
+ * @param params The arguments to pass to the constructor including a populated stopword set.
+ * @throws Exception
+ */
+ AnalyzerPair(String range, Constructor<? extends Analyzer> cons, Object ... params) throws Exception {
+ this(range, cons.newInstance(params), cons.newInstance(useEmptyStopWordSet(params)));
+ }
+ AnalyzerPair(String range, Analyzer stopWordsNotSupported) {
+ this(range, stopWordsNotSupported, stopWordsNotSupported);
+ }
+ private static Object[] useEmptyStopWordSet(Object[] params) {
+ Object rslt[] = new Object[params.length];
+ for (int i=0; i<params.length; i++) {
+ if (params[i] instanceof Set) {
+ rslt[i] = Collections.EMPTY_SET;
+ } else {
+ rslt[i] = params[i];
+ }
+ }
+ return rslt;
+ }
+
+ }
+
+
+ /**
+ * Used for Analyzer classes with a constructor with signature (Version, Set).
+ * @author jeremycarroll
+ *
+ */
+ private static class VersionSetAnalyzerPair extends AnalyzerPair {
+ public VersionSetAnalyzerPair(ConfigOptionsToAnalyzer lro,
+ Class<? extends Analyzer> cls) throws Exception {
+ super(lro.languageRange, getConstructor(cls, Version.class, Set.class), Version.LUCENE_CURRENT, lro.getStopWords());
+ }
+ }
+
+ /**
+ * Used for Analyzer classes which do not support stopwords and have a constructor with signature (Version).
+ * @author jeremycarroll
+ *
+ */
+ private static class VersionAnalyzerPair extends AnalyzerPair {
+ public VersionAnalyzerPair(String range, Class<? extends Analyzer> cls) throws Exception {
+ super(range, getConstructor(cls, Version.class).newInstance(Version.LUCENE_CURRENT));
+ }
+ }
+
+ /**
+ * Special case code for {@link PatternAnalyzer}
+ * @author jeremycarroll
+ *
+ */
+ private static class PatternAnalyzerPair extends AnalyzerPair {
+ public PatternAnalyzerPair(ConfigOptionsToAnalyzer lro, Pattern pattern) throws Exception {
+ super(lro.languageRange, getConstructor(PatternAnalyzer.class,Version.class,Pattern.class,Boolean.TYPE,Set.class),
+ Version.LUCENE_CURRENT,
+ pattern,
+ true,
+ lro.getStopWords());
+ }
+ }
+
+
+
+ /**
+ * This class is initialized with the config options, using the {@link #setProperty(String, String)}
+ * method, for a particular language range and works out which pair of {@link Analyzer}s
+ * to use for that language range.
+ * <p>
+ * Instances of this class are only alive during the execution of
+ * {@link NeedsConfiguringAnalyzerFactory#ConfigurableAnalyzerFactory(FullTextIndex)},
+ * the life-cycle is:
+ * <ol>
+ * <li>The relveant config properties are applied, and are used to populate the fields.
+ * <li>The fields are validated
+ * <li>An {@link AnalyzerPair} is constructed
+ * </ol>
+ *
+ * @author jeremycarroll
+ *
+ */
+ private static class ConfigOptionsToAnalyzer {
+
+ String like;
+ String className;
+ String stopwords;
+ Pattern pattern;
+ final String languageRange;
+ AnalyzerPair result;
+ Pattern wordBoundary;
+ Pattern subWordBoundary;
+ Pattern softHyphens;
+ Boolean alwaysRemoveSoftHyphens;
+
+ public ConfigOptionsToAnalyzer(String languageRange) {
+ this.languageRange = languageRange;
+ }
+
+ /**
+ * This is called only when we have already identified that
+ * the class does support stopwords.
+ * @return
+ */
+ public Set<?> getStopWords() {
+
+ if (doNotUseStopWords())
+ return Collections.EMPTY_SET;
+
+ if (useDefaultStopWords()) {
+ return getStopWordsForClass(className);
+ }
+
+ return getStopWordsForClass(stopwords);
+ }
+
+ boolean doNotUseStopWords() {
+ return AnalyzerOptions.STOPWORDS_VALUE_NONE.equals(stopwords) || (stopwords == null && pattern != null);
+ }
+
+ protected Set<?> getStopWordsForClass(String clazzName) {
+ Class<? extends Analyzer> analyzerClass = getAnalyzerClass(clazzName);
+ try {
+ return (Set<?>) analyzerClass.getMethod("getDefaultStopSet").invoke(null);
+ } catch (Exception e) {
+ if (StandardAnalyzer.class.equals(analyzerClass)) {
+ return StandardAnalyzer.STOP_WORDS_SET;
+ }
+ if (StopAnalyzer.class.equals(analyzerClass)) {
+ return StopAnalyzer.ENGLISH_STOP_WORDS_SET;
+ }
+ throw new RuntimeException("Failed to find stop words from " + clazzName + " for language range "+languageRange);
+ }
+ }
+
+ protected boolean useDefaultStopWords() {
+ return ( stopwords == null && pattern == null ) || AnalyzerOptions.STOPWORDS_VALUE_DEFAULT.equals(stopwords);
+ }
+
+ /**
+ * The first step in the life-cycle, used to initialize the fields.
+ * @return true if the property was recognized.
+ */
+ public boolean setProperty(String shortProperty, String value) {
+ if (shortProperty.equals(AnalyzerOptions.LIKE) ) {
+ like = value;
+ } else if (shortProperty.equals(AnalyzerOptions.ANALYZER_CLASS) ) {
+ className = value;
+ } else if (shortProperty.equals(AnalyzerOptions.STOPWORDS) ) {
+ stopwords = value;
+ } else if (shortProperty.equals(AnalyzerOptions.PATTERN) ) {
+ pattern = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.WORD_BOUNDARY) ) {
+ wordBoundary = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.SUB_WORD_BOUNDARY) ) {
+ subWordBoundary = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.SOFT_HYPHENS) ) {
+ softHyphens = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.ALWAYS_REMOVE_SOFT_HYPHENS) ) {
+ alwaysRemoveSoftHyphens = Boolean.valueOf(value);
+ } else {
+ return false;
+ }
+ return true;
+ }
+
+ /**
+ * The second phase of the life-cycle, used for sanity checking.
+ */
+ public void validate() {
+ if (pattern != null ) {
+ if ( className != null && className != PatternAnalyzer.class.getName()) {
+ throw new RuntimeException("Bad Option: Language range "+languageRange + " with pattern propety for class "+ className);
+ }
+ className = PatternAnalyzer.class.getName();
+ }
+ if (this.wordBoundary != null ) {
+ if ( className != null && className != TermCompletionAnalyzer.class.getName()) {
+ throw new RuntimeException("Bad Option: Language range "+languageRange + " with pattern propety for class "+ className);
+ }
+ className = TermCompletionAnalyzer.class.getName();
+
+ if ( subWordBoundary == null ) {
+ subWordBoundary = AnalyzerOptions.DEFAULT_SUB_WORD_BOUNDARY;
+ }
+ if ( alwaysRemoveSoftHyphens != null && softHyphens == null ) {
+ throw new RuntimeException("Bad option: Language range "+languageRange + ": must specify softHypens when setting alwaysRemoveSoftHyphens");
+ }
+ if (softHyphens != null && alwaysRemoveSoftHyphens == null) {
+ alwaysRemoveSoftHyphens = AnalyzerOptions.DEFAULT_ALWAYS_REMOVE_SOFT_HYPHENS;
+ }
+
+ } else if ( subWordBoundary != null || softHyphens != null || alwaysRemoveSoftHyphens != null ||
+ TermCompletionAnalyzer.class.getName().equals(className) ) {
+ throw new RuntimeException("Bad option: Language range "+languageRange + ": must specify wordBoundary for TermCompletionAnalyzer");
+ }
+
+ if (PatternAnalyzer.class.getName().equals(className) && pattern == null ) {
+ throw new RuntimeException("Bad Option: Language range "+languageRange + " must specify pattern for PatternAnalyzer.");
+ }
+ if ( (like != null) == (className != null) ) {
+ throw new RuntimeException("Bad Option: Language range "+languageRange + " must specify exactly one of implementation class or like.");
+ }
+ if (stopwords != null && like != null) {
+ throw new RuntimeException("Bad Option: Language range "+languageRange + " must not specify stopwords with like.");
+ }
+
+ }
+
+ /**
+ * The third and final phase of the life-cyle used for identifying
+ * the AnalyzerPair.
+ */
+ private AnalyzerPair construct() throws Exception {
+ if (className == null) {
+ return null;
+ }
+ if (pattern != null) {
+ return new PatternAnalyzerPair(this, pattern);
+ }
+ if (softHyphens != null) {
+ return new AnalyzerPair(
+ languageRange,
+ new TermCompletionAnalyzer(
+ wordBoundary,
+ subWordBoundary,
+ softHyphens,
+ alwaysRemoveSoftHyphens));
+ }
+ if (wordBoundary != null) {
+ return new AnalyzerPair(
+ languageRange,
+ new TermCompletionAnalyzer(
+ wordBoundary,
+ subWordBoundary));
+ }
+ final Class<? extends Analyzer> cls = getAnalyzerClass();
+
+ if (hasConstructor(cls, Version.class, Set.class)) {
+
+ // RussianAnalyzer is missing any way to access stop words.
+ if (RussianAnalyzer.class.equals(cls)) {
+ if (useDefaultStopWords()) {
+ return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT), new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
+ }
+ if (doNotUseStopWords()) {
+ return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
+ }
+ }
+ return new VersionSetAnalyzerPair(this, cls);
+ }
+
+ if (stopwords != null && !stopwords.equals(AnalyzerOptions.STOPWORDS_VALUE_NONE)) {
+ throw new RuntimeException("Bad option: language range: " + languageRange + " stopwords are not supported by " + className);
+ }
+ if (hasConstructor(cls, Version.class)) {
+ return new VersionAnalyzerPair(languageRange, cls);
+ }
+
+ if (hasConstructor(cls)) {
+ return new AnalyzerPair(languageRange, cls.newInstance());
+ }
+ throw new RuntimeException("Bad option: cannot find constructor for class " + className + " for language range " + languageRange);
+ }
+
+ /**
+ * Also part of the third phase of the life-cycle, following the {@link AnalyzerOptions#LIKE}
+ * properties.
+ * @param depth
+ * @param max
+ * @param analyzers
+ * @return
+ */
+ AnalyzerPair followLikesToAnalyzerPair(int depth, int max,
+ Map<String, ConfigOptionsToAnalyzer> analyzers) {
+ if (result == null) {
+ if (depth == max) {
+ throw new RuntimeException("Bad configuration: - 'like' loop for language range " + languageRange);
+ }
+ ConfigOptionsToAnalyzer next = analyzers.get(like);
+ if (next == null) {
+ throw new RuntimeException("Bad ...
[truncated message content] |
|
From: <mrp...@us...> - 2014-05-11 15:08:40
|
Revision: 8269
http://sourceforge.net/p/bigdata/code/8269
Author: mrpersonick
Date: 2014-05-11 15:08:38 +0000 (Sun, 11 May 2014)
Log Message:
-----------
static {
log.info(Graph.class.getName());
}
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 22:57:13 UTC (rev 8268)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-11 15:08:38 UTC (rev 8269)
@@ -34,8 +34,8 @@
import org.openrdf.model.URI;
import org.openrdf.model.impl.URIImpl;
-import com.bigdata.blueprints.BigdataGraph;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
+import com.tinkerpop.blueprints.Graph;
/**
* Default dispatch pattern for a core REST API.
@@ -48,7 +48,7 @@
.getLogger(RESTServlet.class);
static {
- log.info(BigdataGraph.class.getName());
+ log.info(Graph.class.getName());
}
/**
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-10 22:57:15
|
Revision: 8268
http://sourceforge.net/p/bigdata/code/8268
Author: mrpersonick
Date: 2014-05-10 22:57:13 +0000 (Sat, 10 May 2014)
Log Message:
-----------
pulling in bigdata/blueprints layer:
static {
log.info(BigdataGraph.class.getName());
}
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 18:32:54 UTC (rev 8267)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 22:57:13 UTC (rev 8268)
@@ -34,6 +34,7 @@
import org.openrdf.model.URI;
import org.openrdf.model.impl.URIImpl;
+import com.bigdata.blueprints.BigdataGraph;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
/**
@@ -46,6 +47,10 @@
private static final transient Logger log = Logger
.getLogger(RESTServlet.class);
+ static {
+ log.info(BigdataGraph.class.getName());
+ }
+
/**
*
*/
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-10 18:32:58
|
Revision: 8267
http://sourceforge.net/p/bigdata/code/8267
Author: mrpersonick
Date: 2014-05-10 18:32:54 +0000 (Sat, 10 May 2014)
Log Message:
-----------
getting back to a clean CI state
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 18:25:09 UTC (rev 8266)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 18:32:54 UTC (rev 8267)
@@ -34,7 +34,6 @@
import org.openrdf.model.URI;
import org.openrdf.model.impl.URIImpl;
-import com.bigdata.blueprints.BigdataGraph;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
/**
@@ -253,8 +252,7 @@
} else if (req.getParameter(BlueprintsServlet.ATTR_BLUEPRINTS) != null) {
-// m_blueprintsServlet.doPost(req, resp);
- log.info(BigdataGraph.class.getName());
+ m_blueprintsServlet.doPost(req, resp);
} else if (req.getParameter("uri") != null) {
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-10 18:25:12
|
Revision: 8266
http://sourceforge.net/p/bigdata/code/8266
Author: mrpersonick
Date: 2014-05-10 18:25:09 +0000 (Sat, 10 May 2014)
Log Message:
-----------
pulling out GraphMLReader
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataVertex.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Added Paths:
-----------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
Removed Paths:
-------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java 2014-05-10 17:51:30 UTC (rev 8265)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java 2014-05-10 18:25:09 UTC (rev 8266)
@@ -53,8 +53,6 @@
import com.tinkerpop.blueprints.Graph;
import com.tinkerpop.blueprints.GraphQuery;
import com.tinkerpop.blueprints.Vertex;
-import com.tinkerpop.blueprints.util.DefaultGraphQuery;
-import com.tinkerpop.blueprints.util.io.graphml.GraphMLReader;
/**
* A base class for a Blueprints wrapper around a bigdata back-end.
@@ -68,26 +66,12 @@
public static final URI EDGE = new URIImpl(BD.NAMESPACE + "Edge");
-// final BigdataSailRepository repo;
-//
-// transient BigdataSailRepositoryConnection cxn;
-
final BlueprintsRDFFactory factory;
-// public BigdataGraph(final BigdataSailRepository repo) {
-// this(repo, BigdataRDFFactory.INSTANCE);
-// }
-
- public BigdataGraph(//final BigdataSailRepository repo,
- final BlueprintsRDFFactory factory) {
-// try {
-// this.repo = repo;
-// this.cxn = repo.getUnisolatedConnection();
-// this.cxn.setAutoCommit(false);
- this.factory = factory;
-// } catch (RepositoryException ex) {
-// throw new RuntimeException(ex);
-// }
+ public BigdataGraph(final BlueprintsRDFFactory factory) {
+
+ this.factory = factory;
+
}
public String toString() {
@@ -98,63 +82,38 @@
* Post a GraphML file to the remote server. (Bulk-upload operation.)
*/
public void loadGraphML(final String file) throws Exception {
- GraphMLReader.inputGraph(this, file);
+// GraphMLReader.inputGraph(this, file);
}
protected abstract RepositoryConnection cxn() throws Exception;
-// public BigdataSailRepositoryConnection getConnection() {
-// return this.cxn;
-// }
-//
-// public BlueprintsRDFFactory getFactory() {
-// return this.factory;
-// }
-
-// public Value getValue(final URI s, final URI p) {
-//
-// try {
-//
-// final RepositoryResult<Statement> result =
-// cxn.getStatements(s, p, null, false);
-//
-// if (result.hasNext()) {
-//
-// final Value o = result.next().getObject();
-//
-// if (result.hasNext()) {
-// throw new RuntimeException(s
-// + ": more than one value for p: " + p
-// + ", did you mean to call getValues()?");
-// }
-//
-// return o;
-//
-// }
-//
-// return null;
-//
-// } catch (Exception ex) {
-// throw new RuntimeException(ex);
-// }
-//
-// }
-
- public Object getProperty(final URI s, final URI p) {
+ /**
+ * Return a property for an edge or vertex.
+ */
+ public Object getProperty(final URI uri, final String property) {
+
+ return getProperty(uri, factory.toPropertyURI(property));
+
+ }
+ /**
+ * Return a property for an edge or vertex.
+ */
+ public Object getProperty(final URI uri, final URI property) {
+
try {
final RepositoryResult<Statement> result =
- cxn().getStatements(s, p, null, false);
+ cxn().getStatements(uri, property, null, false);
if (result.hasNext()) {
final Value value = result.next().getObject();
if (result.hasNext()) {
- throw new RuntimeException(s
- + ": more than one value for p: " + p
- + ", did you mean to call getValues()?");
+ throw new RuntimeException(uri
+ + ": more than one value for p: " + property
+ + ", did you mean to call getProperties()?");
}
if (!(value instanceof Literal)) {
@@ -175,37 +134,15 @@
}
-// public List<Value> getValues(final URI s, final URI p) {
-//
-// try {
-//
-// final RepositoryResult<Statement> result =
-// cxn().getStatements(s, p, null, false);
-//
-// final List<Value> values = new LinkedList<Value>();
-//
-// while (result.hasNext()) {
-//
-// final Value o = result.next().getObject();
-//
-// values.add(o);
-//
-// }
-//
-// return values;
-//
-// } catch (Exception ex) {
-// throw new RuntimeException(ex);
-// }
-//
-// }
-
- public List<Object> getProperties(final URI s, final URI p) {
+ /**
+ * Return a collection of properties for an edge or vertex.
+ */
+ public List<Object> getProperties(final URI uri, final URI property) {
try {
final RepositoryResult<Statement> result =
- cxn().getStatements(s, p, null, false);
+ cxn().getStatements(uri, property, null, false);
final List<Object> props = new LinkedList<Object>();
@@ -231,12 +168,15 @@
}
- public Set<String> getPropertyKeys(final URI s) {
+ /**
+ * Return the property keys used by an edge or vertex.
+ */
+ public Set<String> getPropertyKeys(final URI uri) {
try {
final RepositoryResult<Statement> result =
- cxn().getStatements(s, null, null, false);
+ cxn().getStatements(uri, null, null, false);
final Set<String> properties = new LinkedHashSet<String>();
@@ -267,13 +207,25 @@
}
- public Object removeProperty(final URI s, final URI p) {
+ /**
+ * Remove all values for a particular property on an edge or vertex.
+ */
+ public Object removeProperty(final URI uri, final String property) {
+
+ return removeProperty(uri, factory.toPropertyURI(property));
+
+ }
+
+ /**
+ * Remove all values for a particular property on an edge or vertex.
+ */
+ public Object removeProperty(final URI uri, final URI property) {
try {
- final Object oldVal = getProperty(s, p);
+ final Object oldVal = getProperty(uri, property);
- cxn().remove(s, p, null);
+ cxn().remove(uri, property, null);
return oldVal;
@@ -283,6 +235,8 @@
}
+ /**
+ */
public void setProperty(final URI s, final URI p, final Literal o) {
try {
@@ -593,6 +547,10 @@
}
+ /**
+ * Providing an override implementation for our GraphQuery to avoid the
+ * low-performance scan and filter paradigm. See {@link BigdataGraphQuery}.
+ */
@Override
public GraphQuery query() {
// return new DefaultGraphQuery(this);
@@ -633,42 +591,13 @@
}
}
-// @Override
-// public void commit() {
-// try {
-// cxn().commit();
-// } catch (RepositoryException e) {
-// throw new RuntimeException(e);
-// }
-// }
-//
-// @Override
-// public void rollback() {
-// try {
-// cxn().rollback();
-// cxn.close();
-// cxn = repo.getUnisolatedConnection();
-// cxn.setAutoCommit(false);
-// } catch (RepositoryException e) {
-// throw new RuntimeException(e);
-// }
-// }
-//
-// @Override
-// public void shutdown() {
-// try {
-// cxn.close();
-// repo.shutDown();
-// } catch (RepositoryException e) {
-// throw new RuntimeException(e);
-// }
-// }
-//
-// @Override
-// @Deprecated
-// public void stopTransaction(Conclusion arg0) {
-// }
-
+ /**
+ * Translate a collection of Bigdata statements into an iteration of
+ * Blueprints vertices.
+ *
+ * @author mikepersonick
+ *
+ */
public class VertexIterable implements Iterable<Vertex>, Iterator<Vertex> {
private final CloseableIteration<Statement, ? extends OpenRDFException> stmts;
@@ -723,6 +652,13 @@
}
+ /**
+ * Translate a collection of Bigdata statements into an iteration of
+ * Blueprints edges.
+ *
+ * @author mikepersonick
+ *
+ */
public class EdgeIterable implements Iterable<Edge>, Iterator<Edge> {
private final CloseableIteration<Statement, ? extends OpenRDFException> stmts;
@@ -771,6 +707,12 @@
}
+ /**
+ * Fuse two iterables together into one. Useful for combining IN and OUT
+ * edges for a vertex.
+ *
+ * @author mikepersonick
+ */
public class FusedIterable<T> implements Iterable<T>, Iterator<T> {
private final Iterable<T>[] args;
Deleted: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java 2014-05-10 17:51:30 UTC (rev 8265)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java 2014-05-10 18:25:09 UTC (rev 8266)
@@ -1,449 +0,0 @@
-/**
-Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
-
-Contact:
- SYSTAP, LLC
- 4501 Tower Road
- Greensboro, NC 27410
- lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-*/
-package com.bigdata.blueprints;
-
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.LinkedList;
-import java.util.List;
-
-import org.apache.log4j.Logger;
-import org.openrdf.model.Literal;
-import org.openrdf.model.URI;
-
-import com.tinkerpop.blueprints.Contains;
-import com.tinkerpop.blueprints.Edge;
-import com.tinkerpop.blueprints.GraphQuery;
-import com.tinkerpop.blueprints.Predicate;
-import com.tinkerpop.blueprints.Vertex;
-
-public class BigdataGraphQuery implements GraphQuery {
-
- protected final static transient Logger log = Logger.getLogger(BigdataGraphQuery.class);
-
- protected static List<Class> knownPredicates = Arrays.asList(new Class[] {
- BigdataPredicate.class,
- Compare.class,
- Contains.class,
- com.tinkerpop.blueprints.Compare.class
- });
-
- private final BigdataGraph graph;
-
- private final List<Has> criteria = new LinkedList<Has>();
-
- private transient int limit = Integer.MAX_VALUE;
-
- public BigdataGraphQuery(final BigdataGraph graph) {
- this.graph = graph;
- }
-
- /**
- * Filter out elements that do not have a property with provided key.
- *
- * ?s <key> ?value
- *
- * @param key the key of the property
- * @return the modified query object
- */
- @Override
- public GraphQuery has(final String key) {
- criteria.add(new Has(key));
- return this;
- }
-
- /**
- * Filter out elements that have a property with provided key.
- *
- * ?s ?p ?o .
- * filter not exists { ?s <key> ?value } .
- *
- * @param key the key of the property
- * @return the modified query object
- */
- @Override
- public GraphQuery hasNot(final String key) {
- criteria.add(new HasNot(key));
- return this;
- }
-
- /**
- * Filter out elements that do not have a property value equal to provided value.
- *
- * ?s <key> <value> .
- *
- * @param key the key of the property
- * @param value the value to check against
- * @return the modified query object
- */
- @Override
- public GraphQuery has(final String key, final Object value) {
- criteria.add(new Has(key, value));
- return this;
- }
-
- /**
- * Filter out elements that have a property value equal to provided value.
- *
- * ?s ?p ?o .
- * filter not exists { ?s <key> <value> } .
- *
- * @param key the key of the property
- * @param value the value to check against
- * @return the modified query object
- */
- @Override
- public GraphQuery hasNot(final String key, final Object value) {
- criteria.add(new HasNot(key, value));
- return this;
- }
-
- /**
- * Filter out the element if it does not have a property with a comparable value.
- *
- * @param key the key of the property
- * @param predicate the comparator to use for comparison
- * @param value the value to check against
- * @return the modified query object
- */
- @Override
- public GraphQuery has(final String key, final Predicate predicate, final Object value) {
- if (!knownPredicates.contains(predicate.getClass())) {
- throw new IllegalArgumentException();
- }
- criteria.add(new Has(key, value, BigdataPredicate.toBigdataPredicate(predicate)));
- return this;
- }
-
- /**
- * Filter out the element if it does not have a property with a comparable value.
- *
- * @param key the key of the property
- * @param value the value to check against
- * @param compare the comparator to use for comparison
- * @return the modified query object
- */
- @Override
- @Deprecated
- public <T extends Comparable<T>> GraphQuery has(final String key, final T value,
- final Compare compare) {
- return has(key, compare, value);
- }
-
- /**
- * Filter out the element of its property value is not within the provided interval.
- *
- * @param key the key of the property
- * @param startValue the inclusive start value of the interval
- * @param endValue the exclusive end value of the interval
- * @return the modified query object
- */
- @Override
- public <T extends Comparable<?>> GraphQuery interval(final String key,
- final T startValue, final T endValue) {
- return has(key, BigdataPredicate.GTE, startValue)
- .has(key, BigdataPredicate.LT, endValue);
- }
-
- /**
- * Filter out the element if the take number of incident/adjacent elements to retrieve has already been reached.
- *
- * @param limit the take number of elements to return
- * @return the modified query object
- */
- @Override
- public GraphQuery limit(final int limit) {
- this.limit = limit;
- return this;
- }
-
- /**
- * Execute the query and return the matching edges.
- *
- * @return the unfiltered incident edges
- */
- @Override
- public Iterable<Edge> edges() {
- final String queryStr = toQueryStr(BigdataGraph.EDGE);
- return graph.getEdges(queryStr);
- }
-
- /**
- * Execute the query and return the vertices on the other end of the matching edges.
- *
- * @return the unfiltered adjacent vertices
- */
- @Override
- public Iterable<Vertex> vertices() {
- final String queryStr = toQueryStr(BigdataGraph.VERTEX);
- return graph.getVertices(queryStr, true);
- }
-
- protected String toQueryStr(final URI type) {
-
- final StringBuilder sb = new StringBuilder();
-
- if (type == BigdataGraph.VERTEX) {
- sb.append("construct { ?x rdf:type <"+type+"> . }\n");
- sb.append("{\n select distinct ?x where {\n");
- } else {
- sb.append("construct { ?from ?x ?to . }\n");
- sb.append("{\n select distinct ?from ?x ?to where {\n");
- sb.append(" ?from ?x ?to .\n");
- }
-
- final BlueprintsRDFFactory factory = graph.factory;
-
- boolean hasHas = false;
-
- int i = 1;
- for (Has has : criteria) {
-
- if (log.isTraceEnabled()) {
- log.trace(has);
- }
-
- if (has instanceof HasNot) {
-
- sb.append(" filter not exists { ");
-
- sb.append("?x <").append(factory.toPropertyURI(has.key)).append("> ");
-
- if (has.val != null) {
-
- final String val = factory.toLiteral(has.val).toString();
-
- sb.append(val).append(" .");
-
- } else {
-
- final String var = "?val"+i;
-
- sb.append(var).append(" .");
-
- }
-
- sb.append("}\n");
-
- } else {
-
- hasHas = true;
-
- sb.append(" ?x <").append(factory.toPropertyURI(has.key)).append("> ");
-
- if (has.val != null &&
- (has.pred == null || has.pred == BigdataPredicate.EQ)) {
-
- final Literal val = factory.toLiteral(has.val);
-
- sb.append(val).append(" .\n");
-
- } else {
-
- final String var = "?val"+i;
-
- sb.append(var).append(" .\n");
-
- if (has.pred != null) {
-
- sb.append(toFilterStr(has.pred, var, has.val)).append("\n");
-
- }
-
- }
-
- }
-
- i++;
-
- }
-
- // need a statement pattern for the filter not exists
- if (!hasHas) {
-
- sb.append(" ?x rdf:type <").append(type).append("> .\n");
-
- }
-
-// sb.setLength(sb.length()-1);
-
- sb.append(" }");
-
- if (limit < Integer.MAX_VALUE) {
-
- sb.append(" limit " + factory.toLiteral(limit).getLabel());
-
- }
-
- sb.append("\n}");
-
- if (log.isTraceEnabled()) {
- log.trace("\n"+sb.toString());
- }
-
- return sb.toString();
-
- }
-
- private String toFilterStr(final BigdataPredicate pred, final String var,
- final Object val) {
-
- final BlueprintsRDFFactory factory = graph.factory;
-
- final StringBuilder sb = new StringBuilder();
-
- if (pred == BigdataPredicate.EQ) {
-
- throw new IllegalArgumentException();
-
- } else if (pred == BigdataPredicate.GT || pred == BigdataPredicate.GTE ||
- pred == BigdataPredicate.LT || pred == BigdataPredicate.LTE ||
- pred == BigdataPredicate.NE) {
-
- final Literal l = factory.toLiteral(val);
-
- sb.append(" filter(").append(var);
-
- switch(pred) {
- case GT:
- sb.append(" > "); break;
- case GTE:
- sb.append(" >= "); break;
- case LT:
- sb.append(" < "); break;
- case LTE:
- sb.append(" <= "); break;
- case NE:
- sb.append(" != "); break;
- default:
- break;
- }
-
- sb.append(l).append(") .");
-
- } else if (pred == BigdataPredicate.IN || pred == BigdataPredicate.NIN) {
-
- sb.append(" filter(");
-
- if (pred == BigdataPredicate.NIN) {
- sb.append("!(");
- }
-
- sb.append(var).append(" in (");
-
- final Collection<?> c = (Collection<?>) val;
-
- for (Object o : c) {
-
- final Literal l = factory.toLiteral(o);
-
- sb.append(l).append(", ");
-
- }
-
- sb.setLength(sb.length()-2);
-
- if (pred == BigdataPredicate.NIN) {
- sb.append(")");
- }
-
- sb.append(")) .");
-
- }
-
- return sb.toString();
-
- }
-
- private class Has {
-
- private String key;
-
- private Object val;
-
- private BigdataPredicate pred;
-
- public Has(final String key) {
- this(key, null, null);
- }
-
- public Has(final String key, final Object val) {
- this(key, val, null);
- }
-
- public Has(final String key, final Object val,
- final BigdataPredicate pred) {
-
- if (pred == BigdataPredicate.IN || pred == BigdataPredicate.NIN) {
-
- if (!(val instanceof Collection)) {
- throw new IllegalArgumentException();
- }
-
- if (((Collection<?>) val).size() == 0) {
- throw new IllegalArgumentException();
- }
-
- }
-
- this.key = key;
-
- if (pred == BigdataPredicate.IN && ((Collection<?>) val).size() == 1) {
-
- /*
- * Simple optimization to replace a single value IN with
- * a simple EQ.
- */
- this.val = ((Collection<?>) val).iterator().next();
- this.pred = null;
-
- } else {
-
- this.val = val;
- this.pred = pred;
-
- }
-
- }
-
- public String toString() {
- return "key: " + key + ", val: " + val + ", pred: " + pred;
- }
-
- }
-
- private class HasNot extends Has {
-
- public HasNot(final String key) {
- super(key);
- }
-
- public HasNot(final String key, final Object val) {
- super(key, val);
- }
-
- }
-
-
-
-}
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java 2014-05-10 18:25:09 UTC (rev 8266)
@@ -0,0 +1,489 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.openrdf.model.Literal;
+import org.openrdf.model.URI;
+
+import com.tinkerpop.blueprints.Edge;
+import com.tinkerpop.blueprints.Predicate;
+import com.tinkerpop.blueprints.GraphQuery;
+import com.tinkerpop.blueprints.Vertex;
+
+/**
+ * Translate a low-performance Blueprints GraphQuery into a high-performance
+ * SPARQL query.
+ *
+ * @author mikepersonick
+ *
+ */
+public class BigdataGraphQuery implements GraphQuery {
+
+ protected final static transient Logger log = Logger.getLogger(BigdataGraphQuery.class);
+
+ /**
+ * These are the only Predicate implementations we handle currently.
+ */
+ protected static List<Class> knownPredicates = Arrays.asList(new Class[] {
+ BigdataPredicate.class,
+ com.tinkerpop.blueprints.Query.Compare.class,
+ com.tinkerpop.blueprints.Contains.class,
+ com.tinkerpop.blueprints.Compare.class
+ });
+
+ /**
+ * The graph.
+ */
+ private final BigdataGraph graph;
+
+ /**
+ * The list of criteria. Bigdata's query optimizer will re-order the
+ * criteria based on selectivity and execute for maximum performance and
+ * minimum IO.
+ */
+ private final List<Has> criteria = new LinkedList<Has>();
+
+ /**
+ * Limit the number of results.
+ */
+ private transient int limit = Integer.MAX_VALUE;
+
+ public BigdataGraphQuery(final BigdataGraph graph) {
+ this.graph = graph;
+ }
+
+ /**
+ * Filter out elements that do not have a property with provided key.
+ *
+ * ?s <key> ?value
+ *
+ * @param key the key of the property
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery has(final String key) {
+ criteria.add(new Has(key));
+ return this;
+ }
+
+ /**
+ * Filter out elements that have a property with provided key.
+ *
+ * ?s ?p ?o .
+ * filter not exists { ?s <key> ?value } .
+ *
+ * @param key the key of the property
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery hasNot(final String key) {
+ criteria.add(new HasNot(key));
+ return this;
+ }
+
+ /**
+ * Filter out elements that do not have a property value equal to provided value.
+ *
+ * ?s <key> <value> .
+ *
+ * @param key the key of the property
+ * @param value the value to check against
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery has(final String key, final Object value) {
+ criteria.add(new Has(key, value));
+ return this;
+ }
+
+ /**
+ * Filter out elements that have a property value equal to provided value.
+ *
+ * ?s ?p ?o .
+ * filter not exists { ?s <key> <value> } .
+ *
+ * @param key the key of the property
+ * @param value the value to check against
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery hasNot(final String key, final Object value) {
+ criteria.add(new HasNot(key, value));
+ return this;
+ }
+
+ /**
+ * Filter out the element if it does not have a property with a comparable value.
+ *
+ * @param key the key of the property
+ * @param predicate the comparator to use for comparison
+ * @param value the value to check against
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery has(final String key, final Predicate predicate, final Object value) {
+ if (!knownPredicates.contains(predicate.getClass())) {
+ throw new IllegalArgumentException();
+ }
+ criteria.add(new Has(key, value, BigdataPredicate.toBigdataPredicate(predicate)));
+ return this;
+ }
+
+ /**
+ * Filter out the element if it does not have a property with a comparable value.
+ *
+ * @param key the key of the property
+ * @param value the value to check against
+ * @param compare the comparator to use for comparison
+ * @return the modified query object
+ */
+ @Override
+ @Deprecated
+ public <T extends Comparable<T>> GraphQuery has(final String key, final T value,
+ final Compare compare) {
+ return has(key, compare, value);
+ }
+
+ /**
+ * Filter out the element of its property value is not within the provided interval.
+ *
+ * @param key the key of the property
+ * @param startValue the inclusive start value of the interval
+ * @param endValue the exclusive end value of the interval
+ * @return the modified query object
+ */
+ @Override
+ public <T extends Comparable<?>> GraphQuery interval(final String key,
+ final T startValue, final T endValue) {
+ return has(key, BigdataPredicate.GTE, startValue)
+ .has(key, BigdataPredicate.LT, endValue);
+ }
+
+ /**
+ * Filter out the element if the take number of incident/adjacent elements to retrieve has already been reached.
+ *
+ * @param limit the take number of elements to return
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery limit(final int limit) {
+ this.limit = limit;
+ return this;
+ }
+
+ /**
+ * Execute the query and return the matching edges.
+ *
+ * @return the unfiltered incident edges
+ */
+ @Override
+ public Iterable<Edge> edges() {
+ final String queryStr = toQueryStr(BigdataGraph.EDGE);
+ return graph.getEdges(queryStr);
+ }
+
+ /**
+ * Execute the query and return the vertices on the other end of the matching edges.
+ *
+ * @return the unfiltered adjacent vertices
+ */
+ @Override
+ public Iterable<Vertex> vertices() {
+ final String queryStr = toQueryStr(BigdataGraph.VERTEX);
+ return graph.getVertices(queryStr, true);
+ }
+
+ /**
+ * Generate the SPARQL query.
+ */
+ protected String toQueryStr(final URI type) {
+
+ final StringBuilder sb = new StringBuilder();
+
+ if (type == BigdataGraph.VERTEX) {
+ sb.append("construct { ?x rdf:type <"+type+"> . }\n");
+ sb.append("{\n select distinct ?x where {\n");
+ } else {
+ sb.append("construct { ?from ?x ?to . }\n");
+ sb.append("{\n select distinct ?from ?x ?to where {\n");
+ sb.append(" ?from ?x ?to .\n");
+ }
+
+ final BlueprintsRDFFactory factory = graph.factory;
+
+ boolean hasHas = false;
+
+ int i = 1;
+ for (Has has : criteria) {
+
+ if (log.isTraceEnabled()) {
+ log.trace(has);
+ }
+
+ if (has instanceof HasNot) {
+
+ sb.append(" filter not exists { ");
+
+ sb.append("?x <").append(factory.toPropertyURI(has.key)).append("> ");
+
+ if (has.val != null) {
+
+ final String val = factory.toLiteral(has.val).toString();
+
+ sb.append(val).append(" .");
+
+ } else {
+
+ final String var = "?val"+i;
+
+ sb.append(var).append(" .");
+
+ }
+
+ sb.append("}\n");
+
+ } else {
+
+ hasHas = true;
+
+ sb.append(" ?x <").append(factory.toPropertyURI(has.key)).append("> ");
+
+ if (has.val != null &&
+ (has.pred == null || has.pred == BigdataPredicate.EQ)) {
+
+ final Literal val = factory.toLiteral(has.val);
+
+ sb.append(val).append(" .\n");
+
+ } else {
+
+ final String var = "?val"+i;
+
+ sb.append(var).append(" .\n");
+
+ if (has.pred != null) {
+
+ sb.append(toFilterStr(has.pred, var, has.val)).append("\n");
+
+ }
+
+ }
+
+ }
+
+ i++;
+
+ }
+
+ // need a statement pattern for the filter not exists
+ if (!hasHas) {
+
+ sb.append(" ?x rdf:type <").append(type).append("> .\n");
+
+ }
+
+// sb.setLength(sb.length()-1);
+
+ sb.append(" }");
+
+ if (limit < Integer.MAX_VALUE) {
+
+ sb.append(" limit " + factory.toLiteral(limit).getLabel());
+
+ }
+
+ sb.append("\n}");
+
+ if (log.isTraceEnabled()) {
+ log.trace("\n"+sb.toString());
+ }
+
+ return sb.toString();
+
+ }
+
+ /**
+ * Generate a SPARQL filter string for a particular Predicate.
+ */
+ private String toFilterStr(final BigdataPredicate pred, final String var,
+ final Object val) {
+
+ final BlueprintsRDFFactory factory = graph.factory;
+
+ final StringBuilder sb = new StringBuilder();
+
+ if (pred == BigdataPredicate.EQ) {
+
+ throw new IllegalArgumentException();
+
+ } else if (pred == BigdataPredicate.GT || pred == BigdataPredicate.GTE ||
+ pred == BigdataPredicate.LT || pred == BigdataPredicate.LTE ||
+ pred == BigdataPredicate.NE) {
+
+ final Literal l = factory.toLiteral(val);
+
+ sb.append(" filter(").append(var);
+
+ switch(pred) {
+ case GT:
+ sb.append(" > "); break;
+ case GTE:
+ sb.append(" >= "); break;
+ case LT:
+ sb.append(" < "); break;
+ case LTE:
+ sb.append(" <= "); break;
+ case NE:
+ sb.append(" != "); break;
+ default:
+ break;
+ }
+
+ sb.append(l).append(") .");
+
+ } else if (pred == BigdataPredicate.IN || pred == BigdataPredicate.NIN) {
+
+ sb.append(" filter(");
+
+ if (pred == BigdataPredicate.NIN) {
+ sb.append("!(");
+ }
+
+ sb.append(var).append(" in (");
+
+ final Collection<?> c = (Collection<?>) val;
+
+ for (Object o : c) {
+
+ final Literal l = factory.toLiteral(o);
+
+ sb.append(l).append(", ");
+
+ }
+
+ sb.setLength(sb.length()-2);
+
+ if (pred == BigdataPredicate.NIN) {
+ sb.append(")");
+ }
+
+ sb.append(")) .");
+
+ }
+
+ return sb.toString();
+
+ }
+
+ /**
+ * Standard criterion for filtering by the existence of a property and
+ * optional value.
+ *
+ * @author mikepersonick
+ *
+ */
+ private class Has {
+
+ private String key;
+
+ private Object val;
+
+ private BigdataPredicate pred;
+
+ public Has(final String key) {
+ this(key, null, null);
+ }
+
+ public Has(final String key, final Object val) {
+ this(key, val, null);
+ }
+
+ public Has(final String key, final Object val,
+ final BigdataPredicate pred) {
+
+ if (pred == BigdataPredicate.IN || pred == BigdataPredicate.NIN) {
+
+ if (!(val instanceof Collection)) {
+ throw new IllegalArgumentException();
+ }
+
+ if (((Collection<?>) val).size() == 0) {
+ throw new IllegalArgumentException();
+ }
+
+ }
+
+ this.key = key;
+
+ if (pred == BigdataPredicate.IN && ((Collection<?>) val).size() == 1) {
+
+ /*
+ * Simple optimization to replace a single value IN with
+ * a simple EQ.
+ */
+ this.val = ((Collection<?>) val).iterator().next();
+ this.pred = null;
+
+ } else {
+
+ this.val = val;
+ this.pred = pred;
+
+ }
+
+ }
+
+ public String toString() {
+ return "key: " + key + ", val: " + val + ", pred: " + pred;
+ }
+
+ }
+
+ /**
+ * Criterion for filtering by the non-existence of a property and
+ * optional value. Uses SPARQL filter not exists {}.
+ *
+ * @author mikepersonick
+ *
+ */
+ private class HasNot extends Has {
+
+ public HasNot(final String key) {
+ super(key);
+ }
+
+ public HasNot(final String key, final Object val) {
+ super(key, val);
+ }
+
+ }
+
+
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataVertex.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataVertex.java 2014-05-10 17:51:30 UTC (rev 8265)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataVertex.java 2014-05-10 18:25:09 UTC (rev 8266)
@@ -88,6 +88,11 @@
}
}
+ /**
+ * Not bothering to provide a SPARQL translation for vertex queries at
+ * this time. I suspect that scan and filter works fine when starting from
+ * an individual vertex.
+ */
@Override
public VertexQuery query() {
return new DefaultVertexQuery(this);
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-10 17:51:30 UTC (rev 8265)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-10 18:25:09 UTC (rev 8266)
@@ -31,11 +31,9 @@
import org.apache.log4j.Logger;
-import com.bigdata.blueprints.BigdataGraphBulkLoad;
import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
import com.bigdata.rdf.store.AbstractTripleStore;
-import com.tinkerpop.blueprints.util.io.graphml.GraphMLReader;
/**
* Helper servlet for the blueprints layer.
@@ -110,15 +108,15 @@
conn = getBigdataRDFContext()
.getUnisolatedConnection(namespace);
- final BigdataGraphBulkLoad graph = new BigdataGraphBulkLoad(conn);
+// final BigdataGraphBulkLoad graph = new BigdataGraphBulkLoad(conn);
+//
+// GraphMLReader.inputGraph(graph, req.getInputStream());
+//
+// graph.commit();
+//
+// final long nmodified = graph.getMutationCountLastCommit();
+ final long nmodified = 0;
- GraphMLReader.inputGraph(graph, req.getInputStream());
-
- graph.commit();
-
- final long nmodified = graph.getMutationCountLastCommit();
-// final long nmodified = 0;
-
final long elapsed = System.currentTimeMillis() - begin;
reportModifiedCount(resp, nmodified, elapsed);
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 17:51:30 UTC (rev 8265)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 18:25:09 UTC (rev 8266)
@@ -34,6 +34,7 @@
import org.openrdf.model.URI;
import org.openrdf.model.impl.URIImpl;
+import com.bigdata.blueprints.BigdataGraph;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
/**
@@ -253,6 +254,7 @@
} else if (req.getParameter(BlueprintsServlet.ATTR_BLUEPRINTS) != null) {
// m_blueprintsServlet.doPost(req, resp);
+ log.info(BigdataGraph.class.getName());
} else if (req.getParameter("uri") != null) {
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-10 17:51:35
|
Revision: 8265
http://sourceforge.net/p/bigdata/code/8265
Author: mrpersonick
Date: 2014-05-10 17:51:30 +0000 (Sat, 10 May 2014)
Log Message:
-----------
3. Add in BlueprintsServlet in bigdata-sails/src/java
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-10 15:05:02 UTC (rev 8264)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-10 17:51:30 UTC (rev 8265)
@@ -58,13 +58,6 @@
*/
static final transient String ATTR_BLUEPRINTS = "blueprints";
-// /**
-// * Flag to signify a convert operation. POST an RDF document with a
-// * content type and an accept header for what it should be converted to.
-// */
-// static final transient String ATTR_CONVERT = "convert";
-
-
public BlueprintsServlet() {
}
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 15:05:02 UTC (rev 8264)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/RESTServlet.java 2014-05-10 17:51:30 UTC (rev 8265)
@@ -252,7 +252,7 @@
} else if (req.getParameter(BlueprintsServlet.ATTR_BLUEPRINTS) != null) {
- m_blueprintsServlet.doPost(req, resp);
+// m_blueprintsServlet.doPost(req, resp);
} else if (req.getParameter("uri") != null) {
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-10 15:05:04
|
Revision: 8264
http://sourceforge.net/p/bigdata/code/8264
Author: mrpersonick
Date: 2014-05-10 15:05:02 +0000 (Sat, 10 May 2014)
Log Message:
-----------
3. Add in BlueprintsServlet in bigdata-sails/src/java
Modified Paths:
--------------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
Modified: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-10 11:52:25 UTC (rev 8263)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BlueprintsServlet.java 2014-05-10 15:05:02 UTC (rev 8264)
@@ -31,9 +31,11 @@
import org.apache.log4j.Logger;
+import com.bigdata.blueprints.BigdataGraphBulkLoad;
import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
import com.bigdata.rdf.sail.webapp.client.MiniMime;
import com.bigdata.rdf.store.AbstractTripleStore;
+import com.tinkerpop.blueprints.util.io.graphml.GraphMLReader;
/**
* Helper servlet for the blueprints layer.
@@ -115,15 +117,15 @@
conn = getBigdataRDFContext()
.getUnisolatedConnection(namespace);
-// final BigdataGraphBulkLoad graph = new BigdataGraphBulkLoad(conn);
-//
-// GraphMLReader.inputGraph(graph, req.getInputStream());
-//
-// graph.commit();
-//
-// final long nmodified = graph.getMutationCountLastCommit();
- final long nmodified = 0;
+ final BigdataGraphBulkLoad graph = new BigdataGraphBulkLoad(conn);
+ GraphMLReader.inputGraph(graph, req.getInputStream());
+
+ graph.commit();
+
+ final long nmodified = graph.getMutationCountLastCommit();
+// final long nmodified = 0;
+
final long elapsed = System.currentTimeMillis() - begin;
reportModifiedCount(resp, nmodified, elapsed);
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <jer...@us...> - 2014-05-10 11:52:28
|
Revision: 8263
http://sourceforge.net/p/bigdata/code/8263
Author: jeremy_carroll
Date: 2014-05-10 11:52:25 +0000 (Sat, 10 May 2014)
Log Message:
-----------
Cleaning up of ConfigurableAnalyzerFactory, adding TermCompletionAnalyzer, deprecating DefaultAnalyzerFactory
Finishing of trac 912, work on 915
Unit tests for the old and new behaviors
This merges the branch TEXT_ANALYZERS.
Modified Paths:
--------------
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/DefaultAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractAnalyzerFactoryTest.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractSearchTest.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/NonEnglishExamples.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestAll.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestConfigurableAsDefaultAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestDefaultAnalyzerFactory.java
Added Paths:
-----------
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/TermCompletionAnalyzer.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractDefaultAnalyzerFactoryTest.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestConfigurableAnalyzerFactory.java
branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestUnconfiguredAnalyzerFactory.java
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java 2014-05-10 02:56:35 UTC (rev 8262)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -66,6 +66,7 @@
* Supported classes included all the natural language specific classes from Lucene, and also:
* <ul>
* <li>{@link PatternAnalyzer}
+ * <li>{@link TermCompletionAnalyzer}
* <li>{@link KeywordAnalyzer}
* <li>{@link SimpleAnalyzer}
* <li>{@link StopAnalyzer}
@@ -76,7 +77,6 @@
* <ul>
* <li>no arguments
* <li>{@link Version}
- * <li>{@link Set} (of strings, the stop words)
* <li>{@link Version}, {@link Set}
* </ul>
* is usable. If the class has a static method named <code>getDefaultStopSet()</code> then this is assumed
@@ -89,19 +89,17 @@
* abbreviate to <code>c.b.s.C</code> in this documentation.
* Properties from {@link Options} apply to the factory.
* <p>
- *
- * If there are no such properties at all then the property {@link Options#INCLUDE_DEFAULTS} is set to true,
- * and the behavior of this class is the same as the legacy {@link DefaultAnalyzerFactory}.
- * <p>
* Other properties, from {@link AnalyzerOptions} start with
* <code>c.b.s.C.analyzer.<em>language-range</em></code> where <code><em>language-range</em></code> conforms
- * with the extended language range construct from RFC 4647, section 2.2. These are used to specify
- * an analyzer for the given language range.
+ * with the extended language range construct from RFC 4647, section 2.2.
+ * There is an issue that bigdata does not allow '*' in property names, and we use the character '_' to
+ * substitute for '*' in extended language ranges in property names.
+ * These are used to specify an analyzer for the given language range.
* <p>
* If no analyzer is specified for the language range <code>*</code> then the {@link StandardAnalyzer} is used.
* <p>
* Given any specific language, then the analyzer matching the longest configured language range,
- * measured in number of subtags is used {@link #getAnalyzer(String, boolean)}
+ * measured in number of subtags is returned by {@link #getAnalyzer(String, boolean)}
* In the event of a tie, the alphabetically first language range is used.
* The algorithm to find a match is "Extended Filtering" as defined in section 3.3.2 of RFC 4647.
* <p>
@@ -113,11 +111,13 @@
* <dd>This uses whitespace to tokenize</dd>
* <dt>{@link PatternAnalyzer}</dt>
* <dd>This uses a regular expression to tokenize</dd>
+ * <dt>{@link TermCompletionAnalyzer}</dt>
+ * <dd>This uses up to three regular expressions to specify multiple tokens for each word, to address term completion use cases.</dd>
* <dt>{@link EmptyAnalyzer}</dt>
* <dd>This suppresses the functionality, by treating every expression as a stop word.</dd>
* </dl>
* there are in addition the language specific analyzers that are included
- * by using the option {@link Options#INCLUDE_DEFAULTS}
+ * by using the option {@link Options#NATURAL_LANGUAGE_SUPPORT}
*
*
* @author jeremycarroll
@@ -126,11 +126,26 @@
public class ConfigurableAnalyzerFactory implements IAnalyzerFactory {
final private static transient Logger log = Logger.getLogger(ConfigurableAnalyzerFactory.class);
- static class LanguageRange implements Comparable<LanguageRange> {
+ /**
+ * This is an implementation of RFC 4647 language range,
+ * targetted at the specific needs within bigdata, and only
+ * supporting the extended filtering specified in section 3.3.2
+ * <p>
+ * Language ranges are comparable so that
+ * sorting an array and then matching a language tag against each
+ * member of the array in sequence will give the longest match.
+ * i.e. the longer ranges come first.
+ * @author jeremycarroll
+ *
+ */
+ public static class LanguageRange implements Comparable<LanguageRange> {
private final String range[];
private final String full;
-
+ /**
+ * Note range must be in lower case, this is not verified.
+ * @param range
+ */
public LanguageRange(String range) {
this.range = range.split("-");
full = range;
@@ -174,12 +189,22 @@
return full.hashCode();
}
+ /**
+ * This implements the algoirthm of section 3.3.2 of RFC 4647
+ * as modified with the observation about private use tags
+ * in <a href="http://lists.w3.org/Archives/Public/www-international/2014AprJun/0084">
+ * this message</a>.
+ *
+ *
+ * @param langTag The RFC 5646 Language tag in lower case
+ * @return The result of the algorithm
+ */
public boolean extendedFilterMatch(String langTag) {
return extendedFilterMatch(langTag.toLowerCase(Locale.ROOT).split("-"));
}
// See RFC 4647, 3.3.2
- public boolean extendedFilterMatch(String[] language) {
+ boolean extendedFilterMatch(String[] language) {
// RFC 4647 step 2
if (!matchSubTag(language[0], range[0])) {
return false;
@@ -227,13 +252,14 @@
*/
public interface Options {
/**
- * By setting this option to true, then the behavior of the legacy {@link DefaultAnalyzerFactory}
- * is added, and may be overridden by the settings of the user.
+ * By setting this option to true, then all the known Lucene Analyzers for natural
+ * languages are used for a range of language tags.
+ * These settings may then be overridden by the settings of the user.
* Specifically the following properties are loaded, prior to loading the
* user's specification (with <code>c.b.s.C</code> expanding to
* <code>com.bigdata.search.ConfigurableAnalyzerFactory</code>)
<pre>
-c.b.s.C.analyzer.*.like=eng
+c.b.s.C.analyzer._.like=eng
c.b.s.C.analyzer.por.analyzerClass=org.apache.lucene.analysis.br.BrazilianAnalyzer
c.b.s.C.analyzer.pt.like=por
c.b.s.C.analyzer.zho.analyzerClass=org.apache.lucene.analysis.cn.ChineseAnalyzer
@@ -265,18 +291,13 @@
*
*
*/
- String INCLUDE_DEFAULTS = ConfigurableAnalyzerFactory.class.getName() + ".includeDefaults";
+ String NATURAL_LANGUAGE_SUPPORT = ConfigurableAnalyzerFactory.class.getName() + ".naturalLanguageSupport";
/**
* This is the prefix to all properties configuring the individual analyzers.
*/
String ANALYZER = ConfigurableAnalyzerFactory.class.getName() + ".analyzer.";
-/**
- * If there is no configuration at all, then the defaults are included,
- * but any configuration at all totally replaces the defaults, unless
- * {@link #INCLUDE_DEFAULTS}
- * is explicitly set to true.
- */
- String DEFAULT_INCLUDE_DEFAULTS = "false";
+
+ String DEFAULT_NATURAL_LANGUAGE_SUPPORT = "false";
}
/**
* Options understood by analyzers created by {@link ConfigurableAnalyzerFactory}.
@@ -286,7 +307,9 @@
/**
* If specified this is the fully qualified name of a subclass of {@link Analyzer}
* that has appropriate constructors.
- * Either this or {@link #LIKE} or {@link #PATTERN} must be specified for each language range.
+ * This is set implicitly if some of the options below are selected (for example {@link #PATTERN}).
+ * For each configured language range, if it is not set, either explicitly or implicitly, then
+ * {@link #LIKE} must be specified.
*/
String ANALYZER_CLASS = "analyzerClass";
@@ -326,16 +349,52 @@
String STOPWORDS_VALUE_NONE = "none";
/**
- * If this property is present then the analyzer being used is a
- * {@link PatternAnalyzer} and the value is the pattern to use.
+ * The value of the pattern parameter to
+ * {@link PatternAnalyzer#PatternAnalyzer(Version, Pattern, boolean, Set)}
* (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
* It is an error if a different analyzer class is specified.
*/
- String PATTERN = ".pattern";
+ String PATTERN = "pattern";
+ /**
+ * The value of the wordBoundary parameter to
+ * {@link TermCompletionAnalyzer#TermCompletionAnalyzer(Pattern, Pattern, Pattern, boolean)}
+ * (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
+ * It is an error if a different analyzer class is specified.
+ */
+ String WORD_BOUNDARY = "wordBoundary";
+ /**
+ * The value of the subWordBoundary parameter to
+ * {@link TermCompletionAnalyzer#TermCompletionAnalyzer(Pattern, Pattern, Pattern, boolean)}
+ * (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
+ * It is an error if a different analyzer class is specified.
+ */
+ String SUB_WORD_BOUNDARY = "subWordBoundary";
+ /**
+ * The value of the softHyphens parameter to
+ * {@link TermCompletionAnalyzer#TermCompletionAnalyzer(Pattern, Pattern, Pattern, boolean)}
+ * (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
+ * It is an error if a different analyzer class is specified.
+ */
+ String SOFT_HYPHENS = "softHyphens";
+ /**
+ * The value of the alwaysRemoveSoftHypens parameter to
+ * {@link TermCompletionAnalyzer#TermCompletionAnalyzer(Pattern, Pattern, Pattern, boolean)}
+ * (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
+ * It is an error if a different analyzer class is specified.
+ */
+ String ALWAYS_REMOVE_SOFT_HYPHENS = "alwaysRemoveSoftHyphens";
+
+ boolean DEFAULT_ALWAYS_REMOVE_SOFT_HYPHENS = false;
+
+ /**
+ * The default sub-word boundary is a pattern that never matches,
+ * i.e. there are no sub-word boundaries.
+ */
+ Pattern DEFAULT_SUB_WORD_BOUNDARY = Pattern.compile("(?!)");
}
- private static final String DEFAULT_PROPERTIES =
+ private static final String ALL_LUCENE_NATURAL_LANGUAGES =
"com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.*.like=eng\n" +
"com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.por.analyzerClass=org.apache.lucene.analysis.br.BrazilianAnalyzer\n" +
"com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.pt.like=por\n" +
@@ -365,33 +424,67 @@
"com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.eng.analyzerClass=org.apache.lucene.analysis.standard.StandardAnalyzer\n" +
"com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.en.like=eng\n";
+ private static final String LUCENE_STANDARD_ANALYZER =
+ "com.bigdata.search.ConfigurableAnalyzerFactory.analyzer.*.analyzerClass=org.apache.lucene.analysis.standard.StandardAnalyzer\n";
+
+ /**
+ * This comment describes the implementation of {@link ConfigurableAnalyzerFactory}.
+ * The only method in the interface is {@link ConfigurableAnalyzerFactory#getAnalyzer(String, boolean)},
+ * a map is used from language tag to {@link AnalyzerPair}, where the pair contains
+ * an {@link Analyzer} both with and without stopwords configured (some times these two analyzers are identical,
+ * if, for example, stop words are not supported or not required).
+ * <p>
+ * If there is no entry for the language tag in the map {@link ConfigurableAnalyzerFactory#langTag2AnalyzerPair},
+ * then one is created, by walking down the array {@link ConfigurableAnalyzerFactory#config} of AnalyzerPairs
+ * until a matching one is found.
+ * <p>
+ * The bulk of the code in this class is invoked from the constructor in order to set up this
+ * {@link ConfigurableAnalyzerFactory#config} array. For example, all of the subclasses of {@link AnalyzerPair}s,
+ * are simply to call the appropriate constructor in the appropriate way: the difficulty is that many subclasses
+ * of {@link Analyzer} have constructors with different signatures, and our code needs to navigate each sort.
+ * @author jeremycarroll
+ *
+ */
private static class AnalyzerPair implements Comparable<AnalyzerPair>{
- private final LanguageRange range;
+ final LanguageRange range;
private final Analyzer withStopWords;
private final Analyzer withoutStopWords;
+ public Analyzer getAnalyzer(boolean filterStopwords) {
+ return filterStopwords ? withStopWords : withoutStopWords;
+ }
+
+ public boolean extendedFilterMatch(String[] language) {
+ return range.extendedFilterMatch(language);
+ }
+
AnalyzerPair(String range, Analyzer withStopWords, Analyzer withOutStopWords) {
this.range = new LanguageRange(range);
this.withStopWords = withStopWords;
this.withoutStopWords = withOutStopWords;
}
+ /**
+ * This clone constructor implements {@link AnalyzerOptions#LIKE}.
+ * @param range
+ * @param copyMe
+ */
AnalyzerPair(String range, AnalyzerPair copyMe) {
this.range = new LanguageRange(range);
this.withStopWords = copyMe.withStopWords;
this.withoutStopWords = copyMe.withoutStopWords;
-
}
-
- public Analyzer getAnalyzer(boolean filterStopwords) {
- return filterStopwords ? withStopWords : withoutStopWords;
- }
- @Override
- public String toString() {
- return range.full + "=(" + withStopWords.getClass().getSimpleName() +")";
- }
-
+ /**
+ * If we have a constructor, with arguments including a populated
+ * stop word set, then we can use it to make both the withStopWords
+ * analyzer, and the withoutStopWords analyzer.
+ * @param range
+ * @param cons A Constructor including a {@link java.util.Set} argument
+ * for the stop words.
+ * @param params The arguments to pass to the constructor including a populated stopword set.
+ * @throws Exception
+ */
AnalyzerPair(String range, Constructor<? extends Analyzer> cons, Object ... params) throws Exception {
this(range, cons.newInstance(params), cons.newInstance(useEmptyStopWordSet(params)));
}
@@ -409,38 +502,52 @@
}
return rslt;
}
+
@Override
+ public String toString() {
+ return range.full + "=(" + withStopWords.getClass().getSimpleName() +")";
+ }
+
+ @Override
public int compareTo(AnalyzerPair o) {
return range.compareTo(o.range);
}
-
- public boolean extendedFilterMatch(String[] language) {
- return range.extendedFilterMatch(language);
- }
}
+ /**
+ * Used for Analyzer classes with a constructor with signature (Version, Set).
+ * @author jeremycarroll
+ *
+ */
private static class VersionSetAnalyzerPair extends AnalyzerPair {
public VersionSetAnalyzerPair(ConfigOptionsToAnalyzer lro,
Class<? extends Analyzer> cls) throws Exception {
super(lro.languageRange, getConstructor(cls, Version.class, Set.class), Version.LUCENE_CURRENT, lro.getStopWords());
}
}
-
+
+ /**
+ * Used for Analyzer classes which do not support stopwords and have a constructor with signature (Version).
+ * @author jeremycarroll
+ *
+ */
private static class VersionAnalyzerPair extends AnalyzerPair {
-
public VersionAnalyzerPair(String range, Class<? extends Analyzer> cls) throws Exception {
super(range, getConstructor(cls, Version.class).newInstance(Version.LUCENE_CURRENT));
}
}
-
+ /**
+ * Special case code for {@link PatternAnalyzer}
+ * @author jeremycarroll
+ *
+ */
private static class PatternAnalyzerPair extends AnalyzerPair {
-
- public PatternAnalyzerPair(ConfigOptionsToAnalyzer lro, String pattern) throws Exception {
+ public PatternAnalyzerPair(ConfigOptionsToAnalyzer lro, Pattern pattern) throws Exception {
super(lro.languageRange, getConstructor(PatternAnalyzer.class,Version.class,Pattern.class,Boolean.TYPE,Set.class),
Version.LUCENE_CURRENT,
- Pattern.compile(pattern, Pattern.UNICODE_CHARACTER_CLASS),
+ pattern,
true,
lro.getStopWords());
}
@@ -451,6 +558,16 @@
* This class is initialized with the config options, using the {@link #setProperty(String, String)}
* method, for a particular language range and works out which pair of {@link Analyzer}s
* to use for that language range.
+ * <p>
+ * Instances of this class are only alive during the execution of
+ * {@link ConfigurableAnalyzerFactory#ConfigurableAnalyzerFactory(FullTextIndex)},
+ * the life-cycle is:
+ * <ol>
+ * <li>The relveant config properties are applied, and are used to populate the fields.
+ * <li>The fields are validated
+ * <li>An {@link AnalyzerPair} is constructed
+ * </ol>
+ *
* @author jeremycarroll
*
*/
@@ -459,9 +576,13 @@
String like;
String className;
String stopwords;
- String pattern;
+ Pattern pattern;
final String languageRange;
AnalyzerPair result;
+ Pattern wordBoundary;
+ Pattern subWordBoundary;
+ Pattern softHyphens;
+ Boolean alwaysRemoveSoftHyphens;
public ConfigOptionsToAnalyzer(String languageRange) {
this.languageRange = languageRange;
@@ -474,7 +595,7 @@
*/
public Set<?> getStopWords() {
- if (AnalyzerOptions.STOPWORDS_VALUE_NONE.equals(stopwords))
+ if (doNotUseStopWords())
return Collections.EMPTY_SET;
if (useDefaultStopWords()) {
@@ -484,6 +605,10 @@
return getStopWordsForClass(stopwords);
}
+ boolean doNotUseStopWords() {
+ return AnalyzerOptions.STOPWORDS_VALUE_NONE.equals(stopwords) || (stopwords == null && pattern != null);
+ }
+
protected Set<?> getStopWordsForClass(String clazzName) {
Class<? extends Analyzer> analyzerClass = getAnalyzerClass(clazzName);
try {
@@ -500,9 +625,13 @@
}
protected boolean useDefaultStopWords() {
- return stopwords == null || AnalyzerOptions.STOPWORDS_VALUE_DEFAULT.equals(stopwords);
+ return ( stopwords == null && pattern == null ) || AnalyzerOptions.STOPWORDS_VALUE_DEFAULT.equals(stopwords);
}
+ /**
+ * The first step in the life-cycle, used to initialize the fields.
+ * @return true if the property was recognized.
+ */
public boolean setProperty(String shortProperty, String value) {
if (shortProperty.equals(AnalyzerOptions.LIKE) ) {
like = value;
@@ -511,13 +640,24 @@
} else if (shortProperty.equals(AnalyzerOptions.STOPWORDS) ) {
stopwords = value;
} else if (shortProperty.equals(AnalyzerOptions.PATTERN) ) {
- pattern = value;
+ pattern = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.WORD_BOUNDARY) ) {
+ wordBoundary = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.SUB_WORD_BOUNDARY) ) {
+ subWordBoundary = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.SOFT_HYPHENS) ) {
+ softHyphens = Pattern.compile(value,Pattern.UNICODE_CHARACTER_CLASS);
+ } else if (shortProperty.equals(AnalyzerOptions.ALWAYS_REMOVE_SOFT_HYPHENS) ) {
+ alwaysRemoveSoftHyphens = Boolean.valueOf(value);
} else {
return false;
}
return true;
}
+ /**
+ * The second phase of the life-cycle, used for sanity checking.
+ */
public void validate() {
if (pattern != null ) {
if ( className != null && className != PatternAnalyzer.class.getName()) {
@@ -525,6 +665,27 @@
}
className = PatternAnalyzer.class.getName();
}
+ if (this.wordBoundary != null ) {
+ if ( className != null && className != TermCompletionAnalyzer.class.getName()) {
+ throw new RuntimeException("Bad Option: Language range "+languageRange + " with pattern propety for class "+ className);
+ }
+ className = TermCompletionAnalyzer.class.getName();
+
+ if ( subWordBoundary == null ) {
+ subWordBoundary = AnalyzerOptions.DEFAULT_SUB_WORD_BOUNDARY;
+ }
+ if ( alwaysRemoveSoftHyphens != null && softHyphens == null ) {
+ throw new RuntimeException("Bad option: Language range "+languageRange + ": must specify softHypens when setting alwaysRemoveSoftHyphens");
+ }
+ if (softHyphens != null && alwaysRemoveSoftHyphens == null) {
+ alwaysRemoveSoftHyphens = AnalyzerOptions.DEFAULT_ALWAYS_REMOVE_SOFT_HYPHENS;
+ }
+
+ } else if ( subWordBoundary != null || softHyphens != null || alwaysRemoveSoftHyphens != null ||
+ TermCompletionAnalyzer.class.getName().equals(className) ) {
+ throw new RuntimeException("Bad option: Language range "+languageRange + ": must specify wordBoundary for TermCompletionAnalyzer");
+ }
+
if (PatternAnalyzer.class.getName().equals(className) && pattern == null ) {
throw new RuntimeException("Bad Option: Language range "+languageRange + " must specify pattern for PatternAnalyzer.");
}
@@ -537,21 +698,45 @@
}
+ /**
+ * The third and final phase of the life-cyle used for identifying
+ * the AnalyzerPair.
+ */
private AnalyzerPair construct() throws Exception {
if (className == null) {
return null;
}
if (pattern != null) {
return new PatternAnalyzerPair(this, pattern);
-
- }
+ }
+ if (softHyphens != null) {
+ return new AnalyzerPair(
+ languageRange,
+ new TermCompletionAnalyzer(
+ wordBoundary,
+ subWordBoundary,
+ softHyphens,
+ alwaysRemoveSoftHyphens));
+ }
+ if (wordBoundary != null) {
+ return new AnalyzerPair(
+ languageRange,
+ new TermCompletionAnalyzer(
+ wordBoundary,
+ subWordBoundary));
+ }
final Class<? extends Analyzer> cls = getAnalyzerClass();
if (hasConstructor(cls, Version.class, Set.class)) {
// RussianAnalyzer is missing any way to access stop words.
- if (RussianAnalyzer.class.equals(cls) && useDefaultStopWords()) {
- return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT), new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
+ if (RussianAnalyzer.class.equals(cls)) {
+ if (useDefaultStopWords()) {
+ return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT), new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
+ }
+ if (doNotUseStopWords()) {
+ return new AnalyzerPair(languageRange, new RussianAnalyzer(Version.LUCENE_CURRENT, Collections.EMPTY_SET));
+ }
}
return new VersionSetAnalyzerPair(this, cls);
}
@@ -569,6 +754,29 @@
throw new RuntimeException("Bad option: cannot find constructor for class " + className + " for language range " + languageRange);
}
+ /**
+ * Also part of the third phase of the life-cycle, following the {@link AnalyzerOptions#LIKE}
+ * properties.
+ * @param depth
+ * @param max
+ * @param analyzers
+ * @return
+ */
+ AnalyzerPair followLikesToAnalyzerPair(int depth, int max,
+ Map<String, ConfigOptionsToAnalyzer> analyzers) {
+ if (result == null) {
+ if (depth == max) {
+ throw new RuntimeException("Bad configuration: - 'like' loop for language range " + languageRange);
+ }
+ ConfigOptionsToAnalyzer next = analyzers.get(like);
+ if (next == null) {
+ throw new RuntimeException("Bad option: - 'like' not found for language range " + languageRange+ " (not found: '"+ like +"')");
+ }
+ result = new AnalyzerPair(languageRange, next.followLikesToAnalyzerPair(depth+1, max, analyzers));
+ }
+ return result;
+ }
+
protected Class<? extends Analyzer> getAnalyzerClass() {
return getAnalyzerClass(className);
}
@@ -587,22 +795,6 @@
void setAnalyzerPair(AnalyzerPair ap) {
result = ap;
}
-
- AnalyzerPair followLikesToAnalyzerPair(int depth, int max,
- Map<String, ConfigOptionsToAnalyzer> analyzers) {
- if (result == null) {
- if (depth == max) {
- throw new RuntimeException("Bad configuration: - 'like' loop for language range " + languageRange);
- }
- ConfigOptionsToAnalyzer next = analyzers.get(like);
- if (next == null) {
- throw new RuntimeException("Bad option: - 'like' not found for language range " + languageRange+ " (not found: '"+ like +"')");
- }
- result = new AnalyzerPair(languageRange, next.followLikesToAnalyzerPair(depth+1, max, analyzers));
- }
- return result;
- }
-
}
private final AnalyzerPair config[];
@@ -615,12 +807,19 @@
* strategy so the code will still work on the {@link #MAX_LANG_CACHE_SIZE}+1 th entry.
*/
private static final int MAX_LANG_CACHE_SIZE = 500;
+
private String defaultLanguage;
private final FullTextIndex<?> fullTextIndex;
+ /**
+ * Builds a new ConfigurableAnalyzerFactory.
+ * @param fullTextIndex
+ */
public ConfigurableAnalyzerFactory(final FullTextIndex<?> fullTextIndex) {
+ // A description of the operation of this method is found on AnalyzerPair and
+ // ConfigOptionsToAnalyzer.
// despite our name, we actually make all the analyzers now, and getAnalyzer method is merely a lookup.
if (fullTextIndex == null)
@@ -717,9 +916,9 @@
while (en.hasMoreElements()) {
String prop = (String)en.nextElement();
- if (prop.equals(Options.INCLUDE_DEFAULTS)) continue;
+ if (prop.equals(Options.NATURAL_LANGUAGE_SUPPORT)) continue;
if (prop.startsWith(Options.ANALYZER)) {
- String languageRangeAndProperty[] = prop.substring(Options.ANALYZER.length()).split("[.]");
+ String languageRangeAndProperty[] = prop.substring(Options.ANALYZER.length()).replaceAll("_","*").split("[.]");
if (languageRangeAndProperty.length == 2) {
String languageRange = languageRangeAndProperty[0].toLowerCase(Locale.US); // Turkish "I" could create a problem
@@ -745,25 +944,29 @@
protected Properties initProperties() {
final Properties parentProperties = fullTextIndex.getProperties();
Properties myProps;
- if (Boolean.getBoolean(parentProperties.getProperty(Options.INCLUDE_DEFAULTS, Options.DEFAULT_INCLUDE_DEFAULTS))) {
- myProps = defaultProperties();
+ if (Boolean.valueOf(parentProperties.getProperty(
+ Options.NATURAL_LANGUAGE_SUPPORT,
+ Options.DEFAULT_NATURAL_LANGUAGE_SUPPORT))) {
+
+ myProps = loadPropertyString(ALL_LUCENE_NATURAL_LANGUAGES);
+
+ } else if (hasPropertiesForStarLanguageRange(parentProperties)){
+
+ myProps = new Properties();
+
} else {
- myProps = new Properties();
+
+ myProps = loadPropertyString(LUCENE_STANDARD_ANALYZER);
}
copyRelevantProperties(fullTextIndex.getProperties(), myProps);
-
- if (myProps.isEmpty()) {
- return defaultProperties();
- } else {
- return myProps;
- }
+ return myProps;
}
- protected Properties defaultProperties() {
+ Properties loadPropertyString(String props) {
Properties rslt = new Properties();
try {
- rslt.load(new StringReader(DEFAULT_PROPERTIES));
+ rslt.load(new StringReader(props));
} catch (IOException e) {
throw new RuntimeException("Impossible - well clearly not!", e);
}
@@ -780,6 +983,17 @@
}
}
+ private boolean hasPropertiesForStarLanguageRange(Properties from) {
+ Enumeration<?> en = from.propertyNames();
+ while (en.hasMoreElements()) {
+ String prop = (String)en.nextElement();
+ if (prop.startsWith(Options.ANALYZER+"_.")
+ || prop.startsWith(Options.ANALYZER+"*.")) {
+ return true;
+ }
+ }
+ return false;
+ }
@Override
public Analyzer getAnalyzer(String languageCode, boolean filterStopwords) {
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/DefaultAnalyzerFactory.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/DefaultAnalyzerFactory.java 2014-05-10 02:56:35 UTC (rev 8262)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/DefaultAnalyzerFactory.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -29,7 +29,6 @@
import java.util.Collections;
import java.util.HashMap;
-import java.util.HashSet;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
@@ -52,11 +51,21 @@
import com.bigdata.btree.keys.KeyBuilder;
/**
- * Default implementation registers a bunch of {@link Analyzer}s for various
- * language codes and then serves the appropriate {@link Analyzer} based on
- * the specified language code.
+ * This is the default implementation but should be regarded as legacy since
+ * it fails to use the correct {@link Analyzer} for almost all languages (other than
+ * English). It uses the correct natural language analyzer only for literals tagged with
+ * certain three letter ISO 639 codes:
+ * "por", "deu", "ger", "zho", "chi", "jpn", "kor", "ces", "cze", "dut", "nld", "gre", "ell",
+ * "fra", "fre", "rus" and "tha". All other tags are treated as English.
+ * These codes do not work if they are used with subtags, e.g. "ger-AT" is treated as English.
+ * No two letter code, other than "en" works correctly: note that the W3C and
+ * IETF recommend the use of the two letter forms instead of the three letter forms.
*
* @author <a href="mailto:tho...@us...">Bryan Thompson</a>
+ * @deprecated Using {@link ConfigurableAnalyzerFactory} with
+ * the {@link ConfigurableAnalyzerFactory.Options#NATURAL_LANGUAGE_SUPPORT}
+ * uses the appropriate natural language analyzers for the two letter codes
+ * and for tags which include sub-tags.
* @version $Id$
*/
public class DefaultAnalyzerFactory implements IAnalyzerFactory {
Added: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/TermCompletionAnalyzer.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/TermCompletionAnalyzer.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/java/com/bigdata/search/TermCompletionAnalyzer.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -0,0 +1,248 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+/*
+ * Created on May 8, 2014 by Jeremy J. Carroll, Syapse Inc.
+ */
+package com.bigdata.search;
+
+import java.io.IOException;
+import java.io.Reader;
+import java.io.StringReader;
+import java.nio.CharBuffer;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.lucene.analysis.Analyzer;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.KeywordAnalyzer;
+import org.apache.lucene.analysis.tokenattributes.TermAttribute;
+
+
+/**
+ * An analyzer intended for the term-completion use case; particularly
+ * for technical vocabularies and concept schemes.
+ *
+ * <p>
+ * This analyzer generates several index terms for each word in the input.
+ * These are intended to match short sequences (e.g. three or more) characters
+ * of user-input, to then give the user a drop-down list of matching terms.
+ * <p>
+ * This can be set up to address issues like matching <q>half-time</q> when the user types
+ * <q>tim</q> or if the user types <q>halft</q> (treating the hyphen as a soft hyphen); or
+ * to match <q>TermCompletionAnalyzer</q> when the user types <q>Ana</q>
+ * <p>
+ * In contrast, the Lucene Analyzers are mainly geared around the free text search use
+ * case.
+ * <p>
+ * The intended use cases will typical involve a prefix query of the form:
+ * <pre>
+ * ?t bds:search "prefix*" .
+ * </pre>
+ * to find all literals in the selected graphs, which are indexed by a term starting in <q>prefix</q>,
+ * so the problem this class addresses is finding the appropriate index terms to allow
+ * matching, at sensible points, mid-way through words (such as at hyphens).
+ * <p>
+ * To get maximum effectiveness it maybe best to use private language subtags (see RFC 5647),
+ * e.g. <code>"x-term"</code>
+ * which are mapped to this class by {@link ConfigurableAnalyzerFactory} for
+ * the data being loaded into the store, and linked to some very simple process
+ * like {@link KeywordAnalyzer} for queries which are tagged with a different language tag
+ * that is only used for <code>bds:search</code>, e.g. <code>"x-query"</code>.
+ * The above prefix query then becomes:
+ * <pre>
+ * ?t bds:search "prefix*"@x-query .
+ * </pre>
+ *
+ *
+ *
+ * @author jeremycarroll
+ *
+ */
+public class TermCompletionAnalyzer extends Analyzer {
+
+ private final Pattern wordBoundary;
+ private final Pattern subWordBoundary;
+
+ private final Pattern discard;
+ private final boolean alwaysDiscard;
+
+ /**
+ * Divide the input into words and short tokens
+ * as with {@link #TermCompletionAnalyzer(Pattern, Pattern)}.
+ * Each term is generated, and then an additional term
+ * is generated with softHypens (defined by the pattern),
+ * removed. If the alwaysRemoveSoftHypens flag is true,
+ * then the first term (before the removal) is suppressed.
+ *
+ * @param wordBoundary The definition of space (e.g. " ")
+ * @param subWordBoundary Also index after matches to this (e.g. "-")
+ * @param softHyphens Discard these characters from matches
+ * @param alwaysRemoveSoftHypens If false the discard step is optional.
+ */
+ public TermCompletionAnalyzer(Pattern wordBoundary,
+ Pattern subWordBoundary,
+ Pattern softHyphens,
+ boolean alwaysRemoveSoftHypens) {
+ this.wordBoundary = wordBoundary;
+ this.subWordBoundary = subWordBoundary;
+ if (softHyphens != null) {
+ discard = softHyphens;
+ alwaysDiscard = alwaysRemoveSoftHypens;
+ } else {
+ discard = Pattern.compile("(?!)"); // never matches
+ alwaysDiscard = true;
+ }
+ }
+ /**
+ * Divide the input into words, separated by the wordBoundary,
+ * and return a token for each whole word, and then
+ * generate further tokens for each word by removing prefixes
+ * up to and including each successive match of
+ * subWordBoundary
+ * @param wordBoundary
+ * @param subWordBoundary
+ */
+ public TermCompletionAnalyzer(Pattern wordBoundary,
+ Pattern subWordBoundary) {
+ this(wordBoundary, subWordBoundary, null, true);
+ }
+
+
+ @Override
+ public TokenStream tokenStream(String ignoredFieldName, Reader reader) {
+ return new TermCompletionTokenStream((StringReader)reader);
+ }
+
+ /**
+ * This classes has three processes going on
+ * all driven from the {@link #increment()} method.
+ *
+ * One process is that of iterating over the words in the input:
+ * - the words are identified in the constructor, and the iteration
+ * is performed by {@link #nextWord()}
+ *
+ * - the subword boundaries are identified in {@link #next()}
+ * We then set up {@link #found} to contain the most
+ * recently found subword.
+ *
+ * - the soft hyphen discarding is processed in {@link #maybeDiscardHyphens()}
+ *
+ * - if we are not {@link #alwaysDiscard}ing then {@link #afterDiscard}
+ * can be set to null to return the non-discarded version on the next cycle.
+ *
+ */
+ private class TermCompletionTokenStream extends TokenStream {
+
+ final String[] words;
+ final TermAttribute termAtt;
+
+
+
+ char currentWord[] = new char[]{};
+ Matcher softMatcher;
+ int currentWordIx = -1;
+
+
+ int charPos = 0;
+ private String afterDiscard;
+ private CharBuffer found;
+
+ public TermCompletionTokenStream(StringReader reader) {
+ termAtt = addAttribute(TermAttribute.class);
+ try {
+ reader.mark(Integer.MAX_VALUE);
+ int length = (int) reader.skip(Integer.MAX_VALUE);
+ reader.reset();
+ char fileContent[] = new char[length];
+ reader.read(fileContent);
+ words = wordBoundary.split(new String(fileContent));
+ } catch (IOException e) {
+ throw new RuntimeException("Impossible",e);
+ }
+ }
+
+ @Override
+ public boolean incrementToken() throws IOException {
+ if ( next() ) {
+ if (afterDiscard != null) {
+ int lg = afterDiscard.length();
+ afterDiscard.getChars(0, lg, termAtt.termBuffer(), 0);
+ termAtt.setTermLength(lg);
+ } else {
+ int lg = found.length();
+ found.get(termAtt.termBuffer(), 0, lg);
+ termAtt.setTermLength(lg);
+ }
+ return true;
+ } else {
+ return false;
+ }
+ }
+
+ private boolean next() {
+ if (currentWordIx >= words.length) {
+ return false;
+ }
+ if (!alwaysDiscard) {
+ // Last match was the discarded version,
+ // now do the non-discard version.
+ if (afterDiscard != null) {
+ afterDiscard = null;
+ return true;
+ }
+ }
+ afterDiscard = null;
+ if (charPos + 1 < currentWord.length && softMatcher.find(charPos+1)) {
+ charPos = softMatcher.end();
+ maybeDiscardHyphens();
+ return true;
+ } else {
+ return nextWord();
+ }
+ }
+
+ void maybeDiscardHyphens() {
+ found = CharBuffer.wrap(currentWord, charPos, currentWord.length - charPos);
+ Matcher discarding = discard.matcher(found);
+ if (discarding.find()) {
+ afterDiscard = discarding.replaceAll("");
+ }
+ }
+
+ private boolean nextWord() {
+ currentWordIx++;
+ if (currentWordIx >= words.length) {
+ return false;
+ }
+ currentWord = words[currentWordIx].toCharArray();
+ termAtt.resizeTermBuffer(currentWord.length);
+ charPos = 0;
+ softMatcher = subWordBoundary.matcher(words[currentWordIx]);
+ maybeDiscardHyphens();
+ return true;
+ }
+
+ }
+
+}
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractAnalyzerFactoryTest.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractAnalyzerFactoryTest.java 2014-05-10 02:56:35 UTC (rev 8262)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractAnalyzerFactoryTest.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -22,151 +22,25 @@
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
/*
- * Created on May 7, 2014
+ * Created on May 9, 2014
*/
package com.bigdata.search;
-import java.io.IOException;
-import java.io.StringReader;
-
-import org.apache.lucene.analysis.Analyzer;
-import org.apache.lucene.analysis.TokenStream;
-import org.apache.lucene.analysis.tokenattributes.TermAttribute;
-
public abstract class AbstractAnalyzerFactoryTest extends AbstractSearchTest {
- public AbstractAnalyzerFactoryTest() {
+ public AbstractAnalyzerFactoryTest() {
}
-
- public AbstractAnalyzerFactoryTest(String arg0) {
- super(arg0);
+
+ public AbstractAnalyzerFactoryTest(String arg0) {
+ super(arg0);
}
-
- public void setUp() throws Exception {
- super.setUp();
- init(getExtraProperties());
- }
- abstract String[] getExtraProperties();
-
- private Analyzer getAnalyzer(String lang, boolean filterStopWords) {
- return getNdx().getAnalyzer(lang, filterStopWords);
+
+ @Override
+ public void setUp() throws Exception {
+ super.setUp();
+ init(getExtraProperties());
}
-
- private void comparisonTest(String lang,
- boolean stopWordsSignificant,
- String text,
- String spaceSeparated) throws IOException {
- compareTokenStream(getAnalyzer(lang, stopWordsSignificant), text,
- spaceSeparated.split(" ")); //$NON-NLS-1$
- }
- private void compareTokenStream(Analyzer a, String text, String expected[]) throws IOException {
- TokenStream s = a.tokenStream(null, new StringReader(text));
- int ix = 0;
- while (s.incrementToken()) {
- final TermAttribute term = s.getAttribute(TermAttribute.class);
- final String word = term.term();
- assertTrue(ix < expected.length);
- assertEquals(word, expected[ix++]);
- }
- assertEquals(ix, expected.length);
- }
-
- public void testEnglishFilterStopWords() throws IOException {
- for (String lang: new String[]{ "eng", null, "" }) { //$NON-NLS-1$ //$NON-NLS-2$
- comparisonTest(lang,
- true,
- "The test to end all tests! Forever.", //$NON-NLS-1$
- "test end all tests forever" //$NON-NLS-1$
- );
- }
- }
- public void testEnglishNoFilter() throws IOException {
- for (String lang: new String[]{ "eng", null, "" }) { //$NON-NLS-1$ //$NON-NLS-2$
- comparisonTest(lang,
- false,
- "The test to end all tests! Forever.", //$NON-NLS-1$
- "the test to end all tests forever" //$NON-NLS-1$
- );
- }
- }
-
- // Note we careful use a three letter language code for german.
- // 'de' is more standard, but the DefaultAnalyzerFactory does not
- // implement 'de' correctly.
- public void testGermanFilterStopWords() throws IOException {
- comparisonTest("ger", //$NON-NLS-1$
- true,
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.10") + //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.11"), //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.12") //$NON-NLS-1$
- );
-
- }
+ abstract String[] getExtraProperties();
- // Note we careful use a three letter language code for Russian.
- // 'ru' is more standard, but the DefaultAnalyzerFactory does not
- // implement 'ru' correctly.
- public void testRussianFilterStopWords() throws IOException {
- comparisonTest("rus", //$NON-NLS-1$
- true,
- // I hope this is not offensive text.
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.14") + //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.15"), //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.16") //$NON-NLS-1$
- );
-
- }
- public void testGermanNoStopWords() throws IOException {
- comparisonTest("ger", //$NON-NLS-1$
- false,
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.18") + //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.19"), //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.20") //$NON-NLS-1$
- );
-
- }
- public void testRussianNoStopWords() throws IOException {
- comparisonTest("rus", //$NON-NLS-1$
- false,
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.22") + //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.23"), //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.24") //$NON-NLS-1$
- );
-
- }
- public void testJapanese() throws IOException {
- for (boolean filterStopWords: new Boolean[]{true, false}) {
- comparisonTest("jpn", //$NON-NLS-1$
- filterStopWords,
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.26"), //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.27") + //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.28") + //$NON-NLS-1$
- NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.29")); //$NON-NLS-1$
- }
- }
- public void testConfiguredLanguages() {
- checkConfig("BrazilianAnalyzer", "por", "pt"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
- checkConfig("ChineseAnalyzer", "zho", "chi", "zh"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
- checkConfig("CJKAnalyzer", "jpn", "ja", "kor", "ko"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$
- checkConfig("CzechAnalyzer", "ces", "cze", "cs"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
- checkConfig("DutchAnalyzer", "dut", "nld", "nl"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
- checkConfig("GermanAnalyzer", "deu", "ger", "de"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
- checkConfig("GreekAnalyzer", "gre", "ell", "el"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
- checkConfig("RussianAnalyzer", "rus", "ru"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
- checkConfig("ThaiAnalyzer", "th", "tha"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
- checkConfig("StandardAnalyzer", "en", "eng", "", null); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
- }
-
- private void checkConfig(String classname, String ...langs) {
- for (String lang:langs) {
- // The DefaultAnalyzerFactory only works for language tags of length exactly three.
-// if (lang != null && lang.length()==3)
- {
- assertEquals(classname, getAnalyzer(lang,true).getClass().getSimpleName());
- assertEquals(classname, getAnalyzer(lang+NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.0"),true).getClass().getSimpleName()); //$NON-NLS-1$
- }
- }
-
- }
}
Copied: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractDefaultAnalyzerFactoryTest.java (from rev 8253, branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractAnalyzerFactoryTest.java)
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractDefaultAnalyzerFactoryTest.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractDefaultAnalyzerFactoryTest.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -0,0 +1,133 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+/*
+ * Created on May 7, 2014
+ */
+package com.bigdata.search;
+
+import java.io.IOException;
+
+
+public abstract class AbstractDefaultAnalyzerFactoryTest extends AbstractAnalyzerFactoryTest {
+
+ public AbstractDefaultAnalyzerFactoryTest() {
+ }
+
+ public AbstractDefaultAnalyzerFactoryTest(String arg0) {
+ super(arg0);
+ }
+
+ public void testEnglishFilterStopWords() throws IOException {
+ for (String lang: new String[]{ "eng", null, "" }) { //$NON-NLS-1$ //$NON-NLS-2$
+ comparisonTest(lang,
+ true,
+ "The test to end all tests! Forever.", //$NON-NLS-1$
+ "test end all tests forever" //$NON-NLS-1$
+ );
+ }
+ }
+ public void testEnglishNoFilter() throws IOException {
+ for (String lang: new String[]{ "eng", null, "" }) { //$NON-NLS-1$ //$NON-NLS-2$
+ comparisonTest(lang,
+ false,
+ "The test to end all tests! Forever.", //$NON-NLS-1$
+ "the test to end all tests forever" //$NON-NLS-1$
+ );
+ }
+ }
+
+ // Note we careful use a three letter language code for german.
+ // 'de' is more standard, but the DefaultAnalyzerFactory does not
+ // implement 'de' correctly.
+ public void testGermanFilterStopWords() throws IOException {
+ comparisonTest("ger", //$NON-NLS-1$
+ true,
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.10") + //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.11"), //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.12") //$NON-NLS-1$
+ );
+
+ }
+
+ // Note we careful use a three letter language code for Russian.
+ // 'ru' is more standard, but the DefaultAnalyzerFactory does not
+ // implement 'ru' correctly.
+ public void testRussianFilterStopWords() throws IOException {
+ comparisonTest("rus", //$NON-NLS-1$
+ true,
+ // I hope this is not offensive text.
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.14") + //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.15"), //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.16") //$NON-NLS-1$
+ );
+
+ }
+ public void testGermanNoStopWords() throws IOException {
+ comparisonTest("ger", //$NON-NLS-1$
+ false,
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.18") + //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.19"), //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.20") //$NON-NLS-1$
+ );
+
+ }
+ public void testRussianNoStopWords() throws IOException {
+ comparisonTest("rus", //$NON-NLS-1$
+ false,
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.22") + //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.23"), //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.24") //$NON-NLS-1$
+ );
+
+ }
+ public void testJapanese() throws IOException {
+ for (boolean filterStopWords: new Boolean[]{true, false}) {
+ comparisonTest("jpn", //$NON-NLS-1$
+ filterStopWords,
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.26"), //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.27") + //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.28") + //$NON-NLS-1$
+ NonEnglishExamples.getString("AbstractAnalyzerFactoryTest.29")); //$NON-NLS-1$
+ }
+ }
+ public void testConfiguredLanguages() {
+ checkConfig("BrazilianAnalyzer", "por", "pt"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
+ checkConfig("ChineseAnalyzer", "zho", "chi", "zh"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
+ checkConfig("CJKAnalyzer", "jpn", "ja", "kor", "ko"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$
+ checkConfig("CzechAnalyzer", "ces", "cze", "cs"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
+ checkConfig("DutchAnalyzer", "dut", "nld", "nl"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
+ checkConfig("GermanAnalyzer", "deu", "ger", "de"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
+ checkConfig("GreekAnalyzer", "gre", "ell", "el"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
+ checkConfig("RussianAnalyzer", "rus", "ru"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
+ checkConfig("ThaiAnalyzer", "th", "tha"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
+ checkConfig("StandardAnalyzer", "en", "eng", "", null); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
+ }
+
+ @Override
+ protected void checkConfig(String classname, String ...langs) {
+ checkConfig(isBroken(), classname, langs);
+
+ }
+ abstract boolean isBroken() ;
+}
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractSearchTest.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractSearchTest.java 2014-05-10 02:56:35 UTC (rev 8262)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/AbstractSearchTest.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -26,8 +26,14 @@
*/
package com.bigdata.search;
+import java.io.IOException;
+import java.io.StringReader;
import java.util.Properties;
+import org.apache.lucene.analysis.Analyzer;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.TermAttribute;
+
import com.bigdata.journal.IIndexManager;
import com.bigdata.journal.ITx;
import com.bigdata.journal.ProxyTestCase;
@@ -62,7 +68,7 @@
}
FullTextIndex<Long> createFullTextIndex(String namespace, String ...propertyValuePairs) {
- return createFullTextIndex(namespace, getProperties(), propertyValuePairs);
+ return createFullTextIndex(namespace, (Properties)getProperties().clone(), propertyValuePairs);
}
public void tearDown() throws Exception {
@@ -92,4 +98,65 @@
return properties;
}
+ protected Analyzer getAnalyzer(String lang, boolean filterStopWords) {
+ return getNdx().getAnalyzer(lang, filterStopWords);
+ }
+
+ protected void comparisonTest(String lang, boolean filterStopWords, String text, String spaceSeparated)
+ throws IOException {
+ if (spaceSeparated == null) {
+ String rslt = getTokenStream(getAnalyzer(lang, filterStopWords), text);
+ throw new RuntimeException("Got \"" + rslt+ "\"");
+ }
+ compareTokenStream(getAnalyzer(lang, filterStopWords), text,
+ split(spaceSeparated)); //$NON-NLS-1$
+ }
+
+ private String[] split(String spaceSeparated) {
+ if (spaceSeparated.length()==0) {
+ return new String[0];
+ }
+ return spaceSeparated.split(" ");
+ }
+
+ protected String getTokenStream(Analyzer a, String text) throws IOException {
+ StringBuffer sb = new StringBuffer();
+ TokenStream s = a.tokenStream(null, new StringReader(text));
+ while (s.incrementToken()) {
+ final TermAttribute term = s.getAttribute(TermAttribute.class);
+ if (sb.length()!=0) {
+ sb.append(" ");
+ }
+ sb.append(term.term());
+ }
+ return sb.toString();
+ }
+
+ private void compareTokenStream(Analyzer a, String text, String expected[]) throws IOException {
+ TokenStream s = a.tokenStream(null, new StringReader(text));
+ int ix = 0;
+ while (s.incrementToken()) {
+ final TermAttribute term = s.getAttribute(TermAttribute.class);
+ final String word = term.term();
+ assertTrue(ix < expected.length);
+ assertEquals(expected[ix++], word);
+ }
+ assertEquals(ix, expected.length);
+ }
+
+ protected void checkConfig(boolean threeLetterOnly, String classname, String ...langs) {
+ for (String lang:langs) {
+ // The DefaultAnalyzerFactory only works for language tags of length exactly three.
+ if ((!threeLetterOnly) || (lang != null && lang.length()==3)) {
+ assertEquals(classname, getAnalyzer(lang,true).getClass().getSimpleName());
+ if (!threeLetterOnly) {
+ assertEquals(classname, getAnalyzer(lang+"-x-foobar",true).getClass().getSimpleName()); //$NON-NLS-1$
+ }
+ }
+ }
+ }
+ protected void checkConfig(String classname, String ...langs) {
+ checkConfig(false, classname, langs);
+ }
+
}
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/NonEnglishExamples.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/NonEnglishExamples.java 2014-05-10 02:56:35 UTC (rev 8262)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/NonEnglishExamples.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -1,3 +1,29 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+/*
+ * Created on May 7, 2014 by Jeremy J. Carroll, Syapse Inc.
+ */
package com.bigdata.search;
import java.util.MissingResourceException;
Modified: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestAll.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestAll.java 2014-05-10 02:56:35 UTC (rev 8262)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestAll.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -114,6 +114,8 @@
// which is intended to be the same as the intended
// behavior of DefaultAnalyzerFactory
suite.addTestSuite(TestConfigurableAsDefaultAnalyzerFactory.class);
+ suite.addTestSuite(TestConfigurableAnalyzerFactory.class);
+ suite.addTestSuite(TestUnconfiguredAnalyzerFactory.class);
return suite;
}
Added: branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestConfigurableAnalyzerFactory.java
===================================================================
--- branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestConfigurableAnalyzerFactory.java (rev 0)
+++ branches/BIGDATA_RELEASE_1_3_0/bigdata/src/test/com/bigdata/search/TestConfigurableAnalyzerFactory.java 2014-05-10 11:52:25 UTC (rev 8263)
@@ -0,0 +1,244 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Fr...
[truncated message content] |
|
From: <mrp...@us...> - 2014-05-10 02:56:41
|
Revision: 8262
http://sourceforge.net/p/bigdata/code/8262
Author: mrpersonick
Date: 2014-05-10 02:56:35 +0000 (Sat, 10 May 2014)
Log Message:
-----------
2. Add in bigdata/blueprints source code in bigdata-sails/src/java
Added Paths:
-----------
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataPredicate.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataRDFFactory.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataVertex.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BlueprintsRDFFactory.java
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/edge.rq
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/edges.rq
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/edgesByProperty.rq
branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/vertex.rq
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,107 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import java.util.Arrays;
+import java.util.List;
+
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.model.vocabulary.RDFS;
+
+import com.tinkerpop.blueprints.Direction;
+import com.tinkerpop.blueprints.Edge;
+import com.tinkerpop.blueprints.Vertex;
+
+/**
+ * Edge implementation that wraps an Edge statement and points to a
+ * {@link BigdataGraph} instance.
+ *
+ * @author mikepersonick
+ *
+ */
+public class BigdataEdge extends BigdataElement implements Edge {
+
+ private static final List<String> blacklist = Arrays.asList(new String[] {
+ "id", "", "label"
+ });
+
+ protected final Statement stmt;
+
+ public BigdataEdge(final Statement stmt, final BigdataGraph graph) {
+ super(stmt.getPredicate(), graph);
+
+ this.stmt = stmt;
+ }
+
+ @Override
+ public Object getId() {
+ return graph.factory.fromEdgeURI(uri);
+ }
+
+ @Override
+ public void remove() {
+ graph.removeEdge(this);
+ }
+
+ @Override
+ public String getLabel() {
+ return (String) graph.getProperty(uri, RDFS.LABEL);
+ }
+
+ @Override
+ public Vertex getVertex(final Direction dir) throws IllegalArgumentException {
+
+ if (dir == Direction.BOTH) {
+ throw new IllegalArgumentException();
+ }
+
+ final URI uri = (URI)
+ (dir == Direction.OUT ? stmt.getSubject() : stmt.getObject());
+
+ final String id = graph.factory.fromVertexURI(uri);
+
+ return graph.getVertex(id);
+
+ }
+
+ @Override
+ public void setProperty(final String property, final Object val) {
+
+ if (property == null || blacklist.contains(property)) {
+ throw new IllegalArgumentException();
+ }
+
+ super.setProperty(property, val);
+
+ }
+
+ @Override
+ public String toString() {
+ final URI s = (URI) stmt.getSubject();
+ final URI p = (URI) stmt.getPredicate();
+ final URI o = (URI) stmt.getObject();
+ return "e["+p.getLocalName()+"]["+s.getLocalName()+"->"+o.getLocalName()+"]";
+ }
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataEdge.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,134 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Set;
+
+import org.openrdf.model.Literal;
+import org.openrdf.model.URI;
+
+import com.tinkerpop.blueprints.Element;
+
+/**
+ * Base class for {@link BigdataVertex} and {@link BigdataEdge}. Handles
+ * property-related methods.
+ *
+ * @author mikepersonick
+ *
+ */
+public abstract class BigdataElement implements Element {
+
+ private static final List<String> blacklist = Arrays.asList(new String[] {
+ "id", ""
+ });
+
+ protected final URI uri;
+ protected final BigdataGraph graph;
+
+ public BigdataElement(final URI uri, final BigdataGraph graph) {
+ this.uri = uri;
+ this.graph = graph;
+ }
+
+ @Override
+ @SuppressWarnings("unchecked")
+ public <T> T getProperty(final String property) {
+
+ final URI p = graph.factory.toPropertyURI(property);
+
+ return (T) graph.getProperty(uri, p);
+
+ }
+
+ @Override
+ public Set<String> getPropertyKeys() {
+
+ return graph.getPropertyKeys(uri);
+
+ }
+
+ @Override
+ @SuppressWarnings("unchecked")
+ public <T> T removeProperty(final String property) {
+
+ final URI p = graph.factory.toPropertyURI(property);
+
+ return (T) graph.removeProperty(uri, p);
+
+ }
+
+ @Override
+ public void setProperty(final String property, final Object val) {
+
+ if (property == null || blacklist.contains(property)) {
+ throw new IllegalArgumentException();
+ }
+
+ final URI p = graph.factory.toPropertyURI(property);
+
+ final Literal o = graph.factory.toLiteral(val);
+
+ graph.setProperty(uri, p, o);
+
+ }
+
+ @Override
+ public int hashCode() {
+ final int prime = 31;
+ int result = 1;
+ result = prime * result + ((graph == null) ? 0 : graph.hashCode());
+ result = prime * result + ((uri == null) ? 0 : uri.hashCode());
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (this == obj)
+ return true;
+ if (obj == null)
+ return false;
+ if (getClass() != obj.getClass())
+ return false;
+ BigdataElement other = (BigdataElement) obj;
+ if (graph == null) {
+ if (other.graph != null)
+ return false;
+ } else if (!graph.equals(other.graph))
+ return false;
+ if (uri == null) {
+ if (other.uri != null)
+ return false;
+ } else if (!uri.equals(other.uri))
+ return false;
+ return true;
+ }
+
+ @Override
+ public String toString() {
+ return uri.toString();
+ }
+
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataElement.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,854 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import info.aduna.iteration.CloseableIteration;
+
+import java.util.Iterator;
+import java.util.LinkedHashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Set;
+import java.util.UUID;
+
+import org.apache.commons.io.IOUtils;
+import org.openrdf.OpenRDFException;
+import org.openrdf.model.Literal;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.model.Value;
+import org.openrdf.model.impl.StatementImpl;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.model.vocabulary.RDF;
+import org.openrdf.model.vocabulary.RDFS;
+import org.openrdf.query.GraphQueryResult;
+import org.openrdf.query.QueryLanguage;
+import org.openrdf.repository.RepositoryConnection;
+import org.openrdf.repository.RepositoryResult;
+
+import com.bigdata.rdf.store.BD;
+import com.tinkerpop.blueprints.Direction;
+import com.tinkerpop.blueprints.Edge;
+import com.tinkerpop.blueprints.Features;
+import com.tinkerpop.blueprints.Graph;
+import com.tinkerpop.blueprints.GraphQuery;
+import com.tinkerpop.blueprints.Vertex;
+import com.tinkerpop.blueprints.util.DefaultGraphQuery;
+import com.tinkerpop.blueprints.util.io.graphml.GraphMLReader;
+
+/**
+ * A base class for a Blueprints wrapper around a bigdata back-end.
+ *
+ * @author mikepersonick
+ *
+ */
+public abstract class BigdataGraph implements Graph {
+
+ public static final URI VERTEX = new URIImpl(BD.NAMESPACE + "Vertex");
+
+ public static final URI EDGE = new URIImpl(BD.NAMESPACE + "Edge");
+
+// final BigdataSailRepository repo;
+//
+// transient BigdataSailRepositoryConnection cxn;
+
+ final BlueprintsRDFFactory factory;
+
+// public BigdataGraph(final BigdataSailRepository repo) {
+// this(repo, BigdataRDFFactory.INSTANCE);
+// }
+
+ public BigdataGraph(//final BigdataSailRepository repo,
+ final BlueprintsRDFFactory factory) {
+// try {
+// this.repo = repo;
+// this.cxn = repo.getUnisolatedConnection();
+// this.cxn.setAutoCommit(false);
+ this.factory = factory;
+// } catch (RepositoryException ex) {
+// throw new RuntimeException(ex);
+// }
+ }
+
+ public String toString() {
+ return getClass().getSimpleName().toLowerCase();
+ }
+
+ /**
+ * Post a GraphML file to the remote server. (Bulk-upload operation.)
+ */
+ public void loadGraphML(final String file) throws Exception {
+ GraphMLReader.inputGraph(this, file);
+ }
+
+ protected abstract RepositoryConnection cxn() throws Exception;
+
+// public BigdataSailRepositoryConnection getConnection() {
+// return this.cxn;
+// }
+//
+// public BlueprintsRDFFactory getFactory() {
+// return this.factory;
+// }
+
+// public Value getValue(final URI s, final URI p) {
+//
+// try {
+//
+// final RepositoryResult<Statement> result =
+// cxn.getStatements(s, p, null, false);
+//
+// if (result.hasNext()) {
+//
+// final Value o = result.next().getObject();
+//
+// if (result.hasNext()) {
+// throw new RuntimeException(s
+// + ": more than one value for p: " + p
+// + ", did you mean to call getValues()?");
+// }
+//
+// return o;
+//
+// }
+//
+// return null;
+//
+// } catch (Exception ex) {
+// throw new RuntimeException(ex);
+// }
+//
+// }
+
+ public Object getProperty(final URI s, final URI p) {
+
+ try {
+
+ final RepositoryResult<Statement> result =
+ cxn().getStatements(s, p, null, false);
+
+ if (result.hasNext()) {
+
+ final Value value = result.next().getObject();
+
+ if (result.hasNext()) {
+ throw new RuntimeException(s
+ + ": more than one value for p: " + p
+ + ", did you mean to call getValues()?");
+ }
+
+ if (!(value instanceof Literal)) {
+ throw new RuntimeException("not a property: " + value);
+ }
+
+ final Literal lit = (Literal) value;
+
+ return factory.fromLiteral(lit);
+
+ }
+
+ return null;
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+// public List<Value> getValues(final URI s, final URI p) {
+//
+// try {
+//
+// final RepositoryResult<Statement> result =
+// cxn().getStatements(s, p, null, false);
+//
+// final List<Value> values = new LinkedList<Value>();
+//
+// while (result.hasNext()) {
+//
+// final Value o = result.next().getObject();
+//
+// values.add(o);
+//
+// }
+//
+// return values;
+//
+// } catch (Exception ex) {
+// throw new RuntimeException(ex);
+// }
+//
+// }
+
+ public List<Object> getProperties(final URI s, final URI p) {
+
+ try {
+
+ final RepositoryResult<Statement> result =
+ cxn().getStatements(s, p, null, false);
+
+ final List<Object> props = new LinkedList<Object>();
+
+ while (result.hasNext()) {
+
+ final Value value = result.next().getObject();
+
+ if (!(value instanceof Literal)) {
+ throw new RuntimeException("not a property: " + value);
+ }
+
+ final Literal lit = (Literal) value;
+
+ props.add(factory.fromLiteral(lit));
+
+ }
+
+ return props;
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ public Set<String> getPropertyKeys(final URI s) {
+
+ try {
+
+ final RepositoryResult<Statement> result =
+ cxn().getStatements(s, null, null, false);
+
+ final Set<String> properties = new LinkedHashSet<String>();
+
+ while (result.hasNext()) {
+
+ final Statement stmt = result.next();
+
+ if (!(stmt.getObject() instanceof Literal)) {
+ continue;
+ }
+
+ if (stmt.getPredicate().equals(RDFS.LABEL)) {
+ continue;
+ }
+
+ final String p =
+ factory.fromPropertyURI(stmt.getPredicate());
+
+ properties.add(p);
+
+ }
+
+ return properties;
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ public Object removeProperty(final URI s, final URI p) {
+
+ try {
+
+ final Object oldVal = getProperty(s, p);
+
+ cxn().remove(s, p, null);
+
+ return oldVal;
+
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ }
+
+ public void setProperty(final URI s, final URI p, final Literal o) {
+
+ try {
+
+ cxn().remove(s, p, null);
+
+ cxn().add(s, p, o);
+
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ }
+
+ @Override
+ public Edge addEdge(final Object key, final Vertex from, final Vertex to,
+ final String label) {
+
+ if (label == null) {
+ throw new IllegalArgumentException();
+ }
+
+ final String eid = key != null ? key.toString() : UUID.randomUUID().toString();
+
+ final URI edgeURI = factory.toEdgeURI(eid);
+
+ if (key != null) {
+
+ final Edge edge = getEdge(key);
+
+ if (edge != null) {
+ if (!(edge.getVertex(Direction.OUT).equals(from) &&
+ (edge.getVertex(Direction.OUT).equals(to)))) {
+ throw new IllegalArgumentException("edge already exists: " + key);
+ }
+ }
+
+ }
+
+ try {
+
+// if (cxn().hasStatement(edgeURI, RDF.TYPE, EDGE, false)) {
+// throw new IllegalArgumentException("edge " + eid + " already exists");
+// }
+
+ final URI fromURI = factory.toVertexURI(from.getId().toString());
+ final URI toURI = factory.toVertexURI(to.getId().toString());
+
+ cxn().add(fromURI, edgeURI, toURI);
+ cxn().add(edgeURI, RDF.TYPE, EDGE);
+ cxn().add(edgeURI, RDFS.LABEL, factory.toLiteral(label));
+
+ return new BigdataEdge(new StatementImpl(fromURI, edgeURI, toURI), this);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public Vertex addVertex(final Object key) {
+
+ try {
+
+ final String vid = key != null ?
+ key.toString() : UUID.randomUUID().toString();
+
+ final URI uri = factory.toVertexURI(vid);
+
+// if (cxn().hasStatement(vertexURI, RDF.TYPE, VERTEX, false)) {
+// throw new IllegalArgumentException("vertex " + vid + " already exists");
+// }
+
+ cxn().add(uri, RDF.TYPE, VERTEX);
+
+ return new BigdataVertex(uri, this);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public Edge getEdge(final Object key) {
+
+ if (key == null)
+ throw new IllegalArgumentException();
+
+ try {
+
+ final URI edge = factory.toEdgeURI(key.toString());
+
+ final RepositoryResult<Statement> result =
+ cxn().getStatements(null, edge, null, false);
+
+ if (result.hasNext()) {
+
+ final Statement stmt = result.next();
+
+ if (result.hasNext()) {
+ throw new RuntimeException(
+ "duplicate edge: " + key);
+ }
+
+ return new BigdataEdge(stmt, this);
+
+ }
+
+ return null;
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public Iterable<Edge> getEdges() {
+
+ final URI wild = null;
+ return getEdges(wild, wild);
+
+ }
+
+ public Iterable<Edge> getEdges(final URI s, final URI o, final String... labels) {
+
+ final StringBuilder sb = new StringBuilder();
+ sb.append("construct { ?from ?edge ?to . } where {\n");
+ sb.append("?edge rdf:type bd:Edge . ?from ?edge ?to .\n");
+ if (labels != null && labels.length > 0) {
+ if (labels.length == 1) {
+ sb.append("?edge rdfs:label \"").append(labels[0]).append("\" .\n");
+ } else {
+ sb.append("?edge rdfs:label ?label .\n");
+ sb.append("filter(?label in (");
+ for (String label : labels) {
+ sb.append("\""+label+"\", ");
+ }
+ sb.setLength(sb.length()-2);
+ sb.append(")) .\n");
+ }
+ }
+ sb.append("}");
+
+ final String queryStr = sb.toString()
+ .replace("?from", s != null ? "<"+s+">" : "?from")
+ .replace("?to", o != null ? "<"+o+">" : "?to");
+
+ return getEdges(queryStr);
+
+ }
+
+ Iterable<Edge> getEdges(final String queryStr) {
+
+ try {
+
+ final org.openrdf.query.GraphQuery query =
+ cxn().prepareGraphQuery(QueryLanguage.SPARQL, queryStr);
+
+ final GraphQueryResult stmts = query.evaluate();
+
+ return new EdgeIterable(stmts);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ public Iterable<Vertex> getVertices(final URI s, final URI o,
+ final String... labels) {
+
+ if (s != null && o != null) {
+ throw new IllegalArgumentException();
+ }
+
+ if (s == null && o == null) {
+ throw new IllegalArgumentException();
+ }
+
+ final StringBuilder sb = new StringBuilder();
+ sb.append("construct { ?from ?edge ?to . } where {\n");
+ sb.append("?edge rdf:type bd:Edge . ?from ?edge ?to .\n");
+ if (labels != null && labels.length > 0) {
+ if (labels.length == 1) {
+ sb.append("?edge rdfs:label \"").append(labels[0]).append("\" .\n");
+ } else {
+ sb.append("?edge rdfs:label ?label .\n");
+ sb.append("filter(?label in (");
+ for (String label : labels) {
+ sb.append("\""+label+"\", ");
+ }
+ sb.setLength(sb.length()-2);
+ sb.append(")) .\n");
+ }
+ }
+ sb.append("}");
+
+ final String queryStr = sb.toString()
+ .replace("?from", s != null ? "<"+s+">" : "?from")
+ .replace("?to", o != null ? "<"+o+">" : "?to");
+
+ return getVertices(queryStr, s == null);
+
+ }
+
+ Iterable<Vertex> getVertices(final String queryStr, final boolean subject) {
+
+ try {
+
+ final org.openrdf.query.GraphQuery query =
+ cxn().prepareGraphQuery(QueryLanguage.SPARQL, queryStr);
+
+ final GraphQueryResult stmts = query.evaluate();
+
+ return new VertexIterable(stmts, subject);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ public final <T> Iterable<T> fuse(final Iterable<T>... args) {
+
+ return new FusedIterable<T>(args);
+ }
+
+
+ @Override
+ public Iterable<Edge> getEdges(final String prop, final Object val) {
+
+ final URI p = factory.toPropertyURI(prop);
+ final Literal o = factory.toLiteral(val);
+
+ try {
+
+ final String queryStr = IOUtils.toString(
+ getClass().getResourceAsStream("edgesByProperty.rq"))
+ .replace("?prop", "<"+p+">")
+ .replace("?val", o.toString());
+
+ final org.openrdf.query.GraphQuery query =
+ cxn().prepareGraphQuery(QueryLanguage.SPARQL, queryStr);
+
+ final GraphQueryResult stmts = query.evaluate();
+
+ return new EdgeIterable(stmts);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public Features getFeatures() {
+
+ return FEATURES;
+
+ }
+
+ @Override
+ public Vertex getVertex(final Object key) {
+
+ if (key == null)
+ throw new IllegalArgumentException();
+
+ final URI uri = factory.toVertexURI(key.toString());
+ try {
+ if (cxn().hasStatement(uri, RDF.TYPE, VERTEX, false)) {
+ return new BigdataVertex(uri, this);
+ }
+ return null;
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public Iterable<Vertex> getVertices() {
+
+ try {
+ final RepositoryResult<Statement> result =
+ cxn().getStatements(null, RDF.TYPE, VERTEX, false);
+ return new VertexIterable(result, true);
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public Iterable<Vertex> getVertices(String prop, Object val) {
+
+ final URI p = factory.toPropertyURI(prop);
+ final Literal o = factory.toLiteral(val);
+ try {
+ final RepositoryResult<Statement> result =
+ cxn().getStatements(null, p, o, false);
+ return new VertexIterable(result, true);
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ @Override
+ public GraphQuery query() {
+// return new DefaultGraphQuery(this);
+ return new BigdataGraphQuery(this);
+ }
+
+ @Override
+ public void removeEdge(final Edge edge) {
+ try {
+ final URI uri = factory.toURI(edge);
+ if (!cxn().hasStatement(uri, RDF.TYPE, EDGE, false)) {
+ throw new IllegalStateException();
+ }
+ final URI wild = null;
+ // remove the edge statement
+ cxn().remove(wild, uri, wild);
+ // remove its properties
+ cxn().remove(uri, wild, wild);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void removeVertex(final Vertex vertex) {
+ try {
+ final URI uri = factory.toURI(vertex);
+ if (!cxn().hasStatement(uri, RDF.TYPE, VERTEX, false)) {
+ throw new IllegalStateException();
+ }
+ final URI wild = null;
+ // remove outgoing links and properties
+ cxn().remove(uri, wild, wild);
+ // remove incoming links
+ cxn().remove(wild, wild, uri);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+// @Override
+// public void commit() {
+// try {
+// cxn().commit();
+// } catch (RepositoryException e) {
+// throw new RuntimeException(e);
+// }
+// }
+//
+// @Override
+// public void rollback() {
+// try {
+// cxn().rollback();
+// cxn.close();
+// cxn = repo.getUnisolatedConnection();
+// cxn.setAutoCommit(false);
+// } catch (RepositoryException e) {
+// throw new RuntimeException(e);
+// }
+// }
+//
+// @Override
+// public void shutdown() {
+// try {
+// cxn.close();
+// repo.shutDown();
+// } catch (RepositoryException e) {
+// throw new RuntimeException(e);
+// }
+// }
+//
+// @Override
+// @Deprecated
+// public void stopTransaction(Conclusion arg0) {
+// }
+
+ public class VertexIterable implements Iterable<Vertex>, Iterator<Vertex> {
+
+ private final CloseableIteration<Statement, ? extends OpenRDFException> stmts;
+
+ private final boolean subject;
+
+ private final List<Vertex> cache;
+
+ public VertexIterable(
+ final CloseableIteration<Statement, ? extends OpenRDFException> stmts,
+ final boolean subject) {
+ this.stmts = stmts;
+ this.subject = subject;
+ this.cache = new LinkedList<Vertex>();
+ }
+
+ @Override
+ public boolean hasNext() {
+ try {
+ return stmts.hasNext();
+ } catch (OpenRDFException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public Vertex next() {
+ try {
+ final Statement stmt = stmts.next();
+ final URI v = (URI)
+ (subject ? stmt.getSubject() : stmt.getObject());
+ if (!hasNext()) {
+ stmts.close();
+ }
+ final Vertex vertex = new BigdataVertex(v, BigdataGraph.this);
+ cache.add(vertex);
+ return vertex;
+ } catch (OpenRDFException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void remove() {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterator<Vertex> iterator() {
+ return hasNext() ? this : cache.iterator();
+ }
+
+ }
+
+ public class EdgeIterable implements Iterable<Edge>, Iterator<Edge> {
+
+ private final CloseableIteration<Statement, ? extends OpenRDFException> stmts;
+
+ private final List<Edge> cache;
+
+ public EdgeIterable(
+ final CloseableIteration<Statement, ? extends OpenRDFException> stmts) {
+ this.stmts = stmts;
+ this.cache = new LinkedList<Edge>();
+ }
+
+ @Override
+ public boolean hasNext() {
+ try {
+ return stmts.hasNext();
+ } catch (OpenRDFException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public Edge next() {
+ try {
+ final Statement stmt = stmts.next();
+ if (!hasNext()) {
+ stmts.close();
+ }
+ final Edge edge = new BigdataEdge(stmt, BigdataGraph.this);
+ cache.add(edge);
+ return edge;
+ } catch (OpenRDFException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void remove() {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterator<Edge> iterator() {
+ return hasNext() ? this : cache.iterator();
+ }
+
+ }
+
+ public class FusedIterable<T> implements Iterable<T>, Iterator<T> {
+
+ private final Iterable<T>[] args;
+
+ private transient int i = 0;
+
+ private transient Iterator<T> curr;
+
+ public FusedIterable(final Iterable<T>... args) {
+ this.args = args;
+ this.curr = args[0].iterator();
+ }
+
+ @Override
+ public boolean hasNext() {
+ if (curr.hasNext()) {
+ return true;
+ }
+ while (!curr.hasNext() && i < (args.length-1)) {
+ curr = args[++i].iterator();
+ if (curr.hasNext()) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public T next() {
+ return curr.next();
+ }
+
+ @Override
+ public void remove() {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterator<T> iterator() {
+ return this;
+ }
+
+ }
+
+ protected static final Features FEATURES = new Features();
+
+ static {
+
+ FEATURES.supportsSerializableObjectProperty = false;
+ FEATURES.supportsBooleanProperty = true;
+ FEATURES.supportsDoubleProperty = true;
+ FEATURES.supportsFloatProperty = true;
+ FEATURES.supportsIntegerProperty = true;
+ FEATURES.supportsPrimitiveArrayProperty = false;
+ FEATURES.supportsUniformListProperty = false;
+ FEATURES.supportsMixedListProperty = false;
+ FEATURES.supportsLongProperty = true;
+ FEATURES.supportsMapProperty = false;
+ FEATURES.supportsStringProperty = true;
+
+ FEATURES.supportsDuplicateEdges = true;
+ FEATURES.supportsSelfLoops = true;
+ FEATURES.isPersistent = true;
+ FEATURES.isWrapper = false;
+ FEATURES.supportsVertexIteration = true;
+ FEATURES.supportsEdgeIteration = true;
+ FEATURES.supportsVertexIndex = false;
+ FEATURES.supportsEdgeIndex = false;
+ FEATURES.ignoresSuppliedIds = true;
+ FEATURES.supportsTransactions = false;
+ FEATURES.supportsIndices = true;
+ FEATURES.supportsKeyIndices = true;
+ FEATURES.supportsVertexKeyIndex = true;
+ FEATURES.supportsEdgeKeyIndex = true;
+ FEATURES.supportsEdgeRetrieval = true;
+ FEATURES.supportsVertexProperties = true;
+ FEATURES.supportsEdgeProperties = true;
+ FEATURES.supportsThreadedTransactions = false;
+ }
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraph.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,325 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import java.util.UUID;
+
+import org.openrdf.model.Literal;
+import org.openrdf.model.URI;
+import org.openrdf.model.impl.StatementImpl;
+import org.openrdf.model.vocabulary.RDF;
+import org.openrdf.model.vocabulary.RDFS;
+import org.openrdf.repository.RepositoryConnection;
+
+import com.bigdata.rdf.changesets.IChangeLog;
+import com.bigdata.rdf.changesets.IChangeRecord;
+import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
+import com.tinkerpop.blueprints.Edge;
+import com.tinkerpop.blueprints.GraphQuery;
+import com.tinkerpop.blueprints.TransactionalGraph;
+import com.tinkerpop.blueprints.Vertex;
+
+/**
+ * Simple bulk loader that will insert graph data without any consistency
+ * checking (won't check for duplicate vertex or edge identifiers). Currently
+ * does not overwrite old property values, but we may need to change this.
+ * <p>
+ * Implements {@link IChangeLog} so that we can report a mutation count.
+ *
+ * @author mikepersonick
+ *
+ */
+public class BigdataGraphBulkLoad extends BigdataGraph
+ implements TransactionalGraph, IChangeLog {
+
+ private final BigdataSailRepositoryConnection cxn;
+
+ public BigdataGraphBulkLoad(final BigdataSailRepositoryConnection cxn) {
+ this(cxn, BigdataRDFFactory.INSTANCE);
+ }
+
+ public BigdataGraphBulkLoad(final BigdataSailRepositoryConnection cxn,
+ final BlueprintsRDFFactory factory) {
+ super(factory);
+
+ this.cxn = cxn;
+ this.cxn.addChangeLog(this);
+ }
+
+ protected RepositoryConnection cxn() throws Exception {
+ return cxn;
+ }
+
+ @Override
+ public void commit() {
+ try {
+ cxn.commit();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void rollback() {
+ try {
+ cxn.rollback();
+ cxn.close();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void shutdown() {
+ try {
+ cxn.close();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ @Deprecated
+ public void stopTransaction(Conclusion arg0) {
+ }
+
+
+ static {
+
+// FEATURES.supportsSerializableObjectProperty = false;
+// FEATURES.supportsBooleanProperty = true;
+// FEATURES.supportsDoubleProperty = true;
+// FEATURES.supportsFloatProperty = true;
+// FEATURES.supportsIntegerProperty = true;
+// FEATURES.supportsPrimitiveArrayProperty = false;
+// FEATURES.supportsUniformListProperty = false;
+// FEATURES.supportsMixedListProperty = false;
+// FEATURES.supportsLongProperty = true;
+// FEATURES.supportsMapProperty = false;
+// FEATURES.supportsStringProperty = true;
+//
+// FEATURES.supportsDuplicateEdges = true;
+// FEATURES.supportsSelfLoops = true;
+// FEATURES.isPersistent = true;
+// FEATURES.isWrapper = false;
+// FEATURES.supportsVertexIteration = true;
+// FEATURES.supportsEdgeIteration = true;
+// FEATURES.supportsVertexIndex = false;
+// FEATURES.supportsEdgeIndex = false;
+// FEATURES.ignoresSuppliedIds = true;
+ BigdataGraph.FEATURES.supportsTransactions = true;
+// FEATURES.supportsIndices = true;
+// FEATURES.supportsKeyIndices = true;
+// FEATURES.supportsVertexKeyIndex = true;
+// FEATURES.supportsEdgeKeyIndex = true;
+// FEATURES.supportsEdgeRetrieval = true;
+// FEATURES.supportsVertexProperties = true;
+// FEATURES.supportsEdgeProperties = true;
+// FEATURES.supportsThreadedTransactions = false;
+ }
+
+
+ @Override
+ public Edge getEdge(Object arg0) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterable<Edge> getEdges() {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterable<Edge> getEdges(String arg0, Object arg1) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Vertex getVertex(Object arg0) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterable<Vertex> getVertices() {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public Iterable<Vertex> getVertices(String arg0, Object arg1) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public GraphQuery query() {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void removeEdge(Edge arg0) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void removeVertex(Vertex arg0) {
+ throw new UnsupportedOperationException();
+ }
+
+ /**
+ * Set a property without removing the old value first.
+ */
+ @Override
+ public void setProperty(final URI s, final URI p, final Literal o) {
+
+ try {
+
+// cxn().remove(s, p, null);
+
+ cxn().add(s, p, o);
+
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ }
+
+ /**
+ * Add a vertex without consistency checking (does not check for a duplicate
+ * identifier).
+ */
+ @Override
+ public Vertex addVertex(final Object key) {
+
+ try {
+
+ final String vid = key != null ?
+ key.toString() : UUID.randomUUID().toString();
+
+ final URI uri = factory.toVertexURI(vid);
+
+// if (cxn().hasStatement(vertexURI, RDF.TYPE, VERTEX, false)) {
+// throw new IllegalArgumentException("vertex " + vid + " already exists");
+// }
+
+ cxn().add(uri, RDF.TYPE, VERTEX);
+
+ return new BigdataVertex(uri, this);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ /**
+ * Add an edge without consistency checking (does not check for a duplicate
+ * identifier).
+ */
+ @Override
+ public Edge addEdge(final Object key, final Vertex from, final Vertex to,
+ final String label) {
+
+ if (label == null) {
+ throw new IllegalArgumentException();
+ }
+
+ final String eid = key != null ? key.toString() : UUID.randomUUID().toString();
+
+ final URI edgeURI = factory.toEdgeURI(eid);
+
+// if (key != null) {
+//
+// final Edge edge = getEdge(key);
+//
+// if (edge != null) {
+// if (!(edge.getVertex(Direction.OUT).equals(from) &&
+// (edge.getVertex(Direction.OUT).equals(to)))) {
+// throw new IllegalArgumentException("edge already exists: " + key);
+// }
+// }
+//
+// }
+
+ try {
+
+// if (cxn().hasStatement(edgeURI, RDF.TYPE, EDGE, false)) {
+// throw new IllegalArgumentException("edge " + eid + " already exists");
+// }
+
+ final URI fromURI = factory.toVertexURI(from.getId().toString());
+ final URI toURI = factory.toVertexURI(to.getId().toString());
+
+ cxn().add(fromURI, edgeURI, toURI);
+ cxn().add(edgeURI, RDF.TYPE, EDGE);
+ cxn().add(edgeURI, RDFS.LABEL, factory.toLiteral(label));
+
+ return new BigdataEdge(new StatementImpl(fromURI, edgeURI, toURI), this);
+
+ } catch (Exception ex) {
+ throw new RuntimeException(ex);
+ }
+
+ }
+
+ private transient long mutationCountTotal = 0;
+ private transient long mutationCountCurrentCommit = 0;
+ private transient long mutationCountLastCommit = 0;
+
+ @Override
+ public void changeEvent(final IChangeRecord record) {
+ mutationCountTotal++;
+ mutationCountCurrentCommit++;
+ }
+
+ @Override
+ public void transactionBegin() {
+ }
+
+ @Override
+ public void transactionPrepare() {
+ }
+
+ @Override
+ public void transactionCommited(long commitTime) {
+ mutationCountLastCommit = mutationCountCurrentCommit;
+ mutationCountCurrentCommit = 0;
+ }
+
+ @Override
+ public void transactionAborted() {
+ }
+
+ public long getMutationCountTotal() {
+ return mutationCountTotal;
+ }
+
+ public long getMutationCountCurrentCommit() {
+ return mutationCountCurrentCommit;
+ }
+
+ public long getMutationCountLastCommit() {
+ return mutationCountLastCommit;
+ }
+
+
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphBulkLoad.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,113 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import com.bigdata.rdf.sail.remote.BigdataSailRemoteRepository;
+import com.bigdata.rdf.sail.remote.BigdataSailRemoteRepositoryConnection;
+import com.bigdata.rdf.sail.webapp.client.RemoteRepository;
+
+/**
+ * This is a thin-client implementation of a Blueprints wrapper around the
+ * client library that interacts with the NanoSparqlServer. This is a functional
+ * implementation suitable for writing POCs - it is not a high performance
+ * implementation by any means (currently does not support caching, batched
+ * update, or Blueprints query re-writes). Does have a single "bulk upload"
+ * operation that wraps a method on RemoteRepository that will POST a graphml
+ * file to the blueprints layer of the bigdata server.
+ *
+ * @see {@link BigdataSailRemoteRepository}
+ * @see {@link BigdataSailRemoteRepositoryConnection}
+ * @see {@link RemoteRepository}
+ *
+ * @author mikepersonick
+ *
+ */
+public class BigdataGraphClient extends BigdataGraph {
+
+ final BigdataSailRemoteRepository repo;
+
+ transient BigdataSailRemoteRepositoryConnection cxn;
+
+ public BigdataGraphClient(final String bigdataEndpoint) {
+ this(bigdataEndpoint, BigdataRDFFactory.INSTANCE);
+ }
+
+ public BigdataGraphClient(final String bigdataEndpoint,
+ final BlueprintsRDFFactory factory) {
+ this(new BigdataSailRemoteRepository(bigdataEndpoint), factory);
+ }
+
+ public BigdataGraphClient(final RemoteRepository repo) {
+ this(repo, BigdataRDFFactory.INSTANCE);
+ }
+
+ public BigdataGraphClient(final RemoteRepository repo,
+ final BlueprintsRDFFactory factory) {
+ this(new BigdataSailRemoteRepository(repo), factory);
+ }
+
+ public BigdataGraphClient(final BigdataSailRemoteRepository repo) {
+ this(repo, BigdataRDFFactory.INSTANCE);
+ }
+
+ public BigdataGraphClient(final BigdataSailRemoteRepository repo,
+ final BlueprintsRDFFactory factory) {
+ super(factory);
+
+ this.repo = repo;
+ }
+
+ /**
+ * Post a GraphML file to the remote server. (Bulk-upload operation.)
+ */
+ @Override
+ public void loadGraphML(final String file) throws Exception {
+ this.repo.getRemoteRepository().postGraphML(file);
+ }
+
+ /**
+ * Get a {@link BigdataSailRemoteRepositoryConnection}.
+ */
+ protected BigdataSailRemoteRepositoryConnection cxn() throws Exception {
+ if (cxn == null) {
+ cxn = repo.getConnection();
+ }
+ return cxn;
+ }
+
+ /**
+ * Shutdown the connection and repository (client-side, not server-side).
+ */
+ @Override
+ public void shutdown() {
+ try {
+ if (cxn != null) {
+ cxn.close();
+ }
+ repo.shutDown();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphClient.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,166 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import org.openrdf.repository.RepositoryConnection;
+
+import com.bigdata.rdf.sail.BigdataSail;
+import com.bigdata.rdf.sail.BigdataSailRepository;
+import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
+import com.tinkerpop.blueprints.TransactionalGraph;
+
+/**
+ * This is the most basic possible implementation of the Blueprints Graph API.
+ * It wraps an embedded {@link BigdataSailRepository} and holds open an
+ * unisolated connection to the database for the lifespan of the Graph (until
+ * {@link #shutdown()} is called.
+ *
+ * @author mikepersonick
+ *
+ */
+public class BigdataGraphEmbedded extends BigdataGraph implements TransactionalGraph {
+
+ final BigdataSailRepository repo;
+
+ transient BigdataSailRepositoryConnection cxn;
+
+ /**
+ * Create a Blueprints wrapper around a {@link BigdataSail} instance.
+ */
+ public BigdataGraphEmbedded(final BigdataSail sail) {
+ this(sail, BigdataRDFFactory.INSTANCE);
+ }
+
+ /**
+ * Create a Blueprints wrapper around a {@link BigdataSail} instance with
+ * a non-standard {@link BlueprintsRDFFactory} implementation.
+ */
+ public BigdataGraphEmbedded(final BigdataSail sail,
+ final BlueprintsRDFFactory factory) {
+ this(new BigdataSailRepository(sail), factory);
+ }
+
+ /**
+ * Create a Blueprints wrapper around a {@link BigdataSailRepository}
+ * instance.
+ */
+ public BigdataGraphEmbedded(final BigdataSailRepository repo) {
+ this(repo, BigdataRDFFactory.INSTANCE);
+ }
+
+ /**
+ * Create a Blueprints wrapper around a {@link BigdataSailRepository}
+ * instance with a non-standard {@link BlueprintsRDFFactory} implementation.
+ */
+ public BigdataGraphEmbedded(final BigdataSailRepository repo,
+ final BlueprintsRDFFactory factory) {
+ super(factory);
+
+ this.repo = repo;
+ }
+
+ protected RepositoryConnection cxn() throws Exception {
+ if (cxn == null) {
+ cxn = repo.getUnisolatedConnection();
+ cxn.setAutoCommit(false);
+ }
+ return cxn;
+ }
+
+ @Override
+ public void commit() {
+ try {
+ if (cxn != null)
+ cxn.commit();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void rollback() {
+ try {
+ if (cxn != null) {
+ cxn.rollback();
+ cxn.close();
+ cxn = null;
+ }
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ public void shutdown() {
+ try {
+ if (cxn != null) {
+ cxn.close();
+ }
+ repo.shutDown();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ @Deprecated
+ public void stopTransaction(Conclusion arg0) {
+ }
+
+
+ static {
+
+// FEATURES.supportsSerializableObjectProperty = false;
+// FEATURES.supportsBooleanProperty = true;
+// FEATURES.supportsDoubleProperty = true;
+// FEATURES.supportsFloatProperty = true;
+// FEATURES.supportsIntegerProperty = true;
+// FEATURES.supportsPrimitiveArrayProperty = false;
+// FEATURES.supportsUniformListProperty = false;
+// FEATURES.supportsMixedListProperty = false;
+// FEATURES.supportsLongProperty = true;
+// FEATURES.supportsMapProperty = false;
+// FEATURES.supportsStringProperty = true;
+//
+// FEATURES.supportsDuplicateEdges = true;
+// FEATURES.supportsSelfLoops = true;
+// FEATURES.isPersistent = true;
+// FEATURES.isWrapper = false;
+// FEATURES.supportsVertexIteration = true;
+// FEATURES.supportsEdgeIteration = true;
+// FEATURES.supportsVertexIndex = false;
+// FEATURES.supportsEdgeIndex = false;
+// FEATURES.ignoresSuppliedIds = true;
+ BigdataGraph.FEATURES.supportsTransactions = true;
+// FEATURES.supportsIndices = true;
+// FEATURES.supportsKeyIndices = true;
+// FEATURES.supportsVertexKeyIndex = true;
+// FEATURES.supportsEdgeKeyIndex = true;
+// FEATURES.supportsEdgeRetrieval = true;
+// FEATURES.supportsVertexProperties = true;
+// FEATURES.supportsEdgeProperties = true;
+// FEATURES.supportsThreadedTransactions = false;
+ }
+
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphEmbedded.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,86 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import org.apache.log4j.Logger;
+
+import com.bigdata.rdf.sail.BigdataSail;
+import com.bigdata.rdf.sail.BigdataSailFactory;
+
+
+/**
+ * Helper class to create BigdataGraph instances.
+ *
+ * @author mikepersonick
+ *
+ */
+public class BigdataGraphFactory {
+
+ protected static final transient Logger log = Logger.getLogger(BigdataGraphFactory.class);
+
+ /**
+ * Connect to a remote bigdata instance.
+ */
+ public static BigdataGraph connect(final String serviceEndpoint) {
+ if (serviceEndpoint.endsWith("/bigdata/sparql")) {
+ return new BigdataGraphClient(serviceEndpoint);
+ } else if (serviceEndpoint.endsWith("/bigdata/")) {
+ return new BigdataGraphClient(serviceEndpoint + "sparql");
+ } else if (serviceEndpoint.endsWith("/bigdata")) {
+ return new BigdataGraphClient(serviceEndpoint + "/sparql");
+ } else if (serviceEndpoint.endsWith("/")) {
+ return new BigdataGraphClient(serviceEndpoint + "bigdata/sparql");
+ } else {
+ return new BigdataGraphClient(serviceEndpoint + "/bigdata/sparql");
+ }
+ }
+
+ /**
+ * Open an existing persistent local bigdata instance.
+ */
+ public static BigdataGraph open(final String file) throws Exception {
+ final BigdataSail sail = BigdataSailFactory.openSail(file);
+ sail.initialize();
+ return new BigdataGraphEmbedded(sail);
+ }
+
+ /**
+ * Create a new local in-memory bigdata instance.
+ */
+ public static BigdataGraph create() throws Exception {
+ final BigdataSail sail = BigdataSailFactory.createSail();
+ sail.initialize();
+ return new BigdataGraphEmbedded(sail);
+ }
+
+ /**
+ * Create a new persistent local bigdata instance.
+ */
+ public static BigdataGraph create(final String file)
+ throws Exception {
+ final BigdataSail sail = BigdataSailFactory.createSail(file);
+ sail.initialize();
+ return new BigdataGraphEmbedded(sail);
+ }
+
+}
Property changes on: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphFactory.java
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Added: branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java (rev 0)
+++ branches/BLUEPRINTS/bigdata-sails/src/java/com/bigdata/blueprints/BigdataGraphQuery.java 2014-05-10 02:56:35 UTC (rev 8262)
@@ -0,0 +1,449 @@
+/**
+Copyright (C) SYSTAP, LLC 2006-2014. All rights reserved.
+
+Contact:
+ SYSTAP, LLC
+ 4501 Tower Road
+ Greensboro, NC 27410
+ lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+package com.bigdata.blueprints;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.openrdf.model.Literal;
+import org.openrdf.model.URI;
+
+import com.tinkerpop.blueprints.Contains;
+import com.tinkerpop.blueprints.Edge;
+import com.tinkerpop.blueprints.GraphQuery;
+import com.tinkerpop.blueprints.Predicate;
+import com.tinkerpop.blueprints.Vertex;
+
+public class BigdataGraphQuery implements GraphQuery {
+
+ protected final static transient Logger log = Logger.getLogger(BigdataGraphQuery.class);
+
+ protected static List<Class> knownPredicates = Arrays.asList(new Class[] {
+ BigdataPredicate.class,
+ Compare.class,
+ Contains.class,
+ com.tinkerpop.blueprints.Compare.class
+ });
+
+ private final BigdataGraph graph;
+
+ private final List<Has> criteria = new LinkedList<Has>();
+
+ private transient int limit = Integer.MAX_VALUE;
+
+ public BigdataGraphQuery(final BigdataGraph graph) {
+ this.graph = graph;
+ }
+
+ /**
+ * Filter out elements that do not have a property with provided key.
+ *
+ * ?s <key> ?value
+ *
+ * @param key the key of the property
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery has(final String key) {
+ criteria.add(new Has(key));
+ return this;
+ }
+
+ /**
+ * Filter out elements that have a property with provided key.
+ *
+ * ?s ?p ?o .
+ * filter not exists { ?s <key> ?value } .
+ *
+ * @param key the key of the property
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery hasNot(final String key) {
+ criteria.add(new HasNot(key));
+ return this;
+ }
+
+ /**
+ * Filter out elements that do not have a property value equal to provided value.
+ *
+ * ?s <key> <value> .
+ *
+ * @param key the key of the property
+ * @param value the value to check against
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery has(final String key, final Object value) {
+ criteria.add(new Has(key, value));
+ return this;
+ }
+
+ /**
+ * Filter out elements that have a property value equal to provided value.
+ *
+ * ?s ?p ?o .
+ * filter not exists { ?s <key> <value> } .
+ *
+ * @param key the key of the property
+ * @param value the value to check against
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery hasNot(final String key, final Object value) {
+ criteria.add(new HasNot(key, value));
+ return this;
+ }
+
+ /**
+ * Filter out the element if it does not have a property with a comparable value.
+ *
+ * @param key the key of the property
+ * @param predicate the comparator to use for comparison
+ * @param value the value to check against
+ * @return the modified query object
+ */
+ @Override
+ public GraphQuery has(final String key, final Predicate predicate, final Object value) {
+ if (!knownPredicates.contains(predicate.getClass())) {
+ throw new IllegalArgumentException();
+ }
+ criteria.add(new Has(key, value, BigdataPredicate.toBigdataPredicate(predicate)));
+ return this;
+ }
+
+ /**
+ * Filter out the element if it does not have a property with a comparable value.
+ *
+ * @param key the key of the property
+ * @param value the value to check against
+ * @param compare the comparator to use for comparison
+ * @return the modified query object
+ */
+ @Override
+ @Deprecated
+ public <T extends Comparable<T>> GraphQuery has(final String key, final T value,
+ final Compare compare) {
+ return has(key, compare, value);
+ }
+
+ /**
+ * Filter out the element of its property value is not within the provided interval.
+ *
+ * @param key the key of the property
+ * @param startValue the inclusive start value of the interval
+ * @param endValue the exclusive end value of the interval
+ * @return the modified query object
+ */
+ @Override
+ public <T extends Comparable<?>> GraphQuery interval(final S...
[truncated message content] |
|
From: <jer...@us...> - 2014-05-10 00:58:02
|
Revision: 8261
http://sourceforge.net/p/bigdata/code/8261
Author: jeremy_carroll
Date: 2014-05-10 00:57:58 +0000 (Sat, 10 May 2014)
Log Message:
-----------
Various spelling errors
Modified Paths:
--------------
branches/TEXT_ANALYZERS/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java
Modified: branches/TEXT_ANALYZERS/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java
===================================================================
--- branches/TEXT_ANALYZERS/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java 2014-05-09 23:33:24 UTC (rev 8260)
+++ branches/TEXT_ANALYZERS/bigdata/src/java/com/bigdata/search/ConfigurableAnalyzerFactory.java 2014-05-10 00:57:58 UTC (rev 8261)
@@ -291,13 +291,13 @@
*
*
*/
- String NATURAL_LANGUAGE_SUPPORT = ConfigurableAnalyzerFactory.class.getName() + ".includeDefaults";
+ String NATURAL_LANGUAGE_SUPPORT = ConfigurableAnalyzerFactory.class.getName() + ".naturalLanguageSupport";
/**
* This is the prefix to all properties configuring the individual analyzers.
*/
String ANALYZER = ConfigurableAnalyzerFactory.class.getName() + ".analyzer.";
- String DEFAULT_NATURAL_LAMGUAGE_SUPPORT = "false";
+ String DEFAULT_NATURAL_LANGUAGE_SUPPORT = "false";
}
/**
* Options understood by analyzers created by {@link ConfigurableAnalyzerFactory}.
@@ -375,14 +375,14 @@
* (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
* It is an error if a different analyzer class is specified.
*/
- String SOFT_HYPHENS = "softHypens";
+ String SOFT_HYPHENS = "softHyphens";
/**
* The value of the alwaysRemoveSoftHypens parameter to
* {@link TermCompletionAnalyzer#TermCompletionAnalyzer(Pattern, Pattern, Pattern, boolean)}
* (Note the {@link Pattern#UNICODE_CHARACTER_CLASS} flag is enabled).
* It is an error if a different analyzer class is specified.
*/
- String ALWAYS_REMOVE_SOFT_HYPHENS = "alwaysRemoveSoftHypens";
+ String ALWAYS_REMOVE_SOFT_HYPHENS = "alwaysRemoveSoftHyphens";
boolean DEFAULT_ALWAYS_REMOVE_SOFT_HYPHENS = false;
@@ -946,7 +946,7 @@
Properties myProps;
if (Boolean.valueOf(parentProperties.getProperty(
Options.NATURAL_LANGUAGE_SUPPORT,
- Options.DEFAULT_NATURAL_LAMGUAGE_SUPPORT))) {
+ Options.DEFAULT_NATURAL_LANGUAGE_SUPPORT))) {
myProps = loadPropertyString(ALL_LUCENE_NATURAL_LANGUAGES);
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <mrp...@us...> - 2014-05-09 23:33:27
|
Revision: 8260
http://sourceforge.net/p/bigdata/code/8260
Author: mrpersonick
Date: 2014-05-09 23:33:24 +0000 (Fri, 09 May 2014)
Log Message:
-----------
1. Add in blueprints-core dependency in bigdata-sails
Added Paths:
-----------
branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar
Added: branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar
===================================================================
(Binary files differ)
Index: branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar
===================================================================
--- branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar 2014-05-09 23:08:34 UTC (rev 8259)
+++ branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar 2014-05-09 23:33:24 UTC (rev 8260)
Property changes on: branches/BLUEPRINTS/bigdata-sails/lib/blueprints-core-2.4.0.jar
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/octet-stream
\ No newline at end of property
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
4 messages has been excluded from this view by a project administrator.
