Menu
▾
▴
bigdata-developers — List for bigdata developers
This list is closed, nobody may subscribe to it.
| 2010 |
Jan
|
Feb
(19) |
Mar
(8) |
Apr
(25) |
May
(16) |
Jun
(77) |
Jul
(131) |
Aug
(76) |
Sep
(30) |
Oct
(7) |
Nov
(3) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
(2) |
Jul
(16) |
Aug
(3) |
Sep
(1) |
Oct
|
Nov
(7) |
Dec
(7) |
| 2012 |
Jan
(10) |
Feb
(1) |
Mar
(8) |
Apr
(6) |
May
(1) |
Jun
(3) |
Jul
(1) |
Aug
|
Sep
(1) |
Oct
|
Nov
(8) |
Dec
(2) |
| 2013 |
Jan
(5) |
Feb
(12) |
Mar
(2) |
Apr
(1) |
May
(1) |
Jun
(1) |
Jul
(22) |
Aug
(50) |
Sep
(31) |
Oct
(64) |
Nov
(83) |
Dec
(28) |
| 2014 |
Jan
(31) |
Feb
(18) |
Mar
(27) |
Apr
(39) |
May
(45) |
Jun
(15) |
Jul
(6) |
Aug
(27) |
Sep
(6) |
Oct
(67) |
Nov
(70) |
Dec
(1) |
| 2015 |
Jan
(3) |
Feb
(18) |
Mar
(22) |
Apr
(121) |
May
(42) |
Jun
(17) |
Jul
(8) |
Aug
(11) |
Sep
(26) |
Oct
(15) |
Nov
(66) |
Dec
(38) |
| 2016 |
Jan
(14) |
Feb
(59) |
Mar
(28) |
Apr
(44) |
May
(21) |
Jun
(12) |
Jul
(9) |
Aug
(11) |
Sep
(4) |
Oct
(2) |
Nov
(1) |
Dec
|
| 2017 |
Jan
(20) |
Feb
(7) |
Mar
(4) |
Apr
(18) |
May
(7) |
Jun
(3) |
Jul
(13) |
Aug
(2) |
Sep
(4) |
Oct
(9) |
Nov
(2) |
Dec
(5) |
| 2018 |
Jan
|
Feb
|
Mar
|
Apr
(2) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Jean-Marc V. <jea...@gm...> - 2015-09-14 10:11:56
|
Fine , and thanks ;
this answer is for a SPARQL query,
but what about the embeded API ?
Can one access this union graph by API ?
I guess one accesses this union graph simply by doing this SPARQL query by
API:
CONSTRUCT {
?S ?P ?O
} WHERE {
GRAPH ?G {
?S ?P ?O
}
}
or is there a more efficient way ?
FYI my use case is inferring input forms from ontologies and data in a
number of named graphs:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms_play/README.md
Thanks to Banana-RDF, I should be able to support Jena TDB and BlazeGraph
with the same code.
2015-09-14 11:56 GMT+02:00 Bryan Thompson <br...@sy...>:
> Any default graph query in blazegraph has these semantics unless you use
> FROM to restrict the set of considered graphs.
>
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. Blazegraph is now available with GPU acceleration using our disruptive
> technology to accelerate data-parallel graph analytics and graph query.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Mon, Sep 14, 2015 at 5:54 AM, Jean-Marc Vanel <jea...@gm...
> > wrote:
>
>> In Jena TDB there is the special graph urn:x-arq:UnionGraph
>>
>>
>> https://jena.apache.org/documentation/tdb/datasets.html#special-graph-names
>>
>> that is the RDF merge of all the named graphs in the SPARQL dataset .
>>
>> It is a convenient feature although not standard.
>>
>> Is there an equivalent feature in BlazeGraph ?
>>
>> --
>> Jean-Marc Vanel
>> Déductions SARL - Consulting, services, training,
>> Rule-based programming, Semantic Web
>> http://deductions-software.com/
>> +33 (0)6 89 16 29 52
>> Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>
--
Jean-Marc Vanel
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
http://deductions-software.com/
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
|
|
From: Bryan T. <br...@sy...> - 2015-09-14 09:56:42
|
Any default graph query in blazegraph has these semantics unless you use FROM to restrict the set of considered graphs. ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Mon, Sep 14, 2015 at 5:54 AM, Jean-Marc Vanel <jea...@gm...> wrote: > In Jena TDB there is the special graph urn:x-arq:UnionGraph > > https://jena.apache.org/documentation/tdb/datasets.html#special-graph-names > > that is the RDF merge of all the named graphs in the SPARQL dataset . > > It is a convenient feature although not standard. > > Is there an equivalent feature in BlazeGraph ? > > -- > Jean-Marc Vanel > Déductions SARL - Consulting, services, training, > Rule-based programming, Semantic Web > http://deductions-software.com/ > +33 (0)6 89 16 29 52 > Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Jean-Marc V. <jea...@gm...> - 2015-09-14 09:54:22
|
In Jena TDB there is the special graph urn:x-arq:UnionGraph https://jena.apache.org/documentation/tdb/datasets.html#special-graph-names that is the RDF merge of all the named graphs in the SPARQL dataset . It is a convenient feature although not standard. Is there an equivalent feature in BlazeGraph ? -- Jean-Marc Vanel Déductions SARL - Consulting, services, training, Rule-based programming, Semantic Web http://deductions-software.com/ +33 (0)6 89 16 29 52 Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui |
|
From: Stefan B. <ste...@wu...> - 2015-09-14 08:26:41
|
Hi all! Only last week I started using Blazegraph and was impressed by the performance and support for property path queries. Currently we are evaluating different SPARQL engines and especially exploiting property path queries for a kind of backward-chaining reasoning (see [1] for more details). The queries contain very long and complicated property paths. When evaluating these queries (example query and full stack trace in the end of the message) I get a NullPointerException. The data I loaded with the DataLoader is just the LUBM ontology and just University0_0.owl from LUBM. The query (generated by a query rewriter [2]) returns all instances of lubm:Student under OWL QL semantics and thus encodes all necessary reasoning in the path expressions. Can you see what the actual problem with this query is? Is the query just too big, too many joins? What can I do fix this? Thank you very much! Stefan Bischof [1] S Bischof, M Krötzsch, A Polleres, S Rudolph: Schema-agnostic query rewriting in SPARQL 1.1. ISWC 2014 [2] http://citydata.wu.ac.at/SPR/ Query: PREFIX dc: <http://purl.org/dc/elements/1.1/> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX owl: <http://www.w3.org/2002/07/owl#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX lubm: <http://swat.cse.lehigh.edu/onto/univ-bench.owl#> SELECT * WHERE { { ?_v0 (((((rdfs:subClassOf|owl:equivalentClass)|^owl:equivalentClass)|((owl:intersectionOf/(rdf:rest)*)/rdf:first))|((owl:onProperty/((((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty)|(((owl:inverseOf|^owl:inverseOf)/(((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty))*)/(owl:inverseOf|^owl:inverseOf))))*)/(^owl:onProperty|rdfs:domain)))|((((owl:onProperty/((((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty)|(((owl:inverseOf|^owl:inverseOf)/(((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty))*)/(owl:inverseOf|^owl:inverseOf))))*)/(owl:inverseOf|^owl:inverseOf))/(((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty))*)/rdfs:range))* lubm:Student . { { ?p rdf:type ?_v0} UNION { ?_v1 ((((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty)|(((owl:inverseOf|^owl:inverseOf)/(((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty))*)/(owl:inverseOf|^owl:inverseOf))))*/(^owl:onProperty|rdfs:domain) ?_v0 . ?p ?_v1 _:b0 } } UNION { ?_v1 ((((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty)|(((owl:inverseOf|^owl:inverseOf)/(((rdfs:subPropertyOf|owl:equivalentProperty)|^owl:equivalentProperty))*)/(owl:inverseOf|^owl:inverseOf))))*/rdfs:range ?_v0 . _:b1 ?_v1 ?p } } } Full Stacktrace: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:632) at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:259) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:248) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:138) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:830) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:649) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ... 1 more Caused by: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at com.bigdata.rdf.sail.Bigdata2Sesame2BindingSetIterator.hasNext(Bigdata2Sesame2BindingSetIterator.java:188) at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68) at org.openrdf.query.QueryResults.report(QueryResults.java:155) at org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:76) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$TupleQueryTask.doQuery(BigdataRDFContext.java:1705) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1562) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1527) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:699) ... 4 more Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.checkFuture(BlockingBuffer.java:1523) at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator._hasNext(BlockingBuffer.java:1710) at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.hasNext(BlockingBuffer.java:1563) at com.bigdata.striterator.AbstractChunkedResolverator._hasNext(AbstractChunkedResolverator.java:365) at com.bigdata.striterator.AbstractChunkedResolverator.hasNext(AbstractChunkedResolverator.java:341) at com.bigdata.rdf.sail.Bigdata2Sesame2BindingSetIterator.hasNext(Bigdata2Sesame2BindingSetIterator.java:134) ... 11 more Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.checkFuture(BlockingBuffer.java:1454) ... 16 more Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at com.bigdata.rdf.sail.RunningQueryCloseableIterator.checkFuture(RunningQueryCloseableIterator.java:59) at com.bigdata.rdf.sail.RunningQueryCloseableIterator.close(RunningQueryCloseableIterator.java:73) at com.bigdata.rdf.sail.RunningQueryCloseableIterator.hasNext(RunningQueryCloseableIterator.java:82) at com.bigdata.striterator.ChunkedWrappedIterator.hasNext(ChunkedWrappedIterator.java:197) at com.bigdata.striterator.AbstractChunkedResolverator$ChunkConsumerTask.call(AbstractChunkedResolverator.java:222) at com.bigdata.striterator.AbstractChunkedResolverator$ChunkConsumerTask.call(AbstractChunkedResolverator.java:197) ... 4 more Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at com.bigdata.util.concurrent.Haltable.get(Haltable.java:273) at com.bigdata.bop.engine.AbstractRunningQuery.get(AbstractRunningQuery.java:1511) at com.bigdata.bop.engine.AbstractRunningQuery.get(AbstractRunningQuery.java:104) at com.bigdata.rdf.sail.RunningQueryCloseableIterator.checkFuture(RunningQueryCloseableIterator.java:46) ... 9 more Caused by: java.lang.Exception: task=ChunkTask{query=12f11bc0-b22a-48c8-89f7-a3603bae639a,bopId=58,partitionId=-1,sinkId=75,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1337) at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTaskWrapper.run(ChunkedRunningQuery.java:896) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at com.bigdata.concurrent.FutureTaskMon.run(FutureTaskMon.java:63) at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkFutureTask.run(ChunkedRunningQuery.java:791) ... 3 more Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1317) ... 8 more Caused by: java.lang.RuntimeException: cause=java.lang.NullPointerException, state=JVMHashJoinUtility{open=false,joinType=Normal,joinVars=[],outputDistinctJVs=true,size=20,considered(left=26,right=312,joins=312)} at com.bigdata.bop.join.JVMHashJoinUtility.launderThrowable(JVMHashJoinUtility.java:1406) at com.bigdata.bop.join.JVMHashJoinUtility.acceptSolutions(JVMHashJoinUtility.java:431) at com.bigdata.bop.join.HashIndexOp$ChunkTask.acceptSolutions(HashIndexOp.java:433) at com.bigdata.bop.join.HashIndexOp$ChunkTask.call(HashIndexOp.java:338) at com.bigdata.bop.join.HashIndexOp$ChunkTask.call(HashIndexOp.java:237) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1316) ... 8 more Caused by: java.lang.NullPointerException at com.bigdata.bop.join.JVMHashJoinUtility.acceptSolutions(JVMHashJoinUtility.java:410) ... 13 more |
|
From: andreas81_81 <and...@o2...> - 2015-09-14 07:30:26
|
Hi all,
I have blazegraph 1.5.2 running on my machine (Windows, bundled version) and I've written sample java application (based on the information on blazegraph wiki page) but it does not work. My custom function is exactly the same as provied on the page (security filter), the function is registered, the query seems to be correct ... but blazegraph is unable to find the registered function. I'm running blazegraph with the following command:
java -classpath d:\dev\Blazegraph\org.lhasalimited.blazegraph\target\classes\ -server -Xmx4g -jar bigdata-bundled.jar
(the directory contains class with my custom function). After running a sample application which calls:
FunctionRegistry.Factory factory = new MyFunctionFactory();
URI myFunctionURI = new URIImpl("https://www.ebi.ac.uk/chembl/chox#validate");
FunctionRegistry.add(myFunctionURI, factory);
The application throws:
java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=2125bb5c-9355-4482-8b52-c3036fe60744,bopId=1,partitionId=-1,sinkId=3,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.UnsupportedOperationException: unknown function: https://www.ebi.ac.uk/chembl/chox#validate
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:261)
at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:532)
at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:189)
at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:237)
at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:137)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:595)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:191)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:72)
at com.bigdata.rdf.sail.webapp.HALoadBalancerServlet.forwardToLocalService(HALoadBalancerServlet.java:938)
at com.bigdata.rdf.sail.webapp.HALoadBalancerServlet.service(HALoadBalancerServlet.java:816)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
at java.lang.Thread.run(Thread.java:745)
Do you have any idea what i'm doing wrong?
|
|
From: Bryan T. <br...@sy...> - 2015-09-11 21:36:24
|
Excellent. On Sep 11, 2015 5:28 PM, "Joakim Soderberg" <joa...@bl...> wrote: > Hi, > I can confirm that the workaround did indeed fix the problem. > > Thanks > > *Joakim Soderberg* > > *Sr Scientist, PhD, **Head Content Platform * > > Blippar US, 444 Castro St # 302, Mountain View, CA 94041 > > > LONDON • NEW YORK • SAN FRANCISCO • LA • CHICAGO • AMSTERDAM • ISTANBUL • > DELHI • TOKYO • TORONTO > > *This e-mail is confidential. It may also be legally privileged. If you > are not the addressee you may not copy, forward, disclose or use any part > of it. If you have received this message in error, please delete it and all > copies from your system and notify the sender immediately by return e-mail. > Internet communications cannot be guaranteed to be timely, secure, error or > virus-free. The sender does not accept liability for any errors or > omissions.* > > On Sep 11, 2015, at 9:58 AM, Bryan Thompson <br...@sy...> wrote: > > Ok. We do have a workaround for this. The problem is a deferred free that > is not being properly released. This is identified by > > java.lang.RuntimeException: Problem with entry at -1982077125921865306* com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909) > * > > This issue was first reported by the wiki data service and we developed a > work around for them. The utility is available here: > > > https://jira.blazegraph.com/browse/BLZG-1236?focusedCommentId=19105&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-19105 > > Please let us know if this resolves the issue. > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com/> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. Blazegraph is now available with GPU acceleration using our disruptive > technology to accelerate data-parallel graph analytics and graph query. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Fri, Sep 11, 2015 at 12:43 PM, Joakim Soderberg < > joa...@bl...> wrote: > >> It seems to be a problem with the same entry: >> >> *type* Exception report >> >> *message* *Problem with entry at -1982077125921865306: >> lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, >> nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September >> 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, >> September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, >> September 10, 2015 11:11:10 PM UTC], commitCounter=53, >> commitRecordAddr={off=NATIVE:-451550099,len=422}, >> commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, >> blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, >> metaStartAddr=4614957, storeType=RW, >> uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, >> checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 >> 2:10:45 AM UTC], closeTime=0}* >> >> *description* *The server encountered an internal error that prevented >> it from fulfilling this request.* >> >> *exception* >> >> java.lang.RuntimeException: Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0} >> com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3126) >> com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) >> com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) >> com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) >> com.blippar.action.AddAction.execute(AddAction.java:81) >> com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) >> com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:622) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:729) >> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) >> >> *root cause* >> >> java.lang.RuntimeException: Problem with entry at -1982077125921865306 >> com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909) >> com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) >> com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) >> com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) >> com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) >> com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) >> com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) >> com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) >> com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) >> com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) >> com.blippar.action.AddAction.execute(AddAction.java:81) >> com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) >> com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:622) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:729) >> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) >> >> *root cause* >> >> java.lang.RuntimeException: addr=-472185800 : cause=java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot >> com.bigdata.rwstore.RWStore.getData(RWStore.java:2191) >> com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) >> com.bigdata.rwstore.RWStore.getData(RWStore.java:2034) >> com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) >> com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799) >> com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895) >> com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) >> com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) >> com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) >> com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) >> com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) >> com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) >> com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) >> com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) >> com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) >> com.blippar.action.AddAction.execute(AddAction.java:81) >> com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) >> com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:622) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:729) >> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) >> >> *root cause* >> >> java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot >> com.bigdata.rwstore.RWStore.getData(RWStore.java:2083) >> com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) >> com.bigdata.rwstore.RWStore.getData(RWStore.java:2034) >> com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) >> com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799) >> com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895) >> com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) >> com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) >> com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) >> com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) >> com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) >> com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) >> com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) >> com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) >> com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) >> com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) >> com.blippar.action.AddAction.execute(AddAction.java:81) >> com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) >> com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:622) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:729) >> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) >> >> >> >> >> On Sep 11, 2015, at 9:15 AM, Brad Bebee <be...@sy...> wrote: >> >> Joakim, >> >> Thank you. If you are able to reproduce it, please send as much of the >> stack trace as you can. >> >> Thanks, --Brad >> >> On Fri, Sep 11, 2015 at 12:09 PM, Joakim Soderberg < >> joa...@bl...> wrote: >> >>> Unfortunately not. I will try to reproduce the error. >>> >>> >>> On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy...> wrote: >>> >>> Joakim, >>> >>> Can you send us the full stack trace to diagnose this? >>> >>> Thanks, --Brad >>> >>> On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg < >>> joa...@bl...> wrote: >>> >>>> Hi >>>> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB >>>> RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by >>>> adding to a RepositoryConnection >>>> I get the following error: >>>> >>>> HTTP Status 500 - Problem with entry at -1982077125921865306: >>>> lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, >>>> nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September >>>> 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, >>>> September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, >>>> September 10, 2015 11:11:10 PM UTC], commitCounter=53, >>>> commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, >>>> blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, >>>> metaStartAddr=4614957, storeType=RW, >>>> uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, >>>> offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, >>>> September 1, 2015 2:10:45 AM UTC], closeTime=0} >>>> >>>> After this I can’t add triples again, could the jnl-file be corrupt >>>> now? Can it be repaired? >>>> >>>> /Joakim >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Bigdata-developers mailing list >>>> Big...@li... >>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>>> >>>> >>> >>> >>> -- >>> _______________ >>> Brad Bebee >>> CEO, Managing Partner >>> SYSTAP, LLC >>> e: be...@sy... >>> m: 202.642.7961 >>> f: 571.367.5000 >>> w: www.blazegraph.com >>> >>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. >>> Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new >>> technology to use GPUs to accelerate data-parallel graph analytics. >>> >>> CONFIDENTIALITY NOTICE: This email and its contents and attachments >>> are for the sole use of the intended recipient(s) and are confidential or >>> proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, >>> dissemination or copying of this email or its contents or attachments is >>> prohibited. If you have received this communication in error, please notify >>> the sender by reply email and permanently delete all copies of the email >>> and its contents and attachments. >>> >>> >>> >> >> >> -- >> _______________ >> Brad Bebee >> CEO, Managing Partner >> SYSTAP, LLC >> e: be...@sy... >> m: 202.642.7961 >> f: 571.367.5000 >> w: www.blazegraph.com >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. >> Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Bigdata-developers mailing list >> Big...@li... >> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >> >> > > |
|
From: Joakim S. <joa...@bl...> - 2015-09-11 21:28:24
|
Hi, I can confirm that the workaround did indeed fix the problem. Thanks Joakim Soderberg Sr Scientist, PhD, Head Content Platform Blippar US, 444 Castro St # 302, Mountain View, CA 94041 LONDON • NEW YORK • SAN FRANCISCO • LA • CHICAGO • AMSTERDAM • ISTANBUL • DELHI • TOKYO • TORONTO This e-mail is confidential. It may also be legally privileged. If you are not the addressee you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. > On Sep 11, 2015, at 9:58 AM, Bryan Thompson <br...@sy...> wrote: > > Ok. We do have a workaround for this. The problem is a deferred free that is not being properly released. This is identified by > > java.lang.RuntimeException: Problem with entry at -1982077125921865306 > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909) > This issue was first reported by the wiki data service and we developed a work around for them. The utility is available here: > > https://jira.blazegraph.com/browse/BLZG-1236?focusedCommentId=19105&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-19105 <https://jira.blazegraph.com/browse/BLZG-1236?focusedCommentId=19105&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-19105> > > Please let us know if this resolves the issue. > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... <mailto:br...@sy...> > http://blazegraph.com <http://blazegraph.com/> > http://blog.bigdata.com <http://bigdata.com/> > http://mapgraph.io <http://mapgraph.io/> > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. > > > > On Fri, Sep 11, 2015 at 12:43 PM, Joakim Soderberg <joa...@bl... <mailto:joa...@bl...>> wrote: > It seems to be a problem with the same entry: > > type Exception report > > message Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0} > > description The server encountered an internal error that prevented it from fulfilling this request. > > exception > > java.lang.RuntimeException: Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0} > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3126) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > root cause > > java.lang.RuntimeException: Problem with entry at -1982077125921865306 > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909) > com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) > com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) > com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) > com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) > com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > root cause > > java.lang.RuntimeException: addr=-472185800 : cause=java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot > com.bigdata.rwstore.RWStore.getData(RWStore.java:2191) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.getData(RWStore.java:2034) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895) > com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) > com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) > com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) > com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) > com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > root cause > > java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot > com.bigdata.rwstore.RWStore.getData(RWStore.java:2083) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.getData(RWStore.java:2034) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895) > com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) > com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) > com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) > com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) > com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > > > >> On Sep 11, 2015, at 9:15 AM, Brad Bebee <be...@sy... <mailto:be...@sy...>> wrote: >> >> Joakim, >> >> Thank you. If you are able to reproduce it, please send as much of the stack trace as you can. >> >> Thanks, --Brad >> >> On Fri, Sep 11, 2015 at 12:09 PM, Joakim Soderberg <joa...@bl... <mailto:joa...@bl...>> wrote: >> Unfortunately not. I will try to reproduce the error. >> >> >> >>> On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy... <mailto:be...@sy...>> wrote: >>> >>> Joakim, >>> >>> Can you send us the full stack trace to diagnose this? >>> >>> Thanks, --Brad >>> >>> On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg <joa...@bl... <mailto:joa...@bl...>> wrote: >>> Hi >>> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by adding to a RepositoryConnection >>> I get the following error: >>> >>> HTTP Status 500 - Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0} >>> >>> After this I can’t add triples again, could the jnl-file be corrupt now? Can it be repaired? >>> >>> /Joakim >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Bigdata-developers mailing list >>> Big...@li... <mailto:Big...@li...> >>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers <https://lists.sourceforge.net/lists/listinfo/bigdata-developers> >>> >>> >>> >>> >>> -- >>> _______________ >>> Brad Bebee >>> CEO, Managing Partner >>> SYSTAP, LLC >>> e: be...@sy... <mailto:be...@sy...> >>> m: 202.642.7961 <tel:202.642.7961> >>> f: 571.367.5000 <tel:571.367.5000> >>> w: www.blazegraph.com <http://www.blazegraph.com/> >>> >>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. >>> >>> CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. >>> >>> >> >> >> >> >> -- >> _______________ >> Brad Bebee >> CEO, Managing Partner >> SYSTAP, LLC >> e: be...@sy... <mailto:be...@sy...> >> m: 202.642.7961 <tel:202.642.7961> >> f: 571.367.5000 <tel:571.367.5000> >> w: www.blazegraph.com <http://www.blazegraph.com/> >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. >> >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... <mailto:Big...@li...> > https://lists.sourceforge.net/lists/listinfo/bigdata-developers <https://lists.sourceforge.net/lists/listinfo/bigdata-developers> > > |
|
From: Bryan T. <br...@sy...> - 2015-09-11 16:58:21
|
Ok. We do have a workaround for this. The problem is a deferred free that is not being properly released. This is identified by java.lang.RuntimeException: Problem with entry at -1982077125921865306* com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909) * This issue was first reported by the wiki data service and we developed a work around for them. The utility is available here: https://jira.blazegraph.com/browse/BLZG-1236?focusedCommentId=19105&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-19105 Please let us know if this resolves the issue. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Fri, Sep 11, 2015 at 12:43 PM, Joakim Soderberg < joa...@bl...> wrote: > It seems to be a problem with the same entry: > > *type* Exception report > > *message* *Problem with entry at -1982077125921865306: > lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, > nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September > 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, > September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, > September 10, 2015 11:11:10 PM UTC], commitCounter=53, > commitRecordAddr={off=NATIVE:-451550099,len=422}, > commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, > blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, > metaStartAddr=4614957, storeType=RW, > uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, > checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 > 2:10:45 AM UTC], closeTime=0}* > > *description* *The server encountered an internal error that prevented it > from fulfilling this request.* > > *exception* > > java.lang.RuntimeException: Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0} > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3126) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > > *root cause* > > java.lang.RuntimeException: Problem with entry at -1982077125921865306 > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909) > com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) > com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) > com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) > com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) > com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > > *root cause* > > java.lang.RuntimeException: addr=-472185800 : cause=java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot > com.bigdata.rwstore.RWStore.getData(RWStore.java:2191) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.getData(RWStore.java:2034) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895) > com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) > com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) > com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) > com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) > com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > > *root cause* > > java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot > com.bigdata.rwstore.RWStore.getData(RWStore.java:2083) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.getData(RWStore.java:2034) > com.bigdata.rwstore.RWStore.getData(RWStore.java:1990) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799) > com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895) > com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563) > com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781) > com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489) > com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291) > com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101) > com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124) > com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98) > com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) > com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349) > com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446) > com.blippar.action.AddAction.execute(AddAction.java:81) > com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93) > com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60) > javax.servlet.http.HttpServlet.service(HttpServlet.java:622) > javax.servlet.http.HttpServlet.service(HttpServlet.java:729) > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > > > > > On Sep 11, 2015, at 9:15 AM, Brad Bebee <be...@sy...> wrote: > > Joakim, > > Thank you. If you are able to reproduce it, please send as much of the > stack trace as you can. > > Thanks, --Brad > > On Fri, Sep 11, 2015 at 12:09 PM, Joakim Soderberg < > joa...@bl...> wrote: > >> Unfortunately not. I will try to reproduce the error. >> >> >> On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy...> wrote: >> >> Joakim, >> >> Can you send us the full stack trace to diagnose this? >> >> Thanks, --Brad >> >> On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg < >> joa...@bl...> wrote: >> >>> Hi >>> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB >>> RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by >>> adding to a RepositoryConnection >>> I get the following error: >>> >>> HTTP Status 500 - Problem with entry at -1982077125921865306: >>> lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, >>> nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September >>> 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, >>> September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, >>> September 10, 2015 11:11:10 PM UTC], commitCounter=53, >>> commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, >>> blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, >>> metaStartAddr=4614957, storeType=RW, >>> uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, >>> offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, >>> September 1, 2015 2:10:45 AM UTC], closeTime=0} >>> >>> After this I can’t add triples again, could the jnl-file be corrupt now? >>> Can it be repaired? >>> >>> /Joakim >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Bigdata-developers mailing list >>> Big...@li... >>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>> >>> >> >> >> -- >> _______________ >> Brad Bebee >> CEO, Managing Partner >> SYSTAP, LLC >> e: be...@sy... >> m: 202.642.7961 >> f: 571.367.5000 >> w: www.blazegraph.com >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. >> Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> >> > > > -- > _______________ > Brad Bebee > CEO, Managing Partner > SYSTAP, LLC > e: be...@sy... > m: 202.642.7961 > f: 571.367.5000 > w: www.blazegraph.com > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. > Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Joakim S. <joa...@bl...> - 2015-09-11 16:43:31
|
It seems to be a problem with the same entry:
type Exception report
message Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0}
description The server encountered an internal error that prevented it from fulfilling this request.
exception
java.lang.RuntimeException: Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0}
com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3126)
com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98)
com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349)
com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446)
com.blippar.action.AddAction.execute(AddAction.java:81)
com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93)
com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60)
javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
root cause
java.lang.RuntimeException: Problem with entry at -1982077125921865306
com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4909)
com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563)
com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781)
com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489)
com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291)
com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101)
com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124)
com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98)
com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349)
com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446)
com.blippar.action.AddAction.execute(AddAction.java:81)
com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93)
com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60)
javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
root cause
java.lang.RuntimeException: addr=-472185800 : cause=java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot
com.bigdata.rwstore.RWStore.getData(RWStore.java:2191)
com.bigdata.rwstore.RWStore.getData(RWStore.java:1990)
com.bigdata.rwstore.RWStore.getData(RWStore.java:2034)
com.bigdata.rwstore.RWStore.getData(RWStore.java:1990)
com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799)
com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895)
com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563)
com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781)
com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489)
com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291)
com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101)
com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124)
com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98)
com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349)
com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446)
com.blippar.action.AddAction.execute(AddAction.java:81)
com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93)
com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60)
javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
root cause
java.lang.IllegalStateException: Bad Address: length requested greater than allocated slot
com.bigdata.rwstore.RWStore.getData(RWStore.java:2083)
com.bigdata.rwstore.RWStore.getData(RWStore.java:1990)
com.bigdata.rwstore.RWStore.getData(RWStore.java:2034)
com.bigdata.rwstore.RWStore.getData(RWStore.java:1990)
com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4799)
com.bigdata.rwstore.RWStore.freeDeferrals(RWStore.java:4895)
com.bigdata.rwstore.RWStore.checkDeferredFrees(RWStore.java:3563)
com.bigdata.journal.RWStrategy.checkDeferredFrees(RWStrategy.java:781)
com.bigdata.journal.AbstractJournal$CommitState.writeCommitRecord(AbstractJournal.java:3489)
com.bigdata.journal.AbstractJournal$CommitState.access$2800(AbstractJournal.java:3291)
com.bigdata.journal.AbstractJournal.commitNow(AbstractJournal.java:4101)
com.bigdata.journal.AbstractJournal.commit(AbstractJournal.java:3124)
com.bigdata.rdf.store.LocalTripleStore.commit(LocalTripleStore.java:98)
com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3394)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330)
com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit(BigdataSailRepositoryConnection.java:349)
com.blippar.blazegraph.BlazegraphUtils.loadData(BlazegraphUtils.java:446)
com.blippar.action.AddAction.execute(AddAction.java:81)
com.blippar.servlet.MinervaServlet.doPost(MinervaServlet.java:93)
com.blippar.servlet.MinervaServlet.doGet(MinervaServlet.java:60)
javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> On Sep 11, 2015, at 9:15 AM, Brad Bebee <be...@sy...> wrote:
>
> Joakim,
>
> Thank you. If you are able to reproduce it, please send as much of the stack trace as you can.
>
> Thanks, --Brad
>
> On Fri, Sep 11, 2015 at 12:09 PM, Joakim Soderberg <joa...@bl... <mailto:joa...@bl...>> wrote:
> Unfortunately not. I will try to reproduce the error.
>
>
>
>> On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy... <mailto:be...@sy...>> wrote:
>>
>> Joakim,
>>
>> Can you send us the full stack trace to diagnose this?
>>
>> Thanks, --Brad
>>
>> On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg <joa...@bl... <mailto:joa...@bl...>> wrote:
>> Hi
>> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by adding to a RepositoryConnection
>> I get the following error:
>>
>> HTTP Status 500 - Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0}
>>
>> After this I can’t add triples again, could the jnl-file be corrupt now? Can it be repaired?
>>
>> /Joakim
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li... <mailto:Big...@li...>
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
>>
>>
>>
>>
>> --
>> _______________
>> Brad Bebee
>> CEO, Managing Partner
>> SYSTAP, LLC
>> e: be...@sy... <mailto:be...@sy...>
>> m: 202.642.7961 <tel:202.642.7961>
>> f: 571.367.5000 <tel:571.367.5000>
>> w: www.blazegraph.com <http://www.blazegraph.com/>
>>
>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics.
>>
>> CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments.
>>
>>
>
>
>
>
> --
> _______________
> Brad Bebee
> CEO, Managing Partner
> SYSTAP, LLC
> e: be...@sy... <mailto:be...@sy...>
> m: 202.642.7961
> f: 571.367.5000
> w: www.blazegraph.com <http://www.blazegraph.com/>
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments.
>
>
|
|
From: Bryan T. <br...@sy...> - 2015-09-11 16:31:21
|
Can you tell us a little more about what was happening? Was this all one big data load onto the Journal? Or were there a number of existing commits and a new write on the journal failed? Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Fri, Sep 11, 2015 at 12:09 PM, Joakim Soderberg < joa...@bl...> wrote: > Unfortunately not. I will try to reproduce the error. > > > On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy...> wrote: > > Joakim, > > Can you send us the full stack trace to diagnose this? > > Thanks, --Brad > > On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg < > joa...@bl...> wrote: > >> Hi >> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB >> RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by >> adding to a RepositoryConnection >> I get the following error: >> >> HTTP Status 500 - Problem with entry at -1982077125921865306: >> lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, >> nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September >> 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, >> September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, >> September 10, 2015 11:11:10 PM UTC], commitCounter=53, >> commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, >> blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, >> metaStartAddr=4614957, storeType=RW, >> uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, >> offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, >> September 1, 2015 2:10:45 AM UTC], closeTime=0} >> >> After this I can’t add triples again, could the jnl-file be corrupt now? >> Can it be repaired? >> >> /Joakim >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Bigdata-developers mailing list >> Big...@li... >> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >> >> > > > -- > _______________ > Brad Bebee > CEO, Managing Partner > SYSTAP, LLC > e: be...@sy... > m: 202.642.7961 > f: 571.367.5000 > w: www.blazegraph.com > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Brad B. <be...@sy...> - 2015-09-11 16:15:30
|
Joakim,
Thank you. If you are able to reproduce it, please send as much of the
stack trace as you can.
Thanks, --Brad
On Fri, Sep 11, 2015 at 12:09 PM, Joakim Soderberg <
joa...@bl...> wrote:
> Unfortunately not. I will try to reproduce the error.
>
>
> On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy...> wrote:
>
> Joakim,
>
> Can you send us the full stack trace to diagnose this?
>
> Thanks, --Brad
>
> On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg <
> joa...@bl...> wrote:
>
>> Hi
>> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB
>> RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by
>> adding to a RepositoryConnection
>> I get the following error:
>>
>> HTTP Status 500 - Problem with entry at -1982077125921865306:
>> lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3,
>> nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September
>> 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday,
>> September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday,
>> September 10, 2015 11:11:10 PM UTC], commitCounter=53,
>> commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220},
>> blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444,
>> metaStartAddr=4614957, storeType=RW,
>> uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138,
>> offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday,
>> September 1, 2015 2:10:45 AM UTC], closeTime=0}
>>
>> After this I can’t add triples again, could the jnl-file be corrupt now?
>> Can it be repaired?
>>
>> /Joakim
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>
>
> --
> _______________
> Brad Bebee
> CEO, Managing Partner
> SYSTAP, LLC
> e: be...@sy...
> m: 202.642.7961
> f: 571.367.5000
> w: www.blazegraph.com
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
>
>
--
_______________
Brad Bebee
CEO, Managing Partner
SYSTAP, LLC
e: be...@sy...
m: 202.642.7961
f: 571.367.5000
w: www.blazegraph.com
Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance graph
database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology
to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
|
|
From: Joakim S. <joa...@bl...> - 2015-09-11 16:10:06
|
Unfortunately not. I will try to reproduce the error.
> On Sep 11, 2015, at 5:23 AM, Brad Bebee <be...@sy...> wrote:
>
> Joakim,
>
> Can you send us the full stack trace to diagnose this?
>
> Thanks, --Brad
>
> On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg <joa...@bl... <mailto:joa...@bl...>> wrote:
> Hi
> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by adding to a RepositoryConnection
> I get the following error:
>
> HTTP Status 500 - Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0}
>
> After this I can’t add triples again, could the jnl-file be corrupt now? Can it be repaired?
>
> /Joakim
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li... <mailto:Big...@li...>
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
>
>
>
>
> --
> _______________
> Brad Bebee
> CEO, Managing Partner
> SYSTAP, LLC
> e: be...@sy... <mailto:be...@sy...>
> m: 202.642.7961
> f: 571.367.5000
> w: www.blazegraph.com <http://www.blazegraph.com/>
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments.
>
>
|
|
From: Brad B. <be...@sy...> - 2015-09-11 12:23:49
|
Joakim,
Can you send us the full stack trace to diagnose this?
Thanks, --Brad
On Thu, Sep 10, 2015 at 8:04 PM, Joakim Soderberg <
joa...@bl...> wrote:
> Hi
> I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB RAM.
> The jul-file is 280GB hosting 1.3 B triples. While adding triples by adding
> to a RepositoryConnection
> I get the following error:
>
> HTTP Status 500 - Problem with entry at -1982077125921865306:
> lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3,
> nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September
> 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday,
> September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday,
> September 10, 2015 11:11:10 PM UTC], commitCounter=53,
> commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220},
> blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444,
> metaStartAddr=4614957, storeType=RW,
> uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138,
> offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday,
> September 1, 2015 2:10:45 AM UTC], closeTime=0}
>
> After this I can’t add triples again, could the jnl-file be corrupt now?
> Can it be repaired?
>
> /Joakim
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
--
_______________
Brad Bebee
CEO, Managing Partner
SYSTAP, LLC
e: be...@sy...
m: 202.642.7961
f: 571.367.5000
w: www.blazegraph.com
Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance graph
database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new technology
to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
|
|
From: Joakim S. <joa...@bl...> - 2015-09-11 00:31:52
|
Hi
I am running Blazegraph in embedded mode on a CentOS 4-core with 60GB RAM. The jul-file is 280GB hosting 1.3 B triples. While adding triples by adding to a RepositoryConnection
I get the following error:
HTTP Status 500 - Problem with entry at -1982077125921865306: lastRootBlock=rootBlock{ rootBlock=1, challisField=53, version=3, nextOffset=18367917264487524, localTime=1441926674289 [Thursday, September 10, 2015 11:11:14 PM UTC], firstCommitTime=1441073445445 [Tuesday, September 1, 2015 2:10:45 AM UTC], lastCommitTime=1441926670929 [Thursday, September 10, 2015 11:11:10 PM UTC], commitCounter=53, commitRecordAddr={off=NATIVE:-451550099,len=422}, commitRecordIndexAddr={off=NATIVE:-468869758,len=220}, blockSequence=106022, quorumToken=-1, metaBitsAddr=13435602594695444, metaStartAddr=4614957, storeType=RW, uuid=d64bf2e9-599b-4f41-8bfe-f3b000f73138, offsetBits=42, checksum=371010502, createTime=1441073445016 [Tuesday, September 1, 2015 2:10:45 AM UTC], closeTime=0}
After this I can’t add triples again, could the jnl-file be corrupt now? Can it be repaired?
/Joakim |
|
From: Bryan T. <br...@sy...> - 2015-09-02 06:54:37
|
This is a case where increasing the number of RWStore write cache buffers should improve write performance by deferring the flush of index pages to the disk and allowing more efficient recycling during the operations. Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Mon, Aug 31, 2015 at 1:56 PM, Jeremy J Carroll <jj...@sy...> wrote: > We have had big problems dropping a large graph, I have just been looking > at the logs and we seem to have many entries like > > WARN : 199265281 com.bigdata.journal.Journal.executorService942 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 8048ms : > addrRoot=-1052669065178378143 > WARN : 199301423 com.bigdata.journal.Journal.executorService942 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 36141ms : > addrRoot=-1052669099538116489 > WARN : 199817295 com.bigdata.journal.Journal.executorService1068 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 1 records (#nodes=1, #leaves=0) in 21813ms : > addrRoot=-1052358706546595628 > WARN : 200111575 com.bigdata.journal.Journal.executorService18 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 5953ms : > addrRoot=-1052338112178413386 > WARN : 200197937 com.bigdata.journal.Journal.executorService1070 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 1 records (#nodes=1, #leaves=0) in 10652ms : > addrRoot=-1053867623046964134 > WARN : 200763920 com.bigdata.journal.Journal.executorService942 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 6214ms : > addrRoot=-1052630260148849987 > > can someone offer an interpretation? > > thanks > > Jeremy > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Bryan T. <br...@sy...> - 2015-08-31 19:36:08
|
That is IO Wait (or sometimes GC) associated with flushing high level nodes in the B+Tree to the disk. Given the data I see above, it looks like IO Wait was probably spiking. Are you running against local SSD? If you are on EC2, have you exceeded your write rate throughput for the instances? There are faster ways to drop an entire triple store or quad store. But dropping a large named graph out of a set of named graphs in a quad store will incur a lot of IO. Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Mon, Aug 31, 2015 at 1:56 PM, Jeremy J Carroll <jj...@sy...> wrote: > We have had big problems dropping a large graph, I have just been looking > at the logs and we seem to have many entries like > > WARN : 199265281 com.bigdata.journal.Journal.executorService942 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 8048ms : > addrRoot=-1052669065178378143 > WARN : 199301423 com.bigdata.journal.Journal.executorService942 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 36141ms : > addrRoot=-1052669099538116489 > WARN : 199817295 com.bigdata.journal.Journal.executorService1068 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 1 records (#nodes=1, #leaves=0) in 21813ms : > addrRoot=-1052358706546595628 > WARN : 200111575 com.bigdata.journal.Journal.executorService18 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 5953ms : > addrRoot=-1052338112178413386 > WARN : 200197937 com.bigdata.journal.Journal.executorService1070 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 1 records (#nodes=1, #leaves=0) in 10652ms : > addrRoot=-1053867623046964134 > WARN : 200763920 com.bigdata.journal.Journal.executorService942 > com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): > wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 6214ms : > addrRoot=-1052630260148849987 > > can someone offer an interpretation? > > thanks > > Jeremy > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Jeremy J C. <jj...@sy...> - 2015-08-31 17:56:27
|
We have had big problems dropping a large graph, I have just been looking at the logs and we seem to have many entries like WARN : 199265281 com.bigdata.journal.Journal.executorService942 com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 8048ms : addrRoot=-1052669065178378143 WARN : 199301423 com.bigdata.journal.Journal.executorService942 com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 36141ms : addrRoot=-1052669099538116489 WARN : 199817295 com.bigdata.journal.Journal.executorService1068 com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): wrote: name=.spo.OCSP, 1 records (#nodes=1, #leaves=0) in 21813ms : addrRoot=-1052358706546595628 WARN : 200111575 com.bigdata.journal.Journal.executorService18 com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 5953ms : addrRoot=-1052338112178413386 WARN : 200197937 com.bigdata.journal.Journal.executorService1070 com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): wrote: name=.spo.OCSP, 1 records (#nodes=1, #leaves=0) in 10652ms : addrRoot=-1053867623046964134 WARN : 200763920 com.bigdata.journal.Journal.executorService942 com.bigdata.btree.AbstractBTree.writeNodeRecursive(AbstractBTree.java:3716): wrote: name=.spo.OCSP, 2 records (#nodes=1, #leaves=1) in 6214ms : addrRoot=-1052630260148849987 can someone offer an interpretation? thanks Jeremy |
|
From: Jeremy J C. <jj...@sy...> - 2015-08-28 18:42:31
|
Hi we want to be able to compute people’s ages from their birth date. Does anyone have a custom function implementing https://docs.marklogic.com/fn:subtract-dateTimes-yielding-yearMonthDuration which seems to be a proprietary variant of http://www.w3.org/TR/xpath-functions/#func-subtract-dateTimes <http://www.w3.org/TR/xpath-functions/#func-subtract-dateTimes> (alternatively we believe there is a SPARQL 1.1 incantation, but quite a long one, that would compute this … if someone knows the chant, please share) Jeremy |
|
From: Bryan T. <br...@sy...> - 2015-08-28 16:45:06
|
Why not drop the namespace? ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Fri, Aug 28, 2015 at 12:04 PM, Jeremy J Carroll <jj...@sy...> wrote: > One large graph in quads mode, sounds like we lose. > > Jeremy > > > > On Aug 28, 2015, at 8:34 AM, Bryan Thompson <br...@sy...> wrote: > > It only impacts DELETE/INSERT + WHERE. > > DROP GRAPH for quads needs to write on the indices to remove all edges in > the graph from all 6 quads indices. So it would be close to the cost of > writing 1B quads. > > DROP ALL could be optimized. The minimum work to drop an index is to > visit all nodes so the allocation slots of the children can be freed. > BTree.removeAll() does implement that optimization. > > Are you trying to drop all quads or just one large graph? > > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com/> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. Blazegraph is now available with GPU acceleration using our disruptive > technology to accelerate data-parallel graph analytics and graph query. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Fri, Aug 28, 2015 at 11:15 AM, Jeremy J Carroll <jj...@sy...> wrote: > >> Sorry for the delayed response, I have been on paternity leave. >> >> I take it that this impacts usage via NSS, and the fix is likely in the >> next blazegraph release? >> >> We believe we have seen this performance issue. >> >> May this also impact e.g. DROP GRAPH ? >> (We have been having surprisingly slow performance DROPping a billion >> triple graph, via NSS) >> >> Jeremy >> >> >> >> On Aug 7, 2015, at 6:09 AM, Bryan Thompson <br...@sy...> wrote: >> >> We found a bug in the openrdf library that is having a very strong >> negative impact on SPARQL UPDATE performance for larger UPDATE sets. The >> root cause is MultipleTupleQueryResult using LinkedList.get(index), which >> is a linear scan, for next(). So the iterator performance falls off >> linearly as the scan progresses. >> >> For the impatient, there is a very simple fix: >> >> 1. Clone the MutableTupleQueryResult class in openrdf into a new >> namespace in blazegraph. >> 2. Replace LinkedList with ArrayList (one line change); >> 3. Import that modified version of the class in our AST2BOpUpdate class >> (one line change). >> >> Michael is testing the performance impact of that fix now. >> >> See https://jira.blazegraph.com/browse/BLZG-1404 >> >> Thanks, >> Bryan >> >> >> > > |
|
From: Jeremy J C. <jj...@sy...> - 2015-08-28 16:05:07
|
One large graph in quads mode, sounds like we lose. Jeremy > On Aug 28, 2015, at 8:34 AM, Bryan Thompson <br...@sy...> wrote: > > It only impacts DELETE/INSERT + WHERE. > > DROP GRAPH for quads needs to write on the indices to remove all edges in the graph from all 6 quads indices. So it would be close to the cost of writing 1B quads. > > DROP ALL could be optimized. The minimum work to drop an index is to visit all nodes so the allocation slots of the children can be freed. BTree.removeAll() does implement that optimization. > > Are you trying to drop all quads or just one large graph? > > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... <mailto:br...@sy...> > http://blazegraph.com <http://blazegraph.com/> > http://blog.bigdata.com <http://bigdata.com/> > http://mapgraph.io <http://mapgraph.io/> > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. > > > > On Fri, Aug 28, 2015 at 11:15 AM, Jeremy J Carroll <jj...@sy... <mailto:jj...@sy...>> wrote: > Sorry for the delayed response, I have been on paternity leave. > > I take it that this impacts usage via NSS, and the fix is likely in the next blazegraph release? > > We believe we have seen this performance issue. > > May this also impact e.g. DROP GRAPH ? > (We have been having surprisingly slow performance DROPping a billion triple graph, via NSS) > > Jeremy > > > >> On Aug 7, 2015, at 6:09 AM, Bryan Thompson <br...@sy... <mailto:br...@sy...>> wrote: >> >> We found a bug in the openrdf library that is having a very strong negative impact on SPARQL UPDATE performance for larger UPDATE sets. The root cause is MultipleTupleQueryResult using LinkedList.get(index), which is a linear scan, for next(). So the iterator performance falls off linearly as the scan progresses. >> >> For the impatient, there is a very simple fix: >> >> 1. Clone the MutableTupleQueryResult class in openrdf into a new namespace in blazegraph. >> 2. Replace LinkedList with ArrayList (one line change); >> 3. Import that modified version of the class in our AST2BOpUpdate class (one line change). >> >> Michael is testing the performance impact of that fix now. >> >> See https://jira.blazegraph.com/browse/BLZG-1404 <https://jira.blazegraph.com/browse/BLZG-1404> >> >> Thanks, >> Bryan >> > > |
|
From: Jeremy J C. <jj...@sy...> - 2015-08-28 15:44:53
|
Sorry for the delayed response, I have been on paternity leave. I take it that this impacts usage via NSS, and the fix is likely in the next blazegraph release? We believe we have seen this performance issue. May this also impact e.g. DROP GRAPH ? (We have been having surprisingly slow performance DROPping a billion triple graph, via NSS) Jeremy > On Aug 7, 2015, at 6:09 AM, Bryan Thompson <br...@sy...> wrote: > > We found a bug in the openrdf library that is having a very strong negative impact on SPARQL UPDATE performance for larger UPDATE sets. The root cause is MultipleTupleQueryResult using LinkedList.get(index), which is a linear scan, for next(). So the iterator performance falls off linearly as the scan progresses. > > For the impatient, there is a very simple fix: > > 1. Clone the MutableTupleQueryResult class in openrdf into a new namespace in blazegraph. > 2. Replace LinkedList with ArrayList (one line change); > 3. Import that modified version of the class in our AST2BOpUpdate class (one line change). > > Michael is testing the performance impact of that fix now. > > See https://jira.blazegraph.com/browse/BLZG-1404 <https://jira.blazegraph.com/browse/BLZG-1404> > > Thanks, > Bryan > |
|
From: Bryan T. <br...@sy...> - 2015-08-28 15:34:35
|
It only impacts DELETE/INSERT + WHERE. DROP GRAPH for quads needs to write on the indices to remove all edges in the graph from all 6 quads indices. So it would be close to the cost of writing 1B quads. DROP ALL could be optimized. The minimum work to drop an index is to visit all nodes so the allocation slots of the children can be freed. BTree.removeAll() does implement that optimization. Are you trying to drop all quads or just one large graph? Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Fri, Aug 28, 2015 at 11:15 AM, Jeremy J Carroll <jj...@sy...> wrote: > Sorry for the delayed response, I have been on paternity leave. > > I take it that this impacts usage via NSS, and the fix is likely in the > next blazegraph release? > > We believe we have seen this performance issue. > > May this also impact e.g. DROP GRAPH ? > (We have been having surprisingly slow performance DROPping a billion > triple graph, via NSS) > > Jeremy > > > > On Aug 7, 2015, at 6:09 AM, Bryan Thompson <br...@sy...> wrote: > > We found a bug in the openrdf library that is having a very strong > negative impact on SPARQL UPDATE performance for larger UPDATE sets. The > root cause is MultipleTupleQueryResult using LinkedList.get(index), which > is a linear scan, for next(). So the iterator performance falls off > linearly as the scan progresses. > > For the impatient, there is a very simple fix: > > 1. Clone the MutableTupleQueryResult class in openrdf into a new namespace > in blazegraph. > 2. Replace LinkedList with ArrayList (one line change); > 3. Import that modified version of the class in our AST2BOpUpdate class > (one line change). > > Michael is testing the performance impact of that fix now. > > See https://jira.blazegraph.com/browse/BLZG-1404 > > Thanks, > Bryan > > > |
|
From: Martynas J. <mar...@gr...> - 2015-08-25 07:45:44
|
Probably unrelated, but this looks like an odd pattern since foaf:Person is a class and not a property: ?s <http://xmlns.com/foaf/0.1/Person> ?l On Tue, Aug 25, 2015 at 1:46 AM, Mike Personick <mi...@sy...> wrote: > What is this triple pattern for? > > ?s ?someprop ?l . > > Why not just formulate as: > > SELECT ?s ?l WHERE { > { > ?s <http://www.w3.org/2000/01/rdf-schema#label> ?l > } UNION { > ?s <http://xmlns.com/foaf/0.1/name> ?l > } UNION { > ?s <http://xmlns.com/foaf/0.1/Person> ?l > } > FILTER regex(?l, 'kill', 'i'). > } > LIMIT 20 > > On Mon, Aug 24, 2015 at 2:47 PM, Joakim Soderberg > <joa...@bl...> wrote: >> >> >> If I execute the following SPARQL query: >> >> SELECT ?s ?l WHERE { ?s ?someprop ?l . { ?s >> <http://www.w3.org/2000/01/rdf-schema#label> ?l } . FILTER >> regex(?l,'kill', 'i'). } LIMIT 20 >> I retreive the following expected result: >> >> Binding http://data.linkedmdb.org/resource/film/4115 >> "Dressed to Kill” >> >> Binding http://data.linkedmdb.org/resource/film/77289 >> "Killing Pablo" >> >> Binding http://data.linkedmdb.org/resource/film/928 >> "The Killer” >> >> etc... >> >> But if I combine with UNION >> >> SELECT ?s ?l WHERE { ?s ?someprop ?l . { ?s >> <http://www.w3.org/2000/01/rdf-schema#label> ?l } UNION { ?s >> <http://xmlns.com/foaf/0.1/name> ?l } UNION { ?s >> <http://xmlns.com/foaf/0.1/Person> ?l } . FILTER regex(?l, 'kill', 'i'). } >> LIMIT 20 >> >> I get the unexpected result: >> >> Bindin >> ghttp://musicbrainz.org/artist/00d992b5-d6d1-4130-971b-ab571593f94a#_ >> Binding"'Sloppy' Henry" >> >> Binding >> http://musicbrainz.org/artist/025a2a2e-39f8-4bea-a175-2efbfb6016ec#_ >> "#/Tau" >> >> Binding >> http://musicbrainz.org/artist/02bec9dc-18f4-46af-b18a-a156b8a0583c#_ >> "'Ale'a" >> >> Binding >> http://musicbrainz.org/artist/03bef19b-535e-4b8a-a882-01fcb7324386#_ >> [Alexandros]" >> >> Binding >> http://musicbrainz.org/artist/0bb8cd19-f05e-44c4-8955-4cc810de33d3#_ >> "-Scaldis-" >> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Bigdata-developers mailing list >> Big...@li... >> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > |
|
From: Mike P. <mi...@sy...> - 2015-08-25 00:11:40
|
What is this triple pattern for?
?s ?someprop ?l .
Why not just formulate as:
SELECT ?s ?l WHERE {
{
?s <http://www.w3.org/2000/01/rdf-schema#label> ?l
} UNION {
?s <http://xmlns.com/foaf/0.1/name> ?l
} UNION {
?s <http://xmlns.com/foaf/0.1/Person> ?l
}
FILTER regex(?l, 'kill', 'i').
}
LIMIT 20
On Mon, Aug 24, 2015 at 2:47 PM, Joakim Soderberg <
joa...@bl...> wrote:
>
> If I execute the following SPARQL query:
>
> SELECT ?s ?l WHERE { ?s ?someprop ?l . { ?s <
> http://www.w3.org/2000/01/rdf-schema#label> ?l } . FILTER
> regex(?l,'kill', 'i'). } LIMIT 20
> I retreive the following expected result:
>
> Binding http://data.linkedmdb.org/resource/film/4115
> "Dressed to Kill”
>
> Binding http://data.linkedmdb.org/resource/film/77289
> "Killing Pablo"
>
> Binding http://data.linkedmdb.org/resource/film/928
> "The Killer”
>
> etc...
>
> But if I combine with UNION
>
> SELECT ?s ?l WHERE { ?s ?someprop ?l . { ?s <
> http://www.w3.org/2000/01/rdf-schema#label> ?l } UNION { ?s <
> http://xmlns.com/foaf/0.1/name> ?l } UNION { ?s <
> http://xmlns.com/foaf/0.1/Person> ?l } . FILTER regex(?l, 'kill', 'i'). }
> LIMIT 20
>
> I get the unexpected result:
>
> Bindin
> ghttp://musicbrainz.org/artist/00d992b5-d6d1-4130-971b-ab571593f94a#_
> Binding"'Sloppy' Henry"
>
> Binding
> http://musicbrainz.org/artist/025a2a2e-39f8-4bea-a175-2efbfb6016ec#_
> "#/Tau"
>
> Binding
> http://musicbrainz.org/artist/02bec9dc-18f4-46af-b18a-a156b8a0583c#_
> "'Ale'a"
>
> Binding
> http://musicbrainz.org/artist/03bef19b-535e-4b8a-a882-01fcb7324386#_
> [Alexandros]"
>
> Binding
> http://musicbrainz.org/artist/0bb8cd19-f05e-44c4-8955-4cc810de33d3#_
> "-Scaldis-"
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
|
27 messages has been excluded from this view by a project administrator.
