Activity for Blazegraph (powered by bigdata)
-
Tyler Coy posted a comment on discussion Help
So, looking at my current logs, I'm seeing 16 - 32 second delays from sending the request to getting a response. The data is loading very slowly (2 - 5 docs/sec) with 200 threads loading data. Any idea why this might be? Any settings to make it use more of the resources? The load average is currently 1.53 and that is mostly from PostgreSQL which is running smoothly.
-
The current logs I'm looking at don't report a 30 second load time. I'm investigating further to determine the difference between the current logs and the logs that did report a 30 second load time. I'm thinking that I might have been looking at the wrong logs after updating the branching factors and a couple of other metrics. Will report back with what I find.
-
I have bigdata running on a high performance computer and am not seeing the resource usage I expect. I'm running a program to load RDF documents to the store. When I have just a few threads running, say 6, the store keeps up. However, when I increase to 30, 50, 400 the store begins to have trouble keeping up. At 400 threads there is a 30 second delay between the call to insert data to bigdata and the response. During this time, the load average never exceeds 8 and the RAM is around 30 or 40 GB. This...
-
Rosario Uceda modified a comment on discussion Help
-
Hi Brad, thanks for answering, I was using java -server -Xmx4g -jar blazegraph.jar & as recommended. I've been hacking since, trying this and that. In the end, I brought the .rpm file and installed with yum using root privileges $ yum install blazegraph.rpm now I start it as: $blazegraph start which translates to the command above. I finally got it working. I hope the installation is stable. I've been working with Blazegraph for a while now, and I've never had these problems.
-
Hi Brad, thanks for answering, I was using java -server -Xmx4g -jar blazegraph.jar & as recommended. I've been hacking since, trying this and that. In the end, I brought the .rpm file and installed with yum using root privileges $ yum install blazegraph.rpm now I start it as: $blazegraph start which translates to the command above. I finally got it working. I hope the installation is stable. I've been working with Blazegraph for a while now, and I've never had these problems.
-
Brad Bebee posted a comment on discussion Help
What command are you using to start the server with the Jar file? https://github.com/blazegraph/database/wiki/Quick_Start On Thu, Aug 20, 2020 at 11:57 AM Rosario Uceda rosariou@users.sourceforge.net wrote: BTW, I now installed Java 9 in the server, as the documentation said that it was required. Didn't fix the problem. Blazegraph.jar runtime exception. Doesn't start [Blazegraph] https://sourceforge.net/p/bigdata/discussion/676946/thread/50b0439949/?limit=25#f8cd Sent from sourceforge.net because...
-
BTW, I now installed Java 9 in the server, as the documentation said that it was required. Didn't fix the problem.
-
I just downloaded the latest blazegraph.jar but get the error below. The server is RedHat 7.7 and the current version of java is also below. I upgraded jetty recently, but I thought that the .jar would be self-contained (maybe I'm wrong?) Thanks in advance, $ java -version openjdk version "1.8.0_262" OpenJDK Runtime Environment (build 1.8.0_262-b10) OpenJDK 64-Bit Server VM (build 25.262-b10, mixed mode) --- ERROR MESSAGE ---- WARN : NanoSparqlServer.java:517: Starting NSS WARN : WebAppContext.java:554:...
-
birk posted a comment on discussion Help
@thompsonbry Thanks!
-
Bryan Thompson posted a comment on discussion Help
There is a stopword list. Stop words are highly common words with little semantic value. They are not indexed by search engines. On Mon, Jun 22, 2020 at 01:16 Birk Weiberg birk@users.sourceforge.net wrote: The problem also accoures with other prepositions like "in", "on", "for". reserved words in full text search? https://sourceforge.net/p/bigdata/discussion/676946/thread/f736f344b1/?limit=25#6534 Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bigdata/discussion/676946/...
-
The problem also accoures with other prepositions like "in", "on" and "for".
-
The problem also accoures with other prepositions like "in", "on", "for".
-
Hi! Are there any reserved keywords in the full text search? I encountered the problem that any search containing the word "of" brings no results. ?name bds:search "art of emotions" ; bds:minRelevance "0.1" ; bds:relevance ?score ; bds:matchAllTerms "true" . There is an exact match "Art of Emotions" in the data. Searching for "art emotions" or "emotions art" works. But anything that includes "of" brings no results.
-
Leandro Tabares Martín posted a comment on discussion Help
Hi I imported the Wikidata dataset to Blazegraph using a remote server. I downloaded the .jnl file to my local computer and I'm trying to serve it using a local Blazegraph instance. However, when I run any query (e.g. select * where {?s ?p ?o }) I get no results. What should I do in order to Blazegraph serve the data contained in the .jnl file? Leandro
-
I'm loading the whole wikidata dataset into Blazegraph using a High Performance Computer. I gave 120 GB RAM and 3 processing cores to the job. After almost 24 hours of load the "wikidata.jnl" file has only 28 GB as size. Initially the process was fast, but as the file increased its size the loading speed has decreased. I realize that only 14 GB of RAM are being used. I already implemented the recomendations given in https://github.com/blazegraph/database/wiki/IOOptimization Do you have some recommendation...
-
Blazegraph (powered by bigdata) released /bigdata/README.txt
-
-
-
That would do it! Bryan On Thu, Aug 22, 2019 at 8:43 AM Joe O'Connor joeoc21@users.sourceforge.net wrote: I had seen that in fact, but similarly my override-xml file did nothing. I managed to solve the problem by decompressing the blazegraph-service.war file, editing the web.xml file directly, and recompressing. Timeout after exactly 10 minutes [Blazegraph Wikidata] Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bigdata/discussion/676946/ To unsubscribe from...
-
Joe O'Connor posted a comment on discussion Help
I had seen that in fact, but similarly my override-xml file did nothing. I managed to solve the problem by decompressing the blazegraph-service.war file, editing the web.xml file directly, and recompressing.
-
Did you look at this part of the post? It describes where to create the overrides for the timeouts. https://sourceforge.net/p/bigdata/discussion/676946/thread/d3abb9bf12/#feb4/05e9 A lot of the post is concerned with whether the instance is setup correctly (heap, G1, JVM version, etc.) and what is taking the time. You should also look at that. Sometimes a query has a bad plan which simple takes too long. Sometimes it places too much pressure on the JVM heap. Sometimes it runs out of native memory....
-
I am trying to run a big query on a Wikidata instance, but the query fails everytime after exactly 10 minutes. The query and error output are included in the attached image. The only parameters that I have changed are the java and native heap sizes (both 64g) and the analyticMaxMemoryPerQuery, which I have set to 0 so that it is unbounded. I have not edited any timeout parameters. Is there such a parameter that defaults to timing out after 10 minutes? If so, how can I change it? And if not, how else...
-
I think that is a great question for the W3C SPARQL mailing list.... We are performing reorderings which are certainly legal for the case where there is no inference. In terms of entailment, our approach certainly assumes that the composition of the BGPs does not change the entailments. But I think this is true for the RDFS+ entailments which Blazegraph supports. Bryan On Tue, Jun 4, 2019 at 7:49 AM Median median89@users.sourceforge.net wrote: sparql 1.1 specification. Section 18.7. https://www.w3.org/TR/sparql11-query/#sparqlBGPExtend...
-
Median posted a comment on discussion Help
sparql 1.1 specification. Section 18.7. https://www.w3.org/TR/sparql11-query/#sparqlBGPExtend
-
Can you point to that in the spec? On Tue, Jun 4, 2019 at 02:25 Median median89@users.sourceforge.net wrote: Dear Bryan, thaks for the info. A question comes to my mind: Could SPARQL entailment give rise different results of the aforementioned method of optimization? As entailment is applied to the BGP level, not the join level (according to SPARQL 1.1 specification). Meaning that in query1, the entailment will be applied to answer BGP(TP1), then to answer BGP(TP2), then the results will be joined....
-
Dear Bryan, thaks for the info. A question comes to my mind: Could SPARQL entailment give rise different results of the aforementioned method of optimization? As entailment is applied to the BGP level, not the join level (according to SPARQL 1.1 specification). Meaning that in query1, the entailment will be applied to answer BGP(TP1), then to answer BGP(TP2), then the results will be joined. However, for query2, entailment will be applied to answer BGP(TP1, TP2) at once. Can this produce any difference...
-
Copying Michael. We do rewrite these patterns quite a bit. In the case you are describing, it looks like we are generating a left-to-right evaluation plan with pipelined join operators. We also have an optimization when there is an OPTIONAL with a single triple pattern. Blazegraph does tend to emphasize non-blocking (pipelined) plans when they correspond with the SPARQL bottom up semantics, primarily to obtain the benefit in terms of time to the first result and to keep down the memory associated...
-
Using BlazerGraph at Wikidata SPARQL endpoint, it seems that it always substitutes group graph pattern (joins of BGPs) by a single BGP, when possible, for optimization reasons. May I ask about the reason behind that? Or under which circumstances this can be applied? May SPARQL entailment affect the results? To calrify this, consider the following SPARQL patterns, which are almost identical, but the first is a single BGP, while the second is the join of two (group graph patterns each composed of)...
-
Yes On Fri, Apr 19, 2019 at 10:21 heendung heendung@users.sourceforge.net wrote: Hi Bryan, I have a quick question abut meaning of the value in goodSet. For example, if I have 3 files(i.e. A.ttl.gz, B.ttl.gz, C.ttl.gz) and the output is {failSet=0,goodSet=3}, does that mean those 3 files were succesfully loaded? Thanks! Time to load Wikidata dump into Blazegraph https://sourceforge.net/p/bigdata/discussion/676946/thread/3e98f2b97e/?limit=25#e369 Sent from sourceforge.net because you indicated interest...
-
heendung posted a comment on discussion Help
Hi Bryan, I have a quick question abut meaning of the value in goodSet. For example, if I have 3 files(i.e. A.ttl.gz, B.ttl.gz, C.ttl.gz) and the output is {failSet=0,goodSet=3}, does that mean those 3 files were succesfully loaded? Thanks!
-
just FYI- I followed the steps on the following guideline: https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md , but currently have the above load time issue.
-
just FYI- I followed the steps on the following guideline: https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md
-
Stas, can you comment on the configuration? Maybe post what wikidata uses and the load times? I will note that HDD vs SDD is a huge difference. They are much slower. On Tue, Apr 16, 2019 at 15:19 heendung heendung@users.sourceforge.net wrote: I am loading Wikidata dump files into Blazegraph, but it seems that the loading takes a lot of times: loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs,...
-
I am loading Wikidata dump files into Blazegraph, but it seems that the loading takes a lot of times: loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs, rate= 3050, commitLatency=0ms, {failSet=0,goodSet=111} loading: 883139354 stmts added in 297554.12 secs, rate= 2967, commitLatency=0ms, {failSet=0,goodSet=112} loading: 903383424 stmts added in 315156.664 secs, rate= 2866, commitLatency=0ms,...
-
I am loading Wikidata dump files into Blazegraph, but it seems that the loading takes a lot of times: loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs, rate= 3050, commitLatency=0ms, {failSet=0,goodSet=111} loading: 883139354 stmts added in 297554.12 secs, rate= 2967, commitLatency=0ms, {failSet=0,goodSet=112} loading: 903383424 stmts added in 315156.664 secs, rate= 2866, commitLatency=0ms,...
-
-
Thanks for the response. By the way, I am using HDD, and it seems that loading wikidata dump into blazegraph takes a lot of times(the rate keeps decreasing): loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs, rate= 3050, commitLatency=0ms, {failSet=0,goodSet=111} loading: 883139354 stmts added in 297554.12 secs, rate= 2967, commitLatency=0ms, {failSet=0,goodSet=112} loading: 903383424 stmts added...
-
Thanks for the response. By the way, I am using HDD, and it seems that loading wikidata dump into blazegraph takes a lot of times(the rate keeps decreasing): loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs, rate= 3050, commitLatency=0ms, {failSet=0,goodSet=111} loading: 883139354 stmts added in 297554.12 secs, rate= 2967, commitLatency=0ms, {failSet=0,goodSet=112} loading: 903383424 stmts added...
-
Thanks for the response. By the way, I am using HDD, and it seems that loading wikidata dump into blazegraph takes a lot of times(the rate keeps decreasing): loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs, rate= 3050, commitLatency=0ms, {failSet=0,goodSet=111} loading: 883139354 stmts added in 297554.12 secs, rate= 2967, commitLatency=0ms, {failSet=0,goodSet=112} loading: 903383424 stmts added...
-
Thanks for the response. By the way, I am using HDD, and it seems that loading wikidata dump into blazegraph takes a lot of times(the rate keeps decreasing): loading: 847529288 stmts added in 271240.911 secs, rate= 3124, commitLatency=0ms, {failSet=0,goodSet=110} loading: 868123530 stmts added in 284586.467 secs, rate= 3050, commitLatency=0ms, {failSet=0,goodSet=111} loading: 883139354 stmts added in 297554.12 secs, rate= 2967, commitLatency=0ms, {failSet=0,goodSet=112} loading: 903383424 stmts added...
-
Yes. See https://github.com/blazegraph/database/blob/master/bigdata-core/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/DataLoaderServlet.java#L308 At https://github.com/blazegraph/database/blob/master/bigdata-core/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/DataLoaderServlet.java#L323 it is building the response for the http client. However, https://github.com/blazegraph/database/blob/master/bigdata-core/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/DataLoaderServlet.java#L310 is...
-
I tried to load wikidata dump into blazegraph using the following command: ./loadRestAPI.sh -n wdq -d pwd/data/split/wikidump-000000020.ttl.gz However, I had the following error: Reading properties: /data/kil/wikidata-query-rdf/dist/target/service-0.3.2-SNAPSHOT/RWStore.properties 22:12:13.584 [qtp1190900417-2632] ERROR c.b.r.sail.webapp.BigdataRDFServlet IP:localhost UA:curl/7.47.0 - cause=java.util.concurrent.TimeoutException, query=DATALOADER-SERVLET: wdq 2019-04-11 22:12:13.588:WARN:oejs.HttpChannel:qtp1190900417-2632:...
-
I am not sure how the data is being loaded. If it is running as a single transaction then you will not see any data until that transaction commits., transactional isolation. On Mon, Apr 8, 2019 at 16:06 S Vista vistas@users.sourceforge.net wrote: Hi everyone, I'm new to blazegraph and sparql, sorry if the question is trivial. I have installed Blazegraph (followed instructions here https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual#Standalone_service ) on a Google Cloud (GC) instance....
-
S Vista posted a comment on discussion Help
I'm by no means an expert on this. However, I had a similar (but less sever) problem. I'm suspecting this is because your disk is slow. I run my server on the cloud and the machine I was using ran on standard disk, and it was slow. Changed to SSD and its twice faster. To confirm if this is the case check your loading rate and post it here for the top 10 iterations. Perhaps we can compare notes and confirm that this is the case.
-
Hi everyone, I'm new to blazegraph and sparql, sorry if the question is trivial. I have installed Blazegraph (followed instructions here ) on a Google Cloud (GC) instance. The installation went through without any problems (I'm using the default configuration, nothing is changed on my side). I also have ran runBlazegraph.sh successfully followed by an equally successful running of the munge.sh and loadRestAPI.sh scripts. All went through without errors (the loading is still underway though I'm not...
-
-
-
-
-
-
-
-
-
SRINIVASAN VENKATRAMAN posted a comment on discussion Help
Thanks Stas. I'm not sure if I am doing something wrong here. It has been nearly 48 hours with only 10GB data loaded. I'm running the below script. ./dataLoader.sh -verbose -namespace kb -defaultGraph http://www.wikidata.org /opt/wikidata/RWStore.properties ~/latest-all.nt.gz The property file has the following for full feature mode. com.bigdata.journal.AbstractJournal.file=/opt/wikidata/blazegraph.jnl com.bigdata.journal.AbstractJournal.bufferMode=DiskRW com.bigdata.service.AbstractTransactionService.minReleaseAge=1...
-
Stas Malyshev posted a comment on discussion Help
Hi! I'm loading the latest-all.nt.gz file into Blazegraph with full feature mode. The loader has been running for nearly 20 hours now and the journal file is just over 6GB. Is this normal. Does anyone have experience loading the entire dataset? How long does it usually take and how much disk space is needed? 6G after 20 hours looks a bit low. In general full dump probably would take about 500-600G to load and takes several days to complete for us. -- Stas Malyshev smalyshev@wikimedia.org
-
That is most properly a question for the wikidata people. Adding Stas to the Cc. Bryan On Fri, Mar 8, 2019 at 3:11 AM SRINIVASAN VENKATRAMAN srini6teen@users.sourceforge.net wrote: I'm loading the latest-all.nt.gz file into Blazegraph with full feature mode. The loader has been running for nearly 20 hours now and the journal file is just over 6GB. Is this normal. Does anyone have experience loading the entire dataset? How long does it usually take and how much disk space is needed? Loading the wikidata...
-
I'm loading the latest-all.nt.gz file into Blazegraph with full feature mode. The loader has been running for nearly 20 hours now and the journal file is just over 6GB. Is this normal. Does anyone have experience loading the entire dataset? How long does it usually take and how much disk space is needed?
-
Will be a new release shortly On Thu, Mar 7, 2019 at 08:39 Anton Kulaga antonkulaga@users.sourceforge.net wrote: Is Blazegraph still alive? It has not been updated for ages Sincerely, Anton Kulaga Bioinformatician at Computational Biology of Aging Group 296 Splaiul Independentei, Bucharest, Romania https://maps.google.com/?q=296+Splaiul+Independentei,+Bucharest,+Romania&entry=gmail&source=g, 060031 http://aging-research.group On Thu, 7 Mar 2019 at 17:51, Bryan Thompson thompsonbry@users.sourceforge.net...
-
Anton Kulaga posted a comment on discussion Help
Is Blazegraph still alive? It has not been updated for ages Sincerely, Anton Kulaga Bioinformatician at Computational Biology of Aging Group 296 Splaiul Independentei, Bucharest, Romania, 060031 http://aging-research.group On Thu, 7 Mar 2019 at 17:51, Bryan Thompson thompsonbry@users.sourceforge.net wrote: Blazegraph can host multiple triple or quad stores. The "namespace" refers to the name of a given triple or quad store. The namespace becomes part of the request URL for talking to the SPARQL end...
-
Blazegraph can host multiple triple or quad stores. The "namespace" refers to the name of a given triple or quad store. The namespace becomes part of the request URL for talking to the SPARQL end point. The defaultGraph argument in that case is specifying the name of the named graph to use for triples which do not have a named graph. For quads, the named graph for the triple is always specified. See https://wiki.blazegraph.com/wiki/index.php/REST_API for the REST API for Blazegraph. The multi-tenancy...
-
I am new to Blazegraph and trying to understand the namespace and defaultGraph parameters while loading the data using bulk load api. I loaded the file latest-all.nt.gz using the below command. ./dataLoader.sh -namespace wikidata -defaultGraph http://www.wikidata.org ../bigdata-war-html/src/main/webapp/WEB-INF/RWStore.properties ~/latest-all.nt.gz It is updating the journal file but I hardly understood what namespace and defaultGraph params are used for.
-
The DataLoader executable and the blazegraph server can not both run at the same time. It is true that many of the properties are durable in the Journal but it is easier to simply have the same file in both places than to clearly identify any properties which can be overridden on startup. To address these issues, there is a REST interface which offers the same functionality as the data loader. See [1]. Bryan [1] https://wiki.blazegraph.com/wiki/index.php/REST_API#Bulk_Data_Load On Fri, Feb 22, 2019...
-
suhas sumukh modified a comment on discussion Help
I created a namespace from the Blazegraph Workbench using the following properties: com.bigdata.journal.AbstractJournal.file=blazegraph.jnl com.bigdata.journal.AbstractJournal.bufferMode=DiskRW com.bigdata.rdf.store.AbstractTripleStore.justify=true com.bigdata.rdf.sail.isolatableIndices=false com.bigdata.namespace.yago.spo.com.bigdata.btree.BTree.branchingFactor=1024 com.bigdata.rdf.sail.namespace=ABCD com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore...
-
I created a namespace from the Blazegraph Workbench using the following properties: com.bigdata.journal.AbstractJournal.file=blazegraph.jnl com.bigdata.journal.AbstractJournal.bufferMode=DiskRW com.bigdata.rdf.store.AbstractTripleStore.justify=true com.bigdata.rdf.sail.isolatableIndices=false com.bigdata.namespace.yago.spo.com.bigdata.btree.BTree.branchingFactor=1024 com.bigdata.rdf.sail.namespace=ABCD com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore...
-
I created a namespace from the Blazegraph Workbench using the following properties: com.bigdata.journal.AbstractJournal.file=blazegraph.jnl com.bigdata.journal.AbstractJournal.bufferMode=DiskRW com.bigdata.rdf.store.AbstractTripleStore.justify=true com.bigdata.rdf.sail.isolatableIndices=false com.bigdata.namespace.yago.spo.com.bigdata.btree.BTree.branchingFactor=1024 com.bigdata.rdf.sail.namespace=ABCD com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore...
-
Teh Kok How posted a comment on discussion Help
Hi, I am creating a docker container of BlazeGraph. I need to initialize the fresh instance with a new namespace. Where is the configuration file stored and what's the format of the config file? Any sample? Thanks!
-
Fred Palm modified a comment on discussion Help
I think I am doing something wrong in order to set queryTimeout. Basically, I have created the file override-web.xml in which I wrote: <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_1.xsd" version="3.1"> <context-param> <param-name>queryTimeout</param-name> <param-value>3600000</param-value> </context-param>...
-
I think I am doing something wrong in order to set queryTimeout. Basically, I have created the file override-web.xml in which I wrote: <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_1.xsd" version="3.1"> <context-param> <param-name>queryTimeout</param-name> <param-value>3600000</param-value> </context-param>...
-
https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#web.xml - defines the global timeout for blazegraph queries (queryTimeout -- When non-zero, this will timeout for queries (milliseconds)) https://wiki.blazegraph.com/wiki/index.php/REST_API#GET_or_POST (The maxQueryTimeMillisSpecifies URL query parameter -- defines the maximum time that a query is allowed to run, measured in milliseconds. May be also specified by using HTTP header X-BIGDATA-MAX-QUERY-MILLIS). Bryan On Tue, Nov 20, 2018 at...
-
Ok... I think I got the problem.... After exactly 10 minutes that the query is running, blazegraph throws the following error: 13:29:24.999 [com.bigdata.journal.Journal.executorService43] WARN com.bigdata.util.concurrent.Haltable IP: UA: - com.bigdata.util.concurrent.Haltable@2d05d70a : isFirstCause=true : com.bigdata.bop.engine.QueryTimeoutException: Query deadline is expired. 13:29:25.013 [com.bigdata.journal.Journal.executorService43] ERROR c.b.bop.join.HTreeHashJoinUtility IP: UA: - cause=java.lang.IllegalStateException:...
-
Ok... I think I got the problem.... After exactly 10 minutes that the query is running, blazegraph throws the following exception: [class com.bigdata.bop.engine.QueryEngine.engineService1] WARN com.bigdata.util.concurrent.Haltable IP: UA: - com.bigdata.util.concurrent.Haltable@44a99516 : isFirstCause=true : com.bigdata.bop.engine.QueryTimeoutException: Query deadline is expired. I have tested it with several big queries, and all of them throw the timeout exception exactly after 10 minutes. I think...
-
Ok... I think I got the problem.... After exactly 10 minutes that the query is running, blazegraph throws the following exception: [class com.bigdata.bop.engine.QueryEngine.engineService1] WARN com.bigdata.util.concurrent.Haltable IP: UA: - com.bigdata.util.concurrent.Haltable@44a99516 : isFirstCause=true : com.bigdata.bop.engine.QueryTimeoutException: Query deadline is expired. I have tested it with several big queries, and all of them throw the timeout exception exactly after 10 minutes. I think...
-
Ok... I think I got the problem.... After exactly 10 minutes that the query is running, blazegraph throws the following exception: ERROR c.b.bop.join.HTreeHashJoinUtility IP: UA: - cause=java.lang.IllegalStateException: Address: -5898270 yields index: 89 >= sector:size(): 0, state=HTreeHashJoinUtility{open=true,joinType=Optional,joinVars=[statement],outputDistinctJVs=true,size=3008559,considered(left=0,right=0,joins=0),joinSetSize=0} java.lang.IllegalStateException: Address: -5898270 yields index:...
-
Hello, I still cannot figure out what's the problem. From the log it seems like that there are two issues: 1) time out 2) memory. I don't understand if two errors are connected in somehow, or not. I have also printed at the beggining of the log all the java parameters. I have set large amount of memory for the heap (128GB), so in theory memory should not be a problem. Moreover, I am query the local end point (USER_AGENT="http://localhost:9999/bigdata/sparql"), so I should not get any time out error...
-
Hello, I still cannot figure out what's the problem. From the log it seems like that there are two issues: 1) time out 2) memory. I don't understand if two errors are connected in somehow, or not. I have also printed at the beggining of the log all the java parameters. I have set large amount of memory for the heap (128GB), so in theory memory should not be a problem. Moreover, I am query the local end point (USER_AGENT="http://localhost:9999/bigdata/sparql"), so I should not get any time out error...
-
Fred, Is there perhaps more to that stack trace? The submitted task in that version of Blazegraph is not associated with a timeout (well, not directly). I think that there must be a nested exception involved. Perhaps check the Blazegraph error log. See https://github.com/blazegraph/database/blob/master/bigdata-core/bigdata-sails/src/java/com/bigdata/rdf/sail/webapp/BigdataServlet.java#L250. You can see that it does a blocking get() on the Future. Therefore any timeout exception is being thrown from...
-
I suggest looking at the GC stats for the Blazegraph process. You have reduced the Java heap size considerably since your original posting. G1 likes a large heap because it can avoid stop the world events if it manages to finish a mixed collection before it runs out of managed heap space. So I would suggest 31G (and verify that you have compressed pointers enabled in the JVM) or 128G (your previous value). If there is a large GC pause, then that could cause timeouts. The trace above does not look...
-
I suggest looking at the GC stats for the Blazegraph process. You have reduced the Java heap size considerably since your original posting. G1 likes a large heap because it can avoid stop the world events if it manages to finish a mixed collection before it runs out of managed heap space. So I would suggest 31G (and verify that you have compressed pointers enabled in the JVM) or 128G (your previous value). If there is a large GC pause, then that could cause timeouts. The trace above does not look...
-
Hello everyone, I want to extract from all the humans in Wikidata, the information shown in their page. For example, for Albert Einstein (https://www.wikidata.org/wiki/Q937), I want to extract: <http://www.wikidata.org/entity/Q937> instance of human <http://www.wikidata.org/entity/Q937> image http://commons.wikimedia.org/wiki/Special:FilePath/Albert%20Einstein%20Head.jpg ... <http://www.wikidata.org/entity/Q937> country of citizenship German Empire start time 1879-03-14T00:00:00Z <http://www.wikidata.org/entity/Q937>...
-
Hi Bryan, now it works. In order to solve this problem I have set up the parameters that you suggested me. Moreover, I had to increase 'Dcom.bigdata.rdf.sparql.ast.QueryHints.analyticMaxMemoryPerQuery' as well. Thanks.
-
This is the switch which specifies how much native memory is available to Java. -XX:MaxDirectMemorySize=30000m Try giving it 500G. You do not need the GC logging options. Those are only for debugging. -Xloggc:/var/log/wdqs/wdqs-blazegraph_jvm_gc.%p-%t.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGC Java uses managed memory. The amount of managed memory is controlled by the -Xmx parameter. However, blazegraph will also use native memory made available to java via the max direct memory...
-
Hi Bryan, thanks for your reply. What do you mean by 'more direct memory to be accessed by the JVM and more of that direct memory to be used by the query'? I visited the performance optimization page of blazegraph wiki and I looked at all the parameters. The OS of my machine is Ubuntu 18.04.1 LTS and I installed openjdk version 1.8.0_181. The following are the input parameters that I set when I run blazegraph: java -server -XX:+UseG1GC -Xmx256G -Xloggc:/var/log/wdqs/wdqs-blazegraph_jvm_gc.%p-%t.log...
-
You need to configure to permit more direct memory to be accessed by the JVM and more of that direct memory to be used by the query. Stas can point you at how these configuration options are imposed for wikidata. Or you find that information on the blazegraph wiki under performance optimization Bryan On Fri, Oct 12, 2018, 13:08 Fred Palm fred1234@users.sourceforge.net wrote: Hello everyone, I want to extract from all the humans in Wikidata, the information shown in their page. For example, for Albert...
-
Hello everyone, I want to extract from all the humans in Wikidata, the information shown in their page. For example, for Albert Einstein (https://www.wikidata.org/wiki/Q937), I want to extract: <http://www.wikidata.org/entity/Q937> instance of human <http://www.wikidata.org/entity/Q937> image http://commons.wikimedia.org/wiki/Special:FilePath/Albert%20Einstein%20Head.jpg ... <http://www.wikidata.org/entity/Q937> country of citizenship German Empire start time 1879-03-14T00:00:00Z <http://www.wikidata.org/entity/Q937>...
-
Hello everyone, I want to extract from all the humans in Wikidata, the information shown in their page. For example, for Albert Einstein (https://www.wikidata.org/wiki/Q937), I want to extract: http://www.wikidata.org/entity/Q937 instance of human http://www.wikidata.org/entity/Q937 image http://commons.wikimedia.org/wiki/Special:FilePath/Albert%20Einstein%20Head.jpg ... http://www.wikidata.org/entity/Q937 country of citizenship German Empire start time 1879-03-14T00:00:00Z http://www.wikidata.org/entity/Q937...
-
Massimo Manfredino posted a comment on discussion Help
Bryan, thanks for your reply. I will try to familiarize with the rule API debugging code and writing unit tests if needed. If you have further material to share about dictionary, IV and so on, please do. Best regards Massimo
-
The rule API operates in terms of IVs (internal value representations). It can consider any IVs, though rules are generally written at the level of "types". A few possible gotchas: - rules do not have access to the dictionary. Thus, rules could not create new dictionary encoded valued. So, e.g., unless you inline all strings (not recommended), you could not have rules do things like string concatenation. - rules might require pre resolution of IVs. That is, they might require that the terms are part...
-
Thanks Bryan, i read that page. What i don't understand is if a rule can be built based on actual values of some data properties as in my example. Basically, i have seen in the BG codebase rules that infer something based on the existence of types of triples, regardless of actual values when the object of a statement is a data property. What i need is a more detaild explanation of the Rule API (in particular, what i receive as parameters in the constructor of a rule and if i can imagine to get actual...
-
This is discussed on the Blazegraph wiki - see https://wiki.blazegraph.com/wiki/index.php/InferenceAndTruthMaintenance Thanks, Bryan On Thu, Oct 4, 2018 at 3:48 AM Massimo Manfredino maxmanfred@users.sourceforge.net wrote: Hello, i am quite new to Blazegraph, so not sure how to create an inference rule that does the following: i have a class Profile having a datatype property called email whose value is a string with an email address a class Person having two datatype properties called firstName...
-
Hello, i am quite new to Blazegraph, so not sure how to create an inference rule that does the following: i have a class Profile having a datatype property called email whose value is a string with an email address a class Person having two datatype properties called firstName and lastName whose meaning is obvious when the rule is executed for each individual of Profile y get the value of email property, process it with RegExp (or similar, this part doesn't matter) and extract name and surname from...
-
Hello, i am quite new to Blazegraph, so not sure how to create an inference rule that does the following: i have a class Profile having a datatype property called email whose value is a string with an email address a class Person having two datatype properties called firstName and lastName whose meaning is obvious when the rule is executed for each individual of Profile y get the value of email property, process it with RegExp (or similar, this part doesn't matter) and extract name and surname from...
-
Fles Duan posted a comment on discussion Help
Thank you Brad! Helps a lot!
-
Hi Fles, Blazegraph does support JSON-LD [1]. If you can convert your MediaWiki dump into JSON-LD, you should be able to load it directly. Thanks, --Brad [1] https://www.w3.org/TR/json-ld/ On Wed, Jun 13, 2018 at 2:36 AM Fles Duan flesd@users.sourceforge.net wrote: Hi there, I'm trying to load rdf data in json format into Sparql using BlazeGraph's REST API. Now I keep getting error like that $ curl -X POST http://localhost:9999/bigdata/sparql --data-urlencode 'update=DROP ALL; LOAD <file: path_to_json_file=""...
-
Hi there, I'm trying to load rdf data in json format into Sparql using BlazeGraph's REST API. Now I keep getting error like that $ curl -X POST http://localhost:9999/bigdata/sparql --data-urlencode 'update=DROP ALL; LOAD <file:///path_to_json_file/dump.json>;' java.util.concurrent.ExecutionException: org.openrdf.rio.RDFParseException: Found unexpected object in top level type field Those json files are exported via MediaWiki's script, like: {"type":"item","id":"xx","labels":{"en":{"language":"en","value":"blahblah"}},"descriptions":{},"aliases":{},"claims":{},"sitelinks":{}},...
-
David, I am trying to track the solution down to this one. It is something that we have seen before, but it is proving difficult to find the answer online. I think that this link might help. https://sourceforge.net/p/bigdata/discussion/676946/thread/cbc36a99/?limit=25#dfb4 Your NPE is on this line System.setProperty("jetty.home", jettyXml.getClass().getResource("/war").toExternalForm()); As seen at https://github.com/blazegraph/database/blob/master/blazegraph-jar/src/main/java/com/bigdata/rdf/sail/webapp/StandaloneNanoSparqlServer.java#L142...
-
David Pride posted a comment on discussion Help
This is the invocation I am using , and the pages of error response I get. All pointers gratefully recieved! Many thanks. command.... java -server -Xmx4g -jar blazegraph.jar Response..... INFO: com.bigdata.util.config.LogUtil: Configure: jar:file:/Users/davidpride/Downloads/blazegraph.jar!/log4j.properties BlazeGraph(TM) Graph Engine Flexible Reliable Affordable Web-Scale Computing for the Enterprise Copyright SYSTAP, LLC DBA Blazegraph 2006-2016. All rights reserved. mcl271716.open.ac.uk Thu Apr...
-
David Fernández Aldana posted a comment on discussion Help
Hello Brad: Thanks, for your response, We will reach to you in case we need more information as for now a single server is enough.
-
Hi David, Starting with the 2.0.0 release [1], the HA journal capabilities are no longer included in the open source releases. Feel free to contact us (beebs at blazegraph.com) for information on other options. Thanks, --Brad [1] https://blog.blazegraph.com/?p=977 On Tue, Oct 17, 2017 at 12:46 AM, "David Fernández Aldana" luskaner@users.sf.net wrote: Hello: We are a company that are currently using blazegraph without the HAJournal capabilities, are these capabilities free or require any kind of subscription?...
-
Hello: We are a company that are currently using blazegraph without the HAJournal capabilities, are these capabilities free or require any kind of subscription? Thanks.
1 >
