Menu
▾
▴
bigdata-developers — List for bigdata developers
This list is closed, nobody may subscribe to it.
| 2010 |
Jan
|
Feb
(19) |
Mar
(8) |
Apr
(25) |
May
(16) |
Jun
(77) |
Jul
(131) |
Aug
(76) |
Sep
(30) |
Oct
(7) |
Nov
(3) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
(2) |
Jul
(16) |
Aug
(3) |
Sep
(1) |
Oct
|
Nov
(7) |
Dec
(7) |
| 2012 |
Jan
(10) |
Feb
(1) |
Mar
(8) |
Apr
(6) |
May
(1) |
Jun
(3) |
Jul
(1) |
Aug
|
Sep
(1) |
Oct
|
Nov
(8) |
Dec
(2) |
| 2013 |
Jan
(5) |
Feb
(12) |
Mar
(2) |
Apr
(1) |
May
(1) |
Jun
(1) |
Jul
(22) |
Aug
(50) |
Sep
(31) |
Oct
(64) |
Nov
(83) |
Dec
(28) |
| 2014 |
Jan
(31) |
Feb
(18) |
Mar
(27) |
Apr
(39) |
May
(45) |
Jun
(15) |
Jul
(6) |
Aug
(27) |
Sep
(6) |
Oct
(67) |
Nov
(70) |
Dec
(1) |
| 2015 |
Jan
(3) |
Feb
(18) |
Mar
(22) |
Apr
(121) |
May
(42) |
Jun
(17) |
Jul
(8) |
Aug
(11) |
Sep
(26) |
Oct
(15) |
Nov
(66) |
Dec
(38) |
| 2016 |
Jan
(14) |
Feb
(59) |
Mar
(28) |
Apr
(44) |
May
(21) |
Jun
(12) |
Jul
(9) |
Aug
(11) |
Sep
(4) |
Oct
(2) |
Nov
(1) |
Dec
|
| 2017 |
Jan
(20) |
Feb
(7) |
Mar
(4) |
Apr
(18) |
May
(7) |
Jun
(3) |
Jul
(13) |
Aug
(2) |
Sep
(4) |
Oct
(9) |
Nov
(2) |
Dec
(5) |
| 2018 |
Jan
|
Feb
|
Mar
|
Apr
(2) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Bryan T. <br...@sy...> - 2015-04-22 14:26:04
|
I would wait on this. There will not (should not) be any intermediate commits so what we need to do is log the allocators (and the shadow allocators used during group commit for unisolated index operations). @Martyn: Can you suggest some logging that might capture what is happening with the allocators during the load before Andreas retries this operation? Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 22, 2015 at 8:32 AM, Andreas Kahl <ka...@bs...> wrote: > There were no other concurrent queries. Just the one SPARQL LOAD. > I have deleted the file in the meantime (after a bit of cleaning I had > ~60GB, so the disk was full at that size). > If I can run DumpJournal without a commit, I can easily re-run the Load up > to the java.io.IOException thrown by the full disk. > > Currently I have restarted the LOAD. I will wait until it breaks down > (about 1h) and try to run DumpJournal on it. > > Andreas > > >>> Bryan Thompson <br...@sy...> 22.04.15 14.03 Uhr >>> > Were you running any other operations concurrently against the database? > Other updates or queries? > > In general, it is helpful to get the metadata about the allocators and root > blocks. However, from what you have written, it sounds like you terminated > the process when the disk space filled up. In this case there would only > be the original root blocks and no commit points recorded on the journal. > > If you still have the file, can you run DumpJournal on it and send the > output? The -pages option is not required in this case since we are only > interested in the root blocks and allocators. > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Wed, Apr 22, 2015 at 7:58 AM, Andreas Kahl <ka...@bs...> > wrote: > > > That was a newly created journal. I simply stopped tomcat, deleted > > bigdata.jnl and restarted. > > > > Andreas > > > > >>> Bryan Thompson <br...@sy...> 22.04.15 13.46 Uhr >>> > > Was the data loaded into a new and empty journal or into a pre-existing > > journal? If the latter, what size was the journal and what data were in > > it? > > > > Thanks, > > Bryan > > > > ---- > > Bryan Thompson > > Chief Scientist & Founder > > SYSTAP, LLC > > 4501 Tower Road > > Greensboro, NC 27410 > > br...@sy... > > http://blazegraph.com > > http://blog.bigdata.com <http://bigdata.com> > > http://mapgraph.io > > > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > > technology to use GPUs to accelerate data-parallel graph analytics. > > > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > > for the sole use of the intended recipient(s) and are confidential or > > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > > dissemination or copying of this email or its contents or attachments is > > prohibited. If you have received this communication in error, please > notify > > the sender by reply email and permanently delete all copies of the email > > and its contents and attachments. > > > > On Wed, Apr 22, 2015 at 6:54 AM, Andreas Kahl <ka...@bs...> > > wrote: > > > > > Bryan, > > > > > > yes, I used this command: > > > curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;" > > > -d"namespace=gnd" -d"monitor=true" > http://localhost:8080/bigdata/sparql > > > > > > Best Regards > > > Andreas > > > > > > >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>> > > > Andreas, > > > > > > What command did you use to load the data set? I.e., SPARQL update > > "Load" > > > or something else? > > > > > > Than Hello everyone, > > > > > > > > I currently updated to the current Revision (f4c63e5) of Blazegraph > > from > > > > Git and tried to load a dataset into the updated Webapp. With Bigdata > > > 1.4.0 > > > > this resulted in a journal of ~18GB. Now the process was cancelled > > > because > > > > the disk was full - the journal was beyond 50GB for the same file > with > > > the > > > > same settings. > > > > The only exception was that I activated GroupCommit. > > > > > > > > The dataset can be downloaded here: > > > > > > > > > > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > > > > . > > > > Please find the settings used to load the file below. > > > > > > > > Do I have a misconfiguration, or is there a bug eating all disk > memory? > > > > > > > > Best regards > > > > Andreas > > > > > > > > Namespace-Properties: > > > > curl -H "Accept: text/plain" > > > > http://localhost:8080/bigdata/namespace/gnd/properties > > > > #Wed Apr 22 11:35:31 CEST 2015 > > > > > > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > > > > com.bigdata.relation.container=gnd > > > > com.bigdata.rwstore.RWStore.smallSlotType=1024 > > > > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > > > > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > > > > > > > > > > > > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > > > > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > > > > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > > > > com.bigdata.btree.BTree.branchingFactor=700 > > > > > > > > > > > > > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > > > > com.bigdata.rdf.sail.isolatableIndices=false > > > > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > > > > com.bigdata.rdf.sail.bufferCapacity=2000 > > > > com.bigdata.rdf.sail.truthMaintenance=false > > > > com.bigdata.rdf.sail.namespace=gnd > > > > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > > > > com.bigdata.rdf.store.AbstractTripleStore.quads=false > > > > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > > > > com.bigdata.search.FullTextIndex.fieldsEnabled=false > > > > com.bigdata.relation.namespace=gnd > > > > com.bigdata.j.sail.BigdataSail.bufferCapacity=2000 > > > > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > > > > > > > > > > > > > -- > > > ---- > > > Bryan Thompson > > > Chief Scientist & Founder > > > SYSTAP, LLC > > > 4501 Tower Road > > > Greensboro, NC 27410 > > > br...@sy... > > > http://blazegraph.com > > > http://blog.bigdata.com <http://bigdata.com> > > > http://mapgraph.io > > > > > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > > > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > > > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive > new > > > technology to use GPUs to accelerate data-parallel graph analytics. > > > > > > CONFIDENTIALITY NOTICE: This email and its contents and attachments > are > > > for the sole use of the intended recipient(s) and are confidential or > > > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > > > dissemination or copying of this email or its contents or attachments > is > > > prohibited. If you have received this communication in error, please > > notify > > > the sender by reply email and permanently delete all copies of the > email > > > and its contents and attachments. > > > > > > > > > > > > |
|
From: Andreas K. <ka...@bs...> - 2015-04-22 12:32:41
|
There were no other concurrent queries. Just the one SPARQL LOAD. I have deleted the file in the meantime (after a bit of cleaning I had ~60GB, so the disk was full at that size). If I can run DumpJournal without a commit, I can easily re-run the Load up to the java.io.IOException thrown by the full disk. Currently I have restarted the LOAD. I will wait until it breaks down (about 1h) and try to run DumpJournal on it. Andreas >>> Bryan Thompson <br...@sy...> 22.04.15 14.03 Uhr >>> Were you running any other operations concurrently against the database? Other updates or queries? In general, it is helpful to get the metadata about the allocators and root blocks. However, from what you have written, it sounds like you terminated the process when the disk space filled up. In this case there would only be the original root blocks and no commit points recorded on the journal. If you still have the file, can you run DumpJournal on it and send the output? The -pages option is not required in this case since we are only interested in the root blocks and allocators. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 22, 2015 at 7:58 AM, Andreas Kahl <ka...@bs...> wrote: > That was a newly created journal. I simply stopped tomcat, deleted > bigdata.jnl and restarted. > > Andreas > > >>> Bryan Thompson <br...@sy...> 22.04.15 13.46 Uhr >>> > Was the data loaded into a new and empty journal or into a pre-existing > journal? If the latter, what size was the journal and what data were in > it? > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Wed, Apr 22, 2015 at 6:54 AM, Andreas Kahl <ka...@bs...> > wrote: > > > Bryan, > > > > yes, I used this command: > > curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;" > > -d"namespace=gnd" -d"monitor=true" http://localhost:8080/bigdata/sparql > > > > Best Regards > > Andreas > > > > >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>> > > Andreas, > > > > What command did you use to load the data set? I.e., SPARQL update > "Load" > > or something else? > > > > Than Hello everyone, > > > > > > I currently updated to the current Revision (f4c63e5) of Blazegraph > from > > > Git and tried to load a dataset into the updated Webapp. With Bigdata > > 1.4.0 > > > this resulted in a journal of ~18GB. Now the process was cancelled > > because > > > the disk was full - the journal was beyond 50GB for the same file with > > the > > > same settings. > > > The only exception was that I activated GroupCommit. > > > > > > The dataset can be downloaded here: > > > > > > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > > > . > > > Please find the settings used to load the file below. > > > > > > Do I have a misconfiguration, or is there a bug eating all disk memory? > > > > > > Best regards > > > Andreas > > > > > > Namespace-Properties: > > > curl -H "Accept: text/plain" > > > http://localhost:8080/bigdata/namespace/gnd/properties > > > #Wed Apr 22 11:35:31 CEST 2015 > > > > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > > > com.bigdata.relation.container=gnd > > > com.bigdata.rwstore.RWStore.smallSlotType=1024 > > > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > > > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > > > > > > > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > > > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > > > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > > > com.bigdata.btree.BTree.branchingFactor=700 > > > > > > > > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > > > com.bigdata.rdf.sail.isolatableIndices=false > > > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > > > com.bigdata.rdf.sail.bufferCapacity=2000 > > > com.bigdata.rdf.sail.truthMaintenance=false > > > com.bigdata.rdf.sail.namespace=gnd > > > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > > > com.bigdata.rdf.store.AbstractTripleStore.quads=false > > > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > > > com.bigdata.search.FullTextIndex.fieldsEnabled=false > > > com.bigdata.relation.namespace=gnd > > > com.bigdata.j.sail.BigdataSail.bufferCapacity=2000 > > > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > > > > > > > > > -- > > ---- > > Bryan Thompson > > Chief Scientist & Founder > > SYSTAP, LLC > > 4501 Tower Road > > Greensboro, NC 27410 > > br...@sy... > > http://blazegraph.com > > http://blog.bigdata.com <http://bigdata.com> > > http://mapgraph.io > > > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > > technology to use GPUs to accelerate data-parallel graph analytics. > > > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > > for the sole use of the intended recipient(s) and are confidential or > > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > > dissemination or copying of this email or its contents or attachments is > > prohibited. If you have received this communication in error, please > notify > > the sender by reply email and permanently delete all copies of the email > > and its contents and attachments. > > > > > > |
|
From: Bryan T. <br...@sy...> - 2015-04-22 12:03:43
|
Were you running any other operations concurrently against the database? Other updates or queries? In general, it is helpful to get the metadata about the allocators and root blocks. However, from what you have written, it sounds like you terminated the process when the disk space filled up. In this case there would only be the original root blocks and no commit points recorded on the journal. If you still have the file, can you run DumpJournal on it and send the output? The -pages option is not required in this case since we are only interested in the root blocks and allocators. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 22, 2015 at 7:58 AM, Andreas Kahl <ka...@bs...> wrote: > That was a newly created journal. I simply stopped tomcat, deleted > bigdata.jnl and restarted. > > Andreas > > >>> Bryan Thompson <br...@sy...> 22.04.15 13.46 Uhr >>> > Was the data loaded into a new and empty journal or into a pre-existing > journal? If the latter, what size was the journal and what data were in > it? > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Wed, Apr 22, 2015 at 6:54 AM, Andreas Kahl <ka...@bs...> > wrote: > > > Bryan, > > > > yes, I used this command: > > curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;" > > -d"namespace=gnd" -d"monitor=true" http://localhost:8080/bigdata/sparql > > > > Best Regards > > Andreas > > > > >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>> > > Andreas, > > > > What command did you use to load the data set? I.e., SPARQL update > "Load" > > or something else? > > > > Thanks, > > Bryan > > > > On Wednesday, April 22, 2015, Andreas Kahl <ka...@bs...> wrote: > > > > > Hello everyone, > > > > > > I currently updated to the current Revision (f4c63e5) of Blazegraph > from > > > Git and tried to load a dataset into the updated Webapp. With Bigdata > > 1.4.0 > > > this resulted in a journal of ~18GB. Now the process was cancelled > > because > > > the disk was full - the journal was beyond 50GB for the same file with > > the > > > same settings. > > > The only exception was that I activated GroupCommit. > > > > > > The dataset can be downloaded here: > > > > > > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > > > . > > > Please find the settings used to load the file below. > > > > > > Do I have a misconfiguration, or is there a bug eating all disk memory? > > > > > > Best regards > > > Andreas > > > > > > Namespace-Properties: > > > curl -H "Accept: text/plain" > > > http://localhost:8080/bigdata/namespace/gnd/properties > > > #Wed Apr 22 11:35:31 CEST 2015 > > > > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > > > com.bigdata.relation.container=gnd > > > com.bigdata.rwstore.RWStore.smallSlotType=1024 > > > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > > > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > > > > > > > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > > > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > > > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > > > com.bigdata.btree.BTree.branchingFactor=700 > > > > > > > > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > > > com.bigdata.rdf.sail.isolatableIndices=false > > > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > > > com.bigdata.rdf.sail.bufferCapacity=2000 > > > com.bigdata.rdf.sail.truthMaintenance=false > > > com.bigdata.rdf.sail.namespace=gnd > > > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > > > com.bigdata.rdf.store.AbstractTripleStore.quads=false > > > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > > > com.bigdata.search.FullTextIndex.fieldsEnabled=false > > > com.bigdata.relation.namespace=gnd > > > com.bigdata.j.sail.BigdataSail.bufferCapacity=2000 > > > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > > > > > > > > > -- > > ---- > > Bryan Thompson > > Chief Scientist & Founder > > SYSTAP, LLC > > 4501 Tower Road > > Greensboro, NC 27410 > > br...@sy... > > http://blazegraph.com > > http://blog.bigdata.com <http://bigdata.com> > > http://mapgraph.io > > > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > > technology to use GPUs to accelerate data-parallel graph analytics. > > > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > > for the sole use of the intended recipient(s) and are confidential or > > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > > dissemination or copying of this email or its contents or attachments is > > prohibited. If you have received this communication in error, please > notify > > the sender by reply email and permanently delete all copies of the email > > and its contents and attachments. > > > > > > |
|
From: Andreas K. <ka...@bs...> - 2015-04-22 11:58:11
|
That was a newly created journal. I simply stopped tomcat, deleted bigdata.jnl and restarted. Andreas >>> Bryan Thompson <br...@sy...> 22.04.15 13.46 Uhr >>> Was the data loaded into a new and empty journal or into a pre-existing journal? If the latter, what size was the journal and what data were in it? Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 22, 2015 at 6:54 AM, Andreas Kahl <ka...@bs...> wrote: > Bryan, > > yes, I used this command: > curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;" > -d"namespace=gnd" -d"monitor=true" http://localhost:8080/bigdata/sparql > > Best Regards > Andreas > > >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>> > Andreas, > > What command did you use to load the data set? I.e., SPARQL update "Load" > or something else? > > Thanks, > Bryan > > On Wednesday, April 22, 2015, Andreas Kahl <ka...@bs...> wrote: > > > Hello everyone, > > > > I currently updated to the current Revision (f4c63e5) of Blazegraph from > > Git and tried to load a dataset into the updated Webapp. With Bigdata > 1.4.0 > > this resulted in a journal of ~18GB. Now the process was cancelled > because > > the disk was full - the journal was beyond 50GB for the same file with > the > > same settings. > > The only exception was that I activated GroupCommit. > > > > The dataset can be downloaded here: > > > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > > . > > Please find the settings used to load the file below. > > > > Do I have a misconfiguration, or is there a bug eating all disk memory? > > > > Best regards > > Andreas > > > > Namespace-Properties: > > curl -H "Accept: text/plain" > > http://localhost:8080/bigdata/namespace/gnd/properties > > #Wed Apr 22 11:35:31 CEST 2015 > > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > > com.bigdata.relation.container=gnd > > com.bigdata.rwstore.RWStore.smallSlotType=1024 > > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > > > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > > com.bigdata.btree.BTree.branchingFactor=700 > > > > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > > com.bigdata.rdf.sail.isolatableIndices=false > > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > > com.bigdata.rdf.sail.bufferCapacity=2000 > > com.bigdata.rdf.sail.truthMaintenance=false > > com.bigdata.rdf.sail.namespace=gnd > > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > > com.bigdata.rdf.store.AbstractTripleStore.quads=false > > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > > com.bigdata.search.FullTextIndex.fieldsEnabled=false > > com.bigdata.relation.namespace=gnd > > com.bigdata.j.sail.BigdataSail.bufferCapacity=2000 > > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > > > > > -- > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > |
|
From: Bryan T. <br...@sy...> - 2015-04-22 11:46:29
|
Was the data loaded into a new and empty journal or into a pre-existing journal? If the latter, what size was the journal and what data were in it? Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 22, 2015 at 6:54 AM, Andreas Kahl <ka...@bs...> wrote: > Bryan, > > yes, I used this command: > curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;" > -d"namespace=gnd" -d"monitor=true" http://localhost:8080/bigdata/sparql > > Best Regards > Andreas > > >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>> > Andreas, > > What command did you use to load the data set? I.e., SPARQL update "Load" > or something else? > > Thanks, > Bryan > > On Wednesday, April 22, 2015, Andreas Kahl <ka...@bs...> wrote: > > > Hello everyone, > > > > I currently updated to the current Revision (f4c63e5) of Blazegraph from > > Git and tried to load a dataset into the updated Webapp. With Bigdata > 1.4.0 > > this resulted in a journal of ~18GB. Now the process was cancelled > because > > the disk was full - the journal was beyond 50GB for the same file with > the > > same settings. > > The only exception was that I activated GroupCommit. > > > > The dataset can be downloaded here: > > > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > > . > > Please find the settings used to load the file below. > > > > Do I have a misconfiguration, or is there a bug eating all disk memory? > > > > Best regards > > Andreas > > > > Namespace-Properties: > > curl -H "Accept: text/plain" > > http://localhost:8080/bigdata/namespace/gnd/properties > > #Wed Apr 22 11:35:31 CEST 2015 > > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > > com.bigdata.relation.container=gnd > > com.bigdata.rwstore.RWStore.smallSlotType=1024 > > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > > > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > > com.bigdata.btree.BTree.branchingFactor=700 > > > > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > > com.bigdata.rdf.sail.isolatableIndices=false > > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > > com.bigdata.rdf.sail.bufferCapacity=2000 > > com.bigdata.rdf.sail.truthMaintenance=false > > com.bigdata.rdf.sail.namespace=gnd > > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > > com.bigdata.rdf.store.AbstractTripleStore.quads=false > > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > > com.bigdata.search.FullTextIndex.fieldsEnabled=false > > com.bigdata.relation.namespace=gnd > > com.bigdata.journal.Journal.groupCommit=true > > com.bigdata.btree.writeRetentionQueue.capacity=10000 > > com.bigdata.rdf.sail.BigdataSail.bufferCapacity=2000 > > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > > > > > -- > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > |
|
From: Bryan T. <br...@sy...> - 2015-04-22 10:58:10
|
Andreas, What command did you use to load the data set? I.e., SPARQL update "Load" or something else? Thanks, Bryan On Wednesday, April 22, 2015, Andreas Kahl <ka...@bs...> wrote: > Hello everyone, > > I currently updated to the current Revision (f4c63e5) of Blazegraph from > Git and tried to load a dataset into the updated Webapp. With Bigdata 1.4.0 > this resulted in a journal of ~18GB. Now the process was cancelled because > the disk was full - the journal was beyond 50GB for the same file with the > same settings. > The only exception was that I activated GroupCommit. > > The dataset can be downloaded here: > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > . > Please find the settings used to load the file below. > > Do I have a misconfiguration, or is there a bug eating all disk memory? > > Best regards > Andreas > > Namespace-Properties: > curl -H "Accept: text/plain" > http://localhost:8080/bigdata/namespace/gnd/properties > #Wed Apr 22 11:35:31 CEST 2015 > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > com.bigdata.relation.container=gnd > com.bigdata.rwstore.RWStore.smallSlotType=1024 > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > com.bigdata.btree.BTree.branchingFactor=700 > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > com.bigdata.rdf.sail.isolatableIndices=false > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > com.bigdata.rdf.sail.bufferCapacity=2000 > com.bigdata.rdf.sail.truthMaintenance=false > com.bigdata.rdf.sail.namespace=gnd > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > com.bigdata.rdf.store.AbstractTripleStore.quads=false > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > com.bigdata.search.FullTextIndex.fieldsEnabled=false > com.bigdata.relation.namespace=gnd > com.bigdata.journal.Journal.groupCommit=true > com.bigdata.btree.writeRetentionQueue.capacity=10000 > com.bigdata.rdf.sail.BigdataSail.bufferCapacity=2000 > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > -- ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. |
|
From: Andreas K. <ka...@bs...> - 2015-04-22 10:55:07
|
Bryan, yes, I used this command: curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;" -d"namespace=gnd" -d"monitor=true" http://localhost:8080/bigdata/sparql Best Regards Andreas >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>> Andreas, What command did you use to load the data set? I.e., SPARQL update "Load" or something else? Thanks, Bryan On Wednesday, April 22, 2015, Andreas Kahl <ka...@bs...> wrote: > Hello everyone, > > I currently updated to the current Revision (f4c63e5) of Blazegraph from > Git and tried to load a dataset into the updated Webapp. With Bigdata 1.4.0 > this resulted in a journal of ~18GB. Now the process was cancelled because > the disk was full - the journal was beyond 50GB for the same file with the > same settings. > The only exception was that I activated GroupCommit. > > The dataset can be downloaded here: > http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz > . > Please find the settings used to load the file below. > > Do I have a misconfiguration, or is there a bug eating all disk memory? > > Best regards > Andreas > > Namespace-Properties: > curl -H "Accept: text/plain" > http://localhost:8080/bigdata/namespace/gnd/properties > #Wed Apr 22 11:35:31 CEST 2015 > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700 > com.bigdata.relation.container=gnd > com.bigdata.rwstore.RWStore.smallSlotType=1024 > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl > > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary > com.bigdata.journal.AbstractJournal.initialExtent=209715200 > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true > com.bigdata.btree.BTree.branchingFactor=700 > > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms > com.bigdata.rdf.sail.isolatableIndices=false > com.bigdata.service.AbstractTransactionService.minReleaseAge=1 > com.bigdata.rdf.sail.bufferCapacity=2000 > com.bigdata.rdf.sail.truthMaintenance=false > com.bigdata.rdf.sail.namespace=gnd > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore > com.bigdata.rdf.store.AbstractTripleStore.quads=false > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500 > com.bigdata.search.FullTextIndex.fieldsEnabled=false > com.bigdata.relation.namespace=gnd > com.bigdata.journal.Journal.groupCommit=true > com.bigdata.btree.writeRetentionQueue.capacity=10000 > com.bigdata.rdf.sail.BigdataSail.bufferCapacity=2000 > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false > -- ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. |
|
From: Lee K. <le...@sw...> - 2015-04-22 10:04:19
|
Hi Bryan,
Yes the AST in the test is supposed to be for the query
select (count(*) as ?c) where {
select * where {
select * where { ?s ?p ?o }
} limit 21 offset 0
}
Thanks
On Tue, Apr 21, 2015 at 7:53 PM, Bryan Thompson <br...@sy...> wrote:
> Lee,
>
> I can replicate the problem with your query (as given above) against the
> sparql end point.
>
> Can you state the SPARQL that you are trying to model with this unit
> test? It appears to be not query the same as your SPARQL query above. I
> would like to make sure that it is being translated correctly into the
> AST. I can then look at the expected AST and work backwards and see if I
> believe that the test shows the problem.
>
> Thanks,
> Bryan
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Tue, Apr 21, 2015 at 11:07 AM, Lee Kitching <le...@sw...> wrote:
>
>> Hi Bryan,
>>
>> We allow users to enter their own SPARQL queries and wrap them to do
>> things like pagination so unfortunately we cannot just re-write our queries
>> to do the expansion manually.
>> I applied the fix detailed in the ticket and it fixes the for the query I
>> provided, however it fails to rewrite the following query:
>>
>> SELECT (COUNT(*) as ?c) {
>> SELECT * {
>> SELECT * WHERE { ?s ?p ?o }
>> } LIMIT 21 OFFSET 0
>> }
>>
>> I attempted to debug the issue, and it seems to re-write the * projection
>> in the inner-most subquery but not the subquery with the limit and offset.
>> I created a test based on the
>> existing tests:
>>
>> public void test_wildcardProjectionOptimizer03() {
>>
>> /*
>> * Note: DO NOT share structures in this test!!!!
>> */
>> final IBindingSet[] bsets = new IBindingSet[] {};
>>
>> // The source AST.
>> final QueryRoot given = new QueryRoot(QueryType.SELECT);
>> {
>> final SubqueryRoot selectQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final JoinGroupNode whereClause1 = new JoinGroupNode();
>> final StatementPatternNode spoPattern = new
>> StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"),
>> null, Scope.DEFAULT_CONTEXTS);
>> whereClause1.addChild(spoPattern);
>>
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("*"));
>> selectQuery.setProjection(p);
>> selectQuery.setWhereClause(whereClause1);
>> }
>>
>> final SubqueryRoot sliceQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("*"));
>> sliceQuery.setProjection(p);
>>
>> final JoinGroupNode whereClause = new JoinGroupNode();
>> whereClause.addChild(selectQuery);
>>
>> sliceQuery.setSlice(new SliceNode(0, 21));
>> }
>>
>> final FunctionNode countNode = new FunctionNode(
>> FunctionRegistry.COUNT,
>> Collections.EMPTY_MAP,
>> new VarNode("*"));
>>
>> final ProjectionNode countProjection = new ProjectionNode();
>> countProjection.addProjectionExpression(new
>> AssignmentNode(new VarNode("c"), countNode));
>>
>> JoinGroupNode countWhere = new JoinGroupNode();
>> countWhere.addChild(sliceQuery);
>>
>> given.setProjection(countProjection);
>> given.setWhereClause(countWhere);
>> }
>>
>> final QueryRoot expected = new QueryRoot(QueryType.SELECT);
>> {
>> final SubqueryRoot selectQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final JoinGroupNode whereClause1 = new JoinGroupNode();
>> final StatementPatternNode spoPattern = new
>> StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"),
>> null, Scope.DEFAULT_CONTEXTS);
>> whereClause1.addChild(spoPattern);
>>
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("s"));
>> p.addProjectionVar(new VarNode("p"));
>> p.addProjectionVar(new VarNode("o"));
>> selectQuery.setProjection(p);
>> selectQuery.setWhereClause(whereClause1);
>> }
>>
>> final SubqueryRoot sliceQuery = new
>> SubqueryRoot(QueryType.SELECT);
>> {
>> final ProjectionNode p = new ProjectionNode();
>> p.addProjectionVar(new VarNode("s"));
>> p.addProjectionVar(new VarNode("p"));
>> p.addProjectionVar(new VarNode("o"));
>>
>> sliceQuery.setProjection(p);
>>
>> final JoinGroupNode whereClause = new JoinGroupNode();
>> whereClause.addChild(selectQuery);
>>
>> sliceQuery.setSlice(new SliceNode(0, 21));
>> }
>>
>> final FunctionNode countNode = new FunctionNode(
>> FunctionRegistry.COUNT,
>> Collections.EMPTY_MAP,
>> new VarNode("*"));
>>
>> final ProjectionNode countProjection = new ProjectionNode();
>> countProjection.addProjectionExpression(new
>> AssignmentNode(new VarNode("c"), countNode));
>>
>> JoinGroupNode countWhere = new JoinGroupNode();
>> countWhere.addChild(sliceQuery);
>>
>> expected.setProjection(countProjection);
>> expected.setWhereClause(countWhere);
>> }
>>
>> final IASTOptimizer rewriter = new
>> ASTWildcardProjectionOptimizer();
>>
>> final IQueryNode actual = rewriter.optimize(null/* AST2BOpContext
>> */,
>> given/* queryNode */, bsets);
>>
>> assertSameAST(expected, actual);
>>
>> }
>>
>> however I am having some problems running the tests locally so I don't
>> know if it accurately models the situation.
>>
>> Thanks
>>
>>
>>
>> On Mon, Apr 20, 2015 at 9:05 PM, Bryan Thompson <br...@sy...> wrote:
>>
>>> Lee,
>>>
>>> I've updated the ticket with the code changes and the test changes.
>>> Please try this out and let me know if you have any problems.
>>>
>>> Thanks,
>>> Bryan
>>>
>>> ----
>>> Bryan Thompson
>>> Chief Scientist & Founder
>>> SYSTAP, LLC
>>> 4501 Tower Road
>>> Greensboro, NC 27410
>>> br...@sy...
>>> http://blazegraph.com
>>> http://blog.bigdata.com <http://bigdata.com>
>>> http://mapgraph.io
>>>
>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>>> technology to use GPUs to accelerate data-parallel graph analytics.
>>>
>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments
>>> are for the sole use of the intended recipient(s) and are confidential or
>>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>>> dissemination or copying of this email or its contents or attachments is
>>> prohibited. If you have received this communication in error, please notify
>>> the sender by reply email and permanently delete all copies of the email
>>> and its contents and attachments.
>>>
>>> On Mon, Apr 20, 2015 at 1:20 PM, Lee Kitching <le...@sw...> wrote:
>>>
>>>> Hi,
>>>>
>>>> We are currently evaluating using Blazegraph as our rdf database and
>>>> have run in the issue described at http://trac.bigdata.com/ticket/757.
>>>> The below query causes the AssertionError to be thrown:
>>>>
>>>> SELECT (COUNT(*) as ?c) {
>>>> SELECT ?uri ?graph where {
>>>> {
>>>> SELECT * WHERE {
>>>> GRAPH ?graph {
>>>> ?uri a <http://object> .
>>>> ?uri <http://purl.org/dc/terms/title> ?title .
>>>> }
>>>> MINUS {
>>>> ?uri a <http://other>
>>>> }
>>>> }
>>>> ORDER BY ?title
>>>> }
>>>> }
>>>> }
>>>>
>>>> Some debugging shows that the error is caused by the
>>>> ASTWildcardProjectionOptimizer failing to recurse into the subqueries to
>>>> rewrite the * projection. However this recursion is implemented in the
>>>> BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator
>>>> to
>>>> find child operators and therefore only visits children for nodes with
>>>> an arity > 0.
>>>>
>>>> The root query node for the above query has an empty 'args' collection
>>>> and all the associated components of the top-level query are stored in the
>>>> annotations map. It looks like the iterator should search through the
>>>> annotations rather than the args for query nodes.
>>>>
>>>> As there are a lot of implementations of the BOp interface, it seems
>>>> that changing the postOrderIterator2(BOp) method is unlikely to be the
>>>> correct fix. It seems that either the AST query nodes should override the
>>>> arity() function to return the count of the annotations map, or the
>>>> ASTWildcardProjectionOptimizer should use its own iterator for the nodes of
>>>> the query. The latter option would be the least impactful change but I am
>>>> not familiar with the codebase to understand the correct fix.
>>>>
>>>> Any help in resolving the issue would be appreciated.
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>>>> Develop your own process in accordance with the BPMN 2 standard
>>>> Learn Process modeling best practices with Bonita BPM through live
>>>> exercises
>>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>>>> event?utm_
>>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>>>> _______________________________________________
>>>> Bigdata-developers mailing list
>>>> Big...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>
>>>>
>>>
>>
>
|
|
From: Bryan T. <br...@sy...> - 2015-04-21 18:53:30
|
Lee, I can replicate the problem with your query (as given above) against the sparql end point. Can you state the SPARQL that you are trying to model with this unit test? It appears to be not query the same as your SPARQL query above. I would like to make sure that it is being translated correctly into the AST. I can then look at the expected AST and work backwards and see if I believe that the test shows the problem. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Tue, Apr 21, 2015 at 11:07 AM, Lee Kitching <le...@sw...> wrote: > Hi Bryan, > > We allow users to enter their own SPARQL queries and wrap them to do > things like pagination so unfortunately we cannot just re-write our queries > to do the expansion manually. > I applied the fix detailed in the ticket and it fixes the for the query I > provided, however it fails to rewrite the following query: > > SELECT (COUNT(*) as ?c) { > SELECT * { > SELECT * WHERE { ?s ?p ?o } > } LIMIT 21 OFFSET 0 > } > > I attempted to debug the issue, and it seems to re-write the * projection > in the inner-most subquery but not the subquery with the limit and offset. > I created a test based on the > existing tests: > > public void test_wildcardProjectionOptimizer03() { > > /* > * Note: DO NOT share structures in this test!!!! > */ > final IBindingSet[] bsets = new IBindingSet[] {}; > > // The source AST. > final QueryRoot given = new QueryRoot(QueryType.SELECT); > { > final SubqueryRoot selectQuery = new > SubqueryRoot(QueryType.SELECT); > { > final JoinGroupNode whereClause1 = new JoinGroupNode(); > final StatementPatternNode spoPattern = new > StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"), > null, Scope.DEFAULT_CONTEXTS); > whereClause1.addChild(spoPattern); > > final ProjectionNode p = new ProjectionNode(); > p.addProjectionVar(new VarNode("*")); > selectQuery.setProjection(p); > selectQuery.setWhereClause(whereClause1); > } > > final SubqueryRoot sliceQuery = new > SubqueryRoot(QueryType.SELECT); > { > final ProjectionNode p = new ProjectionNode(); > p.addProjectionVar(new VarNode("*")); > sliceQuery.setProjection(p); > > final JoinGroupNode whereClause = new JoinGroupNode(); > whereClause.addChild(selectQuery); > > sliceQuery.setSlice(new SliceNode(0, 21)); > } > > final FunctionNode countNode = new FunctionNode( > FunctionRegistry.COUNT, > Collections.EMPTY_MAP, > new VarNode("*")); > > final ProjectionNode countProjection = new ProjectionNode(); > countProjection.addProjectionExpression(new AssignmentNode(new > VarNode("c"), countNode)); > > JoinGroupNode countWhere = new JoinGroupNode(); > countWhere.addChild(sliceQuery); > > given.setProjection(countProjection); > given.setWhereClause(countWhere); > } > > final QueryRoot expected = new QueryRoot(QueryType.SELECT); > { > final SubqueryRoot selectQuery = new > SubqueryRoot(QueryType.SELECT); > { > final JoinGroupNode whereClause1 = new JoinGroupNode(); > final StatementPatternNode spoPattern = new > StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"), > null, Scope.DEFAULT_CONTEXTS); > whereClause1.addChild(spoPattern); > > final ProjectionNode p = new ProjectionNode(); > p.addProjectionVar(new VarNode("s")); > p.addProjectionVar(new VarNode("p")); > p.addProjectionVar(new VarNode("o")); > selectQuery.setProjection(p); > selectQuery.setWhereClause(whereClause1); > } > > final SubqueryRoot sliceQuery = new > SubqueryRoot(QueryType.SELECT); > { > final ProjectionNode p = new ProjectionNode(); > p.addProjectionVar(new VarNode("s")); > p.addProjectionVar(new VarNode("p")); > p.addProjectionVar(new VarNode("o")); > > sliceQuery.setProjection(p); > > final JoinGroupNode whereClause = new JoinGroupNode(); > whereClause.addChild(selectQuery); > > sliceQuery.setSlice(new SliceNode(0, 21)); > } > > final FunctionNode countNode = new FunctionNode( > FunctionRegistry.COUNT, > Collections.EMPTY_MAP, > new VarNode("*")); > > final ProjectionNode countProjection = new ProjectionNode(); > countProjection.addProjectionExpression(new AssignmentNode(new > VarNode("c"), countNode)); > > JoinGroupNode countWhere = new JoinGroupNode(); > countWhere.addChild(sliceQuery); > > expected.setProjection(countProjection); > expected.setWhereClause(countWhere); > } > > final IASTOptimizer rewriter = new > ASTWildcardProjectionOptimizer(); > > final IQueryNode actual = rewriter.optimize(null/* AST2BOpContext > */, > given/* queryNode */, bsets); > > assertSameAST(expected, actual); > > } > > however I am having some problems running the tests locally so I don't > know if it accurately models the situation. > > Thanks > > > > On Mon, Apr 20, 2015 at 9:05 PM, Bryan Thompson <br...@sy...> wrote: > >> Lee, >> >> I've updated the ticket with the code changes and the test changes. >> Please try this out and let me know if you have any problems. >> >> Thanks, >> Bryan >> >> ---- >> Bryan Thompson >> Chief Scientist & Founder >> SYSTAP, LLC >> 4501 Tower Road >> Greensboro, NC 27410 >> br...@sy... >> http://blazegraph.com >> http://blog.bigdata.com <http://bigdata.com> >> http://mapgraph.io >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> On Mon, Apr 20, 2015 at 1:20 PM, Lee Kitching <le...@sw...> wrote: >> >>> Hi, >>> >>> We are currently evaluating using Blazegraph as our rdf database and >>> have run in the issue described at http://trac.bigdata.com/ticket/757. >>> The below query causes the AssertionError to be thrown: >>> >>> SELECT (COUNT(*) as ?c) { >>> SELECT ?uri ?graph where { >>> { >>> SELECT * WHERE { >>> GRAPH ?graph { >>> ?uri a <http://object> . >>> ?uri <http://purl.org/dc/terms/title> ?title . >>> } >>> MINUS { >>> ?uri a <http://other> >>> } >>> } >>> ORDER BY ?title >>> } >>> } >>> } >>> >>> Some debugging shows that the error is caused by the >>> ASTWildcardProjectionOptimizer failing to recurse into the subqueries to >>> rewrite the * projection. However this recursion is implemented in the >>> BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator >>> to >>> find child operators and therefore only visits children for nodes with >>> an arity > 0. >>> >>> The root query node for the above query has an empty 'args' collection >>> and all the associated components of the top-level query are stored in the >>> annotations map. It looks like the iterator should search through the >>> annotations rather than the args for query nodes. >>> >>> As there are a lot of implementations of the BOp interface, it seems >>> that changing the postOrderIterator2(BOp) method is unlikely to be the >>> correct fix. It seems that either the AST query nodes should override the >>> arity() function to return the count of the annotations map, or the >>> ASTWildcardProjectionOptimizer should use its own iterator for the nodes of >>> the query. The latter option would be the least impactful change but I am >>> not familiar with the codebase to understand the correct fix. >>> >>> Any help in resolving the issue would be appreciated. >>> >>> >>> ------------------------------------------------------------------------------ >>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>> Develop your own process in accordance with the BPMN 2 standard >>> Learn Process modeling best practices with Bonita BPM through live >>> exercises >>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>> event?utm_ >>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>> _______________________________________________ >>> Bigdata-developers mailing list >>> Big...@li... >>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>> >>> >> > |
|
From: Lee K. <le...@sw...> - 2015-04-21 15:07:52
|
Hi Bryan,
We allow users to enter their own SPARQL queries and wrap them to do things
like pagination so unfortunately we cannot just re-write our queries to do
the expansion manually.
I applied the fix detailed in the ticket and it fixes the for the query I
provided, however it fails to rewrite the following query:
SELECT (COUNT(*) as ?c) {
SELECT * {
SELECT * WHERE { ?s ?p ?o }
} LIMIT 21 OFFSET 0
}
I attempted to debug the issue, and it seems to re-write the * projection
in the inner-most subquery but not the subquery with the limit and offset.
I created a test based on the
existing tests:
public void test_wildcardProjectionOptimizer03() {
/*
* Note: DO NOT share structures in this test!!!!
*/
final IBindingSet[] bsets = new IBindingSet[] {};
// The source AST.
final QueryRoot given = new QueryRoot(QueryType.SELECT);
{
final SubqueryRoot selectQuery = new
SubqueryRoot(QueryType.SELECT);
{
final JoinGroupNode whereClause1 = new JoinGroupNode();
final StatementPatternNode spoPattern = new
StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"),
null, Scope.DEFAULT_CONTEXTS);
whereClause1.addChild(spoPattern);
final ProjectionNode p = new ProjectionNode();
p.addProjectionVar(new VarNode("*"));
selectQuery.setProjection(p);
selectQuery.setWhereClause(whereClause1);
}
final SubqueryRoot sliceQuery = new
SubqueryRoot(QueryType.SELECT);
{
final ProjectionNode p = new ProjectionNode();
p.addProjectionVar(new VarNode("*"));
sliceQuery.setProjection(p);
final JoinGroupNode whereClause = new JoinGroupNode();
whereClause.addChild(selectQuery);
sliceQuery.setSlice(new SliceNode(0, 21));
}
final FunctionNode countNode = new FunctionNode(
FunctionRegistry.COUNT,
Collections.EMPTY_MAP,
new VarNode("*"));
final ProjectionNode countProjection = new ProjectionNode();
countProjection.addProjectionExpression(new AssignmentNode(new
VarNode("c"), countNode));
JoinGroupNode countWhere = new JoinGroupNode();
countWhere.addChild(sliceQuery);
given.setProjection(countProjection);
given.setWhereClause(countWhere);
}
final QueryRoot expected = new QueryRoot(QueryType.SELECT);
{
final SubqueryRoot selectQuery = new
SubqueryRoot(QueryType.SELECT);
{
final JoinGroupNode whereClause1 = new JoinGroupNode();
final StatementPatternNode spoPattern = new
StatementPatternNode(new VarNode("s"), new VarNode("p"), new VarNode("o"),
null, Scope.DEFAULT_CONTEXTS);
whereClause1.addChild(spoPattern);
final ProjectionNode p = new ProjectionNode();
p.addProjectionVar(new VarNode("s"));
p.addProjectionVar(new VarNode("p"));
p.addProjectionVar(new VarNode("o"));
selectQuery.setProjection(p);
selectQuery.setWhereClause(whereClause1);
}
final SubqueryRoot sliceQuery = new
SubqueryRoot(QueryType.SELECT);
{
final ProjectionNode p = new ProjectionNode();
p.addProjectionVar(new VarNode("s"));
p.addProjectionVar(new VarNode("p"));
p.addProjectionVar(new VarNode("o"));
sliceQuery.setProjection(p);
final JoinGroupNode whereClause = new JoinGroupNode();
whereClause.addChild(selectQuery);
sliceQuery.setSlice(new SliceNode(0, 21));
}
final FunctionNode countNode = new FunctionNode(
FunctionRegistry.COUNT,
Collections.EMPTY_MAP,
new VarNode("*"));
final ProjectionNode countProjection = new ProjectionNode();
countProjection.addProjectionExpression(new AssignmentNode(new
VarNode("c"), countNode));
JoinGroupNode countWhere = new JoinGroupNode();
countWhere.addChild(sliceQuery);
expected.setProjection(countProjection);
expected.setWhereClause(countWhere);
}
final IASTOptimizer rewriter = new ASTWildcardProjectionOptimizer();
final IQueryNode actual = rewriter.optimize(null/* AST2BOpContext
*/,
given/* queryNode */, bsets);
assertSameAST(expected, actual);
}
however I am having some problems running the tests locally so I don't know
if it accurately models the situation.
Thanks
On Mon, Apr 20, 2015 at 9:05 PM, Bryan Thompson <br...@sy...> wrote:
> Lee,
>
> I've updated the ticket with the code changes and the test changes. Please
> try this out and let me know if you have any problems.
>
> Thanks,
> Bryan
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Mon, Apr 20, 2015 at 1:20 PM, Lee Kitching <le...@sw...> wrote:
>
>> Hi,
>>
>> We are currently evaluating using Blazegraph as our rdf database and have
>> run in the issue described at http://trac.bigdata.com/ticket/757.
>> The below query causes the AssertionError to be thrown:
>>
>> SELECT (COUNT(*) as ?c) {
>> SELECT ?uri ?graph where {
>> {
>> SELECT * WHERE {
>> GRAPH ?graph {
>> ?uri a <http://object> .
>> ?uri <http://purl.org/dc/terms/title> ?title .
>> }
>> MINUS {
>> ?uri a <http://other>
>> }
>> }
>> ORDER BY ?title
>> }
>> }
>> }
>>
>> Some debugging shows that the error is caused by the
>> ASTWildcardProjectionOptimizer failing to recurse into the subqueries to
>> rewrite the * projection. However this recursion is implemented in the
>> BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator
>> to
>> find child operators and therefore only visits children for nodes with an
>> arity > 0.
>>
>> The root query node for the above query has an empty 'args' collection
>> and all the associated components of the top-level query are stored in the
>> annotations map. It looks like the iterator should search through the
>> annotations rather than the args for query nodes.
>>
>> As there are a lot of implementations of the BOp interface, it seems that
>> changing the postOrderIterator2(BOp) method is unlikely to be the correct
>> fix. It seems that either the AST query nodes should override the arity()
>> function to return the count of the annotations map, or the
>> ASTWildcardProjectionOptimizer should use its own iterator for the nodes of
>> the query. The latter option would be the least impactful change but I am
>> not familiar with the codebase to understand the correct fix.
>>
>> Any help in resolving the issue would be appreciated.
>>
>>
>> ------------------------------------------------------------------------------
>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> Develop your own process in accordance with the BPMN 2 standard
>> Learn Process modeling best practices with Bonita BPM through live
>> exercises
>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> event?utm_
>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>
|
|
From: Bryan T. <br...@sy...> - 2015-04-20 20:05:28
|
Lee, I've updated the ticket with the code changes and the test changes. Please try this out and let me know if you have any problems. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Mon, Apr 20, 2015 at 1:20 PM, Lee Kitching <le...@sw...> wrote: > Hi, > > We are currently evaluating using Blazegraph as our rdf database and have > run in the issue described at http://trac.bigdata.com/ticket/757. > The below query causes the AssertionError to be thrown: > > SELECT (COUNT(*) as ?c) { > SELECT ?uri ?graph where { > { > SELECT * WHERE { > GRAPH ?graph { > ?uri a <http://object> . > ?uri <http://purl.org/dc/terms/title> ?title . > } > MINUS { > ?uri a <http://other> > } > } > ORDER BY ?title > } > } > } > > Some debugging shows that the error is caused by the > ASTWildcardProjectionOptimizer failing to recurse into the subqueries to > rewrite the * projection. However this recursion is implemented in the > BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator > to > find child operators and therefore only visits children for nodes with an > arity > 0. > > The root query node for the above query has an empty 'args' collection and > all the associated components of the top-level query are stored in the > annotations map. It looks like the iterator should search through the > annotations rather than the args for query nodes. > > As there are a lot of implementations of the BOp interface, it seems that > changing the postOrderIterator2(BOp) method is unlikely to be the correct > fix. It seems that either the AST query nodes should override the arity() > function to return the count of the annotations map, or the > ASTWildcardProjectionOptimizer should use its own iterator for the nodes of > the query. The latter option would be the least impactful change but I am > not familiar with the codebase to understand the correct fix. > > Any help in resolving the issue would be appreciated. > > > ------------------------------------------------------------------------------ > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT > Develop your own process in accordance with the BPMN 2 standard > Learn Process modeling best practices with Bonita BPM through live > exercises > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- > event?utm_ > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Bryan T. <br...@sy...> - 2015-04-20 19:14:50
|
Lee,
How much of a blocker is this? You could replace * with the variables in
the given query by hand. Or are your queries being generated such that you
have less control over this?
You could certainly try the following. It will also recurse into the
annotations.
@SuppressWarnings("unchecked")
final Iterator<QueryBase> itr = (Iterator<QueryBase>) new
Striterator(
* BOpUtility.postOrderIteratorWithAnnotations((BOp)
queryRoot*
.getWhereClause())).addTypeFilter(QueryBase.
class);
Thanks,
Bryan
Bryan
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io
Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
On Mon, Apr 20, 2015 at 1:20 PM, Lee Kitching <le...@sw...> wrote:
> Hi,
>
> We are currently evaluating using Blazegraph as our rdf database and have
> run in the issue described at http://trac.bigdata.com/ticket/757.
> The below query causes the AssertionError to be thrown:
>
> SELECT (COUNT(*) as ?c) {
> SELECT ?uri ?graph where {
> {
> SELECT * WHERE {
> GRAPH ?graph {
> ?uri a <http://object> .
> ?uri <http://purl.org/dc/terms/title> ?title .
> }
> MINUS {
> ?uri a <http://other>
> }
> }
> ORDER BY ?title
> }
> }
> }
>
> Some debugging shows that the error is caused by the
> ASTWildcardProjectionOptimizer failing to recurse into the subqueries to
> rewrite the * projection. However this recursion is implemented in the
> BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator
> to
> find child operators and therefore only visits children for nodes with an
> arity > 0.
>
> The root query node for the above query has an empty 'args' collection and
> all the associated components of the top-level query are stored in the
> annotations map. It looks like the iterator should search through the
> annotations rather than the args for query nodes.
>
> As there are a lot of implementations of the BOp interface, it seems that
> changing the postOrderIterator2(BOp) method is unlikely to be the correct
> fix. It seems that either the AST query nodes should override the arity()
> function to return the count of the annotations map, or the
> ASTWildcardProjectionOptimizer should use its own iterator for the nodes of
> the query. The latter option would be the least impactful change but I am
> not familiar with the codebase to understand the correct fix.
>
> Any help in resolving the issue would be appreciated.
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
|
|
From: Lee K. <le...@sw...> - 2015-04-20 18:28:19
|
Hi, We are currently evaluating using Blazegraph as our rdf database and have run in the issue described at http://trac.bigdata.com/ticket/757. The below query causes the AssertionError to be thrown: SELECT (COUNT(*) as ?c) { SELECT ?uri ?graph where { { SELECT * WHERE { GRAPH ?graph { ?uri a <http://object> . ?uri <http://purl.org/dc/terms/title> ?title . } MINUS { ?uri a <http://other> } } ORDER BY ?title } } } Some debugging shows that the error is caused by the ASTWildcardProjectionOptimizer failing to recurse into the subqueries to rewrite the * projection. However this recursion is implemented in the BOpUtility.postOrderIterator(BOp) method - this method uses the argIterator to find child operators and therefore only visits children for nodes with an arity > 0. The root query node for the above query has an empty 'args' collection and all the associated components of the top-level query are stored in the annotations map. It looks like the iterator should search through the annotations rather than the args for query nodes. As there are a lot of implementations of the BOp interface, it seems that changing the postOrderIterator2(BOp) method is unlikely to be the correct fix. It seems that either the AST query nodes should override the arity() function to return the count of the annotations map, or the ASTWildcardProjectionOptimizer should use its own iterator for the nodes of the query. The latter option would be the least impactful change but I am not familiar with the codebase to understand the correct fix. Any help in resolving the issue would be appreciated. |
|
From: Brad B. <be...@sy...> - 2015-04-20 13:06:55
|
Alex, I wanted to pass along this thread as well: http://sourceforge.net/p/bigdata/discussion/676946/thread/156e8e70/?limit=25#2edb. Thanks, --Brad On Wed, Apr 15, 2015 at 9:02 AM, Alex Jouravlev < al...@bu...> wrote: > Hi Brad, > > Good question - why do I want to work directly? For starters, because I > don't know Sesame and didn't know why do I need another system. Plus, I am > mainly in the investigation stage. I wanted to try to do the following: > > - Try to work with temporal knowledge either using the fourth value, > or using RDR. In one of the projects, I need to run a reasoning at > particular time. > - I wanted to emulate the situation when several data modules are > independently version controlled. Say I convert an excel sheet into a set > of quads. Then the sheet is replaced. > - I wanted to experiment your truth maintenance mode. More > specifically, I want to be able to know the impact of new data. > - I want to see how you record inference rules, and how I can edit > them. > - Everything you write about inference with quads > > Also, I am piloting one project that should include a quad store (for > semantic integration of metadata), stanford NLP and some generation of > diagrams. I thought if I have it embedded, it would be easier to develop. > > Cheers, > > Alex > > Alex Jouravlev > Director, Business Abstraction Pty Ltd > Phone: +61-(2)-8003-4830 > Mobile: +61-4-0408-3258 > Web: http://www.businessabstraction.com > LinkedIn: http://au.linkedin.com/in/alexjouravlev/ > > On Wed, Apr 15, 2015 at 10:30 PM, Brad Bebee <be...@sy...> wrote: > >> Alex, >> >> One aspect to consider here is the ultimate scalability of your >> application. Currently, the HA and scale-out features are only support >> through integration with the REST API [1]. If you think you will need >> those features, then you should definitely go with the REST API. There is >> a Java client for that [2] that could be used in Scala with relatively >> little effort. You can implement Stored Queries and other features on the >> server-side to achieve high performance [3]. The upcoming 1.5.2 release at >> the end of this month will have transaction support at the REST and SAIL >> layers [4]. >> >> Did you have features in particular you wanted to use outside of the SAIL >> API? >> >> Thanks, --Brad >> >> [1] http://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#REST_API >> [2] http://wiki.blazegraph.com/wiki/index.php/JettyHttpClient >> [3] http://wiki.blazegraph.com/wiki/index.php/StoredQuery >> [4] http://trac.bigdata.com/ticket/1156 >> >> On Tue, Apr 14, 2015 at 7:43 PM, Alex Jouravlev < >> al...@bu...> wrote: >> >>> Hi everyone, >>> >>> What would be the first steps to start using bigdata in the full mode, >>> embedded into a scala/Lift application? I don't want to use Sesame layer >>> as, if I understand correctly, that would strip out a lot of interesting >>> features. >>> >>> <http://au.linkedin.com/in/alexjouravlev/> >>> >>> Alx >>> >>> >>> ------------------------------------------------------------------------------ >>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>> Develop your own process in accordance with the BPMN 2 standard >>> Learn Process modeling best practices with Bonita BPM through live >>> exercises >>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>> event?utm_ >>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>> _______________________________________________ >>> Bigdata-developers mailing list >>> Big...@li... >>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>> >>> >> >> >> -- >> _______________ >> Brad Bebee >> Managing Partner >> SYSTAP, LLC >> e: be...@sy... >> m: 202.642.7961 >> f: 571.367.5000 >> w: www.systap.com >> >> Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> > > -- _______________ Brad Bebee Managing Partner SYSTAP, LLC e: be...@sy... m: 202.642.7961 f: 571.367.5000 w: www.systap.com Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. |
|
From: Bryan T. <br...@sy...> - 2015-04-16 15:30:13
|
The transaction management API is logically scoped to the database, not the
namespace. Therefore we need to pass in the RemoteRepositoryManager object
to the BigdataSailRemoteRepository so we can access the transaction
management API.
I just want to bring this up since there is ongoing confusion in the
semantics of classes that attempt to simplify life for the "just one
namespace" users (blueprints, sail factory). That simplification is still
possible of course, but internally we need a RemoteRepositoryManager (which
currently extends RemoteRepository).
That extended subclass provides the "default" namespace. I think it would
make sense to make the RemoteRepository a parallel class and then always
use a method to obtain a RemoteRepository for either the default namespace
or a specific namespace:
public RemoteRepository getRepositoryForDefaultNamespace() // proposed
method
public RemoteRepository getRepositoryForNamespace(final String
namespace) // existing method.
See
http://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#Transaction_Management
for the proposed transaction management API.
Bryan
|
|
From: Bryan T. <br...@sy...> - 2015-04-15 23:38:44
|
We do not (yet) support RDF*/SPARQL* (aka RDR) for quads. However, it is easy enough to model quads. <<:bob foaf:age 25>> inGraph <:graph1> This would give you something that had a concept of a named graph container (:graph1) but used only the indices for a triple store (SPO, POS, OSP) rather than the 6 indices for a quad store. Since the namespace would be in a triples mode, it would not understand the GRAPH ?g syntax in SPARQL. (The equivalent semantics could be obtained by writing an RDR triple pattern to pull out the inGraph statements, but you need to watch out for how you handle the same statement appearing in more than one graph. The "default graph" semantics say that the only the distinct triples are visited by the triple pattern. This is in contrast to named graph queries in which the distinct quads are visited. More or less. The actual definition has to do with the RDF merge of the triples in the named graphs that are visible to the query and is all pretty complex.) However, if you have a bunch of statements that naturally can be clustered into "graphs" and you want the metadata to be about the graph and not the individual statements in that graph, then just use quads. We do have RDR support for quads on the roadmap. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 15, 2015 at 4:55 PM, Max Ott <ma...@ni...> wrote: > Hi, > > While the RDR extension would seem to fit my use case very well, it seems > to be limited to a single statement. What I would need is being able to > describe contexts (as in the fourth element of a quad store). Something > along the lines of: > > @prefix : <http://bigdata.com> . > @prefix foaf: <http://xmlns.com/foaf/0.1/> . > @prefix dct: <http://purl.org/dc/elements/1.1/> . > > <<:bob foaf:age 25; foaf:name "Bobby".>> dct:creator < > http://example.com/crawlers#c1> ; > dct:source <http://example.net/bobby.html> . > > Unfortunately, that doesn't seem to be supported. Or am I missing > something? > > Do I just need to forget about RDR in this case and figure out on how to > do this with named graphs and quads? It's really the query side which I'm > focusing now. > > Any help would be greatly appreciated. > -max > > > ------------------------------------------------------------------------------ > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT > Develop your own process in accordance with the BPMN 2 standard > Learn Process modeling best practices with Bonita BPM through live > exercises > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- > event?utm_ > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Jeremy J C. <jj...@sy...> - 2015-04-15 21:24:31
|
I will update later with a different experiment in which medium size (1000 or so triples) INSERTS (with no DELETE) are replaced by LOAD GRAPH calls
Jeremy
> On Apr 14, 2015, at 9:58 AM, Jeremy J Carroll <jj...@sy...> wrote:
>
> I found a CONSTRUCT and LOAD much more performant than a DELETE/INSERT, and was wondering why, and whether there is anything new (to me) about the blazegraph architecture that I should understand.
>
> =====
>
>
> I had a graph for which I wished to rename almost all URIs.
> The graph had about 3M triples
> I was working in AWS on
>
> I constructed a temporary graph with a rename mapping
> and then tried the following update query:
>
> DELETE {
> GRAPH <%(abox)s> {
> ?oldS ?oldP ?oldO
> }
> }
> INSERT {
> GRAPH <%(abox)s> {
> ?newS ?newP ?newO
> }
> }
> WHERE {
> graph <%(abox)s> {
> ?oldS ?oldP ?oldO
> }
> GRAPH <x-eg:temporary-graph> {
> ?oldS <x-eg:replaced-by> ?newS
> }
> GRAPH <x-eg:temporary-graph> {
> ?oldP <x-eg:replaced-by> ?newP
> }
> {
> GRAPH <x-eg:temporary-graph> {
> ?oldO <x-eg:replaced-by> ?newO
> }
> } UNION {
> graph <%(abox)s> {
> ?oldS ?oldP ?oldO
> }
> FILTER ( isLiteral(?oldO) )
> BIND ( ?oldO as ?newO )
> }
> }
>
>
>
> where <%(abox)s> is a variable
>
>
> At the point where we perform this query we have exclusive access to the blaze graph process.
>
> It took over 4 hours, with approx. the first hour showing some change in the query execution stats, and then the last 3 hours showing no change in the stats (the status page in the NSS display is not very useful with these update queries).
> After 4 hours I got bored. Cancel did not work. So I killed blazegraph and restarted.
>
> I then rewrote the code as follows.
>
>
> I wrote a construct query:
>
> CONSTRUCT {
> ?newS ?newP ?newO
> }
> WHERE {
> graph <%(abox)s> {
> ?oldS ?oldP ?oldO
> }
> GRAPH <x-eg:temporary-graph> {
> ?oldS <x-eg:replaced-by> ?newS
> }
> GRAPH <x-eg:temporary-graph> {
> ?oldP <x-eg:replaced-by> ?newP
> }
> {
> GRAPH <x-eg:temporary-graph> {
> ?oldO <x-eg:replaced-by> ?newO
> }
> } UNION {
> graph <%(abox)s> {
> ?oldS ?oldP ?oldO
> }
> FILTER ( isLiteral(?oldO) )
> BIND ( ?oldO as ?newO )
> }
> }
>
> this created a temporary file.
>
> I replaced the DELETE part with
>
> DROP GRAPH <%(abox)s>
>
> and the INSERT with
>
> LOAD <file://%(tmpfile)s> INTO GRAPH <%(abox)s>
>
> ====
>
>
> The rewritten code took only a few minutes (less than 5 in total)
> I was expecting some improvement, but not as much as I saw.
>
> My understanding is that each of the three operations is atomic and isolated, but I lost the guarantee linking the three (which I did not need since I had exclusive lock at a higher level).
>
> Was it the atomicity that cost so much?
>
> Jeremy
>
>
>
>
>
>
>
>
|
|
From: Max O. <ma...@ni...> - 2015-04-15 20:55:07
|
Hi, While the RDR extension would seem to fit my use case very well, it seems to be limited to a single statement. What I would need is being able to describe contexts (as in the fourth element of a quad store). Something along the lines of: @prefix : <http://bigdata.com> . @prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix dct: <http://purl.org/dc/elements/1.1/> . <<:bob foaf:age 25; foaf:name "Bobby".>> dct:creator < http://example.com/crawlers#c1> ; dct:source <http://example.net/bobby.html> . Unfortunately, that doesn't seem to be supported. Or am I missing something? Do I just need to forget about RDR in this case and figure out on how to do this with named graphs and quads? It's really the query side which I'm focusing now. Any help would be greatly appreciated. -max |
|
From: Max O. <ma...@ni...> - 2015-04-15 20:39:37
|
Wonderful. Got it to work now. Thanks, -max On Thu, Apr 16, 2015 at 12:33 AM Brad Bebee <be...@sy...> wrote: > Max, > > There were two issues, which are below. Thank you for pointing out the > Wiki issue. It's been updated and corrected: > http://wiki.blazegraph.com/wiki/index.php/Reification_Done_Right#Reification_Done_Right > . Let us know how it works for you. > > Thanks, --Brad > > 1. The dublin core name space needed to be declared: > > @prefix : <http://bigdata.com> . > @prefix foaf: <http://xmlns.com/foaf/0.1/ > > > > @prefix dct: <http://purl.org/dc/elements/1.1/> . > > :bob foaf:name "Bob" . > <<:bob foaf:age 23>> dct:creator <http://example.com/crawlers#c1> ; > dct:source <http://example.net/homepage-listing.html> . > > 2. The RDR content-type needed to be specified in the CURL request. > > curl -D -H 'Content-Type: application/x-turtle-RDR' --upload-file bob.tut > -X POST ' > http://localhost:9999/bigdata/namespace/test/sparql/test/sparql?context-uri=http://example.org/tbox > ' > > > > On Wed, Apr 15, 2015 at 10:04 AM, Bryan Thompson <br...@sy...> wrote: > >> The RDR specific mime types are also defined here: >> http://wiki.blazegraph.com/wiki/index.php/Reification_Done_Right#RDF_Data_Interchange >> >> Thanks, >> Bryan >> >> ---- >> Bryan Thompson >> Chief Scientist & Founder >> SYSTAP, LLC >> 4501 Tower Road >> Greensboro, NC 27410 >> br...@sy... >> http://blazegraph.com >> http://blog.bigdata.com <http://bigdata.com> >> http://mapgraph.io >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> On Wed, Apr 15, 2015 at 10:03 AM, Bryan Thompson <br...@sy...> >> wrote: >> >>> Max, >>> >>> I've fixed the wiki. >>> >>> The problem is the Content-Type header. I have modified the servlet to >>> report back the Content-Type header in the error message. >>> >>> You are probably specifying Content-Type=application/x-turtle. When I >>> use this value I get the same error message. >>> >>> You need to specify *application/x-turtle-RDR* per >>> http://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#MIME_Types >>> >>> Thanks, >>> Bryan >>> ---- >>> Bryan Thompson >>> Chief Scientist & Founder >>> SYSTAP, LLC >>> 4501 Tower Road >>> Greensboro, NC 27410 >>> br...@sy... >>> http://blazegraph.com >>> http://blog.bigdata.com <http://bigdata.com> >>> http://mapgraph.io >>> >>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >>> technology to use GPUs to accelerate data-parallel graph analytics. >>> >>> CONFIDENTIALITY NOTICE: This email and its contents and attachments >>> are for the sole use of the intended recipient(s) and are confidential or >>> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >>> dissemination or copying of this email or its contents or attachments is >>> prohibited. If you have received this communication in error, please notify >>> the sender by reply email and permanently delete all copies of the email >>> and its contents and attachments. >>> >>> On Tue, Apr 14, 2015 at 10:00 PM, Max Ott <ma...@ni...> wrote: >>> >>>> Hi, >>>> >>>> I have trying to get the RDR extension working as this is the feature >>>> I'm most interested in. So I copied a simple statement from the web site >>>> into a file called 'bob.tut' >>>> >>>> @prefix : <http://bigdata.com> . >>>> @prefix foaf: <http://xmlns.com/foaf/0.1/> . >>>> >>>> :bob foaf:name "Bob" . >>>> <<:bob foaf:age 23>> dct:creator <http://example.com/crawlers#c1> ; >>>> dct:source < >>>> http://example.net/homepage-listing.html> . >>>> >>>> (BTW, the trailing dot on line 2 and ‘;’ on line 5 is missing in the >>>> wiki writeup). >>>> >>>> I then created a namespace “test” with RDR selected which gave me the >>>> following service description: >>>> >>>> …. >>>> <endpoint xmlns="http://www.w3.org/ns/sparql-service-description#" >>>> rdf:resource="http://localhost:9999/bigdata/namespace/test/sparql”/> >>>> … >>>> <feature xmlns=" >>>> http://www.w3.org/ns/sparql-service-description#" rdf:resource=" >>>> http://www.bigdata.com/rdf#/features/KB/Mode/Sids"/> >>>> <feature xmlns=" >>>> http://www.w3.org/ns/sparql-service-description#" rdf:resource=" >>>> http://www.bigdata.com/rdf#/features/KB/TruthMaintenance"/> >>>> >>>> If I now try to upload ‘bob’, I get the following error: >>>> >>>> $ curl -D- -H 'Content-Type: text/turtle' --upload-file bob.tut -X POST >>>> ' >>>> http://localhost:9999/bigdata/namespace/test/sparql/test/sparql?context-uri=http://example.org/tbox' >>>> >>>> HTTP/1.1 100 Continue >>>> >>>> HTTP/1.1 500 Server Error >>>> Content-Type: text/plain >>>> Transfer-Encoding: chunked >>>> Server: Jetty(9.2.3.v20140905) >>>> >>>> INSERT-WITH-BODY: baseURI= >>>> http://192.168.59.3:9999/bigdata/namespace/test/sparql, context-uri=[ >>>> http://example.org/tbox] >>>> java.util.concurrent.ExecutionException: >>>> org.openrdf.rio.RDFParseException: IRI included an unencoded space: '32' >>>> [line 5] >>>> …. >>>> >>>> Looks like the RDF parser didn’t get the memo that I’m sending RDR >>>> data, but I have no idea what I’m doing wrong, nor apparently what I'm >>>> doing - period :) >>>> >>>> Any help would be greatly appreciated. >>>> >>>> Cheers, >>>> -max >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>>> Develop your own process in accordance with the BPMN 2 standard >>>> Learn Process modeling best practices with Bonita BPM through live >>>> exercises >>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>>> event?utm_ >>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>>> _______________________________________________ >>>> Bigdata-developers mailing list >>>> Big...@li... >>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>>> >>>> >>> >> >> >> ------------------------------------------------------------------------------ >> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >> Develop your own process in accordance with the BPMN 2 standard >> Learn Process modeling best practices with Bonita BPM through live >> exercises >> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >> event?utm_ >> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >> _______________________________________________ >> Bigdata-developers mailing list >> Big...@li... >> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >> >> > > > -- > _______________ > Brad Bebee > Managing Partner > SYSTAP, LLC > e: be...@sy... > m: 202.642.7961 > f: 571.367.5000 > w: www.systap.com > > > Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > |
|
From: Bryan T. <br...@sy...> - 2015-04-15 15:09:14
|
You can run any given test suite using "ant -DtestName=<classname> junit". I would generally suggest the appropriate TestAll class near what you have touched. Your pain will be greatly reduced once we get #168 (mavenization) a little further along. This is moving along pretty quickly right now and should be in 1.5.2 a little later this month. Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 15, 2015 at 10:58 AM, Nikolas Everett <ni...@gm...> wrote: > The tests I ran pass. I imagine you should squash the commits into one if > possible. > > I'm not sure how to run all the tests with ant. Its always my complaint > with ant :) I have to look through the build.xml to find the command to run > tests. I couldn't find it in this case. > > I'll give it a more thorough shakedown this afternoon - backporting it and > getting some more use out of it. > > Nik > > On Wed, Apr 15, 2015 at 10:46 AM, Bryan Thompson <br...@sy...> wrote: > >> I've attempted the merge to master. It looks ok for me locally and I have >> committed the changes. Can you verify that it looks good for you? >> >> Thanks, >> Bryan >> >> ---- >> Bryan Thompson >> Chief Scientist & Founder >> SYSTAP, LLC >> 4501 Tower Road >> Greensboro, NC 27410 >> br...@sy... >> http://blazegraph.com >> http://blog.bigdata.com <http://bigdata.com> >> http://mapgraph.io >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> On Wed, Apr 15, 2015 at 10:30 AM, Nikolas Everett <ni...@gm...> >> wrote: >> >>> Sourceforge wasn't letting me update the merge request in any way I'm >>> used to so I made another one: >>> https://sourceforge.net/p/bigdata/git/merge-requests/2/ >>> >>> I spent some time poking around documentation but didn't look too hard >>> so I'm sure I'm doing it wrong. If you know the right way please let me >>> know. >>> >>> The trouble was that when I tried to push a new commit to the branch for >>> my merge request sourceforge rejected it. It looks like sourceforge had >>> amended my commit bringing its branch out of whack with my local branch. >>> That combined with sourceforge being slow and not seeing the way to comment >>> on individual lines is getting me grumpy about sourceforge merge requests. >>> >>> Nik >>> >>> On Tue, Apr 14, 2015 at 12:44 PM, Nikolas Everett <ni...@gm...> >>> wrote: >>> >>>> I didn't! I suspect sourceforge isn't emailing me. I'll have a look >>>> at it this afternoon. >>>> >>>> Nik >>>> >>>> On Tue, Apr 14, 2015 at 12:29 PM, Bryan Thompson <br...@sy...> >>>> wrote: >>>> >>>>> Nik, >>>>> >>>>> Did you see the feedback on that merge request? >>>>> >>>>> Thanks, >>>>> Bryan >>>>> >>>>> ---- >>>>> Bryan Thompson >>>>> Chief Scientist & Founder >>>>> SYSTAP, LLC >>>>> 4501 Tower Road >>>>> Greensboro, NC 27410 >>>>> br...@sy... >>>>> http://blazegraph.com >>>>> http://blog.bigdata.com <http://bigdata.com> >>>>> http://mapgraph.io >>>>> >>>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra >>>>> high-performance graph database that supports both RDF/SPARQL and >>>>> Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is >>>>> our disruptive new technology to use GPUs to accelerate data-parallel graph >>>>> analytics. >>>>> >>>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments >>>>> are for the sole use of the intended recipient(s) and are confidential or >>>>> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >>>>> dissemination or copying of this email or its contents or attachments is >>>>> prohibited. If you have received this communication in error, please notify >>>>> the sender by reply email and permanently delete all copies of the email >>>>> and its contents and attachments. >>>>> >>>>> On Mon, Apr 13, 2015 at 11:09 AM, Nikolas Everett <ni...@gm...> >>>>> wrote: >>>>> >>>>>> I filed 1179 <http://trac.bigdata.com/ticket/1179#ticket> a couple >>>>>> of days ago and got together a patch this morning then read >>>>>> http://wiki.blazegraph.com/wiki/index.php/Contributors and saw that >>>>>> I should email here about it. So here goes: >>>>>> >>>>>> I want inline URIs to have the same power that inline values do. >>>>>> Right now inline values can be modified on the way into and out of the >>>>>> index. Inline URIs can only be modified on the way into the index. That >>>>>> leaves you pretty limited in what you can represent with them. Its OK if >>>>>> you can design the URI scheme around the limitation but we can't. >>>>>> >>>>>> I've put my patch as a gist >>>>>> <https://gist.github.com/nik9000/1dc5248f6035349ca2c1> and I've >>>>>> created a "merge request" >>>>>> <https://sourceforge.net/p/bigdata/git/merge-requests/1/> for it. It >>>>>> gives the InlineURIHandler that built the inline URI a crack at building >>>>>> the localName from the inlined value. The cost is an extra HashMap lookup >>>>>> whenever the uri is un-inlined and the default behavior is unchanged. It >>>>>> doesn't change the on disk representation either. >>>>>> >>>>>> Nik >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>>>>> Develop your own process in accordance with the BPMN 2 standard >>>>>> Learn Process modeling best practices with Bonita BPM through live >>>>>> exercises >>>>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>>>>> event?utm_ >>>>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>>>>> _______________________________________________ >>>>>> Bigdata-developers mailing list >>>>>> Big...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>>>>> >>>>>> >>>>> >>>> >>> >> > |
|
From: Bryan T. <br...@sy...> - 2015-04-15 15:05:58
|
Max, I've fixed the wiki. The problem is the Content-Type header. I have modified the servlet to report back the Content-Type header in the error message. You are probably specifying Content-Type=application/x-turtle. When I use this value I get the same error message. You need to specify *application/x-turtle-RDR* per http://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#MIME_Types Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Tue, Apr 14, 2015 at 10:00 PM, Max Ott <ma...@ni...> wrote: > Hi, > > I have trying to get the RDR extension working as this is the feature I'm > most interested in. So I copied a simple statement from the web site into a > file called 'bob.tut' > > @prefix : <http://bigdata.com> . > @prefix foaf: <http://xmlns.com/foaf/0.1/> . > > :bob foaf:name "Bob" . > <<:bob foaf:age 23>> dct:creator <http://example.com/crawlers#c1> ; > dct:source <http://example.net/homepage-listing.html> > . > > (BTW, the trailing dot on line 2 and ‘;’ on line 5 is missing in the wiki > writeup). > > I then created a namespace “test” with RDR selected which gave me the > following service description: > > …. > <endpoint xmlns="http://www.w3.org/ns/sparql-service-description#" > rdf:resource="http://localhost:9999/bigdata/namespace/test/sparql”/> > … > <feature xmlns="http://www.w3.org/ns/sparql-service-description#" > rdf:resource="http://www.bigdata.com/rdf#/features/KB/Mode/Sids"/> > <feature xmlns="http://www.w3.org/ns/sparql-service-description#" > rdf:resource="http://www.bigdata.com/rdf#/features/KB/TruthMaintenance"/> > > If I now try to upload ‘bob’, I get the following error: > > $ curl -D- -H 'Content-Type: text/turtle' --upload-file bob.tut -X POST ' > http://localhost:9999/bigdata/namespace/test/sparql/test/sparql?context-uri=http://example.org/tbox' > > HTTP/1.1 100 Continue > > HTTP/1.1 500 Server Error > Content-Type: text/plain > Transfer-Encoding: chunked > Server: Jetty(9.2.3.v20140905) > > INSERT-WITH-BODY: baseURI= > http://192.168.59.3:9999/bigdata/namespace/test/sparql, context-uri=[ > http://example.org/tbox] > java.util.concurrent.ExecutionException: > org.openrdf.rio.RDFParseException: IRI included an unencoded space: '32' > [line 5] > …. > > Looks like the RDF parser didn’t get the memo that I’m sending RDR data, > but I have no idea what I’m doing wrong, nor apparently what I'm doing - > period :) > > Any help would be greatly appreciated. > > Cheers, > -max > > > > ------------------------------------------------------------------------------ > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT > Develop your own process in accordance with the BPMN 2 standard > Learn Process modeling best practices with Bonita BPM through live > exercises > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- > event?utm_ > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
|
From: Nikolas E. <ni...@gm...> - 2015-04-15 14:59:24
|
The tests I ran pass. I imagine you should squash the commits into one if possible. I'm not sure how to run all the tests with ant. Its always my complaint with ant :) I have to look through the build.xml to find the command to run tests. I couldn't find it in this case. I'll give it a more thorough shakedown this afternoon - backporting it and getting some more use out of it. Nik On Wed, Apr 15, 2015 at 10:46 AM, Bryan Thompson <br...@sy...> wrote: > I've attempted the merge to master. It looks ok for me locally and I have > committed the changes. Can you verify that it looks good for you? > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Wed, Apr 15, 2015 at 10:30 AM, Nikolas Everett <ni...@gm...> > wrote: > >> Sourceforge wasn't letting me update the merge request in any way I'm >> used to so I made another one: >> https://sourceforge.net/p/bigdata/git/merge-requests/2/ >> >> I spent some time poking around documentation but didn't look too hard so >> I'm sure I'm doing it wrong. If you know the right way please let me know. >> >> The trouble was that when I tried to push a new commit to the branch for >> my merge request sourceforge rejected it. It looks like sourceforge had >> amended my commit bringing its branch out of whack with my local branch. >> That combined with sourceforge being slow and not seeing the way to comment >> on individual lines is getting me grumpy about sourceforge merge requests. >> >> Nik >> >> On Tue, Apr 14, 2015 at 12:44 PM, Nikolas Everett <ni...@gm...> >> wrote: >> >>> I didn't! I suspect sourceforge isn't emailing me. I'll have a look at >>> it this afternoon. >>> >>> Nik >>> >>> On Tue, Apr 14, 2015 at 12:29 PM, Bryan Thompson <br...@sy...> >>> wrote: >>> >>>> Nik, >>>> >>>> Did you see the feedback on that merge request? >>>> >>>> Thanks, >>>> Bryan >>>> >>>> ---- >>>> Bryan Thompson >>>> Chief Scientist & Founder >>>> SYSTAP, LLC >>>> 4501 Tower Road >>>> Greensboro, NC 27410 >>>> br...@sy... >>>> http://blazegraph.com >>>> http://blog.bigdata.com <http://bigdata.com> >>>> http://mapgraph.io >>>> >>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >>>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive >>>> new technology to use GPUs to accelerate data-parallel graph analytics. >>>> >>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments >>>> are for the sole use of the intended recipient(s) and are confidential or >>>> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >>>> dissemination or copying of this email or its contents or attachments is >>>> prohibited. If you have received this communication in error, please notify >>>> the sender by reply email and permanently delete all copies of the email >>>> and its contents and attachments. >>>> >>>> On Mon, Apr 13, 2015 at 11:09 AM, Nikolas Everett <ni...@gm...> >>>> wrote: >>>> >>>>> I filed 1179 <http://trac.bigdata.com/ticket/1179#ticket> a couple of >>>>> days ago and got together a patch this morning then read >>>>> http://wiki.blazegraph.com/wiki/index.php/Contributors and saw that I >>>>> should email here about it. So here goes: >>>>> >>>>> I want inline URIs to have the same power that inline values do. >>>>> Right now inline values can be modified on the way into and out of the >>>>> index. Inline URIs can only be modified on the way into the index. That >>>>> leaves you pretty limited in what you can represent with them. Its OK if >>>>> you can design the URI scheme around the limitation but we can't. >>>>> >>>>> I've put my patch as a gist >>>>> <https://gist.github.com/nik9000/1dc5248f6035349ca2c1> and I've >>>>> created a "merge request" >>>>> <https://sourceforge.net/p/bigdata/git/merge-requests/1/> for it. It >>>>> gives the InlineURIHandler that built the inline URI a crack at building >>>>> the localName from the inlined value. The cost is an extra HashMap lookup >>>>> whenever the uri is un-inlined and the default behavior is unchanged. It >>>>> doesn't change the on disk representation either. >>>>> >>>>> Nik >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>>>> Develop your own process in accordance with the BPMN 2 standard >>>>> Learn Process modeling best practices with Bonita BPM through live >>>>> exercises >>>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>>>> event?utm_ >>>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>>>> _______________________________________________ >>>>> Bigdata-developers mailing list >>>>> Big...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>>>> >>>>> >>>> >>> >> > |
|
From: Bryan T. <br...@sy...> - 2015-04-15 14:47:29
|
I've attempted the merge to master. It looks ok for me locally and I have committed the changes. Can you verify that it looks good for you? Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Apr 15, 2015 at 10:30 AM, Nikolas Everett <ni...@gm...> wrote: > Sourceforge wasn't letting me update the merge request in any way I'm used > to so I made another one: > https://sourceforge.net/p/bigdata/git/merge-requests/2/ > > I spent some time poking around documentation but didn't look too hard so > I'm sure I'm doing it wrong. If you know the right way please let me know. > > The trouble was that when I tried to push a new commit to the branch for > my merge request sourceforge rejected it. It looks like sourceforge had > amended my commit bringing its branch out of whack with my local branch. > That combined with sourceforge being slow and not seeing the way to comment > on individual lines is getting me grumpy about sourceforge merge requests. > > Nik > > On Tue, Apr 14, 2015 at 12:44 PM, Nikolas Everett <ni...@gm...> > wrote: > >> I didn't! I suspect sourceforge isn't emailing me. I'll have a look at >> it this afternoon. >> >> Nik >> >> On Tue, Apr 14, 2015 at 12:29 PM, Bryan Thompson <br...@sy...> >> wrote: >> >>> Nik, >>> >>> Did you see the feedback on that merge request? >>> >>> Thanks, >>> Bryan >>> >>> ---- >>> Bryan Thompson >>> Chief Scientist & Founder >>> SYSTAP, LLC >>> 4501 Tower Road >>> Greensboro, NC 27410 >>> br...@sy... >>> http://blazegraph.com >>> http://blog.bigdata.com <http://bigdata.com> >>> http://mapgraph.io >>> >>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >>> technology to use GPUs to accelerate data-parallel graph analytics. >>> >>> CONFIDENTIALITY NOTICE: This email and its contents and attachments >>> are for the sole use of the intended recipient(s) and are confidential or >>> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >>> dissemination or copying of this email or its contents or attachments is >>> prohibited. If you have received this communication in error, please notify >>> the sender by reply email and permanently delete all copies of the email >>> and its contents and attachments. >>> >>> On Mon, Apr 13, 2015 at 11:09 AM, Nikolas Everett <ni...@gm...> >>> wrote: >>> >>>> I filed 1179 <http://trac.bigdata.com/ticket/1179#ticket> a couple of >>>> days ago and got together a patch this morning then read >>>> http://wiki.blazegraph.com/wiki/index.php/Contributors and saw that I >>>> should email here about it. So here goes: >>>> >>>> I want inline URIs to have the same power that inline values do. Right >>>> now inline values can be modified on the way into and out of the index. >>>> Inline URIs can only be modified on the way into the index. That leaves you >>>> pretty limited in what you can represent with them. Its OK if you can >>>> design the URI scheme around the limitation but we can't. >>>> >>>> I've put my patch as a gist >>>> <https://gist.github.com/nik9000/1dc5248f6035349ca2c1> and I've >>>> created a "merge request" >>>> <https://sourceforge.net/p/bigdata/git/merge-requests/1/> for it. It >>>> gives the InlineURIHandler that built the inline URI a crack at building >>>> the localName from the inlined value. The cost is an extra HashMap lookup >>>> whenever the uri is un-inlined and the default behavior is unchanged. It >>>> doesn't change the on disk representation either. >>>> >>>> Nik >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>>> Develop your own process in accordance with the BPMN 2 standard >>>> Learn Process modeling best practices with Bonita BPM through live >>>> exercises >>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>>> event?utm_ >>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>>> _______________________________________________ >>>> Bigdata-developers mailing list >>>> Big...@li... >>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>>> >>>> >>> >> > |
|
From: Brad B. <be...@sy...> - 2015-04-15 14:33:23
|
Max, There were two issues, which are below. Thank you for pointing out the Wiki issue. It's been updated and corrected: http://wiki.blazegraph.com/wiki/index.php/Reification_Done_Right#Reification_Done_Right . Let us know how it works for you. Thanks, --Brad 1. The dublin core name space needed to be declared: @prefix : <http://bigdata.com> . @prefix foaf: <http://xmlns.com/foaf/0.1/> @prefix dct: <http://purl.org/dc/elements/1.1/> . :bob foaf:name "Bob" . <<:bob foaf:age 23>> dct:creator <http://example.com/crawlers#c1> ; dct:source <http://example.net/homepage-listing.html> . 2. The RDR content-type needed to be specified in the CURL request. curl -D -H 'Content-Type: application/x-turtle-RDR' --upload-file bob.tut -X POST ' http://localhost:9999/bigdata/namespace/test/sparql/test/sparql?context-uri=http://example.org/tbox ' On Wed, Apr 15, 2015 at 10:04 AM, Bryan Thompson <br...@sy...> wrote: > The RDR specific mime types are also defined here: > http://wiki.blazegraph.com/wiki/index.php/Reification_Done_Right#RDF_Data_Interchange > > Thanks, > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@sy... > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new > technology to use GPUs to accelerate data-parallel graph analytics. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Wed, Apr 15, 2015 at 10:03 AM, Bryan Thompson <br...@sy...> wrote: > >> Max, >> >> I've fixed the wiki. >> >> The problem is the Content-Type header. I have modified the servlet to >> report back the Content-Type header in the error message. >> >> You are probably specifying Content-Type=application/x-turtle. When I >> use this value I get the same error message. >> >> You need to specify *application/x-turtle-RDR* per >> http://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#MIME_Types >> >> Thanks, >> Bryan >> ---- >> Bryan Thompson >> Chief Scientist & Founder >> SYSTAP, LLC >> 4501 Tower Road >> Greensboro, NC 27410 >> br...@sy... >> http://blazegraph.com >> http://blog.bigdata.com <http://bigdata.com> >> http://mapgraph.io >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> On Tue, Apr 14, 2015 at 10:00 PM, Max Ott <ma...@ni...> wrote: >> >>> Hi, >>> >>> I have trying to get the RDR extension working as this is the feature >>> I'm most interested in. So I copied a simple statement from the web site >>> into a file called 'bob.tut' >>> >>> @prefix : <http://bigdata.com> . >>> @prefix foaf: <http://xmlns.com/foaf/0.1/> . >>> >>> :bob foaf:name "Bob" . >>> <<:bob foaf:age 23>> dct:creator <http://example.com/crawlers#c1> ; >>> dct:source < >>> http://example.net/homepage-listing.html> . >>> >>> (BTW, the trailing dot on line 2 and ‘;’ on line 5 is missing in the >>> wiki writeup). >>> >>> I then created a namespace “test” with RDR selected which gave me the >>> following service description: >>> >>> …. >>> <endpoint xmlns="http://www.w3.org/ns/sparql-service-description#" >>> rdf:resource="http://localhost:9999/bigdata/namespace/test/sparql”/> >>> … >>> <feature xmlns="http://www.w3.org/ns/sparql-service-description#" >>> rdf:resource="http://www.bigdata.com/rdf#/features/KB/Mode/Sids"/> >>> <feature xmlns="http://www.w3.org/ns/sparql-service-description#" >>> rdf:resource="http://www.bigdata.com/rdf#/features/KB/TruthMaintenance >>> "/> >>> >>> If I now try to upload ‘bob’, I get the following error: >>> >>> $ curl -D- -H 'Content-Type: text/turtle' --upload-file bob.tut -X POST ' >>> http://localhost:9999/bigdata/namespace/test/sparql/test/sparql?context-uri=http://example.org/tbox' >>> >>> HTTP/1.1 100 Continue >>> >>> HTTP/1.1 500 Server Error >>> Content-Type: text/plain >>> Transfer-Encoding: chunked >>> Server: Jetty(9.2.3.v20140905) >>> >>> INSERT-WITH-BODY: baseURI= >>> http://192.168.59.3:9999/bigdata/namespace/test/sparql, context-uri=[ >>> http://example.org/tbox] >>> java.util.concurrent.ExecutionException: >>> org.openrdf.rio.RDFParseException: IRI included an unencoded space: '32' >>> [line 5] >>> …. >>> >>> Looks like the RDF parser didn’t get the memo that I’m sending RDR data, >>> but I have no idea what I’m doing wrong, nor apparently what I'm doing - >>> period :) >>> >>> Any help would be greatly appreciated. >>> >>> Cheers, >>> -max >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>> Develop your own process in accordance with the BPMN 2 standard >>> Learn Process modeling best practices with Bonita BPM through live >>> exercises >>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>> event?utm_ >>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>> _______________________________________________ >>> Bigdata-developers mailing list >>> Big...@li... >>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>> >>> >> > > > ------------------------------------------------------------------------------ > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT > Develop your own process in accordance with the BPMN 2 standard > Learn Process modeling best practices with Bonita BPM through live > exercises > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- > event?utm_ > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > -- _______________ Brad Bebee Managing Partner SYSTAP, LLC e: be...@sy... m: 202.642.7961 f: 571.367.5000 w: www.systap.com Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP, LLC. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. |
|
From: Nikolas E. <ni...@gm...> - 2015-04-15 14:31:24
|
Sourceforge wasn't letting me update the merge request in any way I'm used to so I made another one: https://sourceforge.net/p/bigdata/git/merge-requests/2/ I spent some time poking around documentation but didn't look too hard so I'm sure I'm doing it wrong. If you know the right way please let me know. The trouble was that when I tried to push a new commit to the branch for my merge request sourceforge rejected it. It looks like sourceforge had amended my commit bringing its branch out of whack with my local branch. That combined with sourceforge being slow and not seeing the way to comment on individual lines is getting me grumpy about sourceforge merge requests. Nik On Tue, Apr 14, 2015 at 12:44 PM, Nikolas Everett <ni...@gm...> wrote: > I didn't! I suspect sourceforge isn't emailing me. I'll have a look at > it this afternoon. > > Nik > > On Tue, Apr 14, 2015 at 12:29 PM, Bryan Thompson <br...@sy...> wrote: > >> Nik, >> >> Did you see the feedback on that merge request? >> >> Thanks, >> Bryan >> >> ---- >> Bryan Thompson >> Chief Scientist & Founder >> SYSTAP, LLC >> 4501 Tower Road >> Greensboro, NC 27410 >> br...@sy... >> http://blazegraph.com >> http://blog.bigdata.com <http://bigdata.com> >> http://mapgraph.io >> >> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance >> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints >> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new >> technology to use GPUs to accelerate data-parallel graph analytics. >> >> CONFIDENTIALITY NOTICE: This email and its contents and attachments are >> for the sole use of the intended recipient(s) and are confidential or >> proprietary to SYSTAP. Any unauthorized review, use, disclosure, >> dissemination or copying of this email or its contents or attachments is >> prohibited. If you have received this communication in error, please notify >> the sender by reply email and permanently delete all copies of the email >> and its contents and attachments. >> >> On Mon, Apr 13, 2015 at 11:09 AM, Nikolas Everett <ni...@gm...> >> wrote: >> >>> I filed 1179 <http://trac.bigdata.com/ticket/1179#ticket> a couple of >>> days ago and got together a patch this morning then read >>> http://wiki.blazegraph.com/wiki/index.php/Contributors and saw that I >>> should email here about it. So here goes: >>> >>> I want inline URIs to have the same power that inline values do. Right >>> now inline values can be modified on the way into and out of the index. >>> Inline URIs can only be modified on the way into the index. That leaves you >>> pretty limited in what you can represent with them. Its OK if you can >>> design the URI scheme around the limitation but we can't. >>> >>> I've put my patch as a gist >>> <https://gist.github.com/nik9000/1dc5248f6035349ca2c1> and I've created >>> a "merge request" >>> <https://sourceforge.net/p/bigdata/git/merge-requests/1/> for it. It >>> gives the InlineURIHandler that built the inline URI a crack at building >>> the localName from the inlined value. The cost is an extra HashMap lookup >>> whenever the uri is un-inlined and the default behavior is unchanged. It >>> doesn't change the on disk representation either. >>> >>> Nik >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >>> Develop your own process in accordance with the BPMN 2 standard >>> Learn Process modeling best practices with Bonita BPM through live >>> exercises >>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- >>> event?utm_ >>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >>> _______________________________________________ >>> Bigdata-developers mailing list >>> Big...@li... >>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers >>> >>> >> > |
27 messages has been excluded from this view by a project administrator.
