Re: [Bigdata-developers] Current Revision of Blazegraph: Journal consumes extremely much disk space

[Bryan T. <br...@sy...>]

Was the data loaded into a new and empty journal or into a pre-existing
journal?  If the latter, what size was the journal and what data were in it?

Thanks,
Bryan

----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Wed, Apr 22, 2015 at 6:54 AM, Andreas Kahl <ka...@bs...> wrote:

> Bryan,
>
> yes, I used this command:
> curl -d"update=LOAD <file:///srv/feed-dateien/DNBLOD/GND.rdf.gz>;"
> -d"namespace=gnd" -d"monitor=true" http://localhost:8080/bigdata/sparql
>
> Best Regards
> Andreas
>
> >>> Bryan Thompson <br...@sy...> 22.04.15 12.51 Uhr >>>
> Andreas,
>
> What command did you use to load the data set?  I.e., SPARQL update "Load"
> or something else?
>
> Thanks,
> Bryan
>
> On Wednesday, April 22, 2015, Andreas Kahl <ka...@bs...> wrote:
>
> > Hello everyone,
> >
> > I currently updated to the current Revision (f4c63e5) of Blazegraph from
> > Git and tried to load a dataset into the updated Webapp. With Bigdata
> 1.4.0
> > this resulted in a journal of ~18GB. Now the process was cancelled
> because
> > the disk was full - the journal was beyond 50GB for the same file with
> the
> > same settings.
> > The only exception was that I activated GroupCommit.
> >
> > The dataset can be downloaded here:
> >
> http://datendienst.dnb.de/cgi-bin/mabit.pl?cmd=fetch&userID=opendata&pass=opendata&mabheft=GND.rdf.gz
> > .
> > Please find the settings used to load the file below.
> >
> > Do I have a misconfiguration, or is there a bug eating all disk memory?
> >
> > Best regards
> > Andreas
> >
> > Namespace-Properties:
> > curl -H "Accept: text/plain"
> > http://localhost:8080/bigdata/namespace/gnd/properties
> > #Wed Apr 22 11:35:31 CEST 2015
> > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=700
> > com.bigdata.relation.container=gnd
> > com.bigdata.rwstore.RWStore.smallSlotType=1024
> > com.bigdata.journal.AbstractJournal.bufferMode=DiskRW
> > com.bigdata.journal.AbstractJournal.file=/var/lib/bigdata/bigdata.jnl
> >
> >
> com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=de.bsb_muenchen.bigdata.vocab.B3KatVocabulary
> > com.bigdata.journal.AbstractJournal.initialExtent=209715200
> > com.bigdata.rdf.store.AbstractTripleStore.textIndex=true
> > com.bigdata.btree.BTree.branchingFactor=700
> >
> >
> com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms
> > com.bigdata.rdf.sail.isolatableIndices=false
> > com.bigdata.service.AbstractTransactionService.minReleaseAge=1
> > com.bigdata.rdf.sail.bufferCapacity=2000
> > com.bigdata.rdf.sail.truthMaintenance=false
> > com.bigdata.rdf.sail.namespace=gnd
> > com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore
> > com.bigdata.rdf.store.AbstractTripleStore.quads=false
> > com.bigdata.journal.AbstractJournal.writeCacheBufferCount=500
> > com.bigdata.search.FullTextIndex.fieldsEnabled=false
> > com.bigdata.relation.namespace=gnd
> > com.bigdata.journal.Journal.groupCommit=true
> > com.bigdata.btree.writeRetentionQueue.capacity=10000
> > com.bigdata.rdf.sail.BigdataSail.bufferCapacity=2000
> > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false
> >
>
>
> --
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
>