Menu
▾
▴
Re: [Bigdata-developers] Journal.Properties to bulk upload to a remote server
|
From: Bryan T. <br...@sy...> - 2015-11-04 13:16:07
|
Alex, If you are referring to the DataLoader, it is an embedded utility class. It is not designed to operate with a remote database instance. You can mimic many of the advantages of the DataLoader by increasing BigdataSail.Options.BUFFER_CAPACITY to 100,000. You should also follow the guidelines on the wiki for performance optimization if you are interested in bulk data load. See the section entitled Optimizations and benchmarking <https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#p-Optimizations_and_benchmarking>. E.g., https://wiki.blazegraph.com/wiki/index.php/IOOptimization. Some of the more important optimizations for write throughput are: - Write cache service native buffer pool size. - Use of URI inlining techniques if you have URIs that have numeric or UUID patterns embedded into them. - Fast disk. We have a number of improvements in the development branch that improve load speed, including code to overlap the parser with the index writers. Those will be in the 2.0 release. Thanks, Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@sy... http://blazegraph.com http://blog.blazegraph.com Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Wed, Nov 4, 2015 at 7:48 AM, Alex Muir <ale...@gm...> wrote: > Well I downloaded the blazegraph git examples and extracted the unique > bigdata properties. I don't think any of them are related to specifying a > remote server. > > Perhaps there is another way to specify to upload to a remote server with > the bulk loader? > > com.bigdata.btree.BTree.branchingFactor > com.bigdata.btree.keys.KeyBuilder.collator > com.bigdata.btree.writeRetentionQueue.capacity > com.bigdata.journal.AbstractJournal.bufferMode > com.bigdata.journal.AbstractJournal.file > com.bigdata.journal.AbstractJournal.initialExtent > com.bigdata.journal.AbstractJournal.maximumExtent > com.bigdata.journal.AbstractJournal.writeCacheBufferCount > > com.bigdata.namespace.BSBM_284826.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_284826.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_284826.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_284826.spo.OSP.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_284826.spo.POS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_284826.spo.SPO.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_566496.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_566496.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_566496.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_566496.spo.OSP.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_566496.spo.POS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.BSBM_566496.spo.SPO.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.spo.CSPO.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.spo.OCSP.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.spo.PCSO.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.spo.POCS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.spo.SOPC.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.chem2bio2rdf.spo.SPOC.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.dbpedia.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.dbpedia.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.dbpedia.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.dbpedia.spo.OSP.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.dbpedia.spo.POS.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.dbpedia.spo.SPO.com.bigdata.btree.BTree.branchingFactor > com.bigdata.namespace.kb.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor > com.bigdata.namespace.kb.lex.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.kb.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor > > com.bigdata.namespace.kb.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor > com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor > com.bigdata.rdf.rio.RDFParserOptions.stopAtFirstError > com.bigdata.rdf.sail.BigdataSail.bufferCapacity > com.bigdata.rdf.sail.BigdataSail.truthMaintenance > com.bigdata.rdf.sail.bufferCapacity > com.bigdata.rdf.sail.newEvalStrategy > com.bigdata.rdf.sail.queryTimeExpander > com.bigdata.rdf.sail.truthMaintenance > com.bigdata.rdf.store.AbstractTripleStore.axiomsClass > com.bigdata.rdf.store.AbstractTripleStore.bloomFilter > com.bigdata.rdf.store.AbstractTripleStore.extensionFactoryClass > com.bigdata.rdf.store.AbstractTripleStore.justify > com.bigdata.rdf.store.AbstractTripleStore.quads > com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers > com.bigdata.rdf.store.AbstractTripleStore.textIndex > com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass > com.bigdata.resource.OverflowManager.overflowEnabled > com.bigdata.service.AbstractTransactionService.minReleaseAge > com.bigdata.service.EmbeddedFederation.dataDir > com.bigdata.service.IBigdataClient.collectPlatformStatistics > > > > Regards > Alex > www.tilogeo.com > > On Wed, Nov 4, 2015 at 11:12 AM, Alex Muir <ale...@gm...> wrote: > >> Hi, >> >> I'm interested to bulk upload onto a remote server >> >> https://wiki.blazegraph.com/wiki/index.php/Bulk_Data_Load >> >> I assume that I can specify a remote server in the properties file >> however I'm thus far unable to find more information on what goes in a >> property file from the website. >> >> Is there a page defining all the properties? >> >> Regards >> Alex >> www.tilogeo.com >> > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bigdata-developers mailing list > Big...@li... > https://lists.sourceforge.net/lists/listinfo/bigdata-developers > > |
