Re: [Bigdata-developers] Journal.Properties to bulk upload to a remote server

[Alex M. <ale...@gm...>]

Okay thanks Brad and Bryan,  will use the REST API and look into
optimizations


Regards
Alex
www.tilogeo.com

On Wed, Nov 4, 2015 at 1:19 PM, Brad Bebee <be...@sy...> wrote:

> Alex,
>
> Adding to Bryan's comments, if you have a Blazegraph instance running
> remotely and you want to insert data into it, you can use the REST API to
> post URIs of the files to load.  Assuming you can make those URIs
> resolvable on the remote server, it will resolve the URIs and load them.
>
>
> https://wiki.blazegraph.com/wiki/index.php/REST_API#INSERT_RDF_.28POST_with_URLs.29
>
> Thanks, --Brad
>
> On Wed, Nov 4, 2015 at 8:15 AM, Bryan Thompson <br...@sy...> wrote:
>
>> Alex,
>>
>> If you are referring to the DataLoader, it is an embedded utility class.
>> It is not designed to operate with a remote database instance.
>>
>> You can mimic many of the advantages of the DataLoader by increasing
>> BigdataSail.Options.BUFFER_CAPACITY to 100,000.
>>
>> You should also follow the guidelines on the wiki for performance
>> optimization if you are interested in bulk data load.  See the section
>> entitled Optimizations and benchmarking
>> <https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#p-Optimizations_and_benchmarking>.
>> E.g., https://wiki.blazegraph.com/wiki/index.php/IOOptimization.
>>
>> Some of the more important optimizations for write throughput are:
>>
>> - Write cache service native buffer pool size.
>> - Use of URI inlining techniques if you have URIs that have numeric or
>> UUID patterns embedded into them.
>> - Fast disk.
>>
>> We have a number of improvements in the development branch that improve
>> load speed, including code to overlap the parser with the index writers.
>> Those will be in the 2.0 release.
>>
>> Thanks,
>> Bryan
>>
>>
>> ----
>> Bryan Thompson
>> Chief Scientist & Founder
>> SYSTAP, LLC
>> 4501 Tower Road
>> Greensboro, NC 27410
>> br...@sy...
>> http://blazegraph.com
>> http://blog.blazegraph.com
>>
>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>> APIs.  Blazegraph is now available with GPU acceleration using our disruptive
>> technology to accelerate data-parallel graph analytics and graph query.
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>> dissemination or copying of this email or its contents or attachments is
>> prohibited. If you have received this communication in error, please notify
>> the sender by reply email and permanently delete all copies of the email
>> and its contents and attachments.
>>
>> On Wed, Nov 4, 2015 at 7:48 AM, Alex Muir <ale...@gm...> wrote:
>>
>>> Well I downloaded the blazegraph git examples and extracted the unique
>>> bigdata properties. I don't think any of them are related to specifying a
>>> remote server.
>>>
>>> Perhaps there is another way to specify to upload to a remote server
>>> with the bulk loader?
>>>
>>> com.bigdata.btree.BTree.branchingFactor
>>> com.bigdata.btree.keys.KeyBuilder.collator
>>> com.bigdata.btree.writeRetentionQueue.capacity
>>> com.bigdata.journal.AbstractJournal.bufferMode
>>> com.bigdata.journal.AbstractJournal.file
>>> com.bigdata.journal.AbstractJournal.initialExtent
>>> com.bigdata.journal.AbstractJournal.maximumExtent
>>> com.bigdata.journal.AbstractJournal.writeCacheBufferCount
>>>
>>> com.bigdata.namespace.BSBM_284826.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_284826.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_284826.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_284826.spo.OSP.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_284826.spo.POS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_284826.spo.SPO.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_566496.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_566496.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_566496.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_566496.spo.OSP.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_566496.spo.POS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.BSBM_566496.spo.SPO.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.spo.CSPO.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.spo.OCSP.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.spo.PCSO.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.spo.POCS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.spo.SOPC.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.chem2bio2rdf.spo.SPOC.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.dbpedia.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.dbpedia.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.dbpedia.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.dbpedia.spo.OSP.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.dbpedia.spo.POS.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.dbpedia.spo.SPO.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.kb.lex.BLOBS.com.bigdata.btree.BTree.branchingFactor
>>> com.bigdata.namespace.kb.lex.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.kb.lex.ID2TERM.com.bigdata.btree.BTree.branchingFactor
>>>
>>> com.bigdata.namespace.kb.lex.TERM2ID.com.bigdata.btree.BTree.branchingFactor
>>> com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor
>>> com.bigdata.rdf.rio.RDFParserOptions.stopAtFirstError
>>> com.bigdata.rdf.sail.BigdataSail.bufferCapacity
>>> com.bigdata.rdf.sail.BigdataSail.truthMaintenance
>>> com.bigdata.rdf.sail.bufferCapacity
>>> com.bigdata.rdf.sail.newEvalStrategy
>>> com.bigdata.rdf.sail.queryTimeExpander
>>> com.bigdata.rdf.sail.truthMaintenance
>>> com.bigdata.rdf.store.AbstractTripleStore.axiomsClass
>>> com.bigdata.rdf.store.AbstractTripleStore.bloomFilter
>>> com.bigdata.rdf.store.AbstractTripleStore.extensionFactoryClass
>>> com.bigdata.rdf.store.AbstractTripleStore.justify
>>> com.bigdata.rdf.store.AbstractTripleStore.quads
>>> com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers
>>> com.bigdata.rdf.store.AbstractTripleStore.textIndex
>>> com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass
>>> com.bigdata.resource.OverflowManager.overflowEnabled
>>> com.bigdata.service.AbstractTransactionService.minReleaseAge
>>> com.bigdata.service.EmbeddedFederation.dataDir
>>> com.bigdata.service.IBigdataClient.collectPlatformStatistics
>>>
>>>
>>>
>>> Regards
>>> Alex
>>> www.tilogeo.com
>>>
>>> On Wed, Nov 4, 2015 at 11:12 AM, Alex Muir <ale...@gm...>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm interested to bulk upload onto a remote server
>>>>
>>>> https://wiki.blazegraph.com/wiki/index.php/Bulk_Data_Load
>>>>
>>>> I assume that I can specify a remote server in the properties file
>>>> however I'm thus far unable to find more information on what goes in a
>>>> property file from the website.
>>>>
>>>> Is there a page defining all the properties?
>>>>
>>>> Regards
>>>> Alex
>>>> www.tilogeo.com
>>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Bigdata-developers mailing list
>>> Big...@li...
>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>
>
> --
> _______________
> Brad Bebee
> CEO, Managing Partner
> SYSTAP, LLC
> e:  be...@sy...
> m: 202.642.7961
> f:  571.367.5000
> w:  www.blazegraph.com
>
> Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs.  Mapgraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>