bigdata-developers — List for bigdata developers

Re: [Bigdata-developers] getting mutation count from SPARQL update

[Bryan T. <br...@bl...>]

Jim,

There is a ticket which would address this. It is BLZG-824
<https://jira.blazegraph.com/browse/BLZG-824>.  Could you offer some
feedback on that ticket and we can try to raise bring it into a sprint soon.

Thanks,
Bryan

On Mon, Jun 13, 2016 at 3:50 PM, Jim Balhoff <ba...@gm...> wrote:

> Hi,
>
> Is there a client API that would allow me to get the mutation count from a
> SPARQL update? I see that mutationCount is returned from the HTTP
> interface, but I was hoping to get the same information when submitting a
> SPARQL update via the Sesame API. I see that Update.execute() returns void.
>
> I am trying to run a sequence of SPARQL updates to do some “reasoning”,
> and stop when no further triples are being inserted. I suppose I could
> write these as rules, but for this job it just seemed a little simpler to
> use SPARQL update.
>
> Thanks,
> Jim
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>

Re: [Bigdata-developers] Are numbers stored as binary ?

[Bryan T. <br...@bl...>]

They are.  This is referred to as "inlining".  We inline xsd numerics,
including xsd:integer and xsd:decimal as well as xsd:int, xsd:float, etc.

On Sat, Jun 18, 2016 at 9:56 AM, Jean-Marc Vanel <jea...@gm...>
wrote:

> Hi all
>
> I could not find the answer in the FAQ, nor by googling.
>
>
> The advantages of numbers stored as binary versus strings are:
>
> - less storage
> - less CPU in conversion of string => number when using an API
> - more efficient SPARQL queries involving numbers computations
>
> --
> Jean-Marc Vanel
> Déductions SARL - Consulting, services, training,
> Rule-based programming, Semantic Web
> +33 (0)6 89 16 29 52
> Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning
> reports. http://sdm.link/zohomanageengine
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>

[Bigdata-developers] Are numbers stored as binary ?

[Jean-Marc V. <jea...@gm...>]

Hi all

I could not find the answer in the FAQ, nor by googling.


The advantages of numbers stored as binary versus strings are:

- less storage
- less CPU in conversion of string => number when using an API
- more efficient SPARQL queries involving numbers computations

-- 
Jean-Marc Vanel
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui

Re: [Bigdata-developers] Performance impact of Named Subqueries

[Michael S. <ms...@me...>]

Hello Bryan, Edgar,

there is already a query hint for pipelined (non-blocking) hash joins. For instance, when using

<snip>
SELECT * WHERE {
  ?s <http://p1> ?o1
  {
    SELECT * WHERE {
      ?s <http://p2> ?o2 .
      ?s <http://p3> ?o3 .
    }
  }
  hint:Prior hint:pipelinedHashJoin "true" .  
}
</snip>

the inner SELECT will be executed in a pipelined fashion. The hint works also for complex OPTIONALs, EXISTS, and VALUES clauses. Note that we do not use pipelining for named subqueries (i.e., %INCLUDE … patterns) at time being.

I’ve added documentation to the Wiki at https://wiki.blazegraph.com/wiki/index.php/QueryHints <https://wiki.blazegraph.com/wiki/index.php/QueryHints>. 

Best,
Michael

> On 17 Jun 2016, at 23:46, Bryan Thompson <br...@bl...> wrote:
> 
> Named subqueries are run first in a bottom up evaluation style. Normal subqueries are evaluated as-bound using left to right evaluation. So the plans are quite different. There are now non blocking subquery hash joins that are used if a limit is specified. So they can be turned on using a very large limit. We plan to have a query hint for that soon...
> 
> On Jun 17, 2016 5:40 PM, "Edgar Rodriguez-Diaz" <ed...@sy... <mailto:ed...@sy...>> wrote:
> Hi,
> 
> If I understand correctly from the wiki docs, named subqueries should help by reusing results computed, but I’m noticing some very significant performance difference between a query using named subqueries and the same query but inlining the named subqueries, both queries return the same results but the latter performs at least an order of magnitude better than the former. I’ve seen this in several queries now:
> 
>               SPARQLBenchmark
>         ---------------
>         
>                       name | result | rank | runs |  mean |     sd
>         -------------------|--------|------|------|-------|--------
>                     query1 |   PASS |    1 |    3 | 3.497 |  1.195
>                     query2 |   PASS |    2 |    3 | 16.03 |  1.448
>                     query3 |   PASS |    3 |    3 |  21.6 |  1.386
>               named query2 |   PASS |    4 |    3 | 386.9 |  3.411
>               named query3 |   PASS |    5 |    3 | 397.5 |   6.31
>               named query1 |   PASS |    6 |    3 | 827.3 | 0.7966
>               
>         Each of the above 18 runs were run in random, non-consecutive order. Mean times in seconds.
> 
> So I guess the questions are: 
> What’s known the performance impact and if what I’m experiencing is a known behavior? 
> Is it that named subqueries are blocking and when inlining the subqueries the results can be streamed?
> 
> Cheers, 
> Edgar
> 
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity planning
> reports. http://sdm.link/zohomanageengine <http://sdm.link/zohomanageengine>
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li... <mailto:Big...@li...>
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity planning
> reports. http://sdm.link/zohomanageengine_______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] Performance impact of Named Subqueries

[Bryan T. <br...@bl...>]

There is also a potential to run named subqueries in parallel as busy
plans.  We are not currently doing this.  But when we do, it will shift the
performance envelope again. But the main use case for a named subquery is
when you know that bottom-up evaluation of the subquery will be more
efficient than as-bound evaluation.  Note that you can also enforce a
partial ordering among the named subqueries using an INCLUDE.  Doing so
essentially causes one bottom-up query to be nested inside of another
bottom-up query.  Also note that a named subquery result can (in principle)
be reused at multiple points in a query.

Thanks,
Bryan

On Fri, Jun 17, 2016 at 5:10 PM, Edgar Rodriguez-Diaz <ed...@sy...>
wrote:

> Hi,
>
> If I understand correctly from the wiki docs, named subqueries should help
> by reusing results computed, but I’m noticing some very significant
> performance difference between a query using named subqueries and the same
> query but inlining the named subqueries, both queries return the same
> results but the latter performs at least an order of magnitude better than
> the former. I’ve seen this in several queries now:
>
>               SPARQLBenchmark
>         ---------------
>
>                       name | result | rank | runs |  mean |     sd
>         -------------------|--------|------|------|-------|--------
>                     query1 |   PASS |    1 |    3 | 3.497 |  1.195
>                     query2 |   PASS |    2 |    3 | 16.03 |  1.448
>                     query3 |   PASS |    3 |    3 |  21.6 |  1.386
>               named query2 |   PASS |    4 |    3 | 386.9 |  3.411
>               named query3 |   PASS |    5 |    3 | 397.5 |   6.31
>               named query1 |   PASS |    6 |    3 | 827.3 | 0.7966
>
>         Each of the above 18 runs were run in random, non-consecutive
> order. Mean times in seconds.
>
> So I guess the questions are:
> What’s known the performance impact and if what I’m experiencing is a
> known behavior?
> Is it that named subqueries are blocking and when inlining the subqueries
> the results can be streamed?
>
> Cheers,
> Edgar
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning
> reports. http://sdm.link/zohomanageengine
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>

Re: [Bigdata-developers] Performance impact of Named Subqueries

[Bryan T. <br...@bl...>]

Named subqueries are run first in a bottom up evaluation style. Normal
subqueries are evaluated as-bound using left to right evaluation. So the
plans are quite different. There are now non blocking subquery hash joins
that are used if a limit is specified. So they can be turned on using a
very large limit. We plan to have a query hint for that soon...
On Jun 17, 2016 5:40 PM, "Edgar Rodriguez-Diaz" <ed...@sy...> wrote:

Hi,

If I understand correctly from the wiki docs, named subqueries should help
by reusing results computed, but I’m noticing some very significant
performance difference between a query using named subqueries and the same
query but inlining the named subqueries, both queries return the same
results but the latter performs at least an order of magnitude better than
the former. I’ve seen this in several queries now:

              SPARQLBenchmark
        ---------------

                      name | result | rank | runs |  mean |     sd
        -------------------|--------|------|------|-------|--------
                    query1 |   PASS |    1 |    3 | 3.497 |  1.195
                    query2 |   PASS |    2 |    3 | 16.03 |  1.448
                    query3 |   PASS |    3 |    3 |  21.6 |  1.386
              named query2 |   PASS |    4 |    3 | 386.9 |  3.411
              named query3 |   PASS |    5 |    3 | 397.5 |   6.31
              named query1 |   PASS |    6 |    3 | 827.3 | 0.7966

        Each of the above 18 runs were run in random, non-consecutive
order. Mean times in seconds.

So I guess the questions are:
What’s known the performance impact and if what I’m experiencing is a known
behavior?
Is it that named subqueries are blocking and when inlining the subqueries
the results can be streamed?

Cheers,
Edgar


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning
reports. http://sdm.link/zohomanageengine
_______________________________________________
Bigdata-developers mailing list
Big...@li...
https://lists.sourceforge.net/lists/listinfo/bigdata-developers

[Bigdata-developers] Performance impact of Named Subqueries

[Edgar Rodriguez-D. <ed...@sy...>]

Hi,

If I understand correctly from the wiki docs, named subqueries should help
by reusing results computed, but I’m noticing some very significant
performance difference between a query using named subqueries and the same
query but inlining the named subqueries, both queries return the same
results but the latter performs at least an order of magnitude better than
the former. I’ve seen this in several queries now:

              SPARQLBenchmark
        ---------------

                      name | result | rank | runs |  mean |     sd
        -------------------|--------|------|------|-------|--------
                    query1 |   PASS |    1 |    3 | 3.497 |  1.195
                    query2 |   PASS |    2 |    3 | 16.03 |  1.448
                    query3 |   PASS |    3 |    3 |  21.6 |  1.386
              named query2 |   PASS |    4 |    3 | 386.9 |  3.411
              named query3 |   PASS |    5 |    3 | 397.5 |   6.31
              named query1 |   PASS |    6 |    3 | 827.3 | 0.7966

        Each of the above 18 runs were run in random, non-consecutive
order. Mean times in seconds.

So I guess the questions are:
What’s known the performance impact and if what I’m experiencing is a known
behavior?
Is it that named subqueries are blocking and when inlining the subqueries
the results can be streamed?

Cheers,
Edgar

[Bigdata-developers] getting mutation count from SPARQL update

[Jim B. <ba...@gm...>]

Hi,

Is there a client API that would allow me to get the mutation count from a SPARQL update? I see that mutationCount is returned from the HTTP interface, but I was hoping to get the same information when submitting a SPARQL update via the Sesame API. I see that Update.execute() returns void.

I am trying to run a sequence of SPARQL updates to do some “reasoning”, and stop when no further triples are being inserted. I suppose I could write these as rules, but for this job it just seemed a little simpler to use SPARQL update.

Thanks,
Jim

Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Brad B. <be...@bl...>]

Daniel, Edgar,

Adding to Bryan's comments, the general starting points for improving load
speed are below.  With fast disks, customized branching factors, and custom
vocabularies, we've seen sustained loads for data sets such as Pubchem
(~10B stmts) at 85k stmts/s.

- Setting branching factors in your properties file using the DumpJournal
utility:
https://wiki.blazegraph.com/wiki/index.php/IOOptimization#Branching_Factors.
This involves doing a partial load of the data, running the utilities, and
then updating the configuration.

- Building a custom vocabulary and configuring inlining:
https://wiki.blazegraph.com/wiki/index.php/InlineIVs.    There's an example
Pubchem custom vocabulary at
https://github.com/blazegraph/database/tree/master/vocabularies/src/main/java/com/blazegraph/vocab/pubchem
.

Thanks, --Brad

On Tue, May 31, 2016 at 8:19 PM, Daniel Hernández <da...@de...> wrote:

> Thanks Bryan for your clarification. I will repeat my experiments with
> different
> hardware configurations in the future.
>
> Daniel
>
> El 31/05/16 a las 20:00, Bryan Thompson escribió:
>
> Sata is a non starter for blazegraph unless it is ssd. The lack of write
> reordering combined with high seek latency significantly limits
> performance.  This could be different for other engines. Blazegraph (the
> open source platform) is pretty disk oriented.  The gpu platform is focused
> on high performance in fast memory.
>
> Bryan
> On May 31, 2016 7:00 PM, "Daniel Hernández" <da...@de...> wrote:
>
> Edgar,
>
> I confirm that the loading rate decreases while the database increases its
> size.
>
> I have loaded 500M of triples and it have taken 23h using a triple store
> back-end. I loaded the same amount of quads using the quad store back-end
> and it takes 67h. The resulting databases have 61GB and 120GB,
> respectively. My machine has 2xSATA disks on RAID 1, 32GB of RAM a 2xIntel
> Xeon with Six Core. I use the parameter -Xmx6g when loading (For small
> files, I got better results with 6g than with 5g and 8g).
>
> I have seen that using SSD improves at least 3 times the elapsed loading
> time. However, this could be true for every engine. Edgar, if you improve
> your loading times without changing your machine I will be grateful if you
> tell us how to yo did it.
>
> (By the way, I loaded the same files into Virtuoso and it required
> approximately 4 hours for each file.)
>
> Cheers,
> Daniel
>
> El 31/05/16 a las 18:15, Bryan Thompson escribió:
>
> Edgar,
>
> There is no single configuration for maximum load throughput.  Instead
> there are a variety of steps you can take to improve load performance. For
> example, right sizing the jvm, using fast disk, maximizing inlining, etc.
> Beyond these steps and those detailed on the wiki, we look at the entire
> system to identify and remove bottlenecks.
>
> Thanks,
> Bryan
> On May 31, 2016 5:28 PM, "Edgar Rodriguez-Diaz" <ed...@sy...> wrote:
>
>> A correction here on the data size, it’s not 180G - it’s 18G of a gzip
>> trig file exported by blazegraph; number of triples is correct.
>>
>> > On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz <ed...@sy...>
>> wrote:
>> >
>> > Hi,
>> >
>> > I’ve been trying to use the DataLoader tool for bulk loading a very
>> large file into blazegraph (~180G with ~4 billion triples) with and empty
>> journal file, but I’m noticing a performance degradation on the rate of
>> triples/s loaded. It started at around 55K and after 200 M triples the rate
>> is around 32K, the rate keeps going down consistently.
>> > What is the configuration to get the best performance out of the bulk
>> load into blazegraph?
>> >
>> > Thanks.
>> >
>> > - Edgar
>>
>>
>>
>> ------------------------------------------------------------------------------
>> What NetFlow Analyzer can do for you? Monitors network bandwidth and
>> traffic
>> patterns at an interface-level. Reveals which users, apps, and protocols
>> are
>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>> planning reports.
>> https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>
>
>
> _______________________________________________
> Bigdata-developers mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>


-- 
_______________
Brad Bebee
CEO
Blazegraph
e:  be...@bl...
m: 202.642.7961
w:  www.blazegraph.com

Blazegraph products help to solve the Graph Cache Thrash to achieve large
scale processing for graph and predictive analytics.  Blazegraph is the
creator of the industry’s first GPU-accelerated high-performance database
for large graphs, has been named as one of the “10 Companies and
Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.

Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
performance graph database that supports both RDF/SPARQL and Apache
TinkerPop™ APIs.   Blazegraph GPU
<https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
<https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
technologies that use GPUs to enable extreme scaling that is thousands of
times faster and 40 times more affordable than CPU-based solutions.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
disclosure, dissemination or copying of this email or its contents or
attachments is prohibited. If you have received this communication in
error, please notify the sender by reply email and permanently delete all
copies of the email and its contents and attachments.

Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Daniel H. <da...@de...>]

Thanks Bryan for your clarification. I will repeat my experiments with 
different
hardware configurations in the future.

Daniel

El 31/05/16 a las 20:00, Bryan Thompson escribió:
>
> Sata is a non starter for blazegraph unless it is ssd. The lack of 
> write reordering combined with high seek latency significantly limits 
> performance.  This could be different for other engines. Blazegraph 
> (the open source platform) is pretty disk oriented.  The gpu platform 
> is focused on high performance in fast memory.
>
> Bryan
>
> On May 31, 2016 7:00 PM, "Daniel Hernández" <da...@de... 
> <mailto:da...@de...>> wrote:
>
>     Edgar,
>
>     I confirm that the loading rate decreases while the database
>     increases its size.
>
>     I have loaded 500M of triples and it have taken 23h using a triple
>     store back-end. I loaded the same amount of quads using the quad
>     store back-end and it takes 67h. The resulting databases have 61GB
>     and 120GB, respectively. My machine has 2xSATA disks on RAID 1,
>     32GB of RAM a 2xIntel Xeon with Six Core. I use the parameter
>     -Xmx6g when loading (For small files, I got better results with 6g
>     than with 5g and 8g).
>
>     I have seen that using SSD improves at least 3 times the elapsed
>     loading time. However, this could be true for every engine. Edgar,
>     if you improve your loading times without changing your machine I
>     will be grateful if you tell us how to yo did it.
>
>     (By the way, I loaded the same files into Virtuoso and it required
>     approximately 4 hours for each file.)
>
>     Cheers,
>     Daniel
>
>     El 31/05/16 a las 18:15, Bryan Thompson escribió:
>>
>>     Edgar,
>>
>>     There is no single configuration for maximum load throughput. 
>>     Instead there are a variety of steps you can take to improve load
>>     performance. For example, right sizing the jvm, using fast disk,
>>     maximizing inlining, etc. Beyond these steps and those detailed
>>     on the wiki, we look at the entire system to identify and remove
>>     bottlenecks.
>>
>>     Thanks,
>>     Bryan
>>
>>     On May 31, 2016 5:28 PM, "Edgar Rodriguez-Diaz"
>>     <ed...@sy... <mailto:ed...@sy...>> wrote:
>>
>>         A correction here on the data size, it’s not 180G - it’s 18G
>>         of a gzip trig file exported by blazegraph; number of triples
>>         is correct.
>>
>>         > On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz
>>         <ed...@sy... <mailto:ed...@sy...>> wrote:
>>         >
>>         > Hi,
>>         >
>>         > I’ve been trying to use the DataLoader tool for bulk
>>         loading a very large file into blazegraph (~180G with ~4
>>         billion triples) with and empty journal file, but I’m
>>         noticing a performance degradation on the rate of triples/s
>>         loaded. It started at around 55K and after 200 M triples the
>>         rate is around 32K, the rate keeps going down consistently.
>>         > What is the configuration to get the best performance out
>>         of the bulk load into blazegraph?
>>         >
>>         > Thanks.
>>         >
>>         > - Edgar
>>
>>
>>         ------------------------------------------------------------------------------
>>         What NetFlow Analyzer can do for you? Monitors network
>>         bandwidth and traffic
>>         patterns at an interface-level. Reveals which users, apps,
>>         and protocols are
>>         consuming the most bandwidth. Provides multi-vendor support
>>         for NetFlow,
>>         J-Flow, sFlow and other flows. Make informed decisions using
>>         capacity
>>         planning reports.
>>         https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>>         _______________________________________________
>>         Bigdata-developers mailing list
>>         Big...@li...
>>         <mailto:Big...@li...>
>>         https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>>
>>     ------------------------------------------------------------------------------
>>     What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
>>     patterns at an interface-level. Reveals which users, apps, and protocols are
>>     consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>>     J-Flow, sFlow and other flows. Make informed decisions using capacity
>>     planning reports.https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>>
>>
>>     _______________________________________________
>>     Bigdata-developers mailing list
>>     Big...@li...
>>     <mailto:Big...@li...>
>>     https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
>     ------------------------------------------------------------------------------
>     What NetFlow Analyzer can do for you? Monitors network bandwidth
>     and traffic
>     patterns at an interface-level. Reveals which users, apps, and
>     protocols are
>     consuming the most bandwidth. Provides multi-vendor support for
>     NetFlow,
>     J-Flow, sFlow and other flows. Make informed decisions using capacity
>     planning reports.
>     https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>     _______________________________________________
>     Bigdata-developers mailing list
>     Big...@li...
>     <mailto:Big...@li...>
>     https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>

Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Bryan T. <br...@bl...>]

Sata is a non starter for blazegraph unless it is ssd. The lack of write
reordering combined with high seek latency significantly limits
performance.  This could be different for other engines. Blazegraph (the
open source platform) is pretty disk oriented.  The gpu platform is focused
on high performance in fast memory.

Bryan
On May 31, 2016 7:00 PM, "Daniel Hernández" <da...@de...> wrote:

Edgar,

I confirm that the loading rate decreases while the database increases its
size.

I have loaded 500M of triples and it have taken 23h using a triple store
back-end. I loaded the same amount of quads using the quad store back-end
and it takes 67h. The resulting databases have 61GB and 120GB,
respectively. My machine has 2xSATA disks on RAID 1, 32GB of RAM a 2xIntel
Xeon with Six Core. I use the parameter -Xmx6g when loading (For small
files, I got better results with 6g than with 5g and 8g).

I have seen that using SSD improves at least 3 times the elapsed loading
time. However, this could be true for every engine. Edgar, if you improve
your loading times without changing your machine I will be grateful if you
tell us how to yo did it.

(By the way, I loaded the same files into Virtuoso and it required
approximately 4 hours for each file.)

Cheers,
Daniel

El 31/05/16 a las 18:15, Bryan Thompson escribió:

Edgar,

There is no single configuration for maximum load throughput.  Instead
there are a variety of steps you can take to improve load performance. For
example, right sizing the jvm, using fast disk, maximizing inlining, etc.
Beyond these steps and those detailed on the wiki, we look at the entire
system to identify and remove bottlenecks.

Thanks,
Bryan
On May 31, 2016 5:28 PM, "Edgar Rodriguez-Diaz" <ed...@sy...> wrote:

> A correction here on the data size, it’s not 180G - it’s 18G of a gzip
> trig file exported by blazegraph; number of triples is correct.
>
> > On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz <ed...@sy...>
> wrote:
> >
> > Hi,
> >
> > I’ve been trying to use the DataLoader tool for bulk loading a very
> large file into blazegraph (~180G with ~4 billion triples) with and empty
> journal file, but I’m noticing a performance degradation on the rate of
> triples/s loaded. It started at around 55K and after 200 M triples the rate
> is around 32K, the rate keeps going down consistently.
> > What is the configuration to get the best performance out of the bulk
> load into blazegraph?
> >
> > Thanks.
> >
> > - Edgar
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e



_______________________________________________
Bigdata-developers mailing
lis...@li...https://lists.sourceforge.net/lists/listinfo/bigdata-developers



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Bigdata-developers mailing list
Big...@li...
https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Daniel H. <da...@de...>]

Edgar,

I confirm that the loading rate decreases while the database increases 
its size.

I have loaded 500M of triples and it have taken 23h using a triple store 
back-end. I loaded the same amount of quads using the quad store 
back-end and it takes 67h. The resulting databases have 61GB and 120GB, 
respectively. My machine has 2xSATA disks on RAID 1, 32GB of RAM a 
2xIntel Xeon with Six Core. I use the parameter -Xmx6g when loading (For 
small files, I got better results with 6g than with 5g and 8g).

I have seen that using SSD improves at least 3 times the elapsed loading 
time. However, this could be true for every engine. Edgar, if you 
improve your loading times without changing your machine I will be 
grateful if you tell us how to yo did it.

(By the way, I loaded the same files into Virtuoso and it required 
approximately 4 hours for each file.)

Cheers,
Daniel

El 31/05/16 a las 18:15, Bryan Thompson escribió:
>
> Edgar,
>
> There is no single configuration for maximum load throughput.  Instead 
> there are a variety of steps you can take to improve load performance. 
> For example, right sizing the jvm, using fast disk, maximizing 
> inlining, etc. Beyond these steps and those detailed on the wiki, we 
> look at the entire system to identify and remove bottlenecks.
>
> Thanks,
> Bryan
>
> On May 31, 2016 5:28 PM, "Edgar Rodriguez-Diaz" <ed...@sy... 
> <mailto:ed...@sy...>> wrote:
>
>     A correction here on the data size, it’s not 180G - it’s 18G of a
>     gzip trig file exported by blazegraph; number of triples is correct.
>
>     > On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz
>     <ed...@sy... <mailto:ed...@sy...>> wrote:
>     >
>     > Hi,
>     >
>     > I’ve been trying to use the DataLoader tool for bulk loading a
>     very large file into blazegraph (~180G with ~4 billion triples)
>     with and empty journal file, but I’m noticing a performance
>     degradation on the rate of triples/s loaded. It started at around
>     55K and after 200 M triples the rate is around 32K, the rate keeps
>     going down consistently.
>     > What is the configuration to get the best performance out of the
>     bulk load into blazegraph?
>     >
>     > Thanks.
>     >
>     > - Edgar
>
>
>     ------------------------------------------------------------------------------
>     What NetFlow Analyzer can do for you? Monitors network bandwidth
>     and traffic
>     patterns at an interface-level. Reveals which users, apps, and
>     protocols are
>     consuming the most bandwidth. Provides multi-vendor support for
>     NetFlow,
>     J-Flow, sFlow and other flows. Make informed decisions using capacity
>     planning reports.
>     https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>     _______________________________________________
>     Bigdata-developers mailing list
>     Big...@li...
>     <mailto:Big...@li...>
>     https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>
>
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers

Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Bryan T. <br...@bl...>]

Edgar,

There is no single configuration for maximum load throughput.  Instead
there are a variety of steps you can take to improve load performance. For
example, right sizing the jvm, using fast disk, maximizing inlining, etc.
Beyond these steps and those detailed on the wiki, we look at the entire
system to identify and remove bottlenecks.

Thanks,
Bryan
On May 31, 2016 5:28 PM, "Edgar Rodriguez-Diaz" <ed...@sy...> wrote:

> A correction here on the data size, it’s not 180G - it’s 18G of a gzip
> trig file exported by blazegraph; number of triples is correct.
>
> > On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz <ed...@sy...>
> wrote:
> >
> > Hi,
> >
> > I’ve been trying to use the DataLoader tool for bulk loading a very
> large file into blazegraph (~180G with ~4 billion triples) with and empty
> journal file, but I’m noticing a performance degradation on the rate of
> triples/s loaded. It started at around 55K and after 200 M triples the rate
> is around 32K, the rate keeps going down consistently.
> > What is the configuration to get the best performance out of the bulk
> load into blazegraph?
> >
> > Thanks.
> >
> > - Edgar
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>

Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Edgar Rodriguez-D. <ed...@sy...>]

A correction here on the data size, it’s not 180G - it’s 18G of a gzip trig file exported by blazegraph; number of triples is correct.

> On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz <ed...@sy...> wrote:
> 
> Hi,
> 
> I’ve been trying to use the DataLoader tool for bulk loading a very large file into blazegraph (~180G with ~4 billion triples) with and empty journal file, but I’m noticing a performance degradation on the rate of triples/s loaded. It started at around 55K and after 200 M triples the rate is around 32K, the rate keeps going down consistently.
> What is the configuration to get the best performance out of the bulk load into blazegraph?
> 
> Thanks.
> 
> - Edgar

[Bigdata-developers] Fastest way to load data via Bulk Data Load

[Edgar Rodriguez-D. <ed...@sy...>]

Hi,

I’ve been trying to use the DataLoader tool for bulk loading a very large file into blazegraph (~180G with ~4 billion triples) with and empty journal file, but I’m noticing a performance degradation on the rate of triples/s loaded. It started at around 55K and after 200 M triples the rate is around 32K, the rate keeps going down consistently.
What is the configuration to get the best performance out of the bulk load into blazegraph?

Thanks.

- Edgar

Re: [Bigdata-developers] read only mode for the jar file

[Brad B. <be...@bl...>]

Doug,

Excellent.

Cheers, --Brad

On Fri, May 27, 2016 at 1:49 PM, Douglas Fils <dr...@gm...> wrote:

> Brad,
>   Works perfect.  As noted my error was copying the whole base web.xml and
> making that change in it and using it.   Using your version where just that
> one node is included and changed worked perfectly.
>
>   Thanks for this...  Next I need to update my Dockerfile process, but I
> am sure this is going to work.  Really like blazegraph....  coming from
> some other triple stores and really seem to resonate with this one.
>
> Take care
> Doug
>
>
> On Fri, May 27, 2016 at 12:26 PM Brad Bebee <be...@bl...> wrote:
>
>> Here is the wiki link:
>> https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#Read_Only_Configuration_with_the_Jetty_Override_and_Executable_Jar
>> .
>>
>> On Fri, May 27, 2016 at 1:17 PM, Brad Bebee <be...@bl...> wrote:
>>
>>> Doug,
>>>
>>> You need to wrap it in the web-app tag (see contents below and
>>> attached).  I've updated the Wiki as well with this documentation.   Happy
>>> Blazegraphing... ;-)
>>>
>>> Cheers, --Brad
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>>
>>> <web-app xmlns="http://java.sun.com/xml/ns/javaee"
>>>
>>>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>
>>>       xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
>>> http://java.sun.com/xml/ns/javaee/web-app_3_1.xsd"
>>>
>>>       version="3.1">
>>>
>>>   <context-param>
>>>
>>>    <description>When true, the REST API will not permit mutation
>>> operations.</description>
>>>
>>>    <param-name>readOnly</param-name>
>>>
>>>    <param-value>true</param-value>
>>>
>>>   </context-param>
>>>
>>> </web-app>
>>>
>>> On Fri, May 27, 2016 at 11:57 AM, Douglas Fils <dr...@gm...> wrote:
>>>
>>>> Bryan,
>>>>   I am following this and the docs.  However, when I pull the web.xml
>>>> alter the lines for read only to:
>>>>
>>>>  <context-param>
>>>>    <description>When true, the REST API will not permit mutation
>>>> operations.</description>
>>>>    <param-name>readOnly</param-name>
>>>>    <param-value>true</param-value>
>>>>   </context-param>
>>>>
>>>> I run with:  java -server -Xmx4g -Djetty.overrideWebXml=./override.xml
>>>>  -jar blazegraph.jar
>>>>
>>>> I get errors
>>>>
>>>> WARN : NanoSparqlServer.java:517: Starting NSS
>>>> WARN : AbstractLifeCycle.java:212: FAILED
>>>> org.eclipse.jetty.servlet.ServletHandler@239963d8:
>>>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>>>> REST API,REST API
>>>> java.lang.IllegalStateException: Multiple servlets map to path:
>>>> /sparql: REST API,REST API
>>>>
>>>> WARN : WebAppContext.java:506: Failed startup of context
>>>> o.e.j.w.WebAppContext@4157f54e
>>>> {/blazegraph,jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war,STARTING}{jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war}
>>>> java.lang.IllegalStateException: Multiple servlets map to path:
>>>> /sparql: REST API,REST API
>>>>
>>>>
>>>> sorry to be pest, I feel like I'm missing something simple.
>>>>
>>>> Doug
>>>>
>>>>
>>>>
>>>> On Thu, May 26, 2016 at 2:50 PM Bryan Thompson <br...@bl...>
>>>> wrote:
>>>>
>>>>> See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer
>>>>>
>>>>> Customizing the web.xml
>>>>>
>>>>> You can override the default web.xml values in the executable jar
>>>>> using the jetty.overrideWebXml property. The file you specify should
>>>>> override the values that you'd like to replace. The web.xml values that
>>>>> default with the blazegraph.jar are in web.xml
>>>>> <https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
>>>>> .
>>>>>
>>>>> -Djetty.overrideWebXml=/path/to/override.xml
>>>>>
>>>>> A full example is below.
>>>>>
>>>>> java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml -Dbigdata.propertyFile=/etc/blazegraph/RWS
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Bryan
>>>>>
>>>>> On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...>
>>>>> wrote:
>>>>>
>>>>>> I am using the .jar version though, not a  .war file in an app server.
>>>>>> Am I still able to somehow override or reference a new web.xml file?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
>>>>>> wrote:
>>>>>>
>>>>>>> You can control this in web.xml.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Bryan
>>>>>>>
>>>>>>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>   I am sure this is a simple question but I can't seem to launch a
>>>>>>>> previously created blazegraph instance with the namespaces in "readonly"
>>>>>>>> mode.
>>>>>>>>
>>>>>>>>   Is there a simple way to add a -D option or something to
>>>>>>>>
>>>>>>>> java -server -Xmx4g -jar blazegraph.jar
>>>>>>>>
>>>>>>>> such that the existing namespaces are readonly on the sparql
>>>>>>>> endpoint so that I can expose the server to the net without a fear of
>>>>>>>> someone doing a update/delete etc on the triples?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Doug
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Mobile security can be enabling, not merely restricting. Employees
>>>>>>>> who
>>>>>>>> bring their own devices (BYOD) to work are irked by the imposition
>>>>>>>> of MDM
>>>>>>>> restrictions. Mobile Device Manager Plus allows you to control only
>>>>>>>> the
>>>>>>>> apps on BYO-devices by containerizing them, leaving personal data
>>>>>>>> untouched!
>>>>>>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>>>>>>> _______________________________________________
>>>>>>>> Bigdata-developers mailing list
>>>>>>>> Big...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> What NetFlow Analyzer can do for you? Monitors network bandwidth and
>>>> traffic
>>>> patterns at an interface-level. Reveals which users, apps, and
>>>> protocols are
>>>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>>>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>>>> planning reports.
>>>> https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>>>> _______________________________________________
>>>> Bigdata-developers mailing list
>>>> Big...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>
>>>>
>>>
>>>
>>> --
>>> _______________
>>> Brad Bebee
>>> CEO
>>> Blazegraph
>>> e:  be...@bl...
>>> m: 202.642.7961
>>> w:  www.blazegraph.com
>>>
>>> Blazegraph products help to solve the Graph Cache Thrash to achieve
>>> large scale processing for graph and predictive analytics.  Blazegraph is
>>> the creator of the industry’s first GPU-accelerated high-performance
>>> database for large graphs, has been named as one of the “10 Companies
>>> and Technologies to Watch in 2016”
>>> <http://insideanalysis.com/2016/01/20535/>.
>>>
>>> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
>>> performance graph database that supports both RDF/SPARQL and Apache
>>> TinkerPop™ APIs.   Blazegraph GPU
>>> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
>>> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive
>>> new technologies that use GPUs to enable extreme scaling that is thousands
>>> of times faster and 40 times more affordable than CPU-based solutions.
>>>
>>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments
>>> are for the sole use of the intended recipient(s) and are confidential or
>>> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
>>> disclosure, dissemination or copying of this email or its contents or
>>> attachments is prohibited. If you have received this communication in
>>> error, please notify the sender by reply email and permanently delete all
>>> copies of the email and its contents and attachments.
>>>
>>
>>
>>
>> --
>> _______________
>> Brad Bebee
>> CEO
>> Blazegraph
>> e:  be...@bl...
>> m: 202.642.7961
>> w:  www.blazegraph.com
>>
>> Blazegraph products help to solve the Graph Cache Thrash to achieve large
>> scale processing for graph and predictive analytics.  Blazegraph is the
>> creator of the industry’s first GPU-accelerated high-performance database
>> for large graphs, has been named as one of the “10 Companies and
>> Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.
>>
>>
>> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
>> performance graph database that supports both RDF/SPARQL and Apache
>> TinkerPop™ APIs.   Blazegraph GPU
>> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
>> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive
>> new technologies that use GPUs to enable extreme scaling that is thousands
>> of times faster and 40 times more affordable than CPU-based solutions.
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
>> disclosure, dissemination or copying of this email or its contents or
>> attachments is prohibited. If you have received this communication in
>> error, please notify the sender by reply email and permanently delete all
>> copies of the email and its contents and attachments.
>>
>


-- 
_______________
Brad Bebee
CEO
Blazegraph
e:  be...@bl...
m: 202.642.7961
w:  www.blazegraph.com

Blazegraph products help to solve the Graph Cache Thrash to achieve large
scale processing for graph and predictive analytics.  Blazegraph is the
creator of the industry’s first GPU-accelerated high-performance database
for large graphs, has been named as one of the “10 Companies and
Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.

Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
performance graph database that supports both RDF/SPARQL and Apache
TinkerPop™ APIs.   Blazegraph GPU
<https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
<https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
technologies that use GPUs to enable extreme scaling that is thousands of
times faster and 40 times more affordable than CPU-based solutions.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
disclosure, dissemination or copying of this email or its contents or
attachments is prohibited. If you have received this communication in
error, please notify the sender by reply email and permanently delete all
copies of the email and its contents and attachments.

Re: [Bigdata-developers] read only mode for the jar file

[Douglas F. <dr...@gm...>]

Brad,
  Works perfect.  As noted my error was copying the whole base web.xml and
making that change in it and using it.   Using your version where just that
one node is included and changed worked perfectly.

  Thanks for this...  Next I need to update my Dockerfile process, but I am
sure this is going to work.  Really like blazegraph....  coming from some
other triple stores and really seem to resonate with this one.

Take care
Doug


On Fri, May 27, 2016 at 12:26 PM Brad Bebee <be...@bl...> wrote:

> Here is the wiki link:
> https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#Read_Only_Configuration_with_the_Jetty_Override_and_Executable_Jar
> .
>
> On Fri, May 27, 2016 at 1:17 PM, Brad Bebee <be...@bl...> wrote:
>
>> Doug,
>>
>> You need to wrap it in the web-app tag (see contents below and
>> attached).  I've updated the Wiki as well with this documentation.   Happy
>> Blazegraphing... ;-)
>>
>> Cheers, --Brad
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>>
>> <web-app xmlns="http://java.sun.com/xml/ns/javaee"
>>
>>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>
>>       xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
>> http://java.sun.com/xml/ns/javaee/web-app_3_1.xsd"
>>
>>       version="3.1">
>>
>>   <context-param>
>>
>>    <description>When true, the REST API will not permit mutation
>> operations.</description>
>>
>>    <param-name>readOnly</param-name>
>>
>>    <param-value>true</param-value>
>>
>>   </context-param>
>>
>> </web-app>
>>
>> On Fri, May 27, 2016 at 11:57 AM, Douglas Fils <dr...@gm...> wrote:
>>
>>> Bryan,
>>>   I am following this and the docs.  However, when I pull the web.xml
>>> alter the lines for read only to:
>>>
>>>  <context-param>
>>>    <description>When true, the REST API will not permit mutation
>>> operations.</description>
>>>    <param-name>readOnly</param-name>
>>>    <param-value>true</param-value>
>>>   </context-param>
>>>
>>> I run with:  java -server -Xmx4g -Djetty.overrideWebXml=./override.xml
>>>  -jar blazegraph.jar
>>>
>>> I get errors
>>>
>>> WARN : NanoSparqlServer.java:517: Starting NSS
>>> WARN : AbstractLifeCycle.java:212: FAILED
>>> org.eclipse.jetty.servlet.ServletHandler@239963d8:
>>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>>> REST API,REST API
>>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>>> REST API,REST API
>>>
>>> WARN : WebAppContext.java:506: Failed startup of context
>>> o.e.j.w.WebAppContext@4157f54e
>>> {/blazegraph,jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war,STARTING}{jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war}
>>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>>> REST API,REST API
>>>
>>>
>>> sorry to be pest, I feel like I'm missing something simple.
>>>
>>> Doug
>>>
>>>
>>>
>>> On Thu, May 26, 2016 at 2:50 PM Bryan Thompson <br...@bl...>
>>> wrote:
>>>
>>>> See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer
>>>>
>>>> Customizing the web.xml
>>>>
>>>> You can override the default web.xml values in the executable jar using
>>>> the jetty.overrideWebXml property. The file you specify should override the
>>>> values that you'd like to replace. The web.xml values that default with the
>>>> blazegraph.jar are in web.xml
>>>> <https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
>>>> .
>>>>
>>>> -Djetty.overrideWebXml=/path/to/override.xml
>>>>
>>>> A full example is below.
>>>>
>>>> java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml -Dbigdata.propertyFile=/etc/blazegraph/RWS
>>>>
>>>>
>>>> Thanks,
>>>> Bryan
>>>>
>>>> On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...> wrote:
>>>>
>>>>> I am using the .jar version though, not a  .war file in an app server.
>>>>> Am I still able to somehow override or reference a new web.xml file?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
>>>>> wrote:
>>>>>
>>>>>> You can control this in web.xml.
>>>>>>
>>>>>> Thanks,
>>>>>> Bryan
>>>>>>
>>>>>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>   I am sure this is a simple question but I can't seem to launch a
>>>>>>> previously created blazegraph instance with the namespaces in "readonly"
>>>>>>> mode.
>>>>>>>
>>>>>>>   Is there a simple way to add a -D option or something to
>>>>>>>
>>>>>>> java -server -Xmx4g -jar blazegraph.jar
>>>>>>>
>>>>>>> such that the existing namespaces are readonly on the sparql
>>>>>>> endpoint so that I can expose the server to the net without a fear of
>>>>>>> someone doing a update/delete etc on the triples?
>>>>>>>
>>>>>>> Thanks
>>>>>>> Doug
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Mobile security can be enabling, not merely restricting. Employees
>>>>>>> who
>>>>>>> bring their own devices (BYOD) to work are irked by the imposition
>>>>>>> of MDM
>>>>>>> restrictions. Mobile Device Manager Plus allows you to control only
>>>>>>> the
>>>>>>> apps on BYO-devices by containerizing them, leaving personal data
>>>>>>> untouched!
>>>>>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>>>>>> _______________________________________________
>>>>>>> Bigdata-developers mailing list
>>>>>>> Big...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> What NetFlow Analyzer can do for you? Monitors network bandwidth and
>>> traffic
>>> patterns at an interface-level. Reveals which users, apps, and protocols
>>> are
>>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>>> planning reports.
>>> https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>>> _______________________________________________
>>> Bigdata-developers mailing list
>>> Big...@li...
>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>
>>>
>>
>>
>> --
>> _______________
>> Brad Bebee
>> CEO
>> Blazegraph
>> e:  be...@bl...
>> m: 202.642.7961
>> w:  www.blazegraph.com
>>
>> Blazegraph products help to solve the Graph Cache Thrash to achieve large
>> scale processing for graph and predictive analytics.  Blazegraph is the
>> creator of the industry’s first GPU-accelerated high-performance database
>> for large graphs, has been named as one of the “10 Companies and
>> Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.
>>
>>
>> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
>> performance graph database that supports both RDF/SPARQL and Apache
>> TinkerPop™ APIs.   Blazegraph GPU
>> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
>> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive
>> new technologies that use GPUs to enable extreme scaling that is thousands
>> of times faster and 40 times more affordable than CPU-based solutions.
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
>> disclosure, dissemination or copying of this email or its contents or
>> attachments is prohibited. If you have received this communication in
>> error, please notify the sender by reply email and permanently delete all
>> copies of the email and its contents and attachments.
>>
>
>
>
> --
> _______________
> Brad Bebee
> CEO
> Blazegraph
> e:  be...@bl...
> m: 202.642.7961
> w:  www.blazegraph.com
>
> Blazegraph products help to solve the Graph Cache Thrash to achieve large
> scale processing for graph and predictive analytics.  Blazegraph is the
> creator of the industry’s first GPU-accelerated high-performance database
> for large graphs, has been named as one of the “10 Companies and
> Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.
>
>
> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
> performance graph database that supports both RDF/SPARQL and Apache
> TinkerPop™ APIs.   Blazegraph GPU
> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
> technologies that use GPUs to enable extreme scaling that is thousands of
> times faster and 40 times more affordable than CPU-based solutions.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
> disclosure, dissemination or copying of this email or its contents or
> attachments is prohibited. If you have received this communication in
> error, please notify the sender by reply email and permanently delete all
> copies of the email and its contents and attachments.
>

Re: [Bigdata-developers] read only mode for the jar file

[Brad B. <be...@bl...>]

Here is the wiki link:
https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer#Read_Only_Configuration_with_the_Jetty_Override_and_Executable_Jar
.

On Fri, May 27, 2016 at 1:17 PM, Brad Bebee <be...@bl...> wrote:

> Doug,
>
> You need to wrap it in the web-app tag (see contents below and attached).
> I've updated the Wiki as well with this documentation.   Happy
> Blazegraphing... ;-)
>
> Cheers, --Brad
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <web-app xmlns="http://java.sun.com/xml/ns/javaee"
>
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
>       xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
> http://java.sun.com/xml/ns/javaee/web-app_3_1.xsd"
>
>       version="3.1">
>
>   <context-param>
>
>    <description>When true, the REST API will not permit mutation
> operations.</description>
>
>    <param-name>readOnly</param-name>
>
>    <param-value>true</param-value>
>
>   </context-param>
>
> </web-app>
>
> On Fri, May 27, 2016 at 11:57 AM, Douglas Fils <dr...@gm...> wrote:
>
>> Bryan,
>>   I am following this and the docs.  However, when I pull the web.xml
>> alter the lines for read only to:
>>
>>  <context-param>
>>    <description>When true, the REST API will not permit mutation
>> operations.</description>
>>    <param-name>readOnly</param-name>
>>    <param-value>true</param-value>
>>   </context-param>
>>
>> I run with:  java -server -Xmx4g -Djetty.overrideWebXml=./override.xml
>>  -jar blazegraph.jar
>>
>> I get errors
>>
>> WARN : NanoSparqlServer.java:517: Starting NSS
>> WARN : AbstractLifeCycle.java:212: FAILED
>> org.eclipse.jetty.servlet.ServletHandler@239963d8:
>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>> REST API,REST API
>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>> REST API,REST API
>>
>> WARN : WebAppContext.java:506: Failed startup of context
>> o.e.j.w.WebAppContext@4157f54e
>> {/blazegraph,jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war,STARTING}{jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war}
>> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
>> REST API,REST API
>>
>>
>> sorry to be pest, I feel like I'm missing something simple.
>>
>> Doug
>>
>>
>>
>> On Thu, May 26, 2016 at 2:50 PM Bryan Thompson <br...@bl...>
>> wrote:
>>
>>> See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer
>>>
>>> Customizing the web.xml
>>>
>>> You can override the default web.xml values in the executable jar using
>>> the jetty.overrideWebXml property. The file you specify should override the
>>> values that you'd like to replace. The web.xml values that default with the
>>> blazegraph.jar are in web.xml
>>> <https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
>>> .
>>>
>>> -Djetty.overrideWebXml=/path/to/override.xml
>>>
>>> A full example is below.
>>>
>>> java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml -Dbigdata.propertyFile=/etc/blazegraph/RWS
>>>
>>>
>>> Thanks,
>>> Bryan
>>>
>>> On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...> wrote:
>>>
>>>> I am using the .jar version though, not a  .war file in an app server.
>>>> Am I still able to somehow override or reference a new web.xml file?
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
>>>> wrote:
>>>>
>>>>> You can control this in web.xml.
>>>>>
>>>>> Thanks,
>>>>> Bryan
>>>>>
>>>>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>   I am sure this is a simple question but I can't seem to launch a
>>>>>> previously created blazegraph instance with the namespaces in "readonly"
>>>>>> mode.
>>>>>>
>>>>>>   Is there a simple way to add a -D option or something to
>>>>>>
>>>>>> java -server -Xmx4g -jar blazegraph.jar
>>>>>>
>>>>>> such that the existing namespaces are readonly on the sparql endpoint
>>>>>> so that I can expose the server to the net without a fear of someone doing
>>>>>> a update/delete etc on the triples?
>>>>>>
>>>>>> Thanks
>>>>>> Doug
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Mobile security can be enabling, not merely restricting. Employees who
>>>>>> bring their own devices (BYOD) to work are irked by the imposition of
>>>>>> MDM
>>>>>> restrictions. Mobile Device Manager Plus allows you to control only
>>>>>> the
>>>>>> apps on BYO-devices by containerizing them, leaving personal data
>>>>>> untouched!
>>>>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>>>>> _______________________________________________
>>>>>> Bigdata-developers mailing list
>>>>>> Big...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>>>
>>>>>>
>>>>>
>>>
>>
>> ------------------------------------------------------------------------------
>> What NetFlow Analyzer can do for you? Monitors network bandwidth and
>> traffic
>> patterns at an interface-level. Reveals which users, apps, and protocols
>> are
>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>> planning reports.
>> https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>
>
> --
> _______________
> Brad Bebee
> CEO
> Blazegraph
> e:  be...@bl...
> m: 202.642.7961
> w:  www.blazegraph.com
>
> Blazegraph products help to solve the Graph Cache Thrash to achieve large
> scale processing for graph and predictive analytics.  Blazegraph is the
> creator of the industry’s first GPU-accelerated high-performance database
> for large graphs, has been named as one of the “10 Companies and
> Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.
>
>
> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
> performance graph database that supports both RDF/SPARQL and Apache
> TinkerPop™ APIs.   Blazegraph GPU
> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
> technologies that use GPUs to enable extreme scaling that is thousands of
> times faster and 40 times more affordable than CPU-based solutions.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
> disclosure, dissemination or copying of this email or its contents or
> attachments is prohibited. If you have received this communication in
> error, please notify the sender by reply email and permanently delete all
> copies of the email and its contents and attachments.
>



-- 
_______________
Brad Bebee
CEO
Blazegraph
e:  be...@bl...
m: 202.642.7961
w:  www.blazegraph.com

Blazegraph products help to solve the Graph Cache Thrash to achieve large
scale processing for graph and predictive analytics.  Blazegraph is the
creator of the industry’s first GPU-accelerated high-performance database
for large graphs, has been named as one of the “10 Companies and
Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.

Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
performance graph database that supports both RDF/SPARQL and Apache
TinkerPop™ APIs.   Blazegraph GPU
<https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
<https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
technologies that use GPUs to enable extreme scaling that is thousands of
times faster and 40 times more affordable than CPU-based solutions.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
disclosure, dissemination or copying of this email or its contents or
attachments is prohibited. If you have received this communication in
error, please notify the sender by reply email and permanently delete all
copies of the email and its contents and attachments.

Re: [Bigdata-developers] read only mode for the jar file

[Brad B. <be...@bl...>]

Doug,

You need to wrap it in the web-app tag (see contents below and attached).
I've updated the Wiki as well with this documentation.   Happy
Blazegraphing... ;-)

Cheers, --Brad

<?xml version="1.0" encoding="UTF-8"?>

<web-app xmlns="http://java.sun.com/xml/ns/javaee"

      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

      xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_3_1.xsd"

      version="3.1">

  <context-param>

   <description>When true, the REST API will not permit mutation
operations.</description>

   <param-name>readOnly</param-name>

   <param-value>true</param-value>

  </context-param>

</web-app>

On Fri, May 27, 2016 at 11:57 AM, Douglas Fils <dr...@gm...> wrote:

> Bryan,
>   I am following this and the docs.  However, when I pull the web.xml
> alter the lines for read only to:
>
>  <context-param>
>    <description>When true, the REST API will not permit mutation
> operations.</description>
>    <param-name>readOnly</param-name>
>    <param-value>true</param-value>
>   </context-param>
>
> I run with:  java -server -Xmx4g -Djetty.overrideWebXml=./override.xml
>  -jar blazegraph.jar
>
> I get errors
>
> WARN : NanoSparqlServer.java:517: Starting NSS
> WARN : AbstractLifeCycle.java:212: FAILED
> org.eclipse.jetty.servlet.ServletHandler@239963d8:
> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
> REST API,REST API
> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
> REST API,REST API
>
> WARN : WebAppContext.java:506: Failed startup of context
> o.e.j.w.WebAppContext@4157f54e
> {/blazegraph,jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war,STARTING}{jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war}
> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
> REST API,REST API
>
>
> sorry to be pest, I feel like I'm missing something simple.
>
> Doug
>
>
>
> On Thu, May 26, 2016 at 2:50 PM Bryan Thompson <br...@bl...>
> wrote:
>
>> See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer
>>
>> Customizing the web.xml
>>
>> You can override the default web.xml values in the executable jar using
>> the jetty.overrideWebXml property. The file you specify should override the
>> values that you'd like to replace. The web.xml values that default with the
>> blazegraph.jar are in web.xml
>> <https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
>> .
>>
>> -Djetty.overrideWebXml=/path/to/override.xml
>>
>> A full example is below.
>>
>> java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml -Dbigdata.propertyFile=/etc/blazegraph/RWS
>>
>>
>> Thanks,
>> Bryan
>>
>> On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...> wrote:
>>
>>> I am using the .jar version though, not a  .war file in an app server.
>>> Am I still able to somehow override or reference a new web.xml file?
>>>
>>>
>>>
>>>
>>> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
>>> wrote:
>>>
>>>> You can control this in web.xml.
>>>>
>>>> Thanks,
>>>> Bryan
>>>>
>>>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...> wrote:
>>>>
>>>>> Hello,
>>>>>   I am sure this is a simple question but I can't seem to launch a
>>>>> previously created blazegraph instance with the namespaces in "readonly"
>>>>> mode.
>>>>>
>>>>>   Is there a simple way to add a -D option or something to
>>>>>
>>>>> java -server -Xmx4g -jar blazegraph.jar
>>>>>
>>>>> such that the existing namespaces are readonly on the sparql endpoint
>>>>> so that I can expose the server to the net without a fear of someone doing
>>>>> a update/delete etc on the triples?
>>>>>
>>>>> Thanks
>>>>> Doug
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Mobile security can be enabling, not merely restricting. Employees who
>>>>> bring their own devices (BYOD) to work are irked by the imposition of
>>>>> MDM
>>>>> restrictions. Mobile Device Manager Plus allows you to control only the
>>>>> apps on BYO-devices by containerizing them, leaving personal data
>>>>> untouched!
>>>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>>>> _______________________________________________
>>>>> Bigdata-developers mailing list
>>>>> Big...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>>
>>>>>
>>>>
>>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>


-- 
_______________
Brad Bebee
CEO
Blazegraph
e:  be...@bl...
m: 202.642.7961
w:  www.blazegraph.com

Blazegraph products help to solve the Graph Cache Thrash to achieve large
scale processing for graph and predictive analytics.  Blazegraph is the
creator of the industry’s first GPU-accelerated high-performance database
for large graphs, has been named as one of the “10 Companies and
Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.

Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
performance graph database that supports both RDF/SPARQL and Apache
TinkerPop™ APIs.   Blazegraph GPU
<https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
<https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
technologies that use GPUs to enable extreme scaling that is thousands of
times faster and 40 times more affordable than CPU-based solutions.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
disclosure, dissemination or copying of this email or its contents or
attachments is prohibited. If you have received this communication in
error, please notify the sender by reply email and permanently delete all
copies of the email and its contents and attachments.

Re: [Bigdata-developers] read only mode for the jar file

[Bryan T. <br...@bl...>]

Adding Brad. He is the jetty wizard.

Thanks,
Bryan

On Fri, May 27, 2016 at 11:57 AM, Douglas Fils <dr...@gm...> wrote:

> Bryan,
>   I am following this and the docs.  However, when I pull the web.xml
> alter the lines for read only to:
>
>  <context-param>
>    <description>When true, the REST API will not permit mutation
> operations.</description>
>    <param-name>readOnly</param-name>
>    <param-value>true</param-value>
>   </context-param>
>
> I run with:  java -server -Xmx4g -Djetty.overrideWebXml=./override.xml
>  -jar blazegraph.jar
>
> I get errors
>
> WARN : NanoSparqlServer.java:517: Starting NSS
> WARN : AbstractLifeCycle.java:212: FAILED
> org.eclipse.jetty.servlet.ServletHandler@239963d8:
> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
> REST API,REST API
> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
> REST API,REST API
>
> WARN : WebAppContext.java:506: Failed startup of context
> o.e.j.w.WebAppContext@4157f54e
> {/blazegraph,jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war,STARTING}{jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war}
> java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
> REST API,REST API
>
>
> sorry to be pest, I feel like I'm missing something simple.
>
> Doug
>
>
>
> On Thu, May 26, 2016 at 2:50 PM Bryan Thompson <br...@bl...>
> wrote:
>
>> See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer
>>
>> Customizing the web.xml
>>
>> You can override the default web.xml values in the executable jar using
>> the jetty.overrideWebXml property. The file you specify should override the
>> values that you'd like to replace. The web.xml values that default with the
>> blazegraph.jar are in web.xml
>> <https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
>> .
>>
>> -Djetty.overrideWebXml=/path/to/override.xml
>>
>> A full example is below.
>>
>> java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml -Dbigdata.propertyFile=/etc/blazegraph/RWS
>>
>>
>> Thanks,
>> Bryan
>>
>> On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...> wrote:
>>
>>> I am using the .jar version though, not a  .war file in an app server.
>>> Am I still able to somehow override or reference a new web.xml file?
>>>
>>>
>>>
>>>
>>> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
>>> wrote:
>>>
>>>> You can control this in web.xml.
>>>>
>>>> Thanks,
>>>> Bryan
>>>>
>>>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...> wrote:
>>>>
>>>>> Hello,
>>>>>   I am sure this is a simple question but I can't seem to launch a
>>>>> previously created blazegraph instance with the namespaces in "readonly"
>>>>> mode.
>>>>>
>>>>>   Is there a simple way to add a -D option or something to
>>>>>
>>>>> java -server -Xmx4g -jar blazegraph.jar
>>>>>
>>>>> such that the existing namespaces are readonly on the sparql endpoint
>>>>> so that I can expose the server to the net without a fear of someone doing
>>>>> a update/delete etc on the triples?
>>>>>
>>>>> Thanks
>>>>> Doug
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Mobile security can be enabling, not merely restricting. Employees who
>>>>> bring their own devices (BYOD) to work are irked by the imposition of
>>>>> MDM
>>>>> restrictions. Mobile Device Manager Plus allows you to control only the
>>>>> apps on BYO-devices by containerizing them, leaving personal data
>>>>> untouched!
>>>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>>>> _______________________________________________
>>>>> Bigdata-developers mailing list
>>>>> Big...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>>
>>>>>
>>>>
>>

Re: [Bigdata-developers] read only mode for the jar file

[Douglas F. <dr...@gm...>]

Bryan,
  I am following this and the docs.  However, when I pull the web.xml alter
the lines for read only to:

 <context-param>
   <description>When true, the REST API will not permit mutation
operations.</description>
   <param-name>readOnly</param-name>
   <param-value>true</param-value>
  </context-param>

I run with:  java -server -Xmx4g -Djetty.overrideWebXml=./override.xml
 -jar blazegraph.jar

I get errors

WARN : NanoSparqlServer.java:517: Starting NSS
WARN : AbstractLifeCycle.java:212: FAILED
org.eclipse.jetty.servlet.ServletHandler@239963d8:
java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
REST API,REST API
java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
REST API,REST API

WARN : WebAppContext.java:506: Failed startup of context
o.e.j.w.WebAppContext@4157f54e
{/blazegraph,jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war,STARTING}{jar:file:/Users/dfils/Desktop/blazegraph/blazegraph.jar!/war}
java.lang.IllegalStateException: Multiple servlets map to path: /sparql:
REST API,REST API


sorry to be pest, I feel like I'm missing something simple.

Doug



On Thu, May 26, 2016 at 2:50 PM Bryan Thompson <br...@bl...> wrote:

> See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer
>
> Customizing the web.xml
>
> You can override the default web.xml values in the executable jar using
> the jetty.overrideWebXml property. The file you specify should override the
> values that you'd like to replace. The web.xml values that default with the
> blazegraph.jar are in web.xml
> <https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
> .
>
> -Djetty.overrideWebXml=/path/to/override.xml
>
> A full example is below.
>
> java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml -Dbigdata.propertyFile=/etc/blazegraph/RWS
>
>
> Thanks,
> Bryan
>
> On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...> wrote:
>
>> I am using the .jar version though, not a  .war file in an app server.
>> Am I still able to somehow override or reference a new web.xml file?
>>
>>
>>
>>
>> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
>> wrote:
>>
>>> You can control this in web.xml.
>>>
>>> Thanks,
>>> Bryan
>>>
>>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...> wrote:
>>>
>>>> Hello,
>>>>   I am sure this is a simple question but I can't seem to launch a
>>>> previously created blazegraph instance with the namespaces in "readonly"
>>>> mode.
>>>>
>>>>   Is there a simple way to add a -D option or something to
>>>>
>>>> java -server -Xmx4g -jar blazegraph.jar
>>>>
>>>> such that the existing namespaces are readonly on the sparql endpoint
>>>> so that I can expose the server to the net without a fear of someone doing
>>>> a update/delete etc on the triples?
>>>>
>>>> Thanks
>>>> Doug
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Mobile security can be enabling, not merely restricting. Employees who
>>>> bring their own devices (BYOD) to work are irked by the imposition of
>>>> MDM
>>>> restrictions. Mobile Device Manager Plus allows you to control only the
>>>> apps on BYO-devices by containerizing them, leaving personal data
>>>> untouched!
>>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>>> _______________________________________________
>>>> Bigdata-developers mailing list
>>>> Big...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>>
>>>>
>>>
>

Re: [Bigdata-developers] read only mode for the jar file

[Markus A. <ack...@in...>]

Thank you Brian for providing appropriate pointers to existing
documentation. Since I did not find that a couple of days ago
when I wanted to achieve the same as Doug, I looked at
StandaloneNanoSparqlServer.java and wrote a very simple patch
that adds a "-readOnly" command line switch to the blazegraph.jar.

If you think this would be a worthwhile small addition for the next
release, I am happy to open to add the feature request and patch
suggestion to JIRA.

All the best,

Markus

Re: [Bigdata-developers] read only mode for the jar file

[Bryan T. <br...@bl...>]

See https://wiki.blazegraph.com/wiki/index.php/NanoSparqlServer

Customizing the web.xml

You can override the default web.xml values in the executable jar using the
jetty.overrideWebXml property. The file you specify should override the
values that you'd like to replace. The web.xml values that default with the
blazegraph.jar are in web.xml
<https://github.com/blazegraph/database/blob/master/bigdata-war-html/src/main/webapp/WEB-INF/web.xml>
.

-Djetty.overrideWebXml=/path/to/override.xml

A full example is below.

java -server -Xmx4g -Djetty.overrideWebXml=/path/to/override.xml
-Dbigdata.propertyFile=/etc/blazegraph/RWS


Thanks,
Bryan

On Thu, May 26, 2016 at 2:03 PM, Douglas Fils <dr...@gm...> wrote:

> I am using the .jar version though, not a  .war file in an app server.
> Am I still able to somehow override or reference a new web.xml file?
>
>
>
>
> On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
> wrote:
>
>> You can control this in web.xml.
>>
>> Thanks,
>> Bryan
>>
>> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...> wrote:
>>
>>> Hello,
>>>   I am sure this is a simple question but I can't seem to launch a
>>> previously created blazegraph instance with the namespaces in "readonly"
>>> mode.
>>>
>>>   Is there a simple way to add a -D option or something to
>>>
>>> java -server -Xmx4g -jar blazegraph.jar
>>>
>>> such that the existing namespaces are readonly on the sparql endpoint so
>>> that I can expose the server to the net without a fear of someone doing a
>>> update/delete etc on the triples?
>>>
>>> Thanks
>>> Doug
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Mobile security can be enabling, not merely restricting. Employees who
>>> bring their own devices (BYOD) to work are irked by the imposition of MDM
>>> restrictions. Mobile Device Manager Plus allows you to control only the
>>> apps on BYO-devices by containerizing them, leaving personal data
>>> untouched!
>>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>>> _______________________________________________
>>> Bigdata-developers mailing list
>>> Big...@li...
>>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>>
>>>
>>

Re: [Bigdata-developers] read only mode for the jar file

[Bryan T. <br...@bl...>]

You can control this in web.xml.

Thanks,
Bryan

On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...> wrote:

> Hello,
>   I am sure this is a simple question but I can't seem to launch a
> previously created blazegraph instance with the namespaces in "readonly"
> mode.
>
>   Is there a simple way to add a -D option or something to
>
> java -server -Xmx4g -jar blazegraph.jar
>
> such that the existing namespaces are readonly on the sparql endpoint so
> that I can expose the server to the net without a fear of someone doing a
> update/delete etc on the triples?
>
> Thanks
> Doug
>
>
>
> ------------------------------------------------------------------------------
> Mobile security can be enabling, not merely restricting. Employees who
> bring their own devices (BYOD) to work are irked by the imposition of MDM
> restrictions. Mobile Device Manager Plus allows you to control only the
> apps on BYO-devices by containerizing them, leaving personal data
> untouched!
> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>

Re: [Bigdata-developers] read only mode for the jar file

[Douglas F. <dr...@gm...>]

I am using the .jar version though, not a  .war file in an app server.
Am I still able to somehow override or reference a new web.xml file?




On Thu, May 26, 2016 at 12:56 PM Bryan Thompson <br...@bl...>
wrote:

> You can control this in web.xml.
>
> Thanks,
> Bryan
>
> On Thu, May 26, 2016 at 1:45 PM, Douglas Fils <dr...@gm...> wrote:
>
>> Hello,
>>   I am sure this is a simple question but I can't seem to launch a
>> previously created blazegraph instance with the namespaces in "readonly"
>> mode.
>>
>>   Is there a simple way to add a -D option or something to
>>
>> java -server -Xmx4g -jar blazegraph.jar
>>
>> such that the existing namespaces are readonly on the sparql endpoint so
>> that I can expose the server to the net without a fear of someone doing a
>> update/delete etc on the triples?
>>
>> Thanks
>> Doug
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Mobile security can be enabling, not merely restricting. Employees who
>> bring their own devices (BYOD) to work are irked by the imposition of MDM
>> restrictions. Mobile Device Manager Plus allows you to control only the
>> apps on BYO-devices by containerizing them, leaving personal data
>> untouched!
>> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>