Menu
▾
▴
Re: [Bigdata-developers] SPARQL DELETE, INSERT performance question
|
From: Kevin F. <kf...@ar...> - 2016-08-19 21:04:34
|
Thanks, Bryan.
We're looking at whether we can increase the memory on the machine
(currently 8gb, but has other applications running on it) and see if
that improves matters.
Yours,
Kevin
On 8/18/16 12:32, Bryan Thompson wrote:
> Kevin,
>
> I would not anticipate the kinds of times that you are reporting below
> for these update requests. Assuming that URI and URI/other are not all
> of the triples in your graph, these should be very fast updates.
>
> Can you confirm that the machine has at least those 4GB of RAM available
> beyond the other active processes?
>
> Note that Linux distributions will begin to swap out processes once 50%
> of the physical memory has been used unless you set swappiness to zero
> (a kernel parameter - see the Blazegraph wiki for more information on
> this topic).
>
> Bryan
>
> On Wednesday, August 17, 2016, Kevin Ford <kf...@ar...
> <mailto:kf...@ar...>> wrote:
>
> Dear All,
>
> I have a question about SPARQL DELETE/INSERT performance.
>
> We have a job that issues two DELETE queries followed by an INSERT
> query, per request, to keep our triplestore up to date. The pattern is
> as follows:
>
> DELETE { <URI> ?p ?o };
> DELETE { <URI/other> ?p ?o };
> INSERT {
> <URI> <new> '1' .
> <URI> <new> '2' .
> <URI> <new> '3' .
> <URI/other> <new> '1' .
> }
>
> We've noted two things:
>
> 1) If a store is empty, this is relatively fast.
> 2) IF the store is configured for automatic inferencing, it is slower
> than a store without that feature activated.
>
> Neither of those observations seems particularly surprising, but each of
> those two DELETE statements and the one INSERT takes more than 3 seconds
> against a store with only 4.6 million triples. See below for a sample
> of the output.
>
> Blazegraph is allocated 4g of memory.
>
> We're working within a framework that produces these series of queries
> so, before making any custom modifications, I was wondering if this
> performance is to be expected? If not, would it point to a configuation
> issue of some kind, or something else?
>
> Also, I've noted that issuing a DELETE HTTP request appears to handle
> the deletes faster, but is there a more optimal way to construct those
> three queries (the DELETE/INSERT WHERE pattern did not appear to improve
> matters based on a few isolated tests).
>
> Cordially,
> Kevin
>
>
> $ curl -X POST https://laketsidx/blazegraph/namespace/lakeidx/sparql
> <https://laketsidx/blazegraph/namespace/lakeidx/sparql>
> --data @bg-test-02.sparql -H "Content-type:
> application/x-www-form-urlencoded"
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> "http://www.w3.org/TR/html4/loose.dtd
> <http://www.w3.org/TR/html4/loose.dtd>"><html><head><meta
> http-equiv="Content-Type"
> content="text/html;charset=UTF-8"><title>blazegraph™ by
> SYSTAP</title
> ></head
> ><body<p>totalElapsed=33ms, elapsed=33ms, connFlush=0ms,
> batchResolve=0, whereClause=23ms, deleteClause=9ms, insertClause=23ms</p
> ><hr><p>totalElapsed=2579ms, elapsed=21ms, connFlush=2524ms,
> batchResolve=0, whereClause=21ms, deleteClause=0ms, insertClause=21ms</p
> ><hr><p>totalElapsed=2582ms, elapsed=2ms, connFlush=0ms,
> batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p
> ><hr><p>COMMIT: totalElapsed=3118ms, commitTime=1471453475719,
> mutationCount=62</p
> ></html
>
> $ curl -X POST https://laketsidx/blazegraph/namespace/lakeidx/sparql
> <https://laketsidx/blazegraph/namespace/lakeidx/sparql>
> --data @bg-test-02.sparql -H "Content-type:
> application/x-www-form-urlencoded"
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> "http://www.w3.org/TR/html4/loose.dtd
> <http://www.w3.org/TR/html4/loose.dtd>"><html><head><meta
> http-equiv="Content-Type"
> content="text/html;charset=UTF-8"><title>blazegraph™ by
> SYSTAP</title
> ></head
> ><body<p>totalElapsed=24ms, elapsed=24ms, connFlush=0ms,
> batchResolve=0, whereClause=12ms, deleteClause=11ms,
> insertClause=12ms</p
> ><hr><p>totalElapsed=2966ms, elapsed=12ms, connFlush=2929ms,
> batchResolve=0, whereClause=12ms, deleteClause=0ms, insertClause=12ms</p
> ><hr><p>totalElapsed=2969ms, elapsed=2ms, connFlush=0ms,
> batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p
> ><hr><p>COMMIT: totalElapsed=3669ms, commitTime=1471453483251,
> mutationCount=62</p
> ></html
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li... <javascript:;>
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
>
|
