Menu
▾
▴
Re: [Bigdata-developers] SPARQL DELETE, INSERT performance question
|
From: Kevin F. <kf...@ar...> - 2016-08-24 19:41:48
|
Thanks, Bryan.
Reducing the amount of max memory, thereby avoiding swapping, had a
positive impact.
Yours,
Kevin
On 8/19/16 16:06, Bryan Thompson wrote:
> You might also give it less memory for a small data set. What is
> important is making sure that there is NO swapping. Java handles
> swapping very poorly because it needs to do memory scans during GC.
>
> Thanks,
> Bryan
>
> On Fri, Aug 19, 2016 at 4:36 PM, Kevin Ford <kf...@ar...
> <mailto:kf...@ar...>> wrote:
>
> Thanks, Bryan.
>
> We're looking at whether we can increase the memory on the machine
> (currently 8gb, but has other applications running on it) and see if
> that improves matters.
>
> Yours,
> Kevin
>
>
> On 8/18/16 12:32, Bryan Thompson wrote:
> > Kevin,
> >
> > I would not anticipate the kinds of times that you are reporting below
> > for these update requests. Assuming that URI and URI/other are
> not all
> > of the triples in your graph, these should be very fast updates.
> >
> > Can you confirm that the machine has at least those 4GB of RAM
> available
> > beyond the other active processes?
> >
> > Note that Linux distributions will begin to swap out processes
> once 50%
> > of the physical memory has been used unless you set swappiness to zero
> > (a kernel parameter - see the Blazegraph wiki for more information on
> > this topic).
> >
> > Bryan
> >
> > On Wednesday, August 17, 2016, Kevin Ford <kf...@ar...
> <mailto:kf...@ar...>
> > <mailto:kf...@ar... <mailto:kf...@ar...>>> wrote:
> >
> > Dear All,
> >
> > I have a question about SPARQL DELETE/INSERT performance.
> >
> > We have a job that issues two DELETE queries followed by an INSERT
> > query, per request, to keep our triplestore up to date. The
> pattern is
> > as follows:
> >
> > DELETE { <URI> ?p ?o };
> > DELETE { <URI/other> ?p ?o };
> > INSERT {
> > <URI> <new> '1' .
> > <URI> <new> '2' .
> > <URI> <new> '3' .
> > <URI/other> <new> '1' .
> > }
> >
> > We've noted two things:
> >
> > 1) If a store is empty, this is relatively fast.
> > 2) IF the store is configured for automatic inferencing, it is
> slower
> > than a store without that feature activated.
> >
> > Neither of those observations seems particularly surprising,
> but each of
> > those two DELETE statements and the one INSERT takes more than
> 3 seconds
> > against a store with only 4.6 million triples. See below for
> a sample
> > of the output.
> >
> > Blazegraph is allocated 4g of memory.
> >
> > We're working within a framework that produces these series of
> queries
> > so, before making any custom modifications, I was wondering if
> this
> > performance is to be expected? If not, would it point to a
> configuation
> > issue of some kind, or something else?
> >
> > Also, I've noted that issuing a DELETE HTTP request appears to
> handle
> > the deletes faster, but is there a more optimal way to
> construct those
> > three queries (the DELETE/INSERT WHERE pattern did not appear
> to improve
> > matters based on a few isolated tests).
> >
> > Cordially,
> > Kevin
> >
> >
> > $ curl -X POST
> https://laketsidx/blazegraph/namespace/lakeidx/sparql
> <https://laketsidx/blazegraph/namespace/lakeidx/sparql>
> > <https://laketsidx/blazegraph/namespace/lakeidx/sparql
> <https://laketsidx/blazegraph/namespace/lakeidx/sparql>>
> > --data @bg-test-02.sparql -H "Content-type:
> > application/x-www-form-urlencoded"
> > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> > "http://www.w3.org/TR/html4/loose.dtd
> <http://www.w3.org/TR/html4/loose.dtd>
> > <http://www.w3.org/TR/html4/loose.dtd
> <http://www.w3.org/TR/html4/loose.dtd>>"><html><head><meta
> > http-equiv="Content-Type"
> > content="text/html;charset=UTF-8"><title>blazegraph™ by
> > SYSTAP</title
> > ></head
> > ><body<p>totalElapsed=33ms, elapsed=33ms, connFlush=0ms,
> > batchResolve=0, whereClause=23ms, deleteClause=9ms,
> insertClause=23ms</p
> > ><hr><p>totalElapsed=2579ms, elapsed=21ms, connFlush=2524ms,
> > batchResolve=0, whereClause=21ms, deleteClause=0ms,
> insertClause=21ms</p
> > ><hr><p>totalElapsed=2582ms, elapsed=2ms, connFlush=0ms,
> > batchResolve=0, whereClause=0ms, deleteClause=0ms,
> insertClause=0ms</p
> > ><hr><p>COMMIT: totalElapsed=3118ms, commitTime=1471453475719,
> > mutationCount=62</p
> > ></html
> >
> > $ curl -X POST
> https://laketsidx/blazegraph/namespace/lakeidx/sparql
> <https://laketsidx/blazegraph/namespace/lakeidx/sparql>
> > <https://laketsidx/blazegraph/namespace/lakeidx/sparql
> <https://laketsidx/blazegraph/namespace/lakeidx/sparql>>
> > --data @bg-test-02.sparql -H "Content-type:
> > application/x-www-form-urlencoded"
> > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> > "http://www.w3.org/TR/html4/loose.dtd
> <http://www.w3.org/TR/html4/loose.dtd>
> > <http://www.w3.org/TR/html4/loose.dtd
> <http://www.w3.org/TR/html4/loose.dtd>>"><html><head><meta
> > http-equiv="Content-Type"
> > content="text/html;charset=UTF-8"><title>blazegraph™ by
> > SYSTAP</title
> > ></head
> > ><body<p>totalElapsed=24ms, elapsed=24ms, connFlush=0ms,
> > batchResolve=0, whereClause=12ms, deleteClause=11ms,
> > insertClause=12ms</p
> > ><hr><p>totalElapsed=2966ms, elapsed=12ms, connFlush=2929ms,
> > batchResolve=0, whereClause=12ms, deleteClause=0ms,
> insertClause=12ms</p
> > ><hr><p>totalElapsed=2969ms, elapsed=2ms, connFlush=0ms,
> > batchResolve=0, whereClause=0ms, deleteClause=0ms,
> insertClause=0ms</p
> > ><hr><p>COMMIT: totalElapsed=3669ms, commitTime=1471453483251,
> > mutationCount=62</p
> > ></html
> >
> >
> >
> ------------------------------------------------------------------------------
> > _______________________________________________
> > Bigdata-developers mailing list
> > Big...@li...
> <mailto:Big...@li...> <javascript:;>
> >
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
> >
> <https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>>
> >
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> <mailto:Big...@li...>
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
>
>
|
