Menu
▾
▴
Re: [Bigdata-developers] SPARQL DELETE, INSERT performance question
|
From: Bryan T. <br...@bl...> - 2016-08-19 21:06:11
|
You might also give it less memory for a small data set. What is important
is making sure that there is NO swapping. Java handles swapping very
poorly because it needs to do memory scans during GC.
Thanks,
Bryan
On Fri, Aug 19, 2016 at 4:36 PM, Kevin Ford <kf...@ar...> wrote:
> Thanks, Bryan.
>
> We're looking at whether we can increase the memory on the machine
> (currently 8gb, but has other applications running on it) and see if
> that improves matters.
>
> Yours,
> Kevin
>
>
> On 8/18/16 12:32, Bryan Thompson wrote:
> > Kevin,
> >
> > I would not anticipate the kinds of times that you are reporting below
> > for these update requests. Assuming that URI and URI/other are not all
> > of the triples in your graph, these should be very fast updates.
> >
> > Can you confirm that the machine has at least those 4GB of RAM available
> > beyond the other active processes?
> >
> > Note that Linux distributions will begin to swap out processes once 50%
> > of the physical memory has been used unless you set swappiness to zero
> > (a kernel parameter - see the Blazegraph wiki for more information on
> > this topic).
> >
> > Bryan
> >
> > On Wednesday, August 17, 2016, Kevin Ford <kf...@ar...
> > <mailto:kf...@ar...>> wrote:
> >
> > Dear All,
> >
> > I have a question about SPARQL DELETE/INSERT performance.
> >
> > We have a job that issues two DELETE queries followed by an INSERT
> > query, per request, to keep our triplestore up to date. The pattern
> is
> > as follows:
> >
> > DELETE { <URI> ?p ?o };
> > DELETE { <URI/other> ?p ?o };
> > INSERT {
> > <URI> <new> '1' .
> > <URI> <new> '2' .
> > <URI> <new> '3' .
> > <URI/other> <new> '1' .
> > }
> >
> > We've noted two things:
> >
> > 1) If a store is empty, this is relatively fast.
> > 2) IF the store is configured for automatic inferencing, it is slower
> > than a store without that feature activated.
> >
> > Neither of those observations seems particularly surprising, but
> each of
> > those two DELETE statements and the one INSERT takes more than 3
> seconds
> > against a store with only 4.6 million triples. See below for a
> sample
> > of the output.
> >
> > Blazegraph is allocated 4g of memory.
> >
> > We're working within a framework that produces these series of
> queries
> > so, before making any custom modifications, I was wondering if this
> > performance is to be expected? If not, would it point to a
> configuation
> > issue of some kind, or something else?
> >
> > Also, I've noted that issuing a DELETE HTTP request appears to handle
> > the deletes faster, but is there a more optimal way to construct
> those
> > three queries (the DELETE/INSERT WHERE pattern did not appear to
> improve
> > matters based on a few isolated tests).
> >
> > Cordially,
> > Kevin
> >
> >
> > $ curl -X POST https://laketsidx/blazegraph/namespace/lakeidx/sparql
> > <https://laketsidx/blazegraph/namespace/lakeidx/sparql>
> > --data @bg-test-02.sparql -H "Content-type:
> > application/x-www-form-urlencoded"
> > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> > "http://www.w3.org/TR/html4/loose.dtd
> > <http://www.w3.org/TR/html4/loose.dtd>"><html><head><meta
> > http-equiv="Content-Type"
> > content="text/html;charset=UTF-8"><title>blazegraph™ by
> > SYSTAP</title
> > ></head
> > ><body<p>totalElapsed=33ms, elapsed=33ms, connFlush=0ms,
> > batchResolve=0, whereClause=23ms, deleteClause=9ms,
> insertClause=23ms</p
> > ><hr><p>totalElapsed=2579ms, elapsed=21ms, connFlush=2524ms,
> > batchResolve=0, whereClause=21ms, deleteClause=0ms,
> insertClause=21ms</p
> > ><hr><p>totalElapsed=2582ms, elapsed=2ms, connFlush=0ms,
> > batchResolve=0, whereClause=0ms, deleteClause=0ms,
> insertClause=0ms</p
> > ><hr><p>COMMIT: totalElapsed=3118ms, commitTime=1471453475719,
> > mutationCount=62</p
> > ></html
> >
> > $ curl -X POST https://laketsidx/blazegraph/namespace/lakeidx/sparql
> > <https://laketsidx/blazegraph/namespace/lakeidx/sparql>
> > --data @bg-test-02.sparql -H "Content-type:
> > application/x-www-form-urlencoded"
> > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> > "http://www.w3.org/TR/html4/loose.dtd
> > <http://www.w3.org/TR/html4/loose.dtd>"><html><head><meta
> > http-equiv="Content-Type"
> > content="text/html;charset=UTF-8"><title>blazegraph™ by
> > SYSTAP</title
> > ></head
> > ><body<p>totalElapsed=24ms, elapsed=24ms, connFlush=0ms,
> > batchResolve=0, whereClause=12ms, deleteClause=11ms,
> > insertClause=12ms</p
> > ><hr><p>totalElapsed=2966ms, elapsed=12ms, connFlush=2929ms,
> > batchResolve=0, whereClause=12ms, deleteClause=0ms,
> insertClause=12ms</p
> > ><hr><p>totalElapsed=2969ms, elapsed=2ms, connFlush=0ms,
> > batchResolve=0, whereClause=0ms, deleteClause=0ms,
> insertClause=0ms</p
> > ><hr><p>COMMIT: totalElapsed=3669ms, commitTime=1471453483251,
> > mutationCount=62</p
> > ></html
> >
> >
> > ------------------------------------------------------------
> ------------------
> > _______________________________________________
> > Bigdata-developers mailing list
> > Big...@li... <javascript:;>
> > https://lists.sourceforge.net/lists/listinfo/bigdata-developers
> > <https://lists.sourceforge.net/lists/listinfo/bigdata-developers>
> >
>
> ------------------------------------------------------------
> ------------------
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
|
