Re: [Bigdata-developers] Journal remaining large after performing SPARQL UPDATE queries

[Bryan T. <br...@sy...>]

There are a variety of ways to reduce the size of a journal, however the
journal itself by design does not "shink" when you remove data. Instead the
RWStore will mark that there are free slots on the journal will reuse those
free slots for new allocations.

If you observe a continued growth in the file size, there are generally two
reasons.

1. Simply writing more data, which is to be expected.
2. Holding open a long lived query connection while concurrently writing
more data, which can be somewhat counterintuitive.

The reason why the journal grows for (2) is that recycling of deleted slots
the commit point on which the query is reading can not performed until the
query terminates and the database rolls through another commit point.  In
general, this is completely transparent.  Issues can occasionally arise for
embedded applications that do not correctly close database connections or
for REST API applications where there is a long running query.  Simply
having the workbench open has no effect on this.  It is only an active
query against a historical commit point that would pin the storage for the
historical commit point.

Thanks,
Bryan


----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.blazegraph.com

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  Blazegraph is now available with GPU acceleration using our disruptive
technology to accelerate data-parallel graph analytics and graph query.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Mon, Dec 14, 2015 at 12:06 PM, Reul, Quentin <
que...@wo...> wrote:

> Hi,
>
>
>
> We are using Blazegraph 1.5.2 as part of a project. As part of our
> assessment, we are observing that the size of the Journal is not only
> rapidly increasing, but also that it does not seem to decrease in size
> after performing SPARQL UPDATE queries. For instance, we do a clean up of
> the repositories every so often using
>
>                DELETE {
>
>         ?s ?p ?o .
>
>      } WHERE {
>
>         ?s ?p ?o .
>
>       }
>
> Although this query removes the triples, it does not seem to have any
> impact on the size of the Journal. Looking at the JIRA [1], I see that a
> recommendation is to do a data import / export. Is there any configuration
> that can be applied such that the Journal gets purged after removing
> triples?
>
>
>
> Kind regards,
>
>
>
> [1] https://jira.blazegraph.com/browse/BLZG-1088
>
>
>
> *Quentin Reul *
>
> Advanced Technology | Global Platform Organization | +1 (917) 891 5490
>
> Email: *Que...@wo... <Que...@wo...>*
>
> Skype: quentin_reul
>
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>