Re: [Bigdata-developers] trivial ASK takes over 5s

[Bryan T. <br...@sy...>]

That is possible. There are some barriers around commit processing that
might prevent new tx starts.  So you could have a commit that was taking a
few seconds and during that time new queries might block.  In particular,
AbstractJournal.commitNow() takes the following lock.

        final WriteLock lock = _fieldReadWriteLock.writeLock();

        lock.lock();

That lock is contended by some other code paths.  For example,
getIndexLocal() in the same class needs that lock if there is a cache miss.

We do use a read/write lock there, but once commitNow() gets the write lock
no other thread will be able to proceed.

It is possible that we could defer acquiring that lock. The slowest part of
the commit is flushing the indices to the disk.  If we take a lock that is
specific to the commit, and then wait until we have already flushed the
indices to the disk, we might be able to reduce this latency.  Of course,
such changes raise the possibility of lock ordering problems which could
lead to a deadlock.  So we would have to take a pretty close look at this
before making the change.

Thanks,
Bryan



----
Bryan Thompson
Chief Scientist & Founder
Blazegraph
e: br...@bl...
w: http://blazegraph.com

Blazegraph products help to solve the Graph Cache Thrash to achieve large
scale processing for graph and predictive analytics.  Blazegraph is the
creator of the industry’s first GPU-accelerated high-performance database
for large graphs, has been named as one of the “10 Companies and
Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.

Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
performance graph database that supports both RDF/SPARQL and
Tinkerpop/Blueprints APIs.   Blazegraph GPU
<https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
<https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
technologies that use GPUs to enable extreme scaling that is thousands of
times faster and 40 times more affordable than CPU-based solutions.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
disclosure, dissemination or copying of this email or its contents or
attachments is prohibited. If you have received this communication in
error, please notify the sender by reply email and permanently delete all
copies of the email and its contents and attachments.

On Mon, Mar 21, 2016 at 4:27 PM, Jeremy J Carroll <jj...@sy...> wrote:

> Yes - or blocking some other part of the process ...
>
>
> On Mar 21, 2016, at 12:12 PM, Bryan Thompson <br...@sy...> wrote:
>
> Is the thought then that there might be some barrier that is blocking the
> start of a new query during some part of that large create?
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> Blazegraph
> e: br...@bl...
> w: http://blazegraph.com
>
> Blazegraph products help to solve the Graph Cache Thrash to achieve large
> scale processing for graph and predictive analytics.  Blazegraph is the
> creator of the industry’s first GPU-accelerated high-performance database
> for large graphs, has been named as one of the “10 Companies and
> Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.
>
>
> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
> performance graph database that supports both RDF/SPARQL and
> Tinkerpop/Blueprints APIs.   Blazegraph GPU
> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive new
> technologies that use GPUs to enable extreme scaling that is thousands of
> times faster and 40 times more affordable than CPU-based solutions.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
> disclosure, dissemination or copying of this email or its contents or
> attachments is prohibited. If you have received this communication in
> error, please notify the sender by reply email and permanently delete all
> copies of the email and its contents and attachments.
>
> On Mon, Mar 21, 2016 at 12:40 PM, Jeremy J Carroll <jj...@sy...> wrote:
>
>> The logs I have are:
>> - full GC logs
>> - python level logs of http requests and errors/warnings (of our
>> application server: not the blazegraph http requests)
>>
>> I can map those logs into SPARQL requests approximately, and I see that
>> at the point where I got the long wait, there were a single large create
>> operations that use the
>> INSERT-by-POST operation taking approximately 30s (including python
>> processing) which normally is much more than 50% of blazegraph time.
>>
>> Very minor other activity going on
>>
>> No interesting GC activity
>>
>> Jeremy
>>
>>
>>
>>
>> On Mar 19, 2016, at 11:57 AM, Bryan Thompson <br...@sy...> wrote:
>>
>> Do you have full logs for the server during this interval?
>> Bryan
>>
>> ----
>> Bryan Thompson
>> Chief Scientist & Founder
>> Blazegraph
>> e: br...@bl...
>> w: http://blazegraph.com
>>
>> Blazegraph products help to solve the Graph Cache Thrash to achieve large
>> scale processing for graph and predictive analytics.  Blazegraph is the
>> creator of the industry’s first GPU-accelerated high-performance database
>> for large graphs, has been named as one of the “10 Companies and
>> Technologies to Watch in 2016” <http://insideanalysis.com/2016/01/20535/>.
>>
>>
>> Blazegraph Database <https://www.blazegraph.com/> is our ultra-high
>> performance graph database that supports both RDF/SPARQL and
>> Tinkerpop/Blueprints APIs.   Blazegraph GPU
>> <https://www.blazegraph.com/product/gpu-accelerated/> andBlazegraph DAS
>> <https://www.blazegraph.com/product/gpu-accelerated/>L are disruptive
>> new technologies that use GPUs to enable extreme scaling that is thousands
>> of times faster and 40 times more affordable than CPU-based solutions.
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP, LLC DBA Blazegraph.  Any unauthorized review, use,
>> disclosure, dissemination or copying of this email or its contents or
>> attachments is prohibited. If you have received this communication in
>> error, please notify the sender by reply email and permanently delete all
>> copies of the email and its contents and attachments.
>>
>> On Fri, Mar 18, 2016 at 6:53 PM, Jeremy J Carroll <jj...@sy...> wrote:
>>
>>> I checked the GC logs. We have no GC stoppage of more than a second for
>>> days either side of the slow query.
>>>
>>> The slow query was on March 6th, there was a 3s stoppage on Feb 29th and
>>> a 1.8s stoppage on March 9th
>>>
>>> Any other thoughts?
>>>
>>> Jeremy
>>>
>>>
>>> On Mar 17, 2016, at 2:54 PM, Bryan Thompson <br...@sy...> wrote:
>>>
>>> 5s is probably a major GC event.
>>>
>>>
>>>
>>
>>
>
>