Re: [Bigdata-developers] Help list - Newbie first question

[Igor K. <igo...@ms...>]

Ghislain,

AFAIK, --data-urlencode could not read data from  file in  the  form you've
written. Try following syntax

curl -X POST http://localhost:9999/blazegraph/namespace/<mykb>/sparql
--data-urlencode "query=`cat myQuery.sparql`" --data-urlencode
'format=json' -o myQuery.json

Testing pattern usually should follow expected workload. For standard
application, server does not get restarted often, so it is  'warmed up'
most  of the  time.
During warm-up phase query execution  times might differ from main phase,
so you should  probably add 'warm-up cycle', which is basically execution
of queries mix which follow same pattern as main  application issues.
Running 100-200 queries (or applying  workload for  5-10 minutes) would
warm-up server. After that main  query mix might be  executed. Also note,
that queries and their order, which were executed during warm-up and main
stage is important and might reflect in different results. So there are 2
options: first is to establish fixed mixture of queries for performance
testing, and the second option is to apply a workload for a longer time to
get results averaged over time, so particular fluctuations would not impact
performance measurement. But for the  second option you need to make
queries varying, to reduce impact from caching.


On Mon, Jan 23, 2017 at 4:32 PM, Ghislain ATEMEZING <
ghi...@gm...> wrote:

> Igor, thanks again for the explanations.
> Since I have a lot of queries to do, I'll prefer the cURL option instead.
> I will make an average time of 5 queries, using json header in the POST
> query.
>
> Here is a sample cURL query I am sending:
> curl -i -X POST http://localhost:9999/blazegraph/namespace/<mykb>/sparql
> --data-urlencode 'qu...@my...' --data-urlencode analytic=true
> -H 'Accept: application/json' > myQuery.json
>
> Would that makes sense?
>
> Best,
>
> El lun., 23 ene. 2017 a las 10:24, Igor Kim (<igo...@ms...>)
> escribió:
>
>> Ghislain,
>>
>> Basically CUrl and Workbench do almost the same thing - issue POST query
>> to the server, which should analyze and run it then stream the result back
>> to client.
>> But upon receiving, curl just stores a stream into a file or logs to
>> console, but Workbench has to parse JSON results and show it in a web page,
>> which takes additional time.
>> Note, that JSON is native format for JavaScript code of the Workbench
>> (much easier to parse than XML for example), so using other formats would
>> not improve execution time.
>>
>> The difference will be more obvious for large responses (many rows and
>> large literals).
>>
>> You could use developers console in your browser to see execution times
>> (including transport times, but CUrl will count that time as well).
>>
>> See attached screenshot, the query has taken 52ms while waiting for a
>> response start (prepare a query and start its evaluation),
>> additional 78ms to get whole result (iterate through the results on
>> server and stream it through network using JSON format),
>> but the Workbench have taken additional 322ms to parse and prepare
>> results for the results table on the web page.
>>
>> Also, after execution a query on the Workbench, server's file system
>> caches index data for a time, so repeating requests to the same query might
>> be much faster for subsequent calls using CUrl.
>>
>> As a workaround you might apply 'limit 50' to evaluate less results (only
>> first page), that will reduce overhead time for the Workbench.
>>
>>
>> [image: Inline image 1]
>>
>> On Mon, Jan 23, 2017 at 1:15 PM, Ghislain ATEMEZING <
>> ghi...@gm...> wrote:
>>
>> Hi Igor,
>> Thanks for your answer and sorry if O was not clear enough in my previous
>> post. Basically I thought I can get also the time execution of a query with
>> the ?explain parameter.
>> When comparing the response time of a CUrl in localhost w.r.t the answer
>> on the workbench, I notice it's sometimes 10x higher. Is there a way to get
>> almost the same result somehow?
>>
>> Best,
>> Ghislain
>>
>> Sent from my iPhone, may include typos
>>
>> Le 23 janv. 2017 à 04:51, Igor Kim <igo...@ms...> a écrit :
>>
>> Ghislain,
>>
>> ?explain parameter is a rough equivalent of SQL analyze, so there is no
>> time of response returned, just a query execution plan explanation.
>> Could you expand a little bit on what are you going to use that response
>> time? There is a workaround with using UNION or BIND in the query, which
>> could provide a timestamp for the results:
>>
>> for the whole result:
>>
>> select * from {
>>     {
>>         ?s ....... # statement pattern and filters
>>     }
>>     UNION
>>     {
>>         BIND (now() as ?timestamp) .
>>     }
>> }
>>
>> or for each of the rows:
>>
>> select * from {
>>     ?s ....... # statement pattern and filters
>>     BIND (now() as ?timestamp) .
>> }
>>
>>
>>
>>
>> On Fri, Jan 20, 2017 at 10:07 PM, Ghislain ATEMEZING <
>> ghi...@gm...> wrote:
>>
>> Many thanks Brad!
>> I was looking at the Wiki to see if I can use a parameter in a cUrl
>> request to retrieve the time response of a query. I've seen the ?explain
>> parameter, but not sure to find the that value in the .html output.
>> Is there a way to retrieve that value in a REST-ish  manner?
>>
>> TIA,
>> Ghislain
>>
>> El vie., 20 ene. 2017 a las 17:14, Brad Bebee (<be...@sy...>)
>> escribió:
>>
>> Ghislain,
>>
>> Yes, you may use the list for those topics. In general, before posting,
>> we recommend checking out the Wiki.
>>
>> https://wiki.blazegraph.com/wiki/index.php/Main_Page
>>
>> Thanks, --Brad
>>
>> On Fri, Jan 20, 2017 at 1:38 AM, Ghislain ATEMEZING <
>> ghi...@gm...> wrote:
>>
>> Hi,
>> [Sorry if I am in the wrong list.]
>> I am new to blazegraph and I wanted to know if this list also helps for
>> issues related to the use of the product, setting up issues and so on.
>>
>> TIA
>> Best,
>> Ghislain
>> --
>> -------
>> "Love all, trust a few, do wrong to none" (W. Shakespeare)
>> Web: http://atemezing.org
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>>
>>
>> --
>> _______________
>> Brad Bebee
>> CEO, Managing Partner
>> SYSTAP, LLC
>> e:  be...@sy...
>> m: 202.642.7961
>> f:  571.367.5000
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
>> dissemination or copying of this email or its contents or attachments is
>> prohibited. If you have received this communication in error, please notify
>> the sender by reply email and permanently delete all copies of the email
>> and its contents and attachments.
>>
>> --
>> -------
>> "Love all, trust a few, do wrong to none" (W. Shakespeare)
>> Web: http://atemezing.org
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>>
>>
>>
>> --
>> *Igor Kim*  | Team Leader / Backend  | Maginfo, Ltd
>> Mobile: +7-912-402-4622
>>
>>
>>
>>
>> --
>> *Igor Kim*  | Team Leader / Backend  | Maginfo, Ltd
>> Mobile: +7-912-402-4622
>>
> --
> -------
> "Love all, trust a few, do wrong to none" (W. Shakespeare)
> Web: http://atemezing.org
>



-- 
*Igor Kim*  | Team Leader / Backend  | Maginfo, Ltd
Mobile: +7-912-402-4622