From: Hugh W. <hwi...@op...> - 2013-10-16 11:41:29
|
Hi Nicola, Any fixes for such issues would be made in the develop/7 branch, which at some point will become the stable/7 (probably 7.0.1) branch and they will then fork again, merge again at some other point in time etc. Thus we need to know if these issues exist in the develop/7 branch in order to provide a fix really and there are a number of index and query optimisation fixes scheduled to be added to the develop/7 branch in the coming days thus it would be prudent to test with those, so I shall notify you when they are in or you can monitor the RSS feed at: https://github.com/openlink/virtuoso-opensource/commits/develop%2F7.atom Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers On 16 Oct 2013, at 09:32, Nicola Vitucci <nic...@gm...> wrote: > Hi Hugh, > > yes, I know, but we were planning not to use the dev branch any longer > and switch to the stable version (for stability reasons). So is there > any solution to this incompatibility of data/indices across versions? > Could you maybe try with exactly the same version we used? > > Thank you, > > Nicola > > Il 15/10/2013 18:43, Hugh Williams ha scritto: >> Hi Nicola, >> >> That looks like an actual static tar archive of the sources from Aug, whereas if you clone and checkout the develop/7 branch you will get the latest fixes, which is what I have been testing with ... >> >> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VOSGitUsage >> >> Best Regards >> Hugh Williams >> Professional Services >> OpenLink Software, Inc. // http://www.openlinksw.com/ >> Weblog -- http://www.openlinksw.com/blogs/ >> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >> Twitter -- http://twitter.com/OpenLink >> Google+ -- http://plus.google.com/100570109519069333827/ >> Facebook -- http://www.facebook.com/OpenLinkSoftware >> Universal Data Access, Integration, and Management Technology Providers >> >> On 15 Oct 2013, at 09:19, Nicola Vitucci <nic...@gm...> wrote: >> >>> Hi Hugh, >>> >>> the output of "virtuoso-t -?" is the following: >>> >>> Virtuoso Open Source Edition (Column Store) (multi threaded) >>> Version 7.0.0-rc2.3203-pthreads as of Aug 6 2013 >>> Compiled for Linux (x86_64-unknown-linux-gnu) >>> >>> As for the "git status", I don't have the clone handy but we downloaded >>> the build from here: >>> >>> https://github.com/openlink/virtuoso-opensource/archive/48f0ef879b913c5d3b306c1f83390079c5416fe6.tar.gz >>> >>> Hope this helps. >>> >>> Nicola >>> >>> Il 15/10/2013 04:47, Hugh Williams ha scritto: >>>> Hi Nicola, >>>> >>>> What does the following report as the version of your build: >>>> >>>> ./virtuoso-t -? >>>> >>>> and for your build tree what does the "git status" command return when >>>> run against your build tree, so we can confirm which build is being used ... >>>> >>>> Best Regards >>>> Hugh Williams >>>> Professional Services >>>> OpenLink Software, Inc. // http://www.openlinksw.com/ >>>> Weblog -- http://www.openlinksw.com/blogs/ >>>> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >>>> Twitter -- http://twitter.com/OpenLink >>>> Google+ -- http://plus.google.com/100570109519069333827/ >>>> Facebook -- http://www.facebook.com/OpenLinkSoftware >>>> Universal Data Access, Integration, and Management Technology Providers >>>> >>>> On 14 Oct 2013, at 13:03, Nicola Vitucci <nic...@gm... >>>> <mailto:nic...@gm...>> wrote: >>>> >>>>> Hi Hugh, >>>>> >>>>> the rc2 version we used is 48f0ef879b913c5d3b306c1f83390079c5416fe6 , >>>>> downloaded from <https://github.com/openlink/virtuoso-opensource>. >>>>> >>>>> I also noticed that, if I load the same data on 7.0.0 from scratch, the >>>>> query runs "fast". Is there maybe a problem in how indices are >>>>> interpreted in the two versions, so that a database build using rc2 >>>>> cannot be used correctly on 7.0.0? >>>>> >>>>> Thanks, >>>>> >>>>> Nicola >>>>> >>>>> Il 13/10/2013 03:28, Hugh Williams ha scritto: >>>>>> Hi Nicola, >>>>>> >>>>>> If the profile function is not working for this query, which i have also >>>>>> been able to recreate locally and reported to be fixed, then the explain >>>>>> output is the next best form of profiling. >>>>>> >>>>>> I tried running explain against the latest develop/7 got branch and it >>>>>> gave the following: >>>>>> >>>>>> SQL> explain('SPARQL PREFIX int: <namespace#> CONSTRUCT {?s int:new_prop >>>>>> ?n} WHERE {?s <prop> ?c . ?c rdfs:label ?n}'); >>>>>> REPORT >>>>>> VARCHAR >>>>>> _______________________________________________________________________________ >>>>>> >>>>>> { >>>>>> >>>>>> Precode: >>>>>> 0: { >>>>>> >>>>>> Precode: >>>>>> 0: vector := Call vector ( 1 , 0 , 3 , #new_prop , 1 , 1 ) >>>>>> 5: vector := Call vector (vector) >>>>>> 10: vector := Call vector () >>>>>> 15: BReturn 0 >>>>>> { fork >>>>>> RDF_QUAD_POGS 40 rows(s_1_7_t2.S, s_1_7_t2.O) >>>>>> inlined P = #label >>>>>> RDF_QUAD 0.0018 rows(s_1_7_t1.S) >>>>>> inlined P = #prop O = k_s_1_7_t2.S >>>>>> >>>>>> After code: >>>>>> 0: __ro2lo := Call __ro2lo (s_1_7_t2.O) >>>>>> 5: vector := Call vector (s_1_7_t1.S, __ro2lo) >>>>>> 10: if ($47 "user_aggr_notfirst" = 1 ) then 25 else 14 unkn 14 >>>>>> 14: $47 "user_aggr_notfirst" := := artm 1 >>>>>> 18: user_aggr_ret := Call DB.DBA.SPARQL_CONSTRUCT_INIT ($48 >>>>>> "user_aggr_env") >>>>>> 25: user_aggr_ret := Call DB.DBA.SPARQL_CONSTRUCT_ACC ($48 >>>>>> "user_aggr_env", vector, vector, vector, 1 ) >>>>>> 32: BReturn 0 >>>>>> } >>>>>> skip node 1 <none> set_ctr >>>>>> >>>>>> After code: >>>>>> 0: callret-0 := Call DB.DBA.SPARQL_CONSTRUCT_FIN ($48 >>>>>> "user_aggr_env") >>>>>> 7: BReturn 0 >>>>>> Subquery Select(callret-0) >>>>>> } >>>>>> >>>>>> 8: <none> := Call DB.DBA.RDF_DICT_OF_TRIPLES_TO_THREE_COLS ($100 >>>>>> "scalar") >>>>>> 15: BReturn 0 >>>>>> END Node >>>>>> } >>>>>> >>>>>> 37 Rows. -- 1 msec. >>>>>> SQL> >>>>>> >>>>>> Note the use of the "RDF_QUAD_POGS" index, which was used in your 7.0.0 >>>>>> build but not in the "rc2" build explain output you perform. >>>>>> >>>>>> Which brings me to another question, what is the "rc2" build you refer >>>>>> to, as building from the latest git develop/7 branch my version is: >>>>>> >>>>>> ./virtuoso-t -? >>>>>> Virtuoso Open Source Edition (Column Store) (multi threaded) >>>>>> Version 7.0.1-dev.3203-pthreads as of Oct 13 2013 >>>>>> Compiled for Darwin (x86_64-apple-darwin12.3.0) >>>>>> Copyright (C) 1998-2013 OpenLink Software >>>>>> >>>>>> What is your version ? >>>>>> >>>>>> Best Regards >>>>>> Hugh Williams >>>>>> Professional Services >>>>>> OpenLink Software, Inc. // http://www.openlinksw.com/ >>>>>> Weblog -- http://www.openlinksw.com/blogs/ >>>>>> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >>>>>> Twitter -- http://twitter.com/OpenLink >>>>>> Google+ -- http://plus.google.com/100570109519069333827/ >>>>>> Facebook -- http://www.facebook.com/OpenLinkSoftware >>>>>> Universal Data Access, Integration, and Management Technology Providers >>>>>> >>>>>> On 11 Oct 2013, at 13:04, Nicola Vitucci <nic...@gm... >>>>>> <mailto:nic...@gm...> >>>>>> <mailto:nic...@gm...>> wrote: >>>>>> >>>>>>> Hi Hugh, >>>>>>> >>>>>>> thanks for the reply. Unfortunately I could't PROFILE any of the queries >>>>>>> because both ended up with an "Error 22012: [Virtuoso Driver][Virtuoso >>>>>>> Server]SR083: Division by zero". Any alternatives to do some profiling? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Nicola >>>>>>> >>>>>>> Il 11/10/2013 13:01, Hugh Williams ha scritto: >>>>>>>> Hi Nicola, >>>>>>>> >>>>>>>> Can you please provide the output of running the profile() function >>>>>>>> for >>>>>>>> these two queries ie replace explain with profile, which provides more >>>>>>>> information on the query execution: >>>>>>>> >>>>>>>> http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#readingqueryprofile >>>>>>>> >>>>>>>> There has not really been changes indexes but more in the query >>>>>>>> optimiser were the plans may have changed which the explain and profile >>>>>>>> outputs should help identify for your query ... >>>>>>>> >>>>>>>> Best Regards >>>>>>>> Hugh Williams >>>>>>>> Professional Services >>>>>>>> OpenLink Software, Inc. // http://www.openlinksw.com/ >>>>>>>> Weblog -- http://www.openlinksw.com/blogs/ >>>>>>>> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >>>>>>>> Twitter -- http://twitter.com/OpenLink >>>>>>>> Google+ -- http://plus.google.com/100570109519069333827/ >>>>>>>> Facebook -- http://www.facebook.com/OpenLinkSoftware >>>>>>>> Universal Data Access, Integration, and Management Technology Providers >>>>>>>> >>>>>>>> On 11 Oct 2013, at 09:52, Nicola Vitucci <nic...@gm... >>>>>>>> <mailto:nic...@gm...>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I got a problem with Virtuoso 7.0.0 and I think it's related to >>>>>>>>> the use >>>>>>>>> of indices. My question is: have there been any big changes >>>>>>>>> between the >>>>>>>>> rc2 and the stable version, so that indices are not completely >>>>>>>>> compatible across the two versions? >>>>>>>>> >>>>>>>>> Here is one of the queries I ran on both versions (just the names have >>>>>>>>> been changed): >>>>>>>>> >>>>>>>>> PREFIX int: <namespace#> CONSTRUCT {?s int:new_prop ?n} WHERE {?s >>>>>>>>> <prop> >>>>>>>>> ?c . ?c rdfs:label ?n} >>>>>>>>> >>>>>>>>> and below you can find the result of EXPLAINing the query on the rc2 >>>>>>>>> version first and then on 7.0.0. The query is reasonably fast on rc2 >>>>>>>>> (41659 msec with ResultSetMaxRows = 10000), while it takes an awfully >>>>>>>>> long time (1011741 msec, same ResultSetMaxRows) on 7.0.0. Other >>>>>>>>> queries >>>>>>>>> have a similar problem. >>>>>>>>> >>>>>>>>> Any ideas? >>>>>>>>> >>>>>>>>> Thank you, >>>>>>>>> >>>>>>>>> Nicola >>>>>>>>> >>>>>>>>> ------------------------ >>>>>>>>> Virtuoso rc2 >>>>>>>>> >>>>>>>>> { >>>>>>>>> >>>>>>>>> Precode: >>>>>>>>> 0: { >>>>>>>>> s# 63 cluster outer seq start, set no <V $44 set_ctr in> >>>>>>>>> save ctx:() >>>>>>>>> >>>>>>>>> Precode: >>>>>>>>> 0: <V $27 vector x> := Call vector ( 1 , 0 , 3 , #new_prop , 1 >>>>>>>>> , 1 ) >>>>>>>>> 5: <V $29 vector x> := Call vector (<V $27 vector x>) >>>>>>>>> 10: <V $31 vector x> := Call vector () >>>>>>>>> 15: BReturn 0 >>>>>>>>> s# 89 Fork 46 >>>>>>>>> { >>>>>>>>> s# 70 from DB.DBA.RDF_QUAD by RDF_QUAD 2.5e+05 rows >>>>>>>>> Key RDF_QUAD ASC (<V $35 s_1_7_t1.O an>, <V $34 s_1_7_t1.S rn>) >>>>>>>>> inlined P = #prop >>>>>>>>> row specs: O LIKE <c T�> >>>>>>>>> >>>>>>>>> s# 77 from DB.DBA.RDF_QUAD by RDF_QUAD 1 rows >>>>>>>>> Key RDF_QUAD ASC (<V $38 s_1_7_t2.O an>) >>>>>>>>> inlined P = #label , S = <v $81 k_s_1_7_t1.O S83 rn> >>>>>>>>> vector param casts: <V $35 s_1_7_t1.O an>-> <v $81 k_s_1_7_t1.O >>>>>>>>> S83 rn> >>>>>>>>> >>>>>>>>> >>>>>>>>> After code: >>>>>>>>> 0: <V $40 __ro2lo x> := Call __ro2lo (<V $38 s_1_7_t2.O an>) >>>>>>>>> 5: <V $42 vector x> := Call vector (<r $34 s_1_7_t1.S via S77 >>>>>>>>> S83>, <V $40 __ro2lo x>) >>>>>>>>> 10: if ($47 "user_aggr_notfirst" = 1 ) then 25 else 14 unkn 14 >>>>>>>>> 14: $47 "user_aggr_notfirst" := := artm 1 >>>>>>>>> 18: <V $49 user_aggr_ret x> := Call DB.DBA.SPARQL_CONSTRUCT_INIT >>>>>>>>> ($48 "user_aggr_env") >>>>>>>>> 25: <V $49 user_aggr_ret x> := Call DB.DBA.SPARQL_CONSTRUCT_ACC >>>>>>>>> ($48 "user_aggr_env", <r $29 vector via S77 S83 S70>, <V $42 >>>>>>>>> vector x>, >>>>>>>>> <r $31 vector via S77 S83 S70>, 1 ) >>>>>>>>> 32: BReturn 0 >>>>>>>>> } >>>>>>>>> s# 92 skip node 1 <none> <V $44 set_ctr in> >>>>>>>>> >>>>>>>>> After code: >>>>>>>>> 0: <V $50 callret-0 x> := Call DB.DBA.SPARQL_CONSTRUCT_FIN ($48 >>>>>>>>> "user_aggr_env") >>>>>>>>> 7: BReturn 0 >>>>>>>>> s# 96 Subquery Select(<V $50 callret-0 x>) >>>>>>>>> set no: <r $44 set_ctr via S92> >>>>>>>>> } >>>>>>>>> >>>>>>>>> 8: <none> := Call DB.DBA.RDF_DICT_OF_TRIPLES_TO_THREE_COLS ($100 >>>>>>>>> "scalar") >>>>>>>>> 15: BReturn 0 >>>>>>>>> END Node >>>>>>>>> } >>>>>>>>> >>>>>>>>> ------------------------ >>>>>>>>> Virtuoso 7.0.0 >>>>>>>>> >>>>>>>>> { >>>>>>>>> >>>>>>>>> Precode: >>>>>>>>> 0: { >>>>>>>>> >>>>>>>>> Precode: >>>>>>>>> 0: vector := Call vector ( 1 , 0 , 3 , #new_prop , 1 , 1 ) >>>>>>>>> 5: vector := Call vector (vector) >>>>>>>>> 10: vector := Call vector () >>>>>>>>> 15: BReturn 0 >>>>>>>>> { fork >>>>>>>>> RDF_QUAD 4.2e+08 rows(s_1_7_t2.S, s_1_7_t2.O) >>>>>>>>> inlined P = #label >>>>>>>>> >>>>>>>>> Precode: >>>>>>>>> 0: __ro2lo := Call __ro2lo (s_1_7_t2.O) >>>>>>>>> 5: BReturn 0 >>>>>>>>> RDF_QUAD_POGS 33 rows(s_1_7_t1.S) >>>>>>>>> P = #prop , O = cast >>>>>>>>> >>>>>>>>> After code: >>>>>>>>> 0: vector := Call vector (s_1_7_t1.S, __ro2lo) >>>>>>>>> 5: if ($47 "user_aggr_notfirst" = 1 ) then 20 else 9 unkn 9 >>>>>>>>> 9: $47 "user_aggr_notfirst" := := artm 1 >>>>>>>>> 13: user_aggr_ret := Call DB.DBA.SPARQL_CONSTRUCT_INIT ($48 >>>>>>>>> "user_aggr_env") >>>>>>>>> 20: user_aggr_ret := Call DB.DBA.SPARQL_CONSTRUCT_ACC ($48 >>>>>>>>> "user_aggr_env", vector, vector, vector, 1 ) >>>>>>>>> 27: BReturn 0 >>>>>>>>> } >>>>>>>>> skip node 1 <none> set_ctr >>>>>>>>> >>>>>>>>> After code: >>>>>>>>> 0: callret-0 := Call DB.DBA.SPARQL_CONSTRUCT_FIN ($48 >>>>>>>>> "user_aggr_env") >>>>>>>>> 7: BReturn 0 >>>>>>>>> Subquery Select(callret-0) >>>>>>>>> } >>>>>>>>> >>>>>>>>> 8: <none> := Call DB.DBA.RDF_DICT_OF_TRIPLES_TO_THREE_COLS ($100 >>>>>>>>> "scalar") >>>>>>>>> 15: BReturn 0 >>>>>>>>> END Node >>>>>>>>> } >>>>>>>>> >>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>> October Webinars: Code for Performance >>>>>>>>> Free Intel webinars can help you accelerate application performance. >>>>>>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>>>>>>>> most from >>>>>>>>> the latest Intel processors and coprocessors. See abstracts and >>>>>>>>> register > >>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk >>>>>>>>> _______________________________________________ >>>>>>>>> Virtuoso-users mailing list >>>>>>>>> Vir...@li... >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >>>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------------------------ >>>>>>> October Webinars: Code for Performance >>>>>>> Free Intel webinars can help you accelerate application performance. >>>>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>>>>>> most from >>>>>>> the latest Intel processors and coprocessors. See abstracts and >>>>>>> register > >>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk >>>>>>> _______________________________________________ >>>>>>> Virtuoso-users mailing list >>>>>>> Vir...@li... >>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >>>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> October Webinars: Code for Performance >>>>> Free Intel webinars can help you accelerate application performance. >>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>>>> most from >>>>> the latest Intel processors and coprocessors. See abstracts and register > >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk >>>>> _______________________________________________ >>>>> Virtuoso-users mailing list >>>>> Vir...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >>>> >>> >>> ------------------------------------------------------------------------------ >>> October Webinars: Code for Performance >>> Free Intel webinars can help you accelerate application performance. >>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from >>> the latest Intel processors and coprocessors. See abstracts and register > >>> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Virtuoso-users mailing list >>> Vir...@li... >>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >> >> > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk > _______________________________________________ > Virtuoso-users mailing list > Vir...@li... > https://lists.sourceforge.net/lists/listinfo/virtuoso-users |