From: Dan B. <dan...@gm...> - 2011-03-11 01:17:07
|
Result: Recursion here isn't terrible for performance. Details: Yaron kindly set "SMWResultPrinter::$maxRecursionDepth = 40" on http://scratchpad.referata.com and I did some testing. For some reason, the level of recursion now stops at level 20 (??). I'm doing everything as before: * http://scratchpad.referata.com/wiki/Piston * http://scratchpad.referata.com/wiki/Template:Part_of_what Pages higher in the tree don't hit this limit, and the template 'terminates' correctly, e.g. * http://scratchpad.referata.com/wiki/Foo I benchmarked the performance hit due to recursion by requesting each page in the tree 20 times using curl, and parsing out the '<!-- Served in 0.451 secs. -->' string in the returned URL. Every page request was done with the 'action=purge' CGI setting. The results are quite interesting, suggesting that each recursion adds 0.02 seconds to the 'serve time' on scratchpad.referata.com: * http://imagepaste.nullnetwork.net/img/1299805257test_benchmark.png * http://oi56.tinypic.com/2gx41oy.jpg # duplicate upload About the above plot, upper and lower triangles are +/- s.d. of the mean load time (squares) in 20 trials. After depth 20 the behaviour changes because the template doesn't deal well with this limit condition and a large (null) results set is generated each time (see the bottom of the 'Piston' page linked above). The 'solid' regression includes just the points up to and including depth 20, the 'dashed' regression includes the points above 20. Cheers, Dan. On 10 March 2011 13:54, Dan Bolser <dan...@gm...> wrote: > On 10 March 2011 12:27, Alex M. Hendler <amh...@on...> wrote: >> Hi Dan, >> >> I am not sure what "your own risk!" means, but whenever I see an >> exclamation point in the body of a comment that is not telling you how >> super-cool a feature is, I get a bit scared. > > I don't see that online here: > http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html > > Perhaps it's a version issue, but in any case, I take your point. > > >> That said, I can foresee a problem if you set recursion depth too high >> and someone made an error such that a page that gets returned by the >> query is "part of" the querying page, then you just end up with loopy >> (literally) query results. > > Yeah, of course a page shouldn't be 'part of' itself, but there is > nothing to stop a user doing that. > > >> As for performance, in my experience (still yet to be confirmed by >> others) maxRecursionDepth > 2 may cause odd property inheritance >> behavior if you embed or transclude pages. See >> https://bugzilla.wikimedia.org/show_bug.cgi?id=27445 > > Weird bug! Seems to be a 'feature' of the embed format only with nested queries? > > >> I hope this helps point you in the right direction in terms of doing >> what you need to do (it looks like maybe a template with known levels of >> properties, e.g., [[-Part of::x]] passed through sufficient levels of >> arrays/arraymaps and #if: statements would be a good bet); hopefully > > Yeah, I don't have a fixed number of levels. > > >> someone else will be able to better answer your performance question. >> >> As for SemanticTreeView, it looks like it must be installed with Halo, >> but the Ontoprise folks may have a different view of that. > > > Did you get this from the docs, or have you tried installing? > > Thanks for your help, > Dan. > >> -Alex >> >> >> - >> ----Original Message----- >> From: Dan Bolser <dan...@gm...> >> To: Alex M. Hendler <amh...@on...> >> Cc: Semantic MediaWiki Users <sem...@li...> >> Subject: Re: [Semediawiki-user] Recursive ask using format=template, >> stuck at depth of 3? >> Date: Thu, 10 Mar 2011 10:00:32 +0000 >> >> Thanks Alex. >> >> I read the document you suggested: >> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >> >> but I expected to see why the settings are what they are (i.e. why so >> low), which I don't see. >> >> Do you think those 'odd' issues that you mention would to this simple >> recursive query? Perhaps I'm being naive, but it seems to work so well >> up to the given depth... would performance degrade super-linearly with >> increased depth? >> >> >> Alternatively, the Semantic TreeView extension looks promising: >> >> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_tree_parser_function >> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_generateTree_parser_function >> >> >> but it isn't clear from the documentation if you need to install halo >> or not (some places state halo as a requirement, while other >> information suggests that it isn't) ... I guess I'll just suck it and >> see... The biggest problem is that it expects a tree while I have a >> DAG... Not sure what that'll do to it. >> >> Any other suggested extensions for navigating relations? (Other than >> Special:Browse of course!) >> >> >> Cheers, >> Dan. >> >> On 10 March 2011 07:39, Alex M. Hendler <amh...@on...> wrote: >>> Dan, >>> >>> I believe the setting you're looking for is >>> >>> SMWResultPrinter::$maxRecursionDepth = x; >>> >>> Place that after your enableSemantics statement in LocalSettings.php, >>> where x = [desired depth]-1. >>> >>> The reason you can't get beyond 3 levels on Referata is that (I believe) >>> it uses the default max. depth of 2. includes/SMW_QueryPrinter.php >>> explains the recursion settings very well. >>> >>> The developers can correct me if I am wrong, but I believe that >>> recursion is intentionally not more publicly documented (i.e., outside >>> of source code) because it can do odd and perhaps unpredictable things >>> to semantics as well as performance if you increase it above 2. >>> >>> You can sometimes fool the system into recursing beyond >>> maxRecursionDepth, but that is not something I would encourage anyone to >>> do intentionally (it also prints out very messy results). >>> >>> Hope this helps, >>> Alex >>> >>> ----Original Message----- >>> From: Dan Bolser <dan...@gm...> >>> To: Semantic MediaWiki Users <sem...@li...> >>> Subject: [Semediawiki-user] Recursive ask using format=template, stuck >>> at depth of 3? >>> Date: Wed, 9 Mar 2011 23:58:14 +0000 >>> >>> Hi, >>> >>> I wrote a nice 'recursive' template to traverse a DAG here: >>> http://scratchpad.referata.com/wiki/Template:Part_of_what >>> >>> It works by calling itself with an ask using it's given input and then >>> the result of the ask and so on up the DAG ... (recursion always makes >>> me a little dizzy). >>> >>> It works great except it only calls itself a maximum of 3 times, as >>> evidenced by comparing the results of these two pages: >>> * http://scratchpad.referata.com/wiki/Piston >>> * http://scratchpad.referata.com/wiki/Engine >>> >>> I couldn't find the appropriate setting in the manual to bump this >>> 'maximum recursion' value. The equivalent value for MediaWiki can be >>> found here, but note the default setting is a whopping 40: >>> >>> http://www.mediawiki.org/wiki/Manual:$wgMaxTemplateDepth >>> >>> >>> Any tips appreciated. >>> >>> Cheers, >>> Dan. >>> >>> ------------------------------------------------------------------------------ >>> Colocation vs. Managed Hosting >>> A question and answer guide to determining the best fit >>> for your organization - today and in the future. >>> http://p.sf.net/sfu/internap-sfd2d >>> _______________________________________________ >>> Semediawiki-user mailing list >>> Sem...@li... >>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>> >>> >> >> > |
From: Dan B. <dan...@gm...> - 2011-03-15 13:35:01
|
Anyone know / guess why a) the limit is 20, b) the query goes awry at the limit? Cheers, Dan. On 11 March 2011 01:16, Dan Bolser <dan...@gm...> wrote: > Result: Recursion here isn't terrible for performance. > > Details: > > Yaron kindly set "SMWResultPrinter::$maxRecursionDepth = 40" on > http://scratchpad.referata.com and I did some testing. > > For some reason, the level of recursion now stops at level 20 (??). > I'm doing everything as before: > * http://scratchpad.referata.com/wiki/Piston > * http://scratchpad.referata.com/wiki/Template:Part_of_what > > Pages higher in the tree don't hit this limit, and the template > 'terminates' correctly, e.g. > * http://scratchpad.referata.com/wiki/Foo > > I benchmarked the performance hit due to recursion by requesting each > page in the tree 20 times using curl, and parsing out the '<!-- Served > in 0.451 secs. -->' string in the returned URL. Every page request was > done with the 'action=purge' CGI setting. > > The results are quite interesting, suggesting that each recursion adds > 0.02 seconds to the 'serve time' on scratchpad.referata.com: > * http://imagepaste.nullnetwork.net/img/1299805257test_benchmark.png > * http://oi56.tinypic.com/2gx41oy.jpg # duplicate upload > > > About the above plot, upper and lower triangles are +/- s.d. of the > mean load time (squares) in 20 trials. After depth 20 the behaviour > changes because the template doesn't deal well with this limit > condition and a large (null) results set is generated each time (see > the bottom of the 'Piston' page linked above). The 'solid' regression > includes just the points up to and including depth 20, the 'dashed' > regression includes the points above 20. > > > Cheers, > Dan. > > On 10 March 2011 13:54, Dan Bolser <dan...@gm...> wrote: >> On 10 March 2011 12:27, Alex M. Hendler <amh...@on...> wrote: >>> Hi Dan, >>> >>> I am not sure what "your own risk!" means, but whenever I see an >>> exclamation point in the body of a comment that is not telling you how >>> super-cool a feature is, I get a bit scared. >> >> I don't see that online here: >> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >> >> Perhaps it's a version issue, but in any case, I take your point. >> >> >>> That said, I can foresee a problem if you set recursion depth too high >>> and someone made an error such that a page that gets returned by the >>> query is "part of" the querying page, then you just end up with loopy >>> (literally) query results. >> >> Yeah, of course a page shouldn't be 'part of' itself, but there is >> nothing to stop a user doing that. >> >> >>> As for performance, in my experience (still yet to be confirmed by >>> others) maxRecursionDepth > 2 may cause odd property inheritance >>> behavior if you embed or transclude pages. See >>> https://bugzilla.wikimedia.org/show_bug.cgi?id=27445 >> >> Weird bug! Seems to be a 'feature' of the embed format only with nested queries? >> >> >>> I hope this helps point you in the right direction in terms of doing >>> what you need to do (it looks like maybe a template with known levels of >>> properties, e.g., [[-Part of::x]] passed through sufficient levels of >>> arrays/arraymaps and #if: statements would be a good bet); hopefully >> >> Yeah, I don't have a fixed number of levels. >> >> >>> someone else will be able to better answer your performance question. >>> >>> As for SemanticTreeView, it looks like it must be installed with Halo, >>> but the Ontoprise folks may have a different view of that. >> >> >> Did you get this from the docs, or have you tried installing? >> >> Thanks for your help, >> Dan. >> >>> -Alex >>> >>> >>> - >>> ----Original Message----- >>> From: Dan Bolser <dan...@gm...> >>> To: Alex M. Hendler <amh...@on...> >>> Cc: Semantic MediaWiki Users <sem...@li...> >>> Subject: Re: [Semediawiki-user] Recursive ask using format=template, >>> stuck at depth of 3? >>> Date: Thu, 10 Mar 2011 10:00:32 +0000 >>> >>> Thanks Alex. >>> >>> I read the document you suggested: >>> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >>> >>> but I expected to see why the settings are what they are (i.e. why so >>> low), which I don't see. >>> >>> Do you think those 'odd' issues that you mention would to this simple >>> recursive query? Perhaps I'm being naive, but it seems to work so well >>> up to the given depth... would performance degrade super-linearly with >>> increased depth? >>> >>> >>> Alternatively, the Semantic TreeView extension looks promising: >>> >>> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_tree_parser_function >>> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_generateTree_parser_function >>> >>> >>> but it isn't clear from the documentation if you need to install halo >>> or not (some places state halo as a requirement, while other >>> information suggests that it isn't) ... I guess I'll just suck it and >>> see... The biggest problem is that it expects a tree while I have a >>> DAG... Not sure what that'll do to it. >>> >>> Any other suggested extensions for navigating relations? (Other than >>> Special:Browse of course!) >>> >>> >>> Cheers, >>> Dan. >>> >>> On 10 March 2011 07:39, Alex M. Hendler <amh...@on...> wrote: >>>> Dan, >>>> >>>> I believe the setting you're looking for is >>>> >>>> SMWResultPrinter::$maxRecursionDepth = x; >>>> >>>> Place that after your enableSemantics statement in LocalSettings.php, >>>> where x = [desired depth]-1. >>>> >>>> The reason you can't get beyond 3 levels on Referata is that (I believe) >>>> it uses the default max. depth of 2. includes/SMW_QueryPrinter.php >>>> explains the recursion settings very well. >>>> >>>> The developers can correct me if I am wrong, but I believe that >>>> recursion is intentionally not more publicly documented (i.e., outside >>>> of source code) because it can do odd and perhaps unpredictable things >>>> to semantics as well as performance if you increase it above 2. >>>> >>>> You can sometimes fool the system into recursing beyond >>>> maxRecursionDepth, but that is not something I would encourage anyone to >>>> do intentionally (it also prints out very messy results). >>>> >>>> Hope this helps, >>>> Alex >>>> >>>> ----Original Message----- >>>> From: Dan Bolser <dan...@gm...> >>>> To: Semantic MediaWiki Users <sem...@li...> >>>> Subject: [Semediawiki-user] Recursive ask using format=template, stuck >>>> at depth of 3? >>>> Date: Wed, 9 Mar 2011 23:58:14 +0000 >>>> >>>> Hi, >>>> >>>> I wrote a nice 'recursive' template to traverse a DAG here: >>>> http://scratchpad.referata.com/wiki/Template:Part_of_what >>>> >>>> It works by calling itself with an ask using it's given input and then >>>> the result of the ask and so on up the DAG ... (recursion always makes >>>> me a little dizzy). >>>> >>>> It works great except it only calls itself a maximum of 3 times, as >>>> evidenced by comparing the results of these two pages: >>>> * http://scratchpad.referata.com/wiki/Piston >>>> * http://scratchpad.referata.com/wiki/Engine >>>> >>>> I couldn't find the appropriate setting in the manual to bump this >>>> 'maximum recursion' value. The equivalent value for MediaWiki can be >>>> found here, but note the default setting is a whopping 40: >>>> >>>> http://www.mediawiki.org/wiki/Manual:$wgMaxTemplateDepth >>>> >>>> >>>> Any tips appreciated. >>>> >>>> Cheers, >>>> Dan. >>>> >>>> ------------------------------------------------------------------------------ >>>> Colocation vs. Managed Hosting >>>> A question and answer guide to determining the best fit >>>> for your organization - today and in the future. >>>> http://p.sf.net/sfu/internap-sfd2d >>>> _______________________________________________ >>>> Semediawiki-user mailing list >>>> Sem...@li... >>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>>> >>>> >>> >>> >> > > ------------------------------------------------------------------------------ > Colocation vs. Managed Hosting > A question and answer guide to determining the best fit > for your organization - today and in the future. > http://p.sf.net/sfu/internap-sfd2d > _______________________________________________ > Semediawiki-user mailing list > Sem...@li... > https://lists.sourceforge.net/lists/listinfo/semediawiki-user > |
From: Markus K. <ma...@se...> - 2011-03-15 14:21:02
|
On 15/03/11 13:34, Dan Bolser wrote: > Anyone know / guess why a) the limit is 20, b) the query goes awry at the limit? MediaWiki has a global restriction on the number of parser functions that may be used on one page. Maybe this or a similar MW "emergency break" could be a reason for this limitation. I am not sure if the current embedding limit in SMW is still needed, since there are now a number of additional global controls that prevent arbitrary multiplication of transclusion in such queries (the concern originally was not transclusion performance but multiplication of the same embedded page due to its appearance in a great number of results). It might be that it would be safe to increase the limit by default. Cheers, Markus > > On 11 March 2011 01:16, Dan Bolser<dan...@gm...> wrote: >> Result: Recursion here isn't terrible for performance. >> >> Details: >> >> Yaron kindly set "SMWResultPrinter::$maxRecursionDepth = 40" on >> http://scratchpad.referata.com and I did some testing. >> >> For some reason, the level of recursion now stops at level 20 (??). >> I'm doing everything as before: >> * http://scratchpad.referata.com/wiki/Piston >> * http://scratchpad.referata.com/wiki/Template:Part_of_what >> >> Pages higher in the tree don't hit this limit, and the template >> 'terminates' correctly, e.g. >> * http://scratchpad.referata.com/wiki/Foo >> >> I benchmarked the performance hit due to recursion by requesting each >> page in the tree 20 times using curl, and parsing out the '<!-- Served >> in 0.451 secs. -->' string in the returned URL. Every page request was >> done with the 'action=purge' CGI setting. >> >> The results are quite interesting, suggesting that each recursion adds >> 0.02 seconds to the 'serve time' on scratchpad.referata.com: >> * http://imagepaste.nullnetwork.net/img/1299805257test_benchmark.png >> * http://oi56.tinypic.com/2gx41oy.jpg # duplicate upload >> >> >> About the above plot, upper and lower triangles are +/- s.d. of the >> mean load time (squares) in 20 trials. After depth 20 the behaviour >> changes because the template doesn't deal well with this limit >> condition and a large (null) results set is generated each time (see >> the bottom of the 'Piston' page linked above). The 'solid' regression >> includes just the points up to and including depth 20, the 'dashed' >> regression includes the points above 20. >> >> >> Cheers, >> Dan. >> >> On 10 March 2011 13:54, Dan Bolser<dan...@gm...> wrote: >>> On 10 March 2011 12:27, Alex M. Hendler<amh...@on...> wrote: >>>> Hi Dan, >>>> >>>> I am not sure what "your own risk!" means, but whenever I see an >>>> exclamation point in the body of a comment that is not telling you how >>>> super-cool a feature is, I get a bit scared. >>> >>> I don't see that online here: >>> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >>> >>> Perhaps it's a version issue, but in any case, I take your point. >>> >>> >>>> That said, I can foresee a problem if you set recursion depth too high >>>> and someone made an error such that a page that gets returned by the >>>> query is "part of" the querying page, then you just end up with loopy >>>> (literally) query results. >>> >>> Yeah, of course a page shouldn't be 'part of' itself, but there is >>> nothing to stop a user doing that. >>> >>> >>>> As for performance, in my experience (still yet to be confirmed by >>>> others) maxRecursionDepth> 2 may cause odd property inheritance >>>> behavior if you embed or transclude pages. See >>>> https://bugzilla.wikimedia.org/show_bug.cgi?id=27445 >>> >>> Weird bug! Seems to be a 'feature' of the embed format only with nested queries? >>> >>> >>>> I hope this helps point you in the right direction in terms of doing >>>> what you need to do (it looks like maybe a template with known levels of >>>> properties, e.g., [[-Part of::x]] passed through sufficient levels of >>>> arrays/arraymaps and #if: statements would be a good bet); hopefully >>> >>> Yeah, I don't have a fixed number of levels. >>> >>> >>>> someone else will be able to better answer your performance question. >>>> >>>> As for SemanticTreeView, it looks like it must be installed with Halo, >>>> but the Ontoprise folks may have a different view of that. >>> >>> >>> Did you get this from the docs, or have you tried installing? >>> >>> Thanks for your help, >>> Dan. >>> >>>> -Alex >>>> >>>> >>>> - >>>> ----Original Message----- >>>> From: Dan Bolser<dan...@gm...> >>>> To: Alex M. Hendler<amh...@on...> >>>> Cc: Semantic MediaWiki Users<sem...@li...> >>>> Subject: Re: [Semediawiki-user] Recursive ask using format=template, >>>> stuck at depth of 3? >>>> Date: Thu, 10 Mar 2011 10:00:32 +0000 >>>> >>>> Thanks Alex. >>>> >>>> I read the document you suggested: >>>> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >>>> >>>> but I expected to see why the settings are what they are (i.e. why so >>>> low), which I don't see. >>>> >>>> Do you think those 'odd' issues that you mention would to this simple >>>> recursive query? Perhaps I'm being naive, but it seems to work so well >>>> up to the given depth... would performance degrade super-linearly with >>>> increased depth? >>>> >>>> >>>> Alternatively, the Semantic TreeView extension looks promising: >>>> >>>> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_tree_parser_function >>>> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_generateTree_parser_function >>>> >>>> >>>> but it isn't clear from the documentation if you need to install halo >>>> or not (some places state halo as a requirement, while other >>>> information suggests that it isn't) ... I guess I'll just suck it and >>>> see... The biggest problem is that it expects a tree while I have a >>>> DAG... Not sure what that'll do to it. >>>> >>>> Any other suggested extensions for navigating relations? (Other than >>>> Special:Browse of course!) >>>> >>>> >>>> Cheers, >>>> Dan. >>>> >>>> On 10 March 2011 07:39, Alex M. Hendler<amh...@on...> wrote: >>>>> Dan, >>>>> >>>>> I believe the setting you're looking for is >>>>> >>>>> SMWResultPrinter::$maxRecursionDepth = x; >>>>> >>>>> Place that after your enableSemantics statement in LocalSettings.php, >>>>> where x = [desired depth]-1. >>>>> >>>>> The reason you can't get beyond 3 levels on Referata is that (I believe) >>>>> it uses the default max. depth of 2. includes/SMW_QueryPrinter.php >>>>> explains the recursion settings very well. >>>>> >>>>> The developers can correct me if I am wrong, but I believe that >>>>> recursion is intentionally not more publicly documented (i.e., outside >>>>> of source code) because it can do odd and perhaps unpredictable things >>>>> to semantics as well as performance if you increase it above 2. >>>>> >>>>> You can sometimes fool the system into recursing beyond >>>>> maxRecursionDepth, but that is not something I would encourage anyone to >>>>> do intentionally (it also prints out very messy results). >>>>> >>>>> Hope this helps, >>>>> Alex >>>>> >>>>> ----Original Message----- >>>>> From: Dan Bolser<dan...@gm...> >>>>> To: Semantic MediaWiki Users<sem...@li...> >>>>> Subject: [Semediawiki-user] Recursive ask using format=template, stuck >>>>> at depth of 3? >>>>> Date: Wed, 9 Mar 2011 23:58:14 +0000 >>>>> >>>>> Hi, >>>>> >>>>> I wrote a nice 'recursive' template to traverse a DAG here: >>>>> http://scratchpad.referata.com/wiki/Template:Part_of_what >>>>> >>>>> It works by calling itself with an ask using it's given input and then >>>>> the result of the ask and so on up the DAG ... (recursion always makes >>>>> me a little dizzy). >>>>> >>>>> It works great except it only calls itself a maximum of 3 times, as >>>>> evidenced by comparing the results of these two pages: >>>>> * http://scratchpad.referata.com/wiki/Piston >>>>> * http://scratchpad.referata.com/wiki/Engine >>>>> >>>>> I couldn't find the appropriate setting in the manual to bump this >>>>> 'maximum recursion' value. The equivalent value for MediaWiki can be >>>>> found here, but note the default setting is a whopping 40: >>>>> >>>>> http://www.mediawiki.org/wiki/Manual:$wgMaxTemplateDepth >>>>> >>>>> >>>>> Any tips appreciated. >>>>> >>>>> Cheers, >>>>> Dan. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Colocation vs. Managed Hosting >>>>> A question and answer guide to determining the best fit >>>>> for your organization - today and in the future. >>>>> http://p.sf.net/sfu/internap-sfd2d >>>>> _______________________________________________ >>>>> Semediawiki-user mailing list >>>>> Sem...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>>>> >>>>> >>>> >>>> >>> >> >> ------------------------------------------------------------------------------ >> Colocation vs. Managed Hosting >> A question and answer guide to determining the best fit >> for your organization - today and in the future. >> http://p.sf.net/sfu/internap-sfd2d >> _______________________________________________ >> Semediawiki-user mailing list >> Sem...@li... >> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >> > > ------------------------------------------------------------------------------ > Colocation vs. Managed Hosting > A question and answer guide to determining the best fit > for your organization - today and in the future. > http://p.sf.net/sfu/internap-sfd2d > _______________________________________________ > Semediawiki-devel mailing list > Sem...@li... > https://lists.sourceforge.net/lists/listinfo/semediawiki-devel > |
From: Alex M. H. <amh...@on...> - 2011-03-16 10:35:32
|
Hi Dan, Thanks for sharing the benchmark results; good to know that recursion does not seem to be too bad for performance. I'm sorry that I didn't get back to you sooner - I've been a bit tied up. Responses to your questions are below. Best, Alex -- ontolawgy™ LLC: connect . . . the . . . dots™ http://ontolawgy.com -----Original Message----- From: Dan Bolser <dan...@gm...> To: Alex M. Hendler <amh...@on...> Cc: Semantic MediaWiki Users <sem...@li...> Subject: Benchmarking results: Was "Re: [Semediawiki-user] Recursive ask using format=template, stuck at depth of 3?" Date: Fri, 11 Mar 2011 01:16:59 +0000 Result: Recursion here isn't terrible for performance. Details: Yaron kindly set "SMWResultPrinter::$maxRecursionDepth = 40" on http://scratchpad.referata.com and I did some testing. For some reason, the level of recursion now stops at level 20 (??). I'm doing everything as before: * http://scratchpad.referata.com/wiki/Piston * http://scratchpad.referata.com/wiki/Template:Part_of_what Pages higher in the tree don't hit this limit, and the template 'terminates' correctly, e.g. * http://scratchpad.referata.com/wiki/Foo [snip] On 10 March 2011 13:54, Dan Bolser <dan...@gm...> wrote: > On 10 March 2011 12:27, Alex M. Hendler <amh...@on...> wrote: >> Hi Dan, >> >> I am not sure what "your own risk!" means, but whenever I see an >> exclamation point in the body of a comment that is not telling you how >> super-cool a feature is, I get a bit scared. > > I don't see that online here: > http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html > > Perhaps it's a version issue, but in any case, I take your point. The comments are there at lines 75-76 of the source file. If you click on the blue 0077 line number on the left side, you'll see the comments. > > >> That said, I can foresee a problem if you set recursion depth too high >> and someone made an error such that a page that gets returned by the >> query is "part of" the querying page, then you just end up with loopy >> (literally) query results. > > Yeah, of course a page shouldn't be 'part of' itself, but there is > nothing to stop a user doing that. > > >> As for performance, in my experience (still yet to be confirmed by >> others) maxRecursionDepth > 2 may cause odd property inheritance >> behavior if you embed or transclude pages. See >> https://bugzilla.wikimedia.org/show_bug.cgi?id=27445 > > Weird bug! Seems to be a 'feature' of the embed format only with nested queries? If it is a "feature", the documentation specifically says it does not exist. I can't think of any other context aside from "embed" where this would be an issue; it would be really nice to be able to set it either way by default and override on a per-query/category/namespace basis. > >> I hope this helps point you in the right direction in terms of doing >> what you need to do (it looks like maybe a template with known levels of >> properties, e.g., [[-Part of::x]] passed through sufficient levels of >> arrays/arraymaps and #if: statements would be a good bet); hopefully > > Yeah, I don't have a fixed number of levels. Looks like you've found a great way around that with your template. Good news on the performance, but with recursion basically on auto-pilot, you probably have to very carefully curate your content to prevent a big mess. > >> someone else will be able to better answer your performance question. >> >> As for SemanticTreeView, it looks like it must be installed with Halo, >> but the Ontoprise folks may have a different view of that. > > > Did you get this from the docs, or have you tried installing? I got it from the docs--I am pretty sure that all of the Ontoprise extensions require Halo. For the few that don't "officially" require Halo, I've never been able to get them working without installing Halo. > Thanks for your help, > Dan. > >> -Alex >> >> >> - >> ----Original Message----- >> From: Dan Bolser <dan...@gm...> >> To: Alex M. Hendler <amh...@on...> >> Cc: Semantic MediaWiki Users <sem...@li...> >> Subject: Re: [Semediawiki-user] Recursive ask using format=template, >> stuck at depth of 3? >> Date: Thu, 10 Mar 2011 10:00:32 +0000 >> >> Thanks Alex. >> >> I read the document you suggested: >> http://semantic-mediawiki.org/doc/SMW__QueryPrinter_8php-source.html >> >> but I expected to see why the settings are what they are (i.e. why so >> low), which I don't see. >> >> Do you think those 'odd' issues that you mention would to this simple >> recursive query? Perhaps I'm being naive, but it seems to work so well >> up to the given depth... would performance degrade super-linearly with >> increased depth? >> >> >> Alternatively, the Semantic TreeView extension looks promising: >> >> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_tree_parser_function >> * http://smwforum.ontoprise.com/smwforum/index.php/Help:Description_of_the_generateTree_parser_function >> >> >> but it isn't clear from the documentation if you need to install halo >> or not (some places state halo as a requirement, while other >> information suggests that it isn't) ... I guess I'll just suck it and >> see... The biggest problem is that it expects a tree while I have a >> DAG... Not sure what that'll do to it. >> >> Any other suggested extensions for navigating relations? (Other than >> Special:Browse of course!) >> >> >> Cheers, >> Dan. >> >> On 10 March 2011 07:39, Alex M. Hendler <amh...@on...> wrote: >>> Dan, >>> >>> I believe the setting you're looking for is >>> >>> SMWResultPrinter::$maxRecursionDepth = x; >>> >>> Place that after your enableSemantics statement in LocalSettings.php, >>> where x = [desired depth]-1. >>> >>> The reason you can't get beyond 3 levels on Referata is that (I believe) >>> it uses the default max. depth of 2. includes/SMW_QueryPrinter.php >>> explains the recursion settings very well. >>> >>> The developers can correct me if I am wrong, but I believe that >>> recursion is intentionally not more publicly documented (i.e., outside >>> of source code) because it can do odd and perhaps unpredictable things >>> to semantics as well as performance if you increase it above 2. >>> >>> You can sometimes fool the system into recursing beyond >>> maxRecursionDepth, but that is not something I would encourage anyone to >>> do intentionally (it also prints out very messy results). >>> >>> Hope this helps, >>> Alex >>> >>> ----Original Message----- >>> From: Dan Bolser <dan...@gm...> >>> To: Semantic MediaWiki Users <sem...@li...> >>> Subject: [Semediawiki-user] Recursive ask using format=template, stuck >>> at depth of 3? >>> Date: Wed, 9 Mar 2011 23:58:14 +0000 >>> >>> Hi, >>> >>> I wrote a nice 'recursive' template to traverse a DAG here: >>> http://scratchpad.referata.com/wiki/Template:Part_of_what >>> >>> It works by calling itself with an ask using it's given input and then >>> the result of the ask and so on up the DAG ... (recursion always makes >>> me a little dizzy). >>> >>> It works great except it only calls itself a maximum of 3 times, as >>> evidenced by comparing the results of these two pages: >>> * http://scratchpad.referata.com/wiki/Piston >>> * http://scratchpad.referata.com/wiki/Engine >>> >>> I couldn't find the appropriate setting in the manual to bump this >>> 'maximum recursion' value. The equivalent value for MediaWiki can be >>> found here, but note the default setting is a whopping 40: >>> >>> http://www.mediawiki.org/wiki/Manual:$wgMaxTemplateDepth >>> >>> >>> Any tips appreciated. >>> >>> Cheers, >>> Dan. >>> >>> ------------------------------------------------------------------------------ >>> Colocation vs. Managed Hosting >>> A question and answer guide to determining the best fit >>> for your organization - today and in the future. >>> http://p.sf.net/sfu/internap-sfd2d >>> _______________________________________________ >>> Semediawiki-user mailing list >>> Sem...@li... >>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>> >>> >> >> > |