From: Martin B. <mar...@un...> - 2012-02-05 22:51:39
|
Dear fellow SMW users, in the Wiki-based project "LexLep" http://www.univie.ac.at/lexlep/ (see Special:Version for installed components), we are recieving incomplete results in inline queries (using #ask) as well as on property pages (showing a list of pages where the property is used). There seem to exist two variants of the problem: (1) Protected pages will only show up in inline queries until some time after having saved these pages. They disappear eventually from the results after a while, especially after having done a "data repair" (Special:SMWAdmin). (2) Apart from protected pages, there are also pages missing in query results for no apparent reason. The only thing these pages have in common are queries for rather new properties. There are, for example, the Property:disambiguation and Property:project, which have been created later than other properties and which notoriously show incomplete lists of pages using these properties. As above, newly saved pages are shown as expected but they disappear from the lists after a while or after a "data repair". As far as I understand there is a SMW data table which is used for queries, and there seems to be a problem updating this table. I suppose that updates triggered by jobs do/can not read protected pages and will therefore delete their semantic data from the data table. Also, it appears to me that the number of entries in the data table may be limited, so that a completely "updated" data table will not contain the newest entries (i.e. those exceeding this limit). Please tell me if any of my assumptions are correct, or what may be the reason for the incomplete results. Perhaps it is a known bug or just a matter of changing some settings? I would appreciate any advice what I shall do to solve this problem. Best regards Martin Braun |
From: Neill M. <ne...@nl...> - 2012-02-06 13:27:01
|
Hi Martin. Just an idea. Perhaps your job queue is not empty? It is very easy to get it stacking up hundreds of jobs, especially if you edit a template that is used by a lot of pages. Check out http://www.mediawiki.org/wiki/Manual:Job_queue and http://www.mediawiki.org/wiki/Manual:RunJobs.php Cheers Neill. On 05/02/12 22:51, Martin Braun wrote: > Dear fellow SMW users, > > in the Wiki-based project "LexLep" http://www.univie.ac.at/lexlep/ (see > Special:Version for installed components), we are recieving incomplete > results in inline queries (using #ask) as well as on property pages > (showing a list of pages where the property is used). There seem to > exist two variants of the problem: > > (1) Protected pages will only show up in inline queries until some time > after having saved these pages. They disappear eventually from the > results after a while, especially after having done a "data repair" > (Special:SMWAdmin). > > (2) Apart from protected pages, there are also pages missing in query > results for no apparent reason. The only thing these pages have in > common are queries for rather new properties. There are, for example, > the Property:disambiguation and Property:project, which have been > created later than other properties and which notoriously show > incomplete lists of pages using these properties. As above, newly saved > pages are shown as expected but they disappear from the lists after a > while or after a "data repair". > > As far as I understand there is a SMW data table which is used for > queries, and there seems to be a problem updating this table. I suppose > that updates triggered by jobs do/can not read protected pages and will > therefore delete their semantic data from the data table. Also, it > appears to me that the number of entries in the data table may be > limited, so that a completely "updated" data table will not contain the > newest entries (i.e. those exceeding this limit). > > Please tell me if any of my assumptions are correct, or what may be the > reason for the incomplete results. Perhaps it is a known bug or just a > matter of changing some settings? I would appreciate any advice what I > shall do to solve this problem. > > Best regards > Martin Braun > > ------------------------------------------------------------------------------ > Try before you buy = See our experts in action! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-dev2 > _______________________________________________ > Semediawiki-user mailing list > Sem...@li... > https://lists.sourceforge.net/lists/listinfo/semediawiki-user > |
From: Jeroen De D. <jer...@gm...> - 2012-02-07 14:11:38
|
Hey, > in the Wiki-based project "LexLep" http://www.univie.ac.at/lexlep/ (see Special:Version for installed components) Although I don't remember ever seeing this issue, it's definitely worth upgrading to the latest release of SMW, which might very well fix your problem. And even if it does not, many other fixes and improvements have been made :) Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- |
From: Jeroen De D. <jer...@gm...> - 2012-02-07 19:17:34
|
Hey, > (1) Do SMW update jobs only consider data visible to any user, and ignore data of protected pages? I forgot to comment on this earlier - SMW should not treat protected pages any different then regular pages. If it does, this is definitely a bug. Can you report your findings on https://bugzilla.wikimedia.org/ ? > (2) Is there a limit to the SMW data table (maybe at around 84000 entries)? > (3) May there be a MySQL limit involved, e.g. concerning the table "smw_atts2" (at around 52000 entries)? Not that I know of. AFAIK there are people running SMW installs with MySQL and have over 100k entries. Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- |
From: Martin B. <mar...@un...> - 2012-02-16 11:34:35
|
Dear fellow SMW users, some further information concering my previously described problem: (1) There can be a page missing in inline queries, but also just additionally printed out values. And the very same page may be missing in one query but appearing in another depending on what was searched for. General example: There may be a page called "p1" having the properties [[property_a::value_a1]] [[property_b::value_b1]], and another page "p2" having the properties [[property_a::value_a2]] [[property_b::value_b2]]. When asking {{#ask: [[property_a::+]] | ?property_b=|format=ul }} the result may be * p1 (value_b1) * p2 instead of * p1 (value_b1) * p2 (value_b2) When asking {{#ask: [[property_b::+]] | ?property_a=|format=ul }} the result may be * p1 (value_a1) instead of * p1 (value_a1) * p2 (value_a2) So, it seems that not entire pages are missing from the SMW data table (or at least from the query results) but only some properties or property-value pairs (in the above case [[property_b::value_b2]] ). (2) As a test I tried to simplify the system by removing properties that aren't really necessary. If the problem is caused by some kind of limit, there should be a positive effect. After a "data repair", some previously missing pages reappeared in query results, but astonishingly others seem to have disappeared. (Unfortunately there are too many pages and properties, making it impossible to observe all effects.) (3) I also tried to reproduce the incomplete results of protected pages in another wiki (MW 1.17.0, SMW 1.6), but did not succeed. No page was missing in query results. So either the problem no longer exists in newer versions, or it is even more complex. From your reactions up to now I suppose that no one else ever came across this or a similar problem. But if anyone has an idea or advice (other than an update, naturally), please let me know. Best regards Martin Braun |
From: James H. K. <jam...@gm...> - 2012-02-16 14:43:51
|
Hi, >> incomplete results in inline queries ... after having done a "data repair" We are running MW 1.18 with SMW 1.7 storing around 930K property values of 476 different properties and we have not found any issue in regards to missing SMW data and in case a complete data refresh with Special:SMWAdmin is necessary all data sets are re-published which means that for a period of time data sets are incomplete. In the past we had one issue related to $smwgQMaxLimit, concepts, and caching. $smwgQMaxLimit [1] has a default value of 10000 and caused some confusion of missing data in concepts but as soon as we increased the value to 100000 everything worked just fine. >> Apart from protected pages, there are also pages missing in query results for no >> apparent reason It can happen that you have a query and try to sort it with a particular property but when this sort property does not exists for some of those data sets it will not be displayed. For example, a query {{#ask: [[Foo::Bar]]|format=broadtable}} will contain the complete set for [[Foo::Bar]] while the query {{#ask: [[Foo::Bar]]|format=broadtable|sort=Type}} not necessarily contains the complete set because some elements of [[Foo::Bar]] might not have a property value for the property 'Type' and therefore are not displayed but it does not imply their don't exist. In the meantime if you suspect some foul play you might want to check your display result with an additional output that counts results {{#ask:[[Foo::Bar]]|format=count}} to see how many of those data sets are actually selected and compare it the expected result. [1] http://www.semantic-mediawiki.org/wiki/Help:Configuration#smwgQMaxLimit Cheers, On Thu, Feb 16, 2012 at 8:34 PM, Martin Braun <mar...@un...> wrote: > Dear fellow SMW users, > > some further information concering my previously described problem: > > (1) There can be a page missing in inline queries, but also just > additionally printed out values. And the very same page may be missing > in one query but appearing in another depending on what was searched > for. General example: > > There may be a page called "p1" having the properties > [[property_a::value_a1]] [[property_b::value_b1]], > and another page "p2" having the properties [[property_a::value_a2]] > [[property_b::value_b2]]. When asking {{#ask: [[property_a::+]] | > ?property_b=|format=ul }} the result may be > * p1 (value_b1) > * p2 > instead of > * p1 (value_b1) > * p2 (value_b2) > When asking {{#ask: [[property_b::+]] | ?property_a=|format=ul }} the > result may be > * p1 (value_a1) > instead of > * p1 (value_a1) > * p2 (value_a2) > > So, it seems that not entire pages are missing from the SMW data table > (or at least from the query results) but only some properties or > property-value pairs (in the above case [[property_b::value_b2]] ). > > (2) As a test I tried to simplify the system by removing properties that > aren't really necessary. If the problem is caused by some kind of limit, > there should be a positive effect. After a "data repair", some > previously missing pages reappeared in query results, but astonishingly > others seem to have disappeared. (Unfortunately there are too many pages > and properties, making it impossible to observe all effects.) > > (3) I also tried to reproduce the incomplete results of protected pages > in another wiki (MW 1.17.0, SMW 1.6), but did not succeed. No page was > missing in query results. So either the problem no longer exists in > newer versions, or it is even more complex. > > From your reactions up to now I suppose that no one else ever came > across this or a similar problem. But if anyone has an idea or advice > (other than an update, naturally), please let me know. > > Best regards > Martin Braun > > > ------------------------------------------------------------------------------ > Virtualization & Cloud Management Using Capacity Planning > Cloud computing makes use of virtualization - but cloud computing > also focuses on allowing computing to be delivered as a service. > http://www.accelacomm.com/jaw/sfnl/114/51521223/ > _______________________________________________ > Semediawiki-user mailing list > Sem...@li... > https://lists.sourceforge.net/lists/listinfo/semediawiki-user |