Re: [Semediawiki-user] Semantic Drilldown extension issue

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Hi Yaron,

unfortunately you confirm what I was afraid of but thank you for your feedback, it's very helpful for me.
I'll investigate new solutions because I need using all available properties (can't filter it).

Stephane

De : ya...@gm... [mailto:ya...@gm...] De la part de Yaron Koren
Envoyé : lundi 30 juillet 2012 17:35
À : SCHICKLIN Stephane
Cc : sem...@li...
Objet : Re: [Semediawiki-user] Semantic Drilldown extension issue

Hi,

Yes, it looks like you're just reaching a natural limit on Semantic Drilldown's querying. SD is rather database-intensive, especially with huge amounts of data. One thing you can do to improve performance is to use the "combo box" input type for one or more of the filters (or "date range", if it's a date property). It makes the BrowseData interface less interesting, but it speeds up querying a lot, since the page no longer has to find the number of pages for every filter value.

-Yaron
On Mon, Jul 30, 2012 at 9:37 AM, SCHICKLIN Stephane <ste...@bi...<mailto:ste...@bi...>> wrote:
Hey all,

I'm using the semantic drilldown extension and I'm experiencing a timeout and/or memory problem.

Versions mediawiki and semantic mediawiki that I'm using :
MediaWiki<http://www.mediawiki.org/>

1.16.2

PHP<http://www.php.net/>

5.1.6 (apache2handler)

MySQL<http://www.mysql.com/>

5.0.84

 Semantic MediaWiki<http://semantic-mediawiki.org> (Version 1.5.5.2)
 Semantic Drilldown<http://www.mediawiki.org/wiki/Extension:Semantic_Drilldown> (Version 0.8.1)

I recently exported all the data in a triplestore and my store is about 5,6 millions triples. The semantic drilldown worked very well until the data were subjected to a 5 fold increase (around 1-1,2 millions of triples).
Here is the PHP error log for my problem :
[error] [client xxx.xxx.xxx.xxx] PHP Fatal error:  Maximum execution time of 90 seconds exceeded in [...]/extensions/SemanticDrilldown/specials/SD_BrowseData.php on line 435, referer: http://[...]/index.php?title=Queries

The line 435 is corresponding to the following line : $category_str = $category . " (" . count( array_unique( $category_children ) ) . ")";

So, I tried to edit the php.ini to extend the timeout :
;max_execution_time = 90 ; Maximum execution time of each script, in seconds
max_execution_time = 300

But the new execution of the SemanticDrilldown lead to a MySQL issue :
MySQL
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
32101 mysql     16   0  481m 132m 5636 S 100.1  1.7   1593:56 mysqld

The CPU use was blocked at 100% until I killed the query :
mysql> show processlist;
+--------+----------+--------------------------------------+-------------------------+---------+------+--------------+------------------------------------------------------------------------------------------------------+
| Id     | User     | Host                              | db       | Command | Time | State        | Info                                                                                                 |
+--------+----------+--------------------------------------+-------------------------+---------+------+--------------+------------------------------------------------------------------------------------------------------+
| 998902 | wikiuser | -------------------:---- | wikidb | Query   | 2227 | Sending data | SELECT /*  xxx.xxx.xxx.xxx */ COUNT(DISTINCT sdv.id<http://sdv.id>) FROM semantic_drilldown_values sdv
        LEFT OUTER JO |
| 998903 | wikiuser | -------------------:---- | wikidb | Sleep   | 2531 |              | NULL                                                                                                 |

(Unfortunately the query was truncated...).
I don't know what to try to investigate further.
Moreover, maybe my store is too huge for the Semantic Drilldown extension or maybe the problem is already fixed in the next versions of the extension.
I will really appreciate any kind of input,
Thanks in advance,

Stephane

AVIS : Ce courrier et ses pieces jointes sont destines a leur seul destinataire et peuvent contenir des informations confidentielles appartenant a bioMerieux. Si vous n'etes pas destinataire, vous etes informe que toute lecture, divulgation, ou reproduction de ce message et des pieces jointes est strictement interdite. Si vous avez recu ce message par erreur merci d'en prevenir l'expediteur et de le detruire, ainsi que ses pieces jointes.
NOTICE: This message and attachments are intended only for the use of their addressee and may contain confidential information belonging to bioMerieux. If you are not the intended recipient, you are hereby notified that any reading, dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this message in error, please notify the original sender immediately and delete this message, along with any attachments.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-user mailing list
Sem...@li...<mailto:Sem...@li...>
https://lists.sourceforge.net/lists/listinfo/semediawiki-user

--
WikiWorks · MediaWiki Consulting · http://wikiworks.com

AVIS : Ce courrier et ses pieces jointes sont destines a leur seul destinataire et peuvent contenir des informations confidentielles appartenant a bioMerieux. Si vous n'etes pas destinataire, vous etes informe que toute lecture, divulgation, ou reproduction de ce message et des pieces jointes est strictement interdite. Si vous avez recu ce message par erreur merci d'en prevenir l'expediteur et de le detruire, ainsi que ses pieces jointes.
NOTICE: This message and attachments are intended only for the use of their addressee and may contain confidential information belonging to bioMerieux. If you are not the intended recipient, you are hereby notified that any reading, dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this message in error, please notify the original sender immediately and delete this message, along with any attachments.

Re: [Semediawiki-user] Semantic Drilldown extension issue

Lets you store and query data within the wiki's pages.

Re: [Semediawiki-user] Semantic Drilldown extension issue