Wikipedia Miner Toolkit / Bugs / #4 Memory thrashing in extraction scripts

#4 Memory thrashing in extraction scripts

Status: closed-fixed

Owner: David Milne

Labels: None

Priority: 9

Updated: 2008-12-02

Created: 2008-12-01

Creator: David Milne

Private: No

Since the xml dumps have gotten bigger, the extraction scripts have started failing, or at least thrashing around memory until they slow completely down.
The problem areas are in building the anchor_summary and page_links_in files

Discussion

David Milne - 2008-12-01

status: open --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Milne - 2008-12-02

status: closed --> closed-fixed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Nobody/Anonymous - 2009-08-21

I have exactly the same kind of problem and it's not resolved in the last version of patchWikipediaData.pl

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Memory thrashing in extraction scripts

Group

Searches

Help

#4 Memory thrashing in extraction scripts

Discussion