From: Justin M <d3...@sa...> - 2013-10-28 20:43:30
|
I'm trying to find a way to quickly import generated pages, and have links and semantic data updated. If I generate an XML file, I can use the mediawiki importDump.php to bring in the pages and everything is updated, but at a rather slow 2.5 pages a second. With 14 million pages, that would take months! I can skip updating links on the import, which takes it down to 2 days, but then semantic data isn't updated. So if I then run SMW_refreshData.php, we are right back to months. I've tried messing with a lot of the sql importers like mwdumper, but none seem to work right after 5+ years of no development, and they still aren't going to update Semantic's data. Any ideas on how I can speed things up? I'd really appreciate the help. Thanks! |