OpenWebSpider is an Open Source multi-threaded Web Spider (robot, crawler) and search engine with a lot of interesting features!
Thanks, it's works great with mono ;-)
OpenWebSpider now supports MP3s and PDFs CHANGELOG: OpenWebSpiderCS v0.1.4 - Mysql/NET Connector upgraded to 5.2.5.0 - Enhanced encodings support - New feature: Support to META: "robots" (NOINDEX, NOFOLLOW) - New feature: New configuration file field: crawler_id - New field "crawler_id" in table "hostlist" - New table: crawler_act - New feature: Remote actions over running crawlers [Status, Play, Pause, Kill] - New file support: PDFs [Using PDFBox and IKVM] - New table: pdf - New file support: MP3s [Using UltraID3Lib] - New table: mp3 - New feature: new command-line argument: --pdf - New feature: new command-line argument: --mp3 http://www.openwebspider.org/
OpenWebSpiderCS v0.1.4 - Mysql/NET Connector upgraded to 5.2.5.0 - Enhanced encodings support - New feature: Support to META: "robots" (NOINDEX, NOFOLLOW) - New feature: New configuration file field: crawler_id - New field "crawler_id" in table "hostlist" - New table: crawler_act - New feature: Remote actions over running crawlers [Status, Play, Pause, Kill] - New file support: PDFs [Using PDFBox and IKVM] - New table: pdf - New file support: MP3s [Using UltraID3Lib] - New table: mp3 - New feature: new command-line argument: --pdf - New feature: new command-line argument: --mp3
Copyright © 2009 Geeknet, Inc. All rights reserved. Terms of Use
Thanks for your rating!
Would you also like to write a review?
Thanks for your review!
Get credit for your review by logging in via OpenID. Click your account provider: