Browse Code
HITCrawler is a lightweight, easy to use cralwer. The user can get the more important websites first from the starting Url.
HITCrawler is our annual project when I was a freshman.It has the following advantage: 1. The user can get the more important websites first.
2. The user can set the memory the software will use based on the task and computer configuration information.
3. Easy to use, fast running speed.
However, it has the following disadvantage:
1. It is based on single thread. The crawling speed is much slower than the multithreading cralwer.
2. We don't have a configuration file which can set the seed of the website, the connection time, etc. You have to change them in code.
Project Samples
Categories
CommunicationsLicense
GNU General Public License version 3.0 (GPLv3)Follow HITCrawler
You Might Also Like
Yooz provides the smartest, most powerful, and easiest-to-use cloud-based E-invoicing and Purchase-to-Pay automation solution. It delivers unmatched savings, speed, and security with affordable zero-risk subscriptions to more than 5,000 customers and 300,000 users worldwide.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of HITCrawler!