HITCrawler Icon

HITCrawler

beta

HITCralwer is a lightweight, easy to use cralwer

Add a Review
15 Downloads (This Week)
Last Update:
  Browse Code Git Repository

Screenshots

Description

HITCrawler is a lightweight, easy to use cralwer. The user can get the more important websites first from the starting Url.

HITCrawler is our annual project when I was a freshman.It has the following advantage: 1. The user can get the more important websites first.
2. The user can set the memory the software will use based on the task and computer configuration information.
3. Easy to use, fast running speed.

However, it has the following disadvantage:
1. It is based on single thread. The crawling speed is much slower than the multithreading cralwer.
2. We don't have a configuration file which can set the seed of the website, the connection time, etc. You have to change them in code.

HITCrawler Web Site

Categories

Communications

License

W3C License

Update Notifications





Write a Review

User Reviews

Be the first to post a review of HITCrawler!

Additional Project Details

Languages

English

Intended Audience

Other Audience

Programming Language

C++

Registered

2012-07-29
Screenshots can attract more users to your project.
Features can attract more users to your project.

Icons must be PNG, GIF, or JPEG and less than 1 MiB in size. They will be displayed as 48x48 images.