Heritrix: Internet Archive Web Crawler

4.9 Stars (33)
351 Downloads (This Week)
Last Update:
Download heritrix-3.1.0-dist.zip
Browse All Files
Windows Mac Linux

Description

The archive-crawler project is building Heritrix: a flexible, extensible, robust, and scalable web crawler capable of fetching, archiving, and analyzing the full diversity and breadth of internet-accesible content.

Heritrix: Internet Archive Web Crawler Web Site

Features

  • deeply and thoroughly harvests website content
  • works on any Java platform (Linux recommended)
  • stores content to ARC or ISO WARC aggregate/transcript format
  • web interface for operator control and monitoring of crawls

Update Notifications





User Ratings

★★★★★
★★★★
★★★
★★
32
0
0
0
1
ease 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
features 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
design 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
support 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
Write a Review

User Reviews

There are no 2 star reviews.

Additional Project Details

Languages

English

Intended Audience

Advanced End Users, Developers, Education, Government, Information Technology, Non-Profit Organizations

User Interface

Web-based

Programming Language

Java

Registered

2003-02-11
Screenshots can attract more users to your project.
Features can attract more users to your project.

Icons must be PNG, GIF, or JPEG and less than 1 MiB in size. They will be displayed as 48x48 images.