Heritrix: Internet Archive Web Crawler

4.9 Stars (33)
397 Downloads (This Week)
Last Update:
Download heritrix-3.1.0-dist.zip
Browse All Files
Windows Mac Linux

Description

The archive-crawler project is building Heritrix: a flexible, extensible, robust, and scalable web crawler capable of fetching, archiving, and analyzing the full diversity and breadth of internet-accesible content.

Heritrix: Internet Archive Web Crawler Web Site

Features

  • deeply and thoroughly harvests website content
  • works on any Java platform (Linux recommended)
  • stores content to ARC or ISO WARC aggregate/transcript format
  • web interface for operator control and monitoring of crawls

Update Notifications





User Ratings

★★★★★
★★★★
★★★
★★
32
0
0
0
1
ease 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
features 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
design 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
support 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5
Write a Review

User Reviews

  • rusik1978
    1 of 5 2 of 5 3 of 5 4 of 5 5 of 5

    Thanks for great project! Simply the best.Good,good,good.+1

    Posted 05/19/2013
  • sever78
    1 of 5 2 of 5 3 of 5 4 of 5 5 of 5

    very good project, thanks!Good,good,good.+1

    Posted 05/18/2013
  • urina
    1 of 5 2 of 5 3 of 5 4 of 5 5 of 5

    very good project, thanks!Good,good,good.+1

    Posted 05/17/2013
  • alexvinn11
    1 of 5 2 of 5 3 of 5 4 of 5 5 of 5

    very good project, thanks!Good,good,good.+1

    Posted 04/15/2013
  • jimi78
    1 of 5 2 of 5 3 of 5 4 of 5 5 of 5

    very good project, thanks!

    Posted 04/09/2013
Read more reviews

Additional Project Details

Languages

English

Intended Audience

Advanced End Users, Developers, Education, Government, Information Technology, Non-Profit Organizations

User Interface

Web-based

Programming Language

Java

Registered

2003-02-11
Screenshots can attract more users to your project.
Features can attract more users to your project.

Icons must be PNG, GIF, or JPEG and less than 1 MiB in size. They will be displayed as 48x48 images.