Heritrix: Internet Archive Web Crawler

Tracker: Feature Requests

8 Scripts to generate end-of-job reports - ID: 1010883

Last Update: Comment added ( karl-ia )

Details:

If a crawl terminates abnormally or, as was the case
this morning, its terminated normally but the crawler
refuses to go down, then reports are not generated.
This RFE is for a set of scripts to do what the
StatisticsTracker does at the end-of-job. Such scripts
would need to go against the logs rather than in-memory
structures that keep account of state.

Submitted:

Michael Stack ( stack-sf ) - 2004-08-17 17:18

Priority:

Status:

Closed

Resolution:

None

Assigned:

Dan Avery

Category:

Usability/UI

Group:

None

Visibility:

Public

Comments ( 2 )

Date: 2007-03-14 01:33 Sender: karl-ia This issue is now discussed in the new JIRA tracker at http://webteam.archive.org/jira/browse/HER-822 -- please add further comments at that location.
Date: 2004-10-20 21:27 Sender: danavery Logged In: YES user_id=1086990 Script added as src/scripts/make_reports.pl. Takes a crawl log as input and creates most of the standard reports.

Attached File

Changes ( 6 )

Field	Old Value	Date	By
close_date	-	2004-10-20 21:27	danavery
status_id	Open	2004-10-20 21:27	danavery
assigned_to	ia_igor	2004-09-01 22:41	stack-sf
assigned_to	nobody	2004-09-01 22:19	gojomo
priority	6	2004-09-01 22:18	gojomo
priority	5	2004-09-01 22:01	stack-sf

Heritrix: Internet Archive Web Crawler

Tracker: Feature Requests

Comments ( 2 )

Attached File

No Files Currently Attached

Changes ( 6 )

About SourceForge

Find Software

Develop Software

Community

Help