Heritrix: Internet Archive Web Crawler

Tracker: Bugs

6 timing issue on crawl-start & "run time" stat - ID: 1002356

Last Update: Comment added ( karl-ia )

Details:

Sometimes, for a second at the beginning of a crawl,
the crawl elapsed time shows a false, large interval.
(It's ~ 303,000 hours, about the time since 1970.)

This seems to be due to the fact that the StatTracker
thread hasn't done its initiailization by the time
other threads are asking for data.

The ordering of statracker initiatialization should be
adjusted to avoid this timing issue.

Submitted:

Gordon Mohr ( gojomo ) - 2004-08-03 00:29

Priority:

Status:

Closed

Resolution:

Fixed

Assigned:

Gordon Mohr

Category:

None

Group:

1.6.0

Visibility:

Public

Comments ( 4 )

Date: 2007-03-14 00:15 Sender: karl-ia This issue is now discussed in the new JIRA tracker at http://webteam.archive.org/jira/browse/HER-220 -- please add further comments at that location.
Date: 2005-09-08 23:32 Sender: gojomo Logged In: YES user_id=144912 A crude but simple fix applied... commit comment: Fix for [ 1002356 ] timing issue on crawl-start & "run time" stat * AbstractTracker.java if startTime not yet set (zero), consider elapsed time zero, too Haven't seen since applying this fix, but not 100% certain this resolves... problem was always a matter of loading the console-status during a tiny window of time. Closing unless/until it's seen again with this code in place.
Date: 2005-05-17 20:30 Sender: gojomo Logged In: YES user_id=144912 I've been seeing this again in post 1.4-HEAD. Reopening.
Date: 2004-10-20 21:55 Sender: gojomo Logged In: YES user_id=144912 Fixed in HEAD 8/12/2004, by moving StatTracker start-time initialization to a noteStart() method that is called outside the StatTracker thread, before it is begun.

Attached File

Changes ( 12 )

Field	Old Value	Date	By
artifact_group_id	None	2005-09-23 18:01	gojomo
status_id	Open	2005-09-08 23:32	gojomo
close_date	2004-10-20 21:55	2005-09-08 23:32	gojomo
resolution_id	Accepted	2005-09-08 23:32	gojomo
resolution_id	Fixed	2005-05-17 20:30	gojomo
status_id	Closed	2005-05-17 20:30	gojomo
summary	timing issue on crawl-start & run-time stat	2005-05-17 20:30	gojomo
status_id	Open	2004-10-20 21:55	gojomo
close_date	-	2004-10-20 21:55	gojomo
assigned_to	nobody	2004-10-20 21:55	gojomo
resolution_id	None	2004-10-20 21:55	gojomo
priority	5	2004-09-01 21:57	gojomo

Heritrix: Internet Archive Web Crawler

Tracker: Bugs

Comments ( 4 )

Attached File

No Files Currently Attached

Changes ( 12 )

About SourceForge

Find Software

Develop Software

Community

Help