Share

Heritrix: Internet Archive Web Crawler

Tracker: Feature Requests

5 add 'exhausted' queue count to frontier report - ID: 1200205
Last Update: Comment added ( karl-ia )

Current frontier report has no spot for 'finished'
queues -- those that had been created but currently
have no items. This probably traces back to (1)
previous frontiers didn't keep these empty queues
around; (2) they're not guaranteed finished, just
'finished as far as we know right now, pending future
discoveries'.

Now with 1.4's bdb-backed map we can have those empty
queues paged out to disk, so can keep them around. (You
can deduce how many there are in current reports by
taking total queues and subtracting
active+inactive+retired.)

So, there should be a readout of how many are in this
state. But, let's call them 'exhausted' to avoid any
unintended implication that the queue is done for good.


Gordon Mohr ( gojomo ) - 2005-05-11 22:01

5

Closed

None

Gordon Mohr

None

1.6.0

Public


Comments ( 2 )

Date: 2007-03-14 01:41
Sender: karl-ia


This issue is now discussed in the new JIRA tracker at
http://webteam.archive.org/jira/browse/HER-926 -- please add further
comments at that location.


Date: 2005-05-13 04:02
Sender: gojomoProject Admin

Logged In: YES
user_id=144912

Added. Commit comment:

fix for [ 1119644 ] frontier report
ConcurrentModificationException
implementation of [ 1200205 ] add 'exhausted' queue count to
frontier report
* WorkQueueFrontier.java
- have report work from copy of inProcessBag -- will
eliminate specific ConcurrentModificationException reported
- add 'exhausted' queue count (created but now empty) to
one-line and full reports



Attached File

No Files Currently Attached

Changes ( 3 )

Field Old Value Date By
artifact_group_id None 2005-09-23 21:08 gojomo
close_date - 2005-05-13 04:02 gojomo
status_id Open 2005-05-13 04:02 gojomo