Re: [htdig-dev] Re: RFC on 3.2 indexing v. hopcount

[Gilles D. <gr...@sc...>]

According to Geoff Hutchison:
> I had a brief brainstorm on my run today as far as profiling the 
> indexing. Obviously htword/mifluz performance still needs to improve 
> significantly. But another slowdown relative to 3.1 is from the way 3.2 
> treats hopcounts. To ensure that restricting indexes by hopcount works 
> correctly, the "queue" for URLs is really a priority queue. URLs with 
> lower hopcounts move up the heap. Of course this requires some sorting 
> and some overhead.
> 
> Right now, I don't think this needs to happen *unless* we're restricting 
> indexing based on hopcount. So the proposal is that when we're not 
> restricting by hopcount, the Server objects would switch back to the 
> previous system (i.e. no sorting).
> 
> I think this should shave a few percent off of indexing. Does this seem 
> like an OK idea? Can anyone come up with an example where this would be 
> a Bad Idea(tm)?

I can't think of a problem offhand.  Sounds reasonable to me.  Of course,
you probably understand this aspect of the code better than any of us.

-- 
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)