[Archive-access-discuss] [Fwd: Re: Search multiple versions of one URL - working!]

[James G. <jg...@si...>]

Forwarded by St.Ack's request.

Michael Stack wrote:
 > I'm glad its working for you now.  Suggestions for improving doc. so
 > others don't fall into your little wormhole?
 > Thanks James,
 > St.Ack

The "Getting Started" document was great for initial testing of the  system.

I had a problem with Hadoop early, but I was using the hadoop that came 
packaged with nutch 0.8.1... which turned out to be version 0.4.   I had 
assumed incorrectly that nutch itself would be using a recent version of 
hadoop.

When I wanted to begin working with WERA and keep multiple versions of a 
page around, however, my resources were: St.Ack's response to someone 
else on this list, the bug report about keeping multiple versions of a 
webpage, and revisiting the "Getting Started" document (since it 
contained the listing of commands in order).

So I'd say a guide outlining the steps to take to preserve multiple 
versions of a webpage would have been a plus.

Current documentation about how to do incremental indexing would be nice 
too, as this is something I'll be working on soon (I suppose the old FAQ 
solution applies?).

Outside of documentation, most of my desires from Heritrix/NutchWAX/WERA
would be for automation and integration:
- I'm looking forward to the automatic recrawling that I've seen on the 
roadmap of Heritrix.
- A non-manual way of importing new crawls from Heritrix to NutchWAX 
would be desirable
- It would have been really nice if WERA was Tomcat friendly, so WERA, 
NutchWAX, ArcRetriever, and even Heritrix could coexist on one server.
- It would have also been nice if ArcRetriever had the same args as the 
wayback machine, so that either could be used with NutchWAX.   (though 
perhaps they are compatible and I missed it?)

But, as I said, I realize that most of these tools are pre-version-1.0, 
and I'm happy that they're around to begin with.

jamesG