[Archive-access-discuss] Improvements to proxy-mode in the Open Wayback

[Oskar G. <osk...@kb...>]

Hi everyone!

Let me first introduce me to those of you who don't know me already.
My name is Oskar Grenholm and I work as a programmer at The National Library 
of Sweden. I mainly work with things related to our web archive here.

Lately I have made some minor improvemtents to the way the proxy-mode works in 
the Open Wayback Machine. Those changes have made it possible to surf not 
only the most recent copy of a page in the web archive, but instead any copy 
available. 
This can be done with just the Wayback Machine, but to aid (and perhaps 
simplify) the surfing I have also started working on a Firefox extension that 
will help the user with common tasks often encountered when surfing a web 
archive. Among the things this WAX Toolbar does is providing a search field 
for searching the Wayback Machine for different URL:s OR do a full-text 
search from a NutchWAX index (if one is available of course). You can also 
use the toolbar to switch between proxy-mode and the regular Internet, and 
when in proxy-mode easily go back and forth in time.

The changes made to the Wayback are not many. The main idea is that you have a 
BDB index that holds mappings between id:s (a unique id if the toolbar was 
used, otherwise the ip-address the request was made from) and a preferred 
time to surf at. This timestamp is set either when you choose a page to visit 
from the search interface in the WB or by the WAX Toolbar.
Then for each request made to the proxy the WB will look up this timestamp and 
return the page that is the closest in time.

Patches for these changes are attached to this e-mail. Four of the files are 
earlier existing files that have been modified somewhat and two of them are 
new (BDBMapper.java and Redirect.jsp).

Attached is also a tar-file containing the source for the Firefox extension.
If you untar this and enter the directory you can just run 'ant' and a file 
named WaxToolbar.xpi will be built. That is the actual Firefox extension and 
it can be installed as any other extension (i,e,. double-clicking it from 
within Firefox).
When the extension is installed (and after a re-start of Firefox) a new 
toolbar will be there. In the Tools menu there will also be a WAX Toolbar 
Configuration option. Using this you can set the proxy to use (the WB) and a 
server running NutchWAX.

Finally I have attached an example of a web.xml that can be used when running 
the WB with these new changes and the WAX Toolbar. In it some new stuff has 
been added, namely a parameter specifying the redirect path (the Redirect.jsp 
mentioned above) and a servlet called xmlquery that runs in parallell with 
the normal query interface and is used by the extension to find the times a 
page has been archived.

So, let the feedback begin!

Regards, Oskar.