As I mentioned on the previous developers call, I've been working on using VuFind to index websites (in order to replace the current Web module which is based on a deprecated Google API). I've just removed some Villanova-specific code from my work so far and posted a generic version to JIRA:
This requires a bit of customization to become really useful (you need to modify the indexer to create facets based on your local website structure - there's really no way to do that in a totally generic fashion), but basic functionality is pretty easy to get up and running. The only requirements are that you install Aperture and have a website which generates standard sitemap.xml files.
I'd be interested to hear what people think of this!