There has been a lot of development lately in the 'new' subdirectory (it has a separate ./configure). One of the biggest changes is to use mysql for most of the inverted index, with BerkeleyDB still used for smaller indices. This provides a lot of query flexibility. libwww is integrated as the parser for HTML documents. A release is coming soon, as part of this year's TREC efforts.