Arachnid Web Spider Framework Now Available

The Arachnid Web Spider Framework is now available on SourceForge. The Arachnid Framework is a Java-based framework for writing simple Web spiders. Arachnid uses a simple HTML parser to read web pages and then "walk" through a Web site. Arachnid is an abstract class that makes use of the "visitor" design pattern. Developers merely create a sub-class of Arachnid, implement a few simple methods, and then call the "traverse" method that walks through a Web site. Three example applications are included with Arachnid. In one case, a simple "Site Map" generator is implemented in less than sixty lines of code using Arachnid.

Posted by Robert Platt 2002-10-05