I suggest to add cleanup() function call to
net.javacoding.jspider.core.storage.Storage interface,
StorageSPI interface and and all the objects DAOs. The
reason behind it is to clean up storage after spidering
is done. While it's not a problem with MemoryStorage,
JDBC or any future Storage implementations might need
to clean behind itself. Even current JDBC storage
implementation cleans when jspider is started, thus
leaving megabytes of data in the database in between
runs. In the code Storage can be obtained through
jspider.getContext().getStorage() call. The StorageImpl
then can call StorageSPI.cleanup() which in turn call
each object DAO cleanup() method.
I'm preparing a new release of JSpider which will address this issue. Please stay tuned.