#6 specify files

open
nobody
None
5
2005-08-21
2005-08-21
Anonymous
No

Ability to specify files and have links crawled from
there. This is important for dynamic sites with rewrites.
The links are to static files, but they technically don't
exist on the server for programs to crawl.

Discussion