#24 Request-retry/retries after failed requests

[Anonymous]

It would be good to have an option to tell the crawler to retry a request for a page/document x times after it has failed (like a temporary "connection refused" or similar).

Also see this forum-post:
http://sourceforge.net/p/phpcrawl/discussion/307696/thread/cd94497b