A question about how PHPCrawl works - if it finds the same src (or href) on multiple pages does it make a request to that URL multiple times? For example, if I have two pages which both feature the same missing image, does PHPCrawl follow the first src, get a 404, then request the same src a second time, when it finds it on a second page, or does it remember that URL is a 404 and record it as a 404 the second time without requesting it?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
A question about how PHPCrawl works - if it finds the same src (or href) on multiple pages does it make a request to that URL multiple times? For example, if I have two pages which both feature the same missing image, does PHPCrawl follow the first src, get a 404, then request the same src a second time, when it finds it on a second page, or does it remember that URL is a 404 and record it as a 404 the second time without requesting it?