I am using PHPCrawler 0.81 and i am using HideMyAss Free Web proxy to crawl a website but sometime the crawl had stopped working and i checked the error log i saw this "Crawl error43 :Unable to connect to proxy '194.255.66.56' on port '8080'"
Anybody can suggest a retry mode to make crawl work ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Hi !
I am using PHPCrawler 0.81 and i am using HideMyAss Free Web proxy to crawl a website but sometime the crawl had stopped working and i checked the error log i saw this "Crawl error43 :Unable to connect to proxy '194.255.66.56' on port '8080'"
Anybody can suggest a retry mode to make crawl work ?
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Hi!
You could try to increase the connection-timeout, maybe that will help.
http://phpcrawl.cuab.de/classreferences/PHPCrawler/method_detail_tpl_method_setConnectionTimeout.htm
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Hi !
I often set $crawler->setConnectionTimeout(500); but it still get that error
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
hmm, then that's a really strange behaviour of the proxy-server.
Al least it should answer with an error, but it just refuses the connection.
To make it short: No, there is no implementation of request-retries after a failed/refused request in phpcrawl.
But i'll open a feature request for this.
And right now i don't know a simple quick workaround for this, sorry!
Someone else maybe?
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
thank you very much i will try to debug to see more.