I just examined the probmen with the site you mentioned.
There are two problems here:
The site sends a (very seldom) "Refresh:"-http-header with the first response.
(like "Refresh: 8;URL=/cdn-cgi/l/chk_jschl?pass=1434594773.725-X6AvZ/n2NF")
This isn't supported by phpcrawl (yet). But i'll open a bugreport and it will be
supported in the next release.
THe site uses a seperate site for setting an authentication-cookie, that's a real problem.
There's already a bugport for this, please see here for further details: https://sourceforge.net/p/phpcrawl/bugs/34/
We try to find a solution for this.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
now there are many pages with CloudFlare told you should not get the crawler, eg kissanime.com page, I do not get content.help me, please
Hi!
I just examined the probmen with the site you mentioned.
There are two problems here:
The site sends a (very seldom) "Refresh:"-http-header with the first response.
(like "Refresh: 8;URL=/cdn-cgi/l/chk_jschl?pass=1434594773.725-X6AvZ/n2NF")
This isn't supported by phpcrawl (yet). But i'll open a bugreport and it will be
supported in the next release.
THe site uses a seperate site for setting an authentication-cookie, that's a real problem.
There's already a bugport for this, please see here for further details:
https://sourceforge.net/p/phpcrawl/bugs/34/
We try to find a solution for this.
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
so thank you for your interest.
please support the results
Tks all.