When attempting to crawl a HTTPS site (facebook) - no content was received. I found the issue to be the socket failing to be created - is this a known issue? Is there any workaround?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for the quick response. I just checked, and I am indeed using PHP 5.6 (5.6.2). I'll try rolling back to an earlier version now and report my results back, but it sounds like that should resolve the issue.
Thank you!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I was having massive Problems crawling SSL-Sites after a PHP-version update. After a bit of hacking I found the Error: The socket was returning 'Host unreachable'
Solution:
in PHPCrawlerHTTPRequest.class.php Row 551:
Replace 'SNI_server_name' with 'peer_name'
SNI_server_name is not supported by PHP > 5.6.0.
Works like a charm now, hope this helps someone else. Thank you, Uwe, for your wonderful tool..
Peter
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
Anonymous
-
2020-03-19
Ohhh my god!!!! More than 10 hours searching and more than a dozen of parameters trying to find a workaround and the most simple way was the answer. Thank you!!!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you so much Peter!!!
Just saved my butt with the above suggestion. I expect your post will be worth gold when people start to upgrade to php7!
-Val
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
When attempting to crawl a HTTPS site (facebook) - no content was received. I found the issue to be the socket failing to be created - is this a known issue? Is there any workaround?
Hi!
What php version are you using?
5.6x?
If so, there is a known bug: https://sourceforge.net/p/phpcrawl/bugs/86/
Before php 5.6 it should work without any (known) problems.
Did you install OpenSSH (it's required for ssl-connections).
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Hey there!
Thank you for the quick response. I just checked, and I am indeed using PHP 5.6 (5.6.2). I'll try rolling back to an earlier version now and report my results back, but it sounds like that should resolve the issue.
Thank you!
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
For anyone looking at this, I can confirm that PHP 5.6.x was the problem (5.6.2 for me). Dropping back a version everything seems to work fine!
Hi,
thanks for replying back and the confirmation!
This bug (https and php 5.6) will get fixed in the next release.
If anybody needs a temporary workaround for phpcrawl version 0.83, please let me know
(here or in the bugreport).
Thanks!
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
I was having massive Problems crawling SSL-Sites after a PHP-version update. After a bit of hacking I found the Error: The socket was returning 'Host unreachable'
Solution:
in PHPCrawlerHTTPRequest.class.php Row 551:
Replace 'SNI_server_name' with 'peer_name'
SNI_server_name is not supported by PHP > 5.6.0.
Works like a charm now, hope this helps someone else. Thank you, Uwe, for your wonderful tool..
Peter
Ohhh my god!!!! More than 10 hours searching and more than a dozen of parameters trying to find a workaround and the most simple way was the answer. Thank you!!!
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Thank you so much Peter!!!
Just saved my butt with the above suggestion. I expect your post will be worth gold when people start to upgrade to php7!
-Val
PHP7 support here: https://github.com/crispy-computing-machine/phpcrawl