James Shaver

I have a github repo for phpcrawl that I'm using successfully in PHP8. It's installable by composer, or you can just download and install the package where you want. https://github.com/JamesShaver/phpcrawl

Have you tried hitting the proxy outside of phpcrawl? $url = "https://google.com"; $proxy = "127.0.0.1:1080"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_PROXY, $proxy); curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); curl_setopt($ch, CURLOPT_HEADER, 1); curl_exec($ch); curl_close($ch);

Hi, Sure you can. Use phpcrawl to generate the URLs, then write a script of your own to parse the pages for the data you want. I would suggest using the HTML DOM: $html = str_get_html($file_contents); $elem = $html->find('div[id=content]', 0);

Remember grade school math? It would help us see the issue if you show your work.

That's what I would use too. Not sure you need to specify that a single item is an...

Hi, It depends on what all of your blabla really is, and what you're really trying...

You might want to check that the http code is OK, but otherwise it looks good. function...

Sorry for the late reply... Actually it doesn't ignore pages entirely with query...