I would like to crawl a website and retrieve two values (for example the contents of div 1 and div 2) for each url and export these 3 fields into a csv, each line corresponding to a url and these two values.
How can this be achieved ?
Thank you in advance for your help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Hello,
I would like to crawl a website and retrieve two values (for example the contents of div 1 and div 2) for each url and export these 3 fields into a csv, each line corresponding to a url and these two values.
How can this be achieved ?
Thank you in advance for your help.
Hi,
Sure you can. Use phpcrawl to generate the URLs, then write a script of your own to parse the pages for the data you want.
I would suggest using the HTML DOM:
$html = str_get_html($file_contents);
$elem = $html->find('div[id=content]', 0);