I am trying to use web harvest to scrape data from a website after executing a search query. I tried using http-param per the New York Times example to send the query to the server based on the names and values of the search form. I have been unable to do this. Does anyone have any suggestions. Am I on the right track. Thanks in advance for you help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am trying to use web harvest to scrape data from a website after executing a search query. I tried using http-param per the New York Times example to send the query to the server based on the names and values of the search form. I have been unable to do this. Does anyone have any suggestions. Am I on the right track. Thanks in advance for you help.
Hi Johan,
Can you provide more details like page which you want to scrape and request which you've tried to get the search results?
Thanks,
MC