Hi, I am new to Web Harvester, but quite an experienced C++/C# developer. I have been trying to scrape data from http://www.racingpost.com using C# and the microsoft web browser control. The snag that I have hit is that the data I need is dispayed on the screen, but does not appear in the DOM - I think this is because there is script execution happening. The data I specifically need to capture is a list of greyhound races, including the details of each dog running in the race. I would also like to capture the greyhound's form and the result of the race after it has been published.
If I look in the element inspector in Chrome, I can see that the data exists, but I am unable to access it programatically using with the standard microsoft browser control or even webkit.
Is this something that I could solve by using Web Harvester?
Many thanks in advance
Neil Jones
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm a part time trader who want scrap info on the internet to modeling data by myself. I recently found this tool and already my basic configuration file to scrap data on racingpost, but i don't know how to build all my scraper. Can you point me some tips?
regards
Luis
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I am new to Web Harvester, but quite an experienced C++/C# developer. I have been trying to scrape data from http://www.racingpost.com using C# and the microsoft web browser control. The snag that I have hit is that the data I need is dispayed on the screen, but does not appear in the DOM - I think this is because there is script execution happening. The data I specifically need to capture is a list of greyhound races, including the details of each dog running in the race. I would also like to capture the greyhound's form and the result of the race after it has been published.
Starting with the race card, an example of which can be found here:
http://www.racingpost.com/greyhounds/card.sd
The above link is a list of all races for a day - each race in the list has a link within it to access the details. Clicking through a link in the list, for example:
http://www.racingpost.com/greyhounds/card.sd#resultDay=2013-04-28&raceId=1225577
shows the details of the race. I need to capture the data about each greyhound and oreferably its racing form data too.
If I look in the element inspector in Chrome, I can see that the data exists, but I am unable to access it programatically using with the standard microsoft browser control or even webkit.
Is this something that I could solve by using Web Harvester?
Many thanks in advance
Neil Jones
yes, Harvester can crap them for you.
Hi,
I'm a part time trader who want scrap info on the internet to modeling data by myself. I recently found this tool and already my basic configuration file to scrap data on racingpost, but i don't know how to build all my scraper. Can you point me some tips?
regards
Luis