I am scraping a web with HtmlUnit.
Until yesterday I made click in an anchor that loads into the page new data to read the html generated. But now I don´t know why when I make click with my code I can´t get the new html generated. The html before and after the click is the same.
I tried the anchor link into my browser and it works there.
Here is my code
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(java.util.logging.Level.OFF);
String searchQuery = "BetFair" ;
WebClient client = new WebClient(BrowserVersion.CHROME);
client.getOptions().setCssEnabled(false);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(false);
String searchUrl = "https://www.betfair.es/sport/football";
HtmlPage page = client.getPage(searchUrl);
System.out.println(page.asText());
HtmlDivision marketWrapper =(HtmlDivision) page.getFirstByXPath("//div[@class='markets-wrapper']");
HtmlAnchor htmlAnchor = marketWrapper.getFirstByXPath("//a[@title='La Liga - España']");
HtmlPage leaguePage = htmlAnchor.click();
HtmlElement containerElement = leaguePage.getHtmlElementById("zone-main");
HtmlUnorderedList unOrderedList = containerElement.getFirstByXPath("//ul[@class='event-list']");
String teamsString;
String _1;
String _X;
String _2;
System.out.println(leaguePage.asText());
Will have a look....