I'm getting unussual results
I have the following HTMLUnit script which is supposed to extract the zipcode hrefs.
WebClient webClient = new WebClient();
webClient.getOptions().setJavaScriptEnabled(false);
HtmlPage countyPage = webClient.getPage("https://www.realtor.com/propertyrecord-search/Autauga-County_AL");
HtmlDivision zipCodesDiv = (HtmlDivision) countyPage
.getByXPath("//h3[contains(., \"Zip Codes\")]/following::div[1]").get(0);
System.out.println(zipCodesDiv.asXml());
List<HtmlAnchor> zipcodeLinks = zipCodesDiv.getByXPath("//li/a[contains(@href,'propertyrecord-search/')]");
System.out.println("Zipcode link size: " + zipcodeLinks.size());
for (int zipcodeCount = 0; zipcodeCount < zipcodeLinks.size(); zipcodeCount++) {
System.out.println(zipcodeLinks.get(zipcodeCount));
}
zipCodesDiv contains the div that contains the zipcodes, however
zipCodesDiv.getByXPath("//li/a[contains(@href,'propertyrecord-search/')]");
is returning more content than it belongs in zipCodesDiv, it's also returning cities in Autauga County. Why is this happening? Is this a bug?
Your second XPath is wrong:
// addresses all subnodes of the document node use .// instead