Menu

#2013 xpath returning more content than belongs in HtmlDivision

2.34.1
closed
RBRi
None
1
2019-03-30
2019-03-30
arya
No

I'm getting unussual results

I have the following HTMLUnit script which is supposed to extract the zipcode hrefs.

WebClient webClient = new WebClient();
webClient.getOptions().setJavaScriptEnabled(false);

HtmlPage countyPage = webClient.getPage("https://www.realtor.com/propertyrecord-search/Autauga-County_AL");

HtmlDivision zipCodesDiv = (HtmlDivision) countyPage
        .getByXPath("//h3[contains(., \"Zip Codes\")]/following::div[1]").get(0);

System.out.println(zipCodesDiv.asXml());

List<HtmlAnchor> zipcodeLinks = zipCodesDiv.getByXPath("//li/a[contains(@href,'propertyrecord-search/')]");

System.out.println("Zipcode link size: " + zipcodeLinks.size());
for (int zipcodeCount = 0; zipcodeCount < zipcodeLinks.size(); zipcodeCount++) {
    System.out.println(zipcodeLinks.get(zipcodeCount));
}

zipCodesDiv contains the div that contains the zipcodes, however

zipCodesDiv.getByXPath("//li/a[contains(@href,'propertyrecord-search/')]");

is returning more content than it belongs in zipCodesDiv, it's also returning cities in Autauga County. Why is this happening? Is this a bug?

Discussion

  • RBRi

    RBRi - 2019-03-30

    Your second XPath is wrong:

    // addresses all subnodes of the document node use .// instead

            List<HtmlAnchor> zipcodeLinks = zipCodesDiv.getByXPath(".//li/a[contains(@href,'propertyrecord-search/')]");
    
     
  • RBRi

    RBRi - 2019-03-30
    • status: open --> closed
    • assigned_to: RBRi
     

Log in to post a comment.