Menu

#2018 Not Able to Select drop down into input field

2.35.0
open
nobody
None
1
2019-07-16
2019-06-25
No

Hi There,

I am new to use HTMLUnit and I am working on a POC.

There is a website where if you will click on "ENTER A DELIVERY ADDRESS", it will show you a form. When you will start typing, it will make ajax calls to google and suggest you addresses .
I am trying to use an address and select the suggested address in this website but I am not able to select the drop down even if I am able to click the first child.

My sample code is as below

    WebClient webClient = new WebClient(browserVersion);

    WebClientOptions webClientOption = webClient.getOptions();
    webClientOption.setCssEnabled(false);
    webClientOption.setJavaScriptEnabled(true);
    webClientOption.setUseInsecureSSL(true);
    webClientOption.setTimeout(30000);
    webClientOption.setThrowExceptionOnScriptError(false);
    webClientOption.setThrowExceptionOnFailingStatusCode(false);;
    webClientOption.setPopupBlockerEnabled(false);
    webClientOption.setRedirectEnabled(true);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());

    String searchUrl = "https://drizly.com/cart";

    WebRequest request = new WebRequest(new URL(searchUrl));

     HtmlPage currentPage = webClient.getPage(request);
     HtmlElement element = page.getFirstByXPath("/html/body/div[1]/header/div[2]/div/address/p");
         page = element.click();

          HtmlElement element = page.getFirstByXPath("//*[@id="address_field"]");
          try {
            ((HtmlTextInput) element).click();
            ((HtmlTextInput) element).type("457 Hudson Street, New York, NY, USA");
        } catch (IOException e) {
            e.printStackTrace();
        }

                    try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

     try {
        HtmlDivision firstChild = page.getFirstByXPath("/html/body/div[7]/div");
        page =  firstChild.click();
    } catch (IOException e) {
        e.printStackTrace();
    }

    try {
        Thread.sleep(5000);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
         System.out.println(page.asXml());

     I have attached the sample output html file which I am getting after running the above code.

Please help me in finding out the issue with the code. Thanking you in advance.

Regards,
Abhishek Thakur

1 Attachments

Discussion

  • Abhishek Thakur

    Abhishek Thakur - 2019-06-26

    Hi There,

    I see the main issue is with selecting a dropdown in different div to an autocomeplete field and then some javascript runs which changes other fields in the page.

    Regards,
    Abhishek Thakur

     
  • Abhishek Thakur

    Abhishek Thakur - 2019-06-27

    Hi There,

    Please find below runnable code for the above issue

    package com.clavis.harvest.scrape.scrape;

    import com.gargoylesoftware.htmlunit.*;
    import com.gargoylesoftware.htmlunit.html.HtmlDivision;
    import com.gargoylesoftware.htmlunit.html.HtmlElement;
    import com.gargoylesoftware.htmlunit.html.HtmlPage;
    import com.gargoylesoftware.htmlunit.html.HtmlTextInput;

    import java.io.IOException;
    import java.net.URL;

    public class test {

    public static void main(String[] args) throws IOException {
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
    
        WebClientOptions webClientOption = webClient.getOptions();
        webClientOption.setCssEnabled(false);
        webClientOption.setJavaScriptEnabled(true);
        webClientOption.setUseInsecureSSL(true);
        webClientOption.setTimeout(30000);
        webClientOption.setThrowExceptionOnScriptError(false);
        webClientOption.setThrowExceptionOnFailingStatusCode(false);
        ;
        webClientOption.setPopupBlockerEnabled(false);
        webClientOption.setRedirectEnabled(true);
        webClient.setAjaxController(new NicelyResynchronizingAjaxController());
    
        String searchUrl = "https://drizly.com/cart";
    
        WebRequest request = new WebRequest(new URL(searchUrl));
    
        HtmlPage currentPage = webClient.getPage(request);
        HtmlElement element = currentPage.getFirstByXPath("/html/body/div[1]/header/div[2]/div/address/p");
        currentPage = element.click();
    
        element = currentPage.getFirstByXPath("//*[@id=\"address_field\"]");
        try {
            ((HtmlTextInput) element).click();
            ((HtmlTextInput) element).type("457 Hudson Street, New York, NY, USA");
        } catch (IOException e) {
            e.printStackTrace();
        }
    
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    
        try {
            HtmlDivision firstChild = currentPage.getFirstByXPath("/html/body/div[7]/div");
            currentPage = firstChild.click();
        } catch (IOException e) {
            e.printStackTrace();
        }
    
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println(currentPage.asXml());
    }
    

    }

    Regards,
    Abhishek Thakur

     
  • RBRi

    RBRi - 2019-07-11

    Will have a look at this.

     
  • Abhishek Thakur

    Abhishek Thakur - 2019-07-12

    Thanks @rbri , please share your findings.

    Also, I am facing same issues with other website where click on selected option doesn't change anything. Website example is www.metro.fr .

    Regards,
    Abhishek Thakur

     
  • Abhishek Thakur

    Abhishek Thakur - 2019-07-12

    Also I have noticed that these sites are built in React.

    Regards,
    Abhishek Thakur

     
  • Abhishek Thakur

    Abhishek Thakur - 2019-07-16

    Hi @rbri,

    Just to let you know I am not able to set locations in below sites
    1. Amazon.com
    2. www.grocery.walmart.com
    3. www.metro.fr
    4.https://groceries.asda.com/?cmpid=ahc--ghs--asdacom--hp--nav-_-ghs
    5. https://www.carrefour.es/supermercado/
    6. https://www.lowes.com/

    I see mostly it is due to JavaScript errors in the log. Please help me in getting one or two sites to use location as the POC, which I am working on, to use HTMLUnit is going to fail.

    Regards,
    Abhishek Thakur

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.