#2018 Not Able to Select drop down into input field

[Abhishek Thakur]

Hi There,

I am new to use HTMLUnit and I am working on a POC.

There is a website where if you will click on "ENTER A DELIVERY ADDRESS", it will show you a form. When you will start typing, it will make ajax calls to google and suggest you addresses .
I am trying to use an address and select the suggested address in this website but I am not able to select the drop down even if I am able to click the first child.

My sample code is as below

    WebClient webClient = new WebClient(browserVersion);

    WebClientOptions webClientOption = webClient.getOptions();
    webClientOption.setCssEnabled(false);
    webClientOption.setJavaScriptEnabled(true);
    webClientOption.setUseInsecureSSL(true);
    webClientOption.setTimeout(30000);
    webClientOption.setThrowExceptionOnScriptError(false);
    webClientOption.setThrowExceptionOnFailingStatusCode(false);;
    webClientOption.setPopupBlockerEnabled(false);
    webClientOption.setRedirectEnabled(true);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());

    String searchUrl = "https://drizly.com/cart";

    WebRequest request = new WebRequest(new URL(searchUrl));

     HtmlPage currentPage = webClient.getPage(request);
     HtmlElement element = page.getFirstByXPath("/html/body/div[1]/header/div[2]/div/address/p");
         page = element.click();

          HtmlElement element = page.getFirstByXPath("//*[@id="address_field"]");
          try {
            ((HtmlTextInput) element).click();
            ((HtmlTextInput) element).type("457 Hudson Street, New York, NY, USA");
        } catch (IOException e) {
            e.printStackTrace();
        }

                    try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

     try {
        HtmlDivision firstChild = page.getFirstByXPath("/html/body/div[7]/div");
        page =  firstChild.click();
    } catch (IOException e) {
        e.printStackTrace();
    }

    try {
        Thread.sleep(5000);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
         System.out.println(page.asXml());

     I have attached the sample output html file which I am getting after running the above code.

Please help me in finding out the issue with the code. Thanking you in advance.

Regards,
Abhishek Thakur

[Abhishek Thakur]

Hi There,

I see the main issue is with selecting a dropdown in different div to an autocomeplete field and then some javascript runs which changes other fields in the page.

Regards,
Abhishek Thakur

[Abhishek Thakur]

Hi There,

Please find below runnable code for the above issue

package com.clavis.harvest.scrape.scrape;

import com.gargoylesoftware.htmlunit.*;
import com.gargoylesoftware.htmlunit.html.HtmlDivision;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlTextInput;

import java.io.IOException;
import java.net.URL;

public class test {

public static void main(String[] args) throws IOException {
    WebClient webClient = new WebClient(BrowserVersion.CHROME);

    WebClientOptions webClientOption = webClient.getOptions();
    webClientOption.setCssEnabled(false);
    webClientOption.setJavaScriptEnabled(true);
    webClientOption.setUseInsecureSSL(true);
    webClientOption.setTimeout(30000);
    webClientOption.setThrowExceptionOnScriptError(false);
    webClientOption.setThrowExceptionOnFailingStatusCode(false);
    ;
    webClientOption.setPopupBlockerEnabled(false);
    webClientOption.setRedirectEnabled(true);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());

    String searchUrl = "https://drizly.com/cart";

    WebRequest request = new WebRequest(new URL(searchUrl));

    HtmlPage currentPage = webClient.getPage(request);
    HtmlElement element = currentPage.getFirstByXPath("/html/body/div[1]/header/div[2]/div/address/p");
    currentPage = element.click();

    element = currentPage.getFirstByXPath("//*[@id=\"address_field\"]");
    try {
        ((HtmlTextInput) element).click();
        ((HtmlTextInput) element).type("457 Hudson Street, New York, NY, USA");
    } catch (IOException e) {
        e.printStackTrace();
    }

    try {
        Thread.sleep(5000);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }

    try {
        HtmlDivision firstChild = currentPage.getFirstByXPath("/html/body/div[7]/div");
        currentPage = firstChild.click();
    } catch (IOException e) {
        e.printStackTrace();
    }

    try {
        Thread.sleep(5000);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    System.out.println(currentPage.asXml());
}

}

Regards,
Abhishek Thakur

[RBRi]

Will have a look at this.

[Abhishek Thakur]

Thanks @rbri , please share your findings.

Also, I am facing same issues with other website where click on selected option doesn't change anything. Website example is www.metro.fr .

Regards,
Abhishek Thakur

[Abhishek Thakur]

Also I have noticed that these sites are built in React.

Regards,
Abhishek Thakur

[Abhishek Thakur]

Hi @rbri,

Just to let you know I am not able to set locations in below sites
1. Amazon.com
2. www.grocery.walmart.com
3. www.metro.fr
4.https://groceries.asda.com/?cmpid=ahc--ghs--asdacom--hp--nav-_-ghs
5. https://www.carrefour.es/supermercado/
6. https://www.lowes.com/

I see mostly it is due to JavaScript errors in the log. Please help me in getting one or two sites to use location as the POC, which I am working on, to use HTMLUnit is going to fail.

Regards,
Abhishek Thakur