Hi There,
I am new to use HTMLUnit and I am working on a POC.
There is a website where if you will click on "ENTER A DELIVERY ADDRESS", it will show you a form. When you will start typing, it will make ajax calls to google and suggest you addresses .
I am trying to use an address and select the suggested address in this website but I am not able to select the drop down even if I am able to click the first child.
My sample code is as below
WebClient webClient = new WebClient(browserVersion); WebClientOptions webClientOption = webClient.getOptions(); webClientOption.setCssEnabled(false); webClientOption.setJavaScriptEnabled(true); webClientOption.setUseInsecureSSL(true); webClientOption.setTimeout(30000); webClientOption.setThrowExceptionOnScriptError(false); webClientOption.setThrowExceptionOnFailingStatusCode(false);; webClientOption.setPopupBlockerEnabled(false); webClientOption.setRedirectEnabled(true); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); String searchUrl = "https://drizly.com/cart"; WebRequest request = new WebRequest(new URL(searchUrl)); HtmlPage currentPage = webClient.getPage(request); HtmlElement element = page.getFirstByXPath("/html/body/div[1]/header/div[2]/div/address/p"); page = element.click(); HtmlElement element = page.getFirstByXPath("//*[@id="address_field"]"); try { ((HtmlTextInput) element).click(); ((HtmlTextInput) element).type("457 Hudson Street, New York, NY, USA"); } catch (IOException e) { e.printStackTrace(); } try { Thread.sleep(5000); } catch (InterruptedException e) { e.printStackTrace(); } } try { HtmlDivision firstChild = page.getFirstByXPath("/html/body/div[7]/div"); page = firstChild.click(); } catch (IOException e) { e.printStackTrace(); } try { Thread.sleep(5000); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println(page.asXml()); I have attached the sample output html file which I am getting after running the above code.
Please help me in finding out the issue with the code. Thanking you in advance.
Regards,
Abhishek Thakur
Hi There,
I see the main issue is with selecting a dropdown in different div to an autocomeplete field and then some javascript runs which changes other fields in the page.
Regards,
Abhishek Thakur
Hi There,
Please find below runnable code for the above issue
package com.clavis.harvest.scrape.scrape;
import com.gargoylesoftware.htmlunit.*;
import com.gargoylesoftware.htmlunit.html.HtmlDivision;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlTextInput;
import java.io.IOException;
import java.net.URL;
public class test {
}
Regards,
Abhishek Thakur
Will have a look at this.
Thanks @rbri , please share your findings.
Also, I am facing same issues with other website where click on selected option doesn't change anything. Website example is www.metro.fr .
Regards,
Abhishek Thakur
Also I have noticed that these sites are built in React.
Regards,
Abhishek Thakur
Hi @rbri,
Just to let you know I am not able to set locations in below sites
1. Amazon.com
2. www.grocery.walmart.com
3. www.metro.fr
4.https://groceries.asda.com/?cmpid=ahc--ghs--asdacom--hp--nav-_-ghs
5. https://www.carrefour.es/supermercado/
6. https://www.lowes.com/
I see mostly it is due to JavaScript errors in the log. Please help me in getting one or two sites to use location as the POC, which I am working on, to use HTMLUnit is going to fail.
Regards,
Abhishek Thakur