htmlunit-user

[Htmlunit-user] How to handle lazyload page with htmlunit

[adam <sh...@qq...>]

i can't access lazyload part.i am nearly crazy.
below is my code:

		WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_11);
		webClient.getOptions().setCssEnabled(false);
		webClient.getOptions().setJavaScriptEnabled(true);
		webClient.setJavaScriptTimeout(99999);
		webClient.waitForBackgroundJavaScript(99999);
		webClient.setAjaxController(new NicelyResynchronizingAjaxController());
		HtmlPage productPage = (HtmlPage)
webClient.getPage("http://detail.1688.com/offer/1228752603.html");
		



--
View this message in context: http://htmlunit.10904.n7.nabble.com/How-to-handle-lazyload-page-with-htmlunit-tp36142.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] How to handle lazyload page with htmlunit

[Ahmed A. <asa...@ya...>]

Hi Adam,
Please try simulating CHROME.
Ahmed      From: adam <sh...@qq...>
 To: htm...@li... 
 Sent: Thursday, May 21, 2015 2:23 PM
 Subject: [Htmlunit-user] How to handle lazyload page with htmlunit
   
i can't access lazyload part.i am nearly crazy.
below is my code:

        WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_11);
        webClient.getOptions().setCssEnabled(false);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.setJavaScriptTimeout(99999);
        webClient.waitForBackgroundJavaScript(99999);
        webClient.setAjaxController(new NicelyResynchronizingAjaxController());
        HtmlPage productPage = (HtmlPage)
webClient.getPage("http://detail.1688.com/offer/1228752603.html");

  

Re: [Htmlunit-user] How to handle lazyload page with htmlunit

[Ahmed A. <asa...@ya...>]

Hi Adam,
Please try simulating CHROME.
Ahmed      From: adam <sh...@qq...>
 To: htm...@li... 
 Sent: Thursday, May 21, 2015 2:23 PM
 Subject: [Htmlunit-user] How to handle lazyload page with htmlunit
   
i can't access lazyload part.i am nearly crazy.
below is my code:

        WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_11);
        webClient.getOptions().setCssEnabled(false);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.setJavaScriptTimeout(99999);
        webClient.waitForBackgroundJavaScript(99999);
        webClient.setAjaxController(new NicelyResynchronizingAjaxController());
        HtmlPage productPage = (HtmlPage)
webClient.getPage("http://detail.1688.com/offer/1228752603.html");

  

Re: [Htmlunit-user] How to handle lazyload page with htmlunit

[adam <sh...@qq...>]

asashour,thanks very much~
i've change my code,but it doesn't work.

WebClient webClient = new WebClient(BrowserVersion.CHROME);
        webClient.getOptions().setCssEnabled(false);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.setJavaScriptTimeout(99999);
        webClient.waitForBackgroundJavaScript(99999);
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        
        webClient.setAjaxController(new
NicelyResynchronizingAjaxController());
        try {
			HtmlPage productPage =
(HtmlPage)webClient.getPage("http://detail.1688.com/offer/1228752603.html");
			HtmlElement dealElement = (HtmlElement)
productPage.getFirstByXPath("//*[@id=\"mod-detail-otabs\"]/ul/li[2]/a/span");
			HtmlElement commentElement = (HtmlElement)
productPage.getFirstByXPath("//*[@id=\"commentbody\"]/dl[1]/dd/div[2]");
			
			if(dealElement!=null)
			{
				System.out.println(dealElement.asXml());
			}
			if(commentElement!=null)
			{
				System.out.println(commentElement.asXml());
			}
			System.out.println(productPage.asText());
        }

dealElement and commentElement  return null;



--
View this message in context: http://htmlunit.10904.n7.nabble.com/How-to-handle-lazyload-page-with-htmlunit-tp36142p36150.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] How to handle lazyload page with htmlunit

[asashour <asa...@ya...>]

It seems the content needs scrolling down.

Trying with productPage.getBody().type(KeyboardEvent.DOM_VK_PAGE_DOWN);
doesn't work.

You need to know which triggers the javascript to fill the data.

Some hints in http://htmlunit.sourceforge.net/submittingJSBugs.html

Ahmed



--
View this message in context: http://htmlunit.10904.n7.nabble.com/How-to-handle-lazyload-page-with-htmlunit-tp36142p36151.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] How to handle lazyload page with htmlunit

[adam <sh...@qq...>]

i've submit the bug to http://htmlunit.sourceforge.net/submittingJSBugs.html.
asashour,thank you a lot.



--
View this message in context: http://htmlunit.10904.n7.nabble.com/How-to-handle-lazyload-page-with-htmlunit-tp36142p36363.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] How to handle lazyload page with htmlunit

[adam <sh...@qq...>]

https://sourceforge.net/p/htmlunit/bugs/1693/



--
View this message in context: http://htmlunit.10904.n7.nabble.com/How-to-handle-lazyload-page-with-htmlunit-tp36142p36364.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.