Re: [Htmlunit-user] executing javascript which itself is dynamically loaded

[Marc G. <mgu...@ya...>]

Hi,

you should try following with latest release (or latest sources from SVN):
----------------
	   final WebClient webClient = new WebClient(
                   BrowserVersion.FIREFOX_31);
           webClient.getOptions().setUseInsecureSSL(true);

           HtmlPage page = webClient

.getPage("https://appointment-service.com/#cmt2_lefoodtruck");
           webClient.waitForBackgroundJavaScript(10000);
           webClient.waitForBackgroundJavaScriptStartingBefore(10000);
           System.out.println(page.asXml());
           webClient.close();
----------------

It seems not to produce errors (the only one in your log was >TypeError:
Cannot find function initializePage in object [object Object].<, the
rest is only informative)

Cheers,
Marc.
-- 
HtmlUnit support & consulting from the source
Blog: http://mguillem.wordpress.com

Le 22/04/2015 18:02, Alamgir Farouk a écrit :
> Thank you for your reply.  I changed the 'wait' to after the call. I can
> see that some of the javascript is executed as the 'title' element
> changes to "Le Food Truck" However, we seem to be hitting errors and I
> could not debug.
> 
> Our site is optimized for fast loading on a phone, so we have our
> initial javascript run and load more javascript css. Basically, the page
> loads from https://appointment-service.com  and the javascript downloads
> the rest of the data via REST api calls to https://communitylive.ws .
> Pretty standard stuff.
> 
> The farthest I could go was with the following configuration.
> -------------------
>     final WebClient webClient = new WebClient(
>                     BrowserVersion.FIREFOX_31);
> 
>             webClient
>                     .setAjaxController(new
> NicelyResynchronizingAjaxController());
>             webClient.getOptions().setCssEnabled(false);
>            
> webClient.getOptions().setThrowExceptionOnFailingStatusCode(true);
>             webClient.getOptions().setThrowExceptionOnScriptError(true);
>             webClient.getOptions().setRedirectEnabled(false);
>             webClient.getOptions().setAppletEnabled(false);
>             webClient.getOptions().setJavaScriptEnabled(true);
>             webClient.getOptions().setPopupBlockerEnabled(true);
>             webClient.getOptions().setPrintContentOnFailingStatusCode(true);
>             webClient.getOptions().setUseInsecureSSL(true);
> 
>             HtmlPage page = webClient
>                    
> .getPage("https://appointment-service.com/#cmt2_lefoodtruck");
>             webClient.waitForBackgroundJavaScript(10000);
>             webClient.waitForBackgroundJavaScriptStartingBefore(10000);
>             // just wait
>             try {
>                 for (int i = 0; i < 20; i++) {
>                     synchronized (page) {
>                         page.wait(100);
>                     }
>                 }
>             } catch (Throwable e) {
>                 e.printStackTrace();
>             }
>             System.out.println(page.asXml());
>             webClient.close();
> ----------------
> 
> Now I get errors like javascript function initializePage() not found
> along with a few other errors.  (below). We can reorder the loading of
> scripts and have an un-optimized version if necessary, only for creating
> snapshots.  But we are still not being able to get the whole page to work.
> 
> Thanks, -Alamgir
> 
> ---------- errors ---------
> 08:54:54,677  WARN IncorrectnessListenerImpl:38 – Obsolete content type
> encountered: 'text/javascript'.
> 08:54:56,166 ERROR StrictErrorReporter:80 – runtimeError: message=[An
> invalid or illegal selector was specified (selector: '*,:x' error:
> Invalid selector: *:x).]
> sourceName=[https://appointment-service.com/scripts/main.0280c46b5c658e80.js]
> line=[1] lineSource=[null] lineOffset=[0]
> 08:54:56,404  WARN HtmlScript:468 – Script is not JavaScript (type:
> text/vbscript, language: ). Skipping execution.
> 08:54:57,606 ERROR JavaScriptJobManagerImpl:418 – Job run failed with
> unexpected RuntimeException: TypeError: Cannot find function
> initializePage in object [object Object].
> (https://appointment-service.com/scripts/main.0280c46b5c658e80.js#1)
> ======= EXCEPTION START ========
> EcmaError: lineNumber=[1] column=[0] lineSource=[  function (e, i) {]
> name=[TypeError]
> sourceName=[https://appointment-service.com/scripts/main.0280c46b5c658e80.js]
> message=[TypeError: Cannot find function initializePage in object
> [object Object].
> (https://appointment-service.com/scripts/main.0280c46b5c658e80.js#1)]
> com.gargoylesoftware.htmlunit.ScriptException: TypeError: *Cannot find
> function initializePage in object [object Object].*
> (https://appointment-service.com/scripts/main.0280c46b5c658e80.js#1)
> 
> 
> 
>> Date: Wed, 22 Apr 2015 15:04:54 +0200
>> From: mgu...@ya...
>> To: htm...@li...
>> Subject: Re: [Htmlunit-user] executing javascript which itself is
> dynamically loaded
>>
>> Hi,
>>
>> your code doesn't let HtmlUnit time to do anything with the page as you
>> close it too early. I would rather try:
>>
>> final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_31);
>> HtmlPage page =
>> webClient.getPage("http://appointment-service.com/cmt2_lefoodtruck");
>>
>> webClient.waitForBackgroundJavaScript(40000);
>> // or Thread.sleep(10000); // in case the problem comes from
>> waitForBackgroundJavaScript
>> System.out.println(page.asXml());
>> webClient.close();
>>
>>
>> Cheers,
>> Marc.
>> --
>> HtmlUnit support & consulting from the source
>> Blog: http://mguillem.wordpress.com
>>
>> Le 21/04/2015 12:29, Alamgir Farouk a écrit :
>> > our site (test) is
>> > http://appointment-service.com/cmt2_lefoodtruck
>> > (ignore the names, they have no meaning)
>> >
>> > In order to optimize loading on a mobile phone, we download the bulk of
>> > they javascript and css dynamically, via a small javascript function. I
>> > am not sure if this is the problem or something else is wrong, but when
>> > I try to load this site using HtmlUnit, it appears that the javascript
>> > is not being executed.
>> > (java excerpt)
>> > final WebClient webClient = new
>> > WebClient(BrowserVersion.INTERNET_EXPLORER_11);
>> > webClient.waitForBackgroundJavaScript(40000);
>> > HtmlPage page =
>> > webClient.getPage("http://appointment-service.com/cmt2_lefoodtruck");
>> > System.out.println(page.asXml());
>> > webClient.close();
>> >
>> > What are our options? We are trying to create html snapshots for Google
>> > indexing.
>> >
>> > Thanks
>>
>>
>>
>>