Re: [Htmlunit-user] executing javascript which itself is dynamically loaded

[Alamgir F. <afa...@ho...>]

Thank you for your reply.  I changed the 'wait' to after the call. I can see that some of the javascript is executed as the 'title' element changes to "Le Food Truck" However, we seem to be hitting errors and I could not debug. 

Our site is optimized for fast loading on a phone, so we have our initial javascript run and load more javascript css. Basically, the page loads from https://appointment-service.com  and the javascript downloads the rest of the data via REST api calls to https://communitylive.ws . Pretty standard stuff. 

The farthest I could go was with the following configuration. 
-------------------
    final WebClient webClient = new WebClient(
                    BrowserVersion.FIREFOX_31);

            webClient
                    .setAjaxController(new NicelyResynchronizingAjaxController());
            webClient.getOptions().setCssEnabled(false);
            webClient.getOptions().setThrowExceptionOnFailingStatusCode(true);
            webClient.getOptions().setThrowExceptionOnScriptError(true);
            webClient.getOptions().setRedirectEnabled(false);
            webClient.getOptions().setAppletEnabled(false);
            webClient.getOptions().setJavaScriptEnabled(true);
            webClient.getOptions().setPopupBlockerEnabled(true);
            webClient.getOptions().setPrintContentOnFailingStatusCode(true);
            webClient.getOptions().setUseInsecureSSL(true);

            HtmlPage page = webClient
                    .getPage("https://appointment-service.com/#cmt2_lefoodtruck");
            webClient.waitForBackgroundJavaScript(10000);
            webClient.waitForBackgroundJavaScriptStartingBefore(10000);
            // just wait
            try {
                for (int i = 0; i < 20; i++) {
                    synchronized (page) {
                        page.wait(100);
                    }
                }
            } catch (Throwable e) {
                e.printStackTrace();
            }
            System.out.println(page.asXml());
            webClient.close();
----------------

Now I get errors like javascript function initializePage() not found along with a few other errors.  (below). We can reorder the loading of scripts and have an un-optimized version if necessary, only for creating snapshots.  But we are still not being able to get the whole page to work. 

Thanks, -Alamgir

---------- errors ---------
08:54:54,677  WARN IncorrectnessListenerImpl:38 – Obsolete content type encountered: 'text/javascript'.
08:54:56,166 ERROR StrictErrorReporter:80 – runtimeError: message=[An invalid or illegal selector was specified (selector: '*,:x' error: Invalid selector: *:x).] sourceName=[https://appointment-service.com/scripts/main.0280c46b5c658e80.js] line=[1] lineSource=[null] lineOffset=[0]
08:54:56,404  WARN HtmlScript:468 – Script is not JavaScript (type: text/vbscript, language: ). Skipping execution.
08:54:57,606 ERROR JavaScriptJobManagerImpl:418 – Job run failed with unexpected RuntimeException: TypeError: Cannot find function initializePage in object [object Object]. (https://appointment-service.com/scripts/main.0280c46b5c658e80.js#1)
======= EXCEPTION START ========
EcmaError: lineNumber=[1] column=[0] lineSource=[  function (e, i) {] name=[TypeError] sourceName=[https://appointment-service.com/scripts/main.0280c46b5c658e80.js] message=[TypeError: Cannot find function initializePage in object [object Object]. (https://appointment-service.com/scripts/main.0280c46b5c658e80.js#1)]
com.gargoylesoftware.htmlunit.ScriptException: TypeError: Cannot find function initializePage in object [object Object]. (https://appointment-service.com/scripts/main.0280c46b5c658e80.js#1)

> Date: Wed, 22 Apr 2015 15:04:54 +0200
> From: mgu...@ya...
> To: htm...@li...
> Subject: Re: [Htmlunit-user] executing javascript which itself is dynamically loaded
> 
> Hi,
> 
> your code doesn't let HtmlUnit time to do anything with the page as you
> close it too early. I would rather try:
> 
> final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_31);
> HtmlPage page =
> webClient.getPage("http://appointment-service.com/cmt2_lefoodtruck");
> 
> webClient.waitForBackgroundJavaScript(40000);
> // or Thread.sleep(10000); // in case the problem comes from
> waitForBackgroundJavaScript
> System.out.println(page.asXml());
> webClient.close();
> 
> 
> Cheers,
> Marc.
> -- 
> HtmlUnit support & consulting from the source
> Blog: http://mguillem.wordpress.com
> 
> Le 21/04/2015 12:29, Alamgir Farouk a écrit :
> > our site (test) is
> > http://appointment-service.com/cmt2_lefoodtruck
> > (ignore the names, they have no meaning)
> > 
> > In order to optimize loading on a mobile phone, we download the bulk of
> > they javascript and css dynamically, via a small javascript function. I
> > am not sure if this is the problem or something else is wrong, but when
> > I try to load this site using HtmlUnit, it appears that the javascript
> > is not being executed.
> > (java excerpt)
> >             final WebClient webClient = new
> > WebClient(BrowserVersion.INTERNET_EXPLORER_11);
> >             webClient.waitForBackgroundJavaScript(40000);
> >             HtmlPage page =
> > webClient.getPage("http://appointment-service.com/cmt2_lefoodtruck");
> >             System.out.println(page.asXml());
> >             webClient.close();
> > 
> > What are our options?  We are trying to create html snapshots for Google
> > indexing.
> > 
> > Thanks
> 
> 
> 
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user