[Htmlunit-user] HtmlUnit Support for JavaScript

[Satyanarayana K <kri...@ya...>]

Hi,

I am aware that HtmlUnit is perhaps the only open-source Library for HTML Parsing (Java or Non-Java) that provides native support for javaScript execution. However, I wish to highlight certain aspects for discussion / my own clarification. 

This posting would perhaps be best addressed by the HtmlUnit Development Team:

1. I am somewhat confused by a response that is often posted against some of the issues reported - the response that says the problem or issue would be solved by setting WebClient - javaScriptEnabled property to false. My question is: a. If the page content is driven by javaScript, how does setting WebClient - javaScriptEnabled property to false help or solve the problem? b. If the page content is NOT driven by javaScript, why should the WebClient - javaScriptEnabled property setting matter at all?

2.  I have experienced frequent difficulties in loading javaScript-driven pages, esp those that involved cross-domain-messaging. The expected procedure for reporting javaScript bugs (isolating minimal test case) appears very complex to me as I have very little exposure to or experience with javaScript. I have been able to identify that the issues are traceable to cross-domain-messaging. I would like to know, quite simply: Does HtmlUnit execute cross-domain-messaging calls? This cross-domain-messaging appears to be getting pervasive,  a feature that deserves to be supported by HtmlUnit if not already supported.

3. In the documentation of Selenium WebDriver, I have seen that the javaScript engine used by HtmlUnit is the one that was used by FireFox version 3.6 or thereabouts (The default instantiation of HtmlUnitDriver under Selenium is without javaScript enabled). Today FireFox is in version 24 or so. It appears the javaScript engine used by HtmlUnit is obsolete technology. Is this the reason for the cross-domain-messaging problems?

I use HtmlUnit in web-scraping applications. The above issues happen to be critical for such applications.

Thanks for your time,
Satya