Menu ▾ ▴

#1041 Can't parse html when using MockWebConnection

2.7
closed
Ahmed Ashour
None
5
2013-12-06
2010-03-06
shirley wilder
No

I have a html page like this:

<title></title> <script type="text/javascript" src="http://localhost/test.js"></script>

When i use webClient.getPage("http://localhost/test.html"),it works well.
But when i use MockWebConnection and set default response using the code above for it,
there's a exception throw out:

EcmaError: lineNumber=[1] column=[0] lineSource=[<no source="">]</no> name=[ReferenceError] sourceName=[http://localhost/test.js] message=[ReferenceError: "XML" is not defined. (http://localhost/test.js#1)]
com.gargoylesoftware.htmlunit.ScriptException: ReferenceError: "XML" is not defined. (http://localhost/test.js#1)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:527)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:537)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:538)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:425)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:980)
at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:353)
at com.gargoylesoftware.htmlunit.html.HtmlScript$1.execute(HtmlScript.java:215)
at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:235)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:718)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:676)
at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1136)
at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1038)
at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:329)
at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:2999)
at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:1991)
at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:895)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:896)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:350)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:304)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:134)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:101)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:420)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:303)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:360)

Discussion

  • Ahmed Ashour

    Ahmed Ashour - 2010-03-08

    Hi Shirley,

    The below case works fine, please provide minimal test case that shows your error.

    @Test
public void test() throws Exception {
    final String html = "<html><head>\n"

        + "<script src='http://localhost/test.js'></script>\n"
        + "</head><body onload='test()'>\n"
        + "</body></html>";

    final String html2 = "function test() {\n"

        + "  alert('hello');\n"
        + "}";

    MockWebConnection connection = new MockWebConnection();
    connection.setDefaultResponse(html);
    connection.setResponse(new URL("http://localhost/test.js"), html2);

    final WebClient webClient = new WebClient();
    webClient.setWebConnection(connection);
    webClient.getPage("http://localhost/test.html");
}
     

  • shirley wilder

    shirley wilder - 2010-03-09

    There're two questions:
    1.Is there any method that i only need to set the default response,and htmlunit automatic deal with other url request?
    2.If i have to Manual deal with other js request, when the file size is bigger than 6.5k,what can i do?
    thanks!

     

  • shirley wilder

    shirley wilder - 2010-03-09

    Hi asashour,
    I'm sorry,the second question i can deal with it. The file size is 65K,not 6.5K,it's a Negligence.But doesn't matter, htmlunit provides the method to solve this problem.
    But the first question,do you have any time to take a look and deal with it ?

    Thanks!

     

  • Ahmed Ashour

    Ahmed Ashour - 2010-03-09

    Hi Shirley,

    Is there any method that i only need to set the default response,and
    htmlunit automatic deal with other url request

    .setDefaultResponse() handles any requests not specifically defined by .setResponse()

    See the example provided earlier.

     

  • shirley wilder

    shirley wilder - 2010-03-09

    Hi asashour,

    I'm sorry.
    I dont think it can handle any requests not specifically defined by setResponse().

    This is test code:

        final String html = "<html><head>\n"

            + "<script src='http://localhost/test.js'></script>\n"
            + "</head><body onload='test()'>\n" + "</body></html>";
    MockWebConnection connection = new MockWebConnection();
    connection.setDefaultResponse(html);
    try {
        final WebClient webClient = new WebClient();
        webClient.setWebConnection(connection);
        webClient.getPage("http://localhost/test.html");
    } catch (MalformedURLException e) {
        e.printStackTrace();
    } catch (FailingHttpStatusCodeException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    But it still throws an exception like Original：

    cmaError: lineNumber=[1] column=[0] lineSource=[<no source="">]</no> name=[ReferenceError] sourceName=[http://localhost/test.js] message=[ReferenceError: "XML" is not defined. (http://localhost/test.js#1)]
    com.gargoylesoftware.htmlunit.ScriptException: ReferenceError: "XML" is not defined. (http://localhost/test.js#1)

    So， only when I specifically define the "http://localhost/test.js" by setResponse(),
    it can work well.
    What I need is It can work without using setResponse() Method to define this url.

    Thanks!

     

  • Ahmed Ashour

    Ahmed Ashour - 2010-03-09

    Well, the case is still invalid.

    How can you put the response of both test.html and test.js to have the same html content?

    In real browsers (and HtmlUnit), it reads test.html correctly, then reads the <script> which points to test.js. Now, you put the content of test.js to have the HTML (not JavaScript), that's why you have the error that 'XML' is not defined for the JavaScript.</p> <p>You must have two different content for .html and .js, you can't simply combine them.</p> <p>Hope that clarifies</p></script>

     

Log in to post a comment.

MongoDB Logo MongoDB