Re: [Htmlunit-user] first time user

[May, B. L. <bl...@cu...>]

Bump! Still need to figure this out…

From: May, Benjamin L.
Sent: Thursday, July 28, 2016 7:45 AM
To: 'Ahmed Ashour' <asa...@ya...>; 'htm...@li...' <htm...@li...>
Subject: RE: [Htmlunit-user] first time user

Hey Ahmed,

I recompiled today using the newly-released 2.23 and it still doesn’t work.

If you run the code snippet below by itself it works but the content of the page is an error message. If you run the full code, which involves several prior pages which are saving session variables somewhere on the server side and it actually attempts to run the real results page, it still does a stack overflow. The full code (with as much extraneous code and dependencies removed) is attached. It’s ugly but this is a one-time scrape so it doesn’t have to be pretty! :)

-Ben

From: Ahmed Ashour [mailto:asa...@ya...]
Sent: Wednesday, July 27, 2016 9:00 AM
To: htm...@li...<mailto:htm...@li...>
Subject: Re: [Htmlunit-user] first time user

Hi May,

Which version do you use?

With latest snapshot (a release is imminent), there are no errors.

Ahmed

________________________________
From: "May, Benjamin L." <bl...@cu...<mailto:bl...@cu...>>
To: "'htm...@li...'" <htm...@li...<mailto:htm...@li...>>
Sent: Wednesday, July 27, 2016 4:16 AM
Subject: [Htmlunit-user] first time user

Hey all! First time using this library, just trying to get a page that requires javascript... Code snippet:

    WebClient webClient = new WebClient(BrowserVersion.CHROME);

    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setRedirectEnabled(true);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.getOptions().setPrintContentOnFailingStatusCode(true);
    webClient.getOptions().setThrowExceptionOnScriptError(false);

    String link = "http://nycprop.nyc.gov/nycproperty/statements/flk/jsp/stmtassessflk.jsp?statementId=104487723";
    HtmlPage page = webClient.getPage(link);
    webClient.waitForBackgroundJavaScript(30 * 1000);
    String pageAsText = page.asText();
    System.out.println(pageAsText);

And I am getting a HUGE stackoverflow exception. If you try and view the URL in a browser with JS disabled you get a message about needing to turn it on, but there's barely any javascript in the page source from what I can see...

TIA!