Re: [Htmlunit-user] Help Extracting Schedule from a Website

[Ahmed A. <asa...@ya...>]

Hi Rich,
Please be more specific.
Using latest snapshot, INTERNET_EXPLORER_8, and .asText(), I get the below, which is similar to what is in the website.

Alabama LegislatureFirst Special Session 2015                  HomeSession InformationCode of AlabamaConstitutionHouse of RepresentativesSenateHelpContact UsLegislative DayConsoleBillsResolutionsConfirmationsCommittee MeetingsInterim Meetings & AnnouncementsReportsAdjourn ConveneLegislative Audio/VideoBill Status for HB1 ()Not Official Information from the Legislative OfficesDocument Quick LinksLegislative ProcessLegislative GlossaryAlabama LegislatureThe Legislature convenes in regular annual sessions on the first Tuesday in February, except (1) in the first year of the four-year term, when the session will begin on the first Tuesday in March, and (2) in the last year of a four-year term, when the session will begin on the second Tuesday in January. The length of the regular session is limited to 30 meeting days within a period of 105 calendar days. There are usually two meeting or "legislative" days per week, with other days devoted to committee meetings. Special sessions of the Legislature may be called by the Governor, with the Proclamation listing the subjects which the Governor wishes considered. These sessions are limited to 12 legislative days within a 30 calendar day span. In a regular session, bills may be enacted on any subject. In a special session, legislation must be enacted only on those subjects which the Governor announces in his proclamation or "call." Anything not in the "call" requires a two-thirds vote of each house to be enacted.Alabama Legislature

      From: Rich Goldman <ri...@um...>
 To: Ahmed Ashour <asa...@ya...>; "htm...@li..." <htm...@li...> 
 Sent: Thursday, August 13, 2015 12:30 AM
 Subject: Re: [Htmlunit-user] Help Extracting Schedule from a Website
   


This message is eligible for Automatic Cleanup! (ri...@um...) Add cleanup rule | More info
 
If you go to the webpage in a browser you'll get all sorts of information on a piece of legislation.  This code simply returns the template without any data/html populated.  Does this make sense?

On Wednesday, August 12, 2015, Ahmed Ashour <asa...@ya...> wrote:

Hi,
You can also use Thread.sleep().
What is missing (not loaded)?
Please read http://htmlunit.sourceforge.net/submittingJSBugs.html
Ahmed
      From: Rich Goldman <ri...@um...>
 To: htm...@li... 
 Sent: Wednesday, August 12, 2015 10:22 PM
 Subject: Re: [Htmlunit-user] Help Extracting Schedule from a Website
   


I am trying to get the populated HTML of another site but it is not loading, despite putting the wait time to 30 seconds.
The address is http://alisondb.legislature.state.al.us/Alison/SESSBillResult.aspx?BILL=HB1&WIN_TYPE=SELECTED_STATUS 
Are you all able to get page.asXml();  to produce populated html for this address?
I've updated to 2.18 and I've tried putting the waitForBackgroundJavaScript in multiple places without success. My code is:
public String getWebsiteTextWithJavaScript(String url) {  WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_6);  HtmlPage page = null;  try {   webClient.waitForBackgroundJavaScript(30000);      page = webClient.getPage(url);  } catch (FailingHttpStatusCodeException e) {   // TODO Auto-generated catch block   e.printStackTrace();  } catch (MalformedURLException e) {   // TODO Auto-generated catch block   e.printStackTrace();  } catch (IOException e) {   // TODO Auto-generated catch block   e.printStackTrace();  }  //  // // Thread.sleep(10000);  webClient.waitForBackgroundJavaScript(30000);  String text = page.asXml();  webClient.waitForBackgroundJavaScript(30000);    page.cleanUp();  webClient.closeAllWindows();
  return text; }   

------------------------------------------------------------------------------

_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user