Menu

#1732 Htmlunit can't load js

2.18
closed
nobody
None
1
2016-05-06
2015-12-04
henry
No

http://i.sso.sina.com.cn/js/ssologin.js is avaliable, but htmlunit can't get this js, a bug? I read the source code, find this code:

final HttpRoute route = determineRoute(target, wrapper, localcontext);
return this.execChain.execute(route, wrapper, localcontext, execAware);

the problem maybe is route? route url is "http://i.sso.sina.com.cn:80" can't be accessed, is Forbidden 403. Anyone can help me?

十二月 04, 2015 9:42:06 上午 com.gargoylesoftware.htmlunit.html.HtmlPage loadExternalJavaScriptFile
严重: Error loading JavaScript from [http://i.sso.sina.com.cn/js/ssologin.js].
com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 404 Not Found for http://i.sso.sina.com.cn/js/ssologin.js
at com.gargoylesoftware.htmlunit.WebClient.throwFailingHttpStatusCodeExceptionIfNecessary(WebClient.java:537)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1041)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:974)
at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:399)
at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:277)
at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:293)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:799)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:756)
at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3126)
at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2093)
at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:920)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:1039)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:252)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:198)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:271)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:159)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:478)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:352)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:417)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:402)
at com.meetyou.spider.worker.GetCookie.getSinaCookie(GetCookie.java:34)
at com.meetyou.spider.worker.GetCookie.main(GetCookie.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Exception in thread "main" com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 404 Not Found for http://i.sso.sina.com.cn/js/ssologin.js
at com.gargoylesoftware.htmlunit.WebClient.throwFailingHttpStatusCodeExceptionIfNecessary(WebClient.java:537)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1041)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:974)
at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:399)
at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:277)
at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:293)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:799)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:756)
at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3126)
at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2093)
at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:920)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:1039)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:252)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:198)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:271)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:159)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:478)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:352)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:417)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:402)

Discussion

  • RBRi

    RBRi - 2015-12-04

    Can you please post your code.

     
    • henry

      henry - 2015-12-08
      @Override
      protected void login() {
          try {
              String loginUrl = "http://mail.sina.com.cn/?from=mail";
              webClient.getOptions().setJavaScriptEnabled(true);
              webClient.getOptions().setCssEnabled(true);
              webClient.getOptions().setTimeout(5000);
              webClient.setAjaxController(new NicelyResynchronizingAjaxController());
              webClient.addRequestHeader("User-Agent", "Baiduspider+(+http://www.baidu.com/search/spider.htm)");
              HtmlPage htmlpage = webClient.getPage(loginUrl);
              List<DomElement> btns = htmlpage.getElementsByTagName("a");
      
              HtmlElement ue = (HtmlElement)htmlpage.getElementById("freename");
              ue.click();
              ue.type("deoz058dqpg5@163.com");
              HtmlElement pe = (HtmlElement)htmlpage.getElementById("freepassword");
              pe.click();
              pe.type("asd123123a");
      
      
              for(DomElement be : btns) {
                  String at = be.getAttribute("action-type");
                  String cl = be.getAttribute("class");
                  if("btn_submit".equals(at)) {
                      Page resultpage = be.click();
                      break;
                  } else if("loginBtn".equals(cl)) {
                      Page resultpage = be.click();
                      break;
                  }
              }
          } catch (Exception e) {
          }
      }
      
       
  • RBRi

    RBRi - 2015-12-08

    Hi Henry,

    did some research based on your source; for me it works fine. I'm using the latest HtmlUnit and did the test by simulating FF38 (new WebClient(BrowserVersion.FIREFOX_38).

     
    • henry

      henry - 2015-12-09

      I use htmlunit 2.19,this is my pom:
      <dependency>
      <groupid>net.sourceforge.htmlunit</groupid>
      <artifactid>htmlunit</artifactid>
      <version>2.19</version>
      </dependency>
      but can't work, is the maven repo a problem?

       
  • RBRi

    RBRi - 2015-12-09

    see http://mvnrepository.com/artifact/net.sourceforge.htmlunit/htmlunit/2.19

    Please provide more details. What do you mean by: 'can't work'.
    Please verify your WebClient initialziation. You have to provide a browserverion like 'new WebClient(BrowserVersion.FIREFOX_38)'. Otherwise HtmlUnit will simulate IE8 and i fear you page will nocht work with IE8.

     
  • RBRi

    RBRi - 2015-12-17

    Any update on this?

     
  • RBRi

    RBRi - 2016-05-06

    No updates from the reporter for a long time; will close this

     
  • RBRi

    RBRi - 2016-05-06
    • labels: bug -->
    • status: open --> closed
    • assigned_to: HtmlUnit --> nobody
     

Log in to post a comment.

MongoDB Logo MongoDB