Menu
▾
▴
htmlunit-user — Discussion of the use of HtmlUnit
You can subscribe to this list here.
| 2003 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(6) |
Jul
(17) |
Aug
(18) |
Sep
(22) |
Oct
(16) |
Nov
(6) |
Dec
(11) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2004 |
Jan
(11) |
Feb
(10) |
Mar
(34) |
Apr
(26) |
May
(6) |
Jun
(22) |
Jul
(14) |
Aug
(4) |
Sep
(47) |
Oct
(69) |
Nov
(23) |
Dec
(21) |
| 2005 |
Jan
(53) |
Feb
(33) |
Mar
(92) |
Apr
(65) |
May
(63) |
Jun
(57) |
Jul
(43) |
Aug
(132) |
Sep
(61) |
Oct
(75) |
Nov
(60) |
Dec
(130) |
| 2006 |
Jan
(74) |
Feb
(87) |
Mar
(101) |
Apr
(58) |
May
(54) |
Jun
(42) |
Jul
(31) |
Aug
(67) |
Sep
(61) |
Oct
(71) |
Nov
(28) |
Dec
(58) |
| 2007 |
Jan
(53) |
Feb
(50) |
Mar
(96) |
Apr
(66) |
May
(55) |
Jun
(130) |
Jul
(99) |
Aug
(115) |
Sep
(37) |
Oct
(78) |
Nov
(24) |
Dec
(70) |
| 2008 |
Jan
(94) |
Feb
(85) |
Mar
(197) |
Apr
(274) |
May
(119) |
Jun
(143) |
Jul
(193) |
Aug
(99) |
Sep
(160) |
Oct
(120) |
Nov
(178) |
Dec
(109) |
| 2009 |
Jan
(238) |
Feb
(169) |
Mar
(115) |
Apr
(109) |
May
(131) |
Jun
(167) |
Jul
(144) |
Aug
(193) |
Sep
(155) |
Oct
(154) |
Nov
(97) |
Dec
(127) |
| 2010 |
Jan
(108) |
Feb
(127) |
Mar
(176) |
Apr
(113) |
May
(130) |
Jun
(200) |
Jul
(115) |
Aug
(80) |
Sep
(92) |
Oct
(101) |
Nov
(124) |
Dec
(53) |
| 2011 |
Jan
(67) |
Feb
(144) |
Mar
(88) |
Apr
(60) |
May
(89) |
Jun
(54) |
Jul
(68) |
Aug
(81) |
Sep
(48) |
Oct
(40) |
Nov
(10) |
Dec
(20) |
| 2012 |
Jan
(21) |
Feb
(28) |
Mar
(17) |
Apr
(35) |
May
(41) |
Jun
(44) |
Jul
(68) |
Aug
(67) |
Sep
(89) |
Oct
(58) |
Nov
(47) |
Dec
(56) |
| 2013 |
Jan
(49) |
Feb
(28) |
Mar
(46) |
Apr
(31) |
May
(28) |
Jun
(37) |
Jul
(34) |
Aug
(52) |
Sep
(42) |
Oct
(108) |
Nov
(59) |
Dec
(56) |
| 2014 |
Jan
(41) |
Feb
(72) |
Mar
(46) |
Apr
(21) |
May
(19) |
Jun
(17) |
Jul
(15) |
Aug
(40) |
Sep
(11) |
Oct
(3) |
Nov
(5) |
Dec
(31) |
| 2015 |
Jan
(11) |
Feb
(12) |
Mar
(19) |
Apr
(19) |
May
(38) |
Jun
(54) |
Jul
(14) |
Aug
(42) |
Sep
(14) |
Oct
(16) |
Nov
(26) |
Dec
(14) |
| 2016 |
Jan
(3) |
Feb
(1) |
Mar
(24) |
Apr
(5) |
May
(15) |
Jun
(14) |
Jul
(33) |
Aug
(19) |
Sep
(8) |
Oct
(10) |
Nov
|
Dec
(2) |
| 2017 |
Jan
(16) |
Feb
(12) |
Mar
(23) |
Apr
(8) |
May
(11) |
Jun
(20) |
Jul
(21) |
Aug
(20) |
Sep
|
Oct
(6) |
Nov
(9) |
Dec
(2) |
| 2018 |
Jan
(7) |
Feb
(5) |
Mar
(6) |
Apr
(5) |
May
(1) |
Jun
(2) |
Jul
(2) |
Aug
|
Sep
(4) |
Oct
(3) |
Nov
|
Dec
(4) |
| 2019 |
Jan
(2) |
Feb
(2) |
Mar
(3) |
Apr
(4) |
May
|
Jun
(4) |
Jul
(9) |
Aug
(2) |
Sep
|
Oct
(4) |
Nov
(1) |
Dec
(7) |
| 2020 |
Jan
(2) |
Feb
(6) |
Mar
(9) |
Apr
(1) |
May
(1) |
Jun
(15) |
Jul
(1) |
Aug
(1) |
Sep
(2) |
Oct
(6) |
Nov
(3) |
Dec
(5) |
| 2021 |
Jan
(3) |
Feb
(1) |
Mar
(2) |
Apr
(1) |
May
|
Jun
(1) |
Jul
(1) |
Aug
(3) |
Sep
(1) |
Oct
|
Nov
(1) |
Dec
|
| 2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
(1) |
Jul
(4) |
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
(6) |
| 2025 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Ahmed A. <asa...@ya...> - 2015-03-08 05:43:40
|
Hi, Please continue discussing this in the bug report opened. Generally, it is much simpler for all parties to isolate a minimal test case as hinted in http://htmlunit.sourceforge.net/submittingJSBugs.html Ahmed From: stand4love <sta...@16...> To: htmlunit-user <htm...@li...> Sent: Sunday, March 8, 2015 3:11 AM Subject: Re: [Htmlunit-user] 'DomNode has not been set for this SimpleScriptable' error occurs when submit the form Hi, Can you point me to the right direction to resolve this ? Then I will try to fix this by myself. Thanks. 2015-03-08 stand4love ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Htmlunit-user mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
From: stand4love <sta...@16...> - 2015-03-08 02:11:34
|
Hi, Can you point me to the right direction to resolve this ? Then I will try to fix this by myself. Thanks. 2015-03-08 stand4love |
|
From: stand4love <sta...@16...> - 2015-03-06 13:19:44
|
Hi, (the message is not shown correctly in my previous email, so I resend it) I scrape the code from my project, and create a minimum test project. Thank you very much if you can take a look at this issue. You can load it into eclipse, and then run the App#main method. It will generate log in the appLogs directory inside the project (it already has one generated). And when the following line appears in the console: DEBUG com.metest.metest.App - Make request: https://omeo.alipay.com/service/checkcode?sessionID=386e26a43c02761f36e51a7ecb31836e&t=0.5163806172962445 You can see a verifyCode.jpg is generated in the project's root directory, you need to enter the verify code into verifyCode.txt file, which will be picked up by the code (the code will wait for 40 seconds, which gives you enough time to enter the verify code). And I believe you need to open the https://auth.alipay.com/login/index.htm page with UTF-8 encoding in order to view page & js properly. It is always better to have a case that others don't spend time to reach what you already have. I guess the Chinese character thingy really needs to be fixed, please verify if your case is similar to https://sourceforge.net/p/htmlunit/bugs/1623/, and lets' try to find a fix there. I have error processing https://a.alipayobjects.com/authcenter/login/1.2.2/js/login.js, and real chrome shows garbage characters. Please provide your local complete project and your complete java code to take this further. 2015-03-06 stand4love |
|
From: stand4love <sta...@16...> - 2015-03-06 12:09:21
|
Hi, I scrape the code from my project, and create a minimum test project. Thank you very much if you can take a look at this issue. You can load it into eclipse, and then run the App#main method. It will generate log in the appLogs directory inside the project (it already has one generated). And when the following line appears in the console: DEBUG com.metest.metest.App - Make request: https://omeo.alipay.com/service/checkcode?sessionID=386e26a43c02761f36e51a7ecb31836e&t=0.5163806172962445 You can see a verifyCode.jpg is generated in the project's root directory, you need to enter the verify code into verifyCode.txt file, which will be picked up by the code (the code will wait for 40 seconds, which gives you enough time to enter the verify code). Here is the screenshot of the login page, and from top to bottom, they are username, password and verifyCode. The orange button is the submit button. And I believe you need to open the https://auth.alipay.com/login/index.htm page with UTF-8 encoding in order to view page & js properly. It is always better to have a case that others don't spend time to reach what you already have. I guess the Chinese character thingy really needs to be fixed, please verify if your case is similar to https://sourceforge.net/p/htmlunit/bugs/1623/, and lets' try to find a fix there. I have error processing https://a.alipayobjects.com/authcenter/login/1.2.2/js/login.js, and real chrome shows garbage characters. Please provide your local complete project and your complete java code to take this further. 2015-03-06 stand4love |
|
From: Ahmed A. <asa...@ya...> - 2015-03-06 08:47:01
|
Hi again, >> BTW: How to quote your answer in the email ? You can quote it like that. It is always better to have a case that others don't spend time to reach what you already have. I guess the Chinese character thingy really needs to be fixed, please verify if your case is similar to https://sourceforge.net/p/htmlunit/bugs/1623/, and lets' try to find a fix there. I have error processing https://a.alipayobjects.com/authcenter/login/1.2.2/js/login.js, and real chrome shows garbage characters. Please provide your local complete project and your complete java code to take this further. You can also submit a bug report. Ahmed From: stand4love <sta...@16...> To: htmlunit-user <htm...@li...> Sent: Friday, March 6, 2015 9:36 AM Subject: Re: [Htmlunit-user] 'DomNode has not been set for this SimpleScriptable' error occurs when submit the form Let me provide a bit more info: The login page (https://auth.alipay.com/login/index.htm) needs to be accessed using mobile user-agent: Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5 Then the page will show username and password inputs (rather than activex control). Then I fill the username, password, and the verification code, and submit the form: HtmlTextInput nameInput = (HtmlTextInput) loginPage.querySelector("#J-input-user"); nameInput.focus(); nameInput.setValueAttribute(username); nameInput.blur(); HtmlPasswordInput pwdInput = (HtmlPasswordInput) loginPage.querySelector("#password_input"); pwdInput.focus(); pwdInput.setValueAttribute(password); pwdInput.blur(); // enter verify-code HtmlTextInput verifyCodeInput = (HtmlTextInput) loginPage.querySelector("#J-input-checkcode"); verifyCodeInput.focus(); verifyCodeInput.type(verifyCode); verifyCodeInput.blur(); HtmlSubmitInput buttonInput = (HtmlSubmitInput) loginPage.querySelector("#J-login-btn"); webClient.waitForBackgroundJavaScript(10000); HtmlPage mainPage = buttonInput.click(); webClient.waitForBackgroundJavaScript(10000); Then that error occurs. (Note that due to the chinese character issues, now all the js files are loaded from local file system, I created a custom WebConnectionWrapper to intercept the js requests). 2015-03-06 stand4love ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Htmlunit-user mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
From: stand4love <sta...@16...> - 2015-03-06 08:36:55
|
Let me provide a bit more info: The login page (https://auth.alipay.com/login/index.htm) needs to be accessed using mobile user-agent: Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5 Then the page will show username and password inputs (rather than activex control). Then I fill the username, password, and the verification code, and submit the form: HtmlTextInput nameInput = (HtmlTextInput) loginPage.querySelector("#J-input-user"); nameInput.focus(); nameInput.setValueAttribute(username); nameInput.blur(); HtmlPasswordInput pwdInput = (HtmlPasswordInput) loginPage.querySelector("#password_input"); pwdInput.focus(); pwdInput.setValueAttribute(password); pwdInput.blur(); // enter verify-code HtmlTextInput verifyCodeInput = (HtmlTextInput) loginPage.querySelector("#J-input-checkcode"); verifyCodeInput.focus(); verifyCodeInput.type(verifyCode); verifyCodeInput.blur(); HtmlSubmitInput buttonInput = (HtmlSubmitInput) loginPage.querySelector("#J-login-btn"); webClient.waitForBackgroundJavaScript(10000); HtmlPage mainPage = buttonInput.click(); webClient.waitForBackgroundJavaScript(10000); Then that error occurs. (Note that due to the chinese character issues, now all the js files are loaded from local file system, I created a custom WebConnectionWrapper to intercept the js requests). 2015-03-06 stand4love |
|
From: stand4love <sta...@16...> - 2015-03-06 08:22:48
|
I'm trying login this page: https://auth.alipay.com/login/index.htm The code is very simple, but now I'm unable to submit the form. BTW: How to quote your answer in the email ? 2015-03-06 stand4love |
|
From: Ahmed A. <asa...@ya...> - 2015-03-06 07:55:01
|
Hi, Can you provide a complete case, please? Ahmed From: stand4love <sta...@16...> To: htmlunit-user <htm...@li...> Sent: Friday, March 6, 2015 8:26 AM Subject: [Htmlunit-user] 'DomNode has not been set for this SimpleScriptable' error occurs when submit the form Hi, Do you have any idea why this is happening ? It occurs when I call the submitButton's click method, and the HTML form is generated dynamically. Thanks. com.gargoylesoftware.htmlunit.ScriptException: Exception invoking submitat com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:791) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:606) [htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:540) [htmlunit-core-js-2.16-SNAPSHOT.jar:na]at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:723) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:698) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:995) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventListeners(EventListenersContainer.java:193) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:255) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:822) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:756) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:920) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:606) [htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:540) [htmlunit-core-js-2.16-SNAPSHOT.jar:na]at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:925) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:896) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:111) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlSubmitInput.doClickStateUpdate(HtmlSubmitInput.java:99) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1321) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1268) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1216) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.fenqiguanjia.usercrawler.crawlers.complex.TianMaoFenQiCrawler.tryLoginUser(TianMaoFenQiCrawler.java:211) [classes/:na]at com.fenqiguanjia.usercrawler.crawlers.BaseComplexCrawler.tryLoginUser(BaseComplexCrawler.java:63) [classes/:na]at com.fenqiguanjia.usercrawler.crawlers.BaseCrawler.crawlAccount(BaseCrawler.java:185) [classes/:na]at com.fenqiguanjia.usercrawler.webservices.impl.OrderBillCrawlerWebserviceImpl$1.call(OrderBillCrawlerWebserviceImpl.java:93) [classes/:na]at com.fenqiguanjia.usercrawler.webservices.impl.OrderBillCrawlerWebserviceImpl$1.call(OrderBillCrawlerWebserviceImpl.java:1) [classes/:na]at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_65]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_65]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_65]at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]Caused by: java.lang.RuntimeException: Exception invoking submitat net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:193) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:452) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.NativeArray.iterativeMethod(NativeArray.java:1609) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.NativeArray.execIdCall(NativeArray.java:338) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.IdFunctionObject.call(IdFunctionObject.java:97) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:430) [htmlunit-core-js-2.16-SNAPSHOT.jar:na]at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3155) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:109) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:716) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:776) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]... 28 common frames omittedCaused by: java.lang.IllegalStateException: DomNode has not been set for this SimpleScriptable: com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElementat com.gargoylesoftware.htmlunit.javascript.SimpleScriptable.getDomNodeOrDie(SimpleScriptable.java:149) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.Element.getDomNodeOrDie(Element.java:359) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.getDomNodeOrDie(HTMLElement.java:2031) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement.submit(HTMLFormElement.java:342) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT]at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_65]at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_65]at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_65]at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_65]at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:167) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na]... 44 common frames omitted 2015-03-06 stand4love ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Htmlunit-user mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
From: stand4love <sta...@16...> - 2015-03-06 07:32:16
|
Hi, Do you have any idea why this is happening ? It occurs when I call the submitButton's click method, and the HTML form is generated dynamically. Thanks. com.gargoylesoftware.htmlunit.ScriptException: Exception invoking submit at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:791) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:606) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:540) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:723) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:698) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:995) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventListeners(EventListenersContainer.java:193) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:255) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:822) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:756) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:920) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:606) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:540) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:925) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:896) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:111) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlSubmitInput.doClickStateUpdate(HtmlSubmitInput.java:99) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1321) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1268) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1216) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.fenqiguanjia.usercrawler.crawlers.complex.TianMaoFenQiCrawler.tryLoginUser(TianMaoFenQiCrawler.java:211) [classes/:na] at com.fenqiguanjia.usercrawler.crawlers.BaseComplexCrawler.tryLoginUser(BaseComplexCrawler.java:63) [classes/:na] at com.fenqiguanjia.usercrawler.crawlers.BaseCrawler.crawlAccount(BaseCrawler.java:185) [classes/:na] at com.fenqiguanjia.usercrawler.webservices.impl.OrderBillCrawlerWebserviceImpl$1.call(OrderBillCrawlerWebserviceImpl.java:93) [classes/:na] at com.fenqiguanjia.usercrawler.webservices.impl.OrderBillCrawlerWebserviceImpl$1.call(OrderBillCrawlerWebserviceImpl.java:1) [classes/:na] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] Caused by: java.lang.RuntimeException: Exception invoking submit at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:193) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:452) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.NativeArray.iterativeMethod(NativeArray.java:1609) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.NativeArray.execIdCall(NativeArray.java:338) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.IdFunctionObject.call(IdFunctionObject.java:97) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:111) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:430) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3155) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:109) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:716) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:776) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] ... 28 common frames omitted Caused by: java.lang.IllegalStateException: DomNode has not been set for this SimpleScriptable: com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement at com.gargoylesoftware.htmlunit.javascript.SimpleScriptable.getDomNodeOrDie(SimpleScriptable.java:149) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.Element.getDomNodeOrDie(Element.java:359) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.getDomNodeOrDie(HTMLElement.java:2031) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement.submit(HTMLFormElement.java:342) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_65] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_65] at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_65] at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:167) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] ... 44 common frames omitted 2015-03-06 stand4love |
|
From: Rich G. <ri...@um...> - 2015-03-06 01:22:10
|
Sorry, ignore this...another newbie error.
-Rich
On Thu, Mar 5, 2015 at 1:09 PM, Rich Goldman <ri...@um...> wrote:
> One followup:
>
> I'm trying to get the HTML (mainly the link urls) included in some of the
> agendas but it's not coming through using:
>
> WebClient webClient = new WebClient(BrowserVersion.CHROME);
> HtmlPage page = webClient.getPage("
> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/04/2015");
> Thread.sleep(10000);
> System.out.println(page.asXml());
>
> Is there something I need to do in order to get/keep the html rendering?
>
> I get (3/4/2015) HF0416-A15-0112.pdf instead of that text with the link to
> the pdf file...
> -Rich
>
> On Wed, Mar 4, 2015 at 11:45 AM, Ahmed Ashour <asa...@ya...> wrote:
>
>> Hi Rich,
>>
>> Well, waitForBackground is actually better than Thread.sleep(), and it
>> works.
>>
>> It didn't work with you before, because you used it 'before'
>> webClient.getPage(), however it should be 'after', to allow JavaScript/AJAX
>> to run.
>>
>> Hope that clarifies,
>>
>> Ahmed
>> ------------------------------
>> *From:* Rich Goldman <ri...@um...>
>> *To:* htm...@li...
>> *Sent:* Wednesday, March 4, 2015 4:56 PM
>> *Subject:* Re: [Htmlunit-user] Help Extracting Schedule from a Website
>>
>> I think I was confused between using Thread.sleep(10000)
>> and webClient.waitForBackgroundJavaScript(10000).
>>
>> Thanks again.
>> -Rich
>>
>>
>>
>> On Wed, Mar 4, 2015 at 10:21 AM, Alain BUFERNE <alb...@gm...>
>> wrote:
>>
>> By using HtmlUnit, you generally just program what a normal human being
>> will do to use the webSite. Since you just need information send by the
>> server in response of clickt this, select that, you don't need to execute
>> Js code .
>>
>> 2015-03-04 7:05 GMT+01:00 Rich Goldman <ri...@um...>:
>>
>> Doing a bit more digging, it seems the javascript functions for
>> populating the agenda items are in:
>> http://www.house.leg.state.mn.us/schedules/ScheduleElements0.js?v=1.12
>>
>> I don't know enough javascript to know how to execute these functions
>> appropriately though.
>> -Rich
>>
>> On Wed, Mar 4, 2015 at 12:41 AM, Rich Goldman <ri...@um...> wrote:
>>
>> I'm trying to get the schedule information posted at:
>>
>> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015
>>
>> The content is loaded dynamically (presumably via AJAX) and I've tried
>> the following code:
>>
>>
>> final WebClient webClient = new
>> WebClient(BrowserVersion.CHROME);
>> webClient.waitForBackgroundJavaScript(10000);
>> final HtmlPage page = webClient
>> .getPage("
>> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015");
>> String javaScriptCode = "SchedJSx.Init();";
>>
>> ScriptResult result = page.executeJavaScript(javaScriptCode);
>> result.getJavaScriptResult();
>> System.out.println("result: " + result.getJavaScriptResult());
>>
>> I can get some of the dynamic content:
>> Friday, March 06, 2015
>> 10:30 AM
>> Health and Human Services Reform
>> Chair: Rep. Tara Mack
>> Location: Basement State Office Building
>> Note:
>> ***Additional bills may be added
>>
>> but not the agenda/bill list.
>>
>> I feel like I'm missing something simple that I'm now aware of as a
>> newbie. I would appreciate a skilled HTML Unit user looking at the source
>> code of the source website and pointing out what I'm missing so I can
>> extract the agenda for this meeting as well.
>>
>> Thanks for any help you can provide.
>> -Rich
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub
>> for all
>> things parallel software development, from weekly thought leadership
>> blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub
>> for all
>> things parallel software development, from weekly thought leadership
>> blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub
>> for all
>> things parallel software development, from weekly thought leadership
>> blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>>
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub
>> for all
>> things parallel software development, from weekly thought leadership
>> blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>
|
|
From: Rich G. <ri...@um...> - 2015-03-05 18:09:53
|
One followup:
I'm trying to get the HTML (mainly the link urls) included in some of the
agendas but it's not coming through using:
WebClient webClient = new WebClient(BrowserVersion.CHROME);
HtmlPage page = webClient.getPage("
http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/04/2015");
Thread.sleep(10000);
System.out.println(page.asXml());
Is there something I need to do in order to get/keep the html rendering?
I get (3/4/2015) HF0416-A15-0112.pdf instead of that text with the link to
the pdf file...
-Rich
On Wed, Mar 4, 2015 at 11:45 AM, Ahmed Ashour <asa...@ya...> wrote:
> Hi Rich,
>
> Well, waitForBackground is actually better than Thread.sleep(), and it
> works.
>
> It didn't work with you before, because you used it 'before'
> webClient.getPage(), however it should be 'after', to allow JavaScript/AJAX
> to run.
>
> Hope that clarifies,
>
> Ahmed
> ------------------------------
> *From:* Rich Goldman <ri...@um...>
> *To:* htm...@li...
> *Sent:* Wednesday, March 4, 2015 4:56 PM
> *Subject:* Re: [Htmlunit-user] Help Extracting Schedule from a Website
>
> I think I was confused between using Thread.sleep(10000)
> and webClient.waitForBackgroundJavaScript(10000).
>
> Thanks again.
> -Rich
>
>
>
> On Wed, Mar 4, 2015 at 10:21 AM, Alain BUFERNE <alb...@gm...>
> wrote:
>
> By using HtmlUnit, you generally just program what a normal human being
> will do to use the webSite. Since you just need information send by the
> server in response of clickt this, select that, you don't need to execute
> Js code .
>
> 2015-03-04 7:05 GMT+01:00 Rich Goldman <ri...@um...>:
>
> Doing a bit more digging, it seems the javascript functions for populating
> the agenda items are in:
> http://www.house.leg.state.mn.us/schedules/ScheduleElements0.js?v=1.12
>
> I don't know enough javascript to know how to execute these functions
> appropriately though.
> -Rich
>
> On Wed, Mar 4, 2015 at 12:41 AM, Rich Goldman <ri...@um...> wrote:
>
> I'm trying to get the schedule information posted at:
>
> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015
>
> The content is loaded dynamically (presumably via AJAX) and I've tried the
> following code:
>
>
> final WebClient webClient = new
> WebClient(BrowserVersion.CHROME);
> webClient.waitForBackgroundJavaScript(10000);
> final HtmlPage page = webClient
> .getPage("
> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015");
> String javaScriptCode = "SchedJSx.Init();";
>
> ScriptResult result = page.executeJavaScript(javaScriptCode);
> result.getJavaScriptResult();
> System.out.println("result: " + result.getJavaScriptResult());
>
> I can get some of the dynamic content:
> Friday, March 06, 2015
> 10:30 AM
> Health and Human Services Reform
> Chair: Rep. Tara Mack
> Location: Basement State Office Building
> Note:
> ***Additional bills may be added
>
> but not the agenda/bill list.
>
> I feel like I'm missing something simple that I'm now aware of as a
> newbie. I would appreciate a skilled HTML Unit user looking at the source
> code of the source website and pointing out what I'm missing so I can
> extract the agenda for this meeting as well.
>
> Thanks for any help you can provide.
> -Rich
>
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
>
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
|
|
From: Ahmed A. <asa...@ya...> - 2015-03-04 16:45:46
|
Hi Rich, Well, waitForBackground is actually better than Thread.sleep(), and it works. It didn't work with you before, because you used it 'before' webClient.getPage(), however it should be 'after', to allow JavaScript/AJAX to run. Hope that clarifies, Ahmed From: Rich Goldman <ri...@um...> To: htm...@li... Sent: Wednesday, March 4, 2015 4:56 PM Subject: Re: [Htmlunit-user] Help Extracting Schedule from a Website I think I was confused between using Thread.sleep(10000) and webClient.waitForBackgroundJavaScript(10000). Thanks again. -Rich On Wed, Mar 4, 2015 at 10:21 AM, Alain BUFERNE <alb...@gm...> wrote: By using HtmlUnit, you generally just program what a normal human being will do to use the webSite. Since you just need information send by the server in response of clickt this, select that, you don't need to execute Js code . 2015-03-04 7:05 GMT+01:00 Rich Goldman <ri...@um...>: Doing a bit more digging, it seems the javascript functions for populating the agenda items are in: http://www.house.leg.state.mn.us/schedules/ScheduleElements0.js?v=1.12 I don't know enough javascript to know how to execute these functions appropriately though. -Rich On Wed, Mar 4, 2015 at 12:41 AM, Rich Goldman <ri...@um...> wrote: I'm trying to get the schedule information posted at: http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015 The content is loaded dynamically (presumably via AJAX) and I've tried the following code: final WebClient webClient = new WebClient(BrowserVersion.CHROME); webClient.waitForBackgroundJavaScript(10000); final HtmlPage page = webClient .getPage("http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015"); String javaScriptCode = "SchedJSx.Init();"; ScriptResult result = page.executeJavaScript(javaScriptCode); result.getJavaScriptResult(); System.out.println("result: " + result.getJavaScriptResult()); I can get some of the dynamic content:Friday, March 06, 201510:30 AMHealth and Human Services ReformChair: Rep. Tara MackLocation: Basement State Office BuildingNote:***Additional bills may be added but not the agenda/bill list. I feel like I'm missing something simple that I'm now aware of as a newbie. I would appreciate a skilled HTML Unit user looking at the source code of the source website and pointing out what I'm missing so I can extract the agenda for this meeting as well. Thanks for any help you can provide. -Rich ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Htmlunit-user mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlunit-user ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Htmlunit-user mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlunit-user ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Htmlunit-user mailing list Htm...@li... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
From: Rich G. <ri...@um...> - 2015-03-04 15:57:20
|
I think I was confused between using Thread.sleep(10000)
and webClient.waitForBackgroundJavaScript(10000).
Thanks again.
-Rich
On Wed, Mar 4, 2015 at 10:21 AM, Alain BUFERNE <alb...@gm...>
wrote:
> By using HtmlUnit, you generally just program what a normal human being
> will do to use the webSite. Since you just need information send by the
> server in response of clickt this, select that, you don't need to execute
> Js code .
>
> 2015-03-04 7:05 GMT+01:00 Rich Goldman <ri...@um...>:
>
>> Doing a bit more digging, it seems the javascript functions for
>> populating the agenda items are in:
>> http://www.house.leg.state.mn.us/schedules/ScheduleElements0.js?v=1.12
>>
>> I don't know enough javascript to know how to execute these functions
>> appropriately though.
>> -Rich
>>
>> On Wed, Mar 4, 2015 at 12:41 AM, Rich Goldman <ri...@um...> wrote:
>>
>>> I'm trying to get the schedule information posted at:
>>>
>>> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015
>>>
>>> The content is loaded dynamically (presumably via AJAX) and I've tried
>>> the following code:
>>>
>>>
>>> final WebClient webClient = new
>>> WebClient(BrowserVersion.CHROME);
>>> webClient.waitForBackgroundJavaScript(10000);
>>> final HtmlPage page = webClient
>>> .getPage("
>>> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015");
>>> String javaScriptCode = "SchedJSx.Init();";
>>>
>>> ScriptResult result = page.executeJavaScript(javaScriptCode);
>>> result.getJavaScriptResult();
>>> System.out.println("result: " + result.getJavaScriptResult());
>>>
>>> I can get some of the dynamic content:
>>> Friday, March 06, 2015
>>> 10:30 AM
>>> Health and Human Services Reform
>>> Chair: Rep. Tara Mack
>>> Location: Basement State Office Building
>>> Note:
>>> ***Additional bills may be added
>>>
>>> but not the agenda/bill list.
>>>
>>> I feel like I'm missing something simple that I'm now aware of as a
>>> newbie. I would appreciate a skilled HTML Unit user looking at the source
>>> code of the source website and pointing out what I'm missing so I can
>>> extract the agenda for this meeting as well.
>>>
>>> Thanks for any help you can provide.
>>> -Rich
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub
>> for all
>> things parallel software development, from weekly thought leadership
>> blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
|
|
From: Alain B. <alb...@gm...> - 2015-03-04 15:22:02
|
By using HtmlUnit, you generally just program what a normal human being will do to use the webSite. Since you just need information send by the server in response of clickt this, select that, you don't need to execute Js code . 2015-03-04 7:05 GMT+01:00 Rich Goldman <ri...@um...>: > Doing a bit more digging, it seems the javascript functions for populating > the agenda items are in: > http://www.house.leg.state.mn.us/schedules/ScheduleElements0.js?v=1.12 > > I don't know enough javascript to know how to execute these functions > appropriately though. > -Rich > > On Wed, Mar 4, 2015 at 12:41 AM, Rich Goldman <ri...@um...> wrote: > >> I'm trying to get the schedule information posted at: >> >> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015 >> >> The content is loaded dynamically (presumably via AJAX) and I've tried >> the following code: >> >> >> final WebClient webClient = new >> WebClient(BrowserVersion.CHROME); >> webClient.waitForBackgroundJavaScript(10000); >> final HtmlPage page = webClient >> .getPage(" >> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015"); >> String javaScriptCode = "SchedJSx.Init();"; >> >> ScriptResult result = page.executeJavaScript(javaScriptCode); >> result.getJavaScriptResult(); >> System.out.println("result: " + result.getJavaScriptResult()); >> >> I can get some of the dynamic content: >> Friday, March 06, 2015 >> 10:30 AM >> Health and Human Services Reform >> Chair: Rep. Tara Mack >> Location: Basement State Office Building >> Note: >> ***Additional bills may be added >> >> but not the agenda/bill list. >> >> I feel like I'm missing something simple that I'm now aware of as a >> newbie. I would appreciate a skilled HTML Unit user looking at the source >> code of the source website and pointing out what I'm missing so I can >> extract the agenda for this meeting as well. >> >> Thanks for any help you can provide. >> -Rich >> > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for > all > things parallel software development, from weekly thought leadership blogs > to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > Htmlunit-user mailing list > Htm...@li... > https://lists.sourceforge.net/lists/listinfo/htmlunit-user > > |
|
From: Rich G. <ri...@um...> - 2015-03-04 13:58:13
|
Cool. That did it. Thank you.
-Rich
On Wed, Mar 4, 2015 at 5:31 AM, Ahmed Ashour <asa...@ya...> wrote:
> Hi Rich,
>
> You don't need to execute JavaScript, it is automatically handled.
>
> You just need to wait, how about:
>
> HtmlPage page = webClient.getPage("
> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015");
>
> Thread.sleep(10_000);
> System.out.println(page.asText());
>
>
> Ahmed
> ------------------------------
> *From:* Rich Goldman <ri...@um...>
> *To:* htm...@li...
> *Sent:* Wednesday, March 4, 2015 6:41 AM
> *Subject:* [Htmlunit-user] Help Extracting Schedule from a Website
>
> I'm trying to get the schedule information posted at:
>
> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015
>
> The content is loaded dynamically (presumably via AJAX) and I've tried the
> following code:
>
>
> final WebClient webClient = new
> WebClient(BrowserVersion.CHROME);
> webClient.waitForBackgroundJavaScript(10000);
> final HtmlPage page = webClient
> .getPage("
> http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015");
> String javaScriptCode = "SchedJSx.Init();";
>
> ScriptResult result = page.executeJavaScript(javaScriptCode);
> result.getJavaScriptResult();
> System.out.println("result: " + result.getJavaScriptResult());
>
> I can get some of the dynamic content:
> Friday, March 06, 2015
> 10:30 AM
> Health and Human Services Reform
> Chair: Rep. Tara Mack
> Location: Basement State Office Building
> Note:
> ***Additional bills may be added
>
> but not the agenda/bill list.
>
> I feel like I'm missing something simple that I'm now aware of as a
> newbie. I would appreciate a skilled HTML Unit user looking at the source
> code of the source website and pointing out what I'm missing so I can
> extract the agenda for this meeting as well.
>
> Thanks for any help you can provide.
> -Rich
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
|
|
From: Ahmed A. <asa...@ya...> - 2015-03-04 10:31:45
|
Hi Rich,
You don't need to execute JavaScript, it is automatically handled.
You just need to wait, how about:
HtmlPage page = webClient.getPage("http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015");
Thread.sleep(10_000); System.out.println(page.asText());
Ahmed From: Rich Goldman <ri...@um...>
To: htm...@li...
Sent: Wednesday, March 4, 2015 6:41 AM
Subject: [Htmlunit-user] Help Extracting Schedule from a Website
I'm trying to get the schedule information posted at:
http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015
The content is loaded dynamically (presumably via AJAX) and I've tried the following code:
final WebClient webClient = new WebClient(BrowserVersion.CHROME); webClient.waitForBackgroundJavaScript(10000); final HtmlPage page = webClient .getPage("http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015"); String javaScriptCode = "SchedJSx.Init();";
ScriptResult result = page.executeJavaScript(javaScriptCode); result.getJavaScriptResult(); System.out.println("result: " + result.getJavaScriptResult());
I can get some of the dynamic content:Friday, March 06, 201510:30 AMHealth and Human Services ReformChair: Rep. Tara MackLocation: Basement State Office BuildingNote:***Additional bills may be added
but not the agenda/bill list.
I feel like I'm missing something simple that I'm now aware of as a newbie. I would appreciate a skilled HTML Unit user looking at the source code of the source website and pointing out what I'm missing so I can extract the agenda for this meeting as well.
Thanks for any help you can provide. -Rich
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user
|
|
From: Rich G. <ri...@um...> - 2015-03-04 06:06:20
|
Doing a bit more digging, it seems the javascript functions for populating the agenda items are in: http://www.house.leg.state.mn.us/schedules/ScheduleElements0.js?v=1.12 I don't know enough javascript to know how to execute these functions appropriately though. -Rich On Wed, Mar 4, 2015 at 12:41 AM, Rich Goldman <ri...@um...> wrote: > I'm trying to get the schedule information posted at: > > http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015 > > The content is loaded dynamically (presumably via AJAX) and I've tried the > following code: > > > final WebClient webClient = new > WebClient(BrowserVersion.CHROME); > webClient.waitForBackgroundJavaScript(10000); > final HtmlPage page = webClient > .getPage(" > http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015"); > String javaScriptCode = "SchedJSx.Init();"; > > ScriptResult result = page.executeJavaScript(javaScriptCode); > result.getJavaScriptResult(); > System.out.println("result: " + result.getJavaScriptResult()); > > I can get some of the dynamic content: > Friday, March 06, 2015 > 10:30 AM > Health and Human Services Reform > Chair: Rep. Tara Mack > Location: Basement State Office Building > Note: > ***Additional bills may be added > > but not the agenda/bill list. > > I feel like I'm missing something simple that I'm now aware of as a > newbie. I would appreciate a skilled HTML Unit user looking at the source > code of the source website and pointing out what I'm missing so I can > extract the agenda for this meeting as well. > > Thanks for any help you can provide. > -Rich > |
|
From: Rich G. <ri...@um...> - 2015-03-04 05:42:34
|
I'm trying to get the schedule information posted at: http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015 The content is loaded dynamically (presumably via AJAX) and I've tried the following code: final WebClient webClient = new WebClient(BrowserVersion.CHROME); webClient.waitForBackgroundJavaScript(10000); final HtmlPage page = webClient .getPage(" http://www.house.leg.state.mn.us/schedules/schedule.aspx#03/06/2015"); String javaScriptCode = "SchedJSx.Init();"; ScriptResult result = page.executeJavaScript(javaScriptCode); result.getJavaScriptResult(); System.out.println("result: " + result.getJavaScriptResult()); I can get some of the dynamic content: Friday, March 06, 2015 10:30 AM Health and Human Services Reform Chair: Rep. Tara Mack Location: Basement State Office Building Note: ***Additional bills may be added but not the agenda/bill list. I feel like I'm missing something simple that I'm now aware of as a newbie. I would appreciate a skilled HTML Unit user looking at the source code of the source website and pointing out what I'm missing so I can extract the agenda for this meeting as well. Thanks for any help you can provide. -Rich |
|
From: asashour <asa...@ya...> - 2015-03-03 12:47:25
|
Hi, Please note 2.16 is expected to be released around end of March 2015. Ahmed -- View this message in context: http://htmlunit.10904.n7.nabble.com/When-will-the-htmlunit-2-16-be-released-tp35440p35503.html Sent from the HtmlUnit - General mailing list archive at Nabble.com. |
|
From: Ahmed A. <asa...@ya...> - 2015-02-27 04:22:05
|
Hi, I guess there is an issue with Chinese characters, this may be similar to https://sourceforge.net/p/htmlunit/bugs/1623/ Ahmed From: stand4love <sta...@16...> To: htmlunit-user <htm...@li...> Sent: Friday, February 27, 2015 3:45 AM Subject: [Htmlunit-user] Error occurs when parsing js in htmlunit-2.16-SNAPSHOT Error occurs when parsing the https://a.alipayobjects.com:443/authcenter/login/1.2.2/js/login.js page when open this page in htmlunit 2.16 snapshot: https://www.alipay.com/ Can anyone take a look at this ? And here is the exception stacktrace: com.gargoylesoftware.htmlunit.ScriptException: missing name after . operator (https://a.alipayobjects.com:443/authcenter/login/1.2.2/js/login.js#1) |
|
From: stand4love <sta...@16...> - 2015-02-27 02:46:29
|
Error occurs when parsing the https://a.alipayobjects.com:443/authcenter/login/1.2.2/js/login.js page when open this page in htmlunit 2.16 snapshot: https://www.alipay.com/ Can anyone take a look at this ? And here is the exception stacktrace: com.gargoylesoftware.htmlunit.ScriptException: missing name after . operator (https://a.alipayobjects.com:443/authcenter/login/1.2.2/js/login.js#1) at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:791) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:606) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:540) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.compile(JavaScriptEngine.java:631) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1161) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:1049) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:395) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:276) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.doProcessPostponedActions(JavaScriptEngine.java:836) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.access$500(JavaScriptEngine.java:98) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:786) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:606) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:540) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:677) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:652) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptIfPossible(HtmlPage.java:969) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlScript.executeInlineScriptIfNeeded(HtmlScript.java:351) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:411) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:276) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:290) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:793) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) [xercesImpl-2.11.0.jar:na] at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:751) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170) [nekohtml-1.9.21.jar:1.9.21] at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072) [nekohtml-1.9.21.jar:1.9.21] at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206) [nekohtml-1.9.21.jar:na] at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330) [nekohtml-1.9.21.jar:na] at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3126) [nekohtml-1.9.21.jar:1.9.21] at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2093) [nekohtml-1.9.21.jar:1.9.21] at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:920) [nekohtml-1.9.21.jar:1.9.21] at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499) [nekohtml-1.9.21.jar:1.9.21] at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452) [nekohtml-1.9.21.jar:1.9.21] at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) [xercesImpl-2.11.0.jar:na] at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:1017) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:248) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:194) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:471) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:345) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:184) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:122) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1987) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:237) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:478) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:345) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:410) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:395) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.fenqiguanjia.usercrawler.crawlers.complex.TianMaoFenQiCrawler.tryLoginUser(TianMaoFenQiCrawler.java:74) [classes/:na] at com.fenqiguanjia.usercrawler.crawlers.BaseComplexCrawler.tryLoginUser(BaseComplexCrawler.java:59) [classes/:na] at com.fenqiguanjia.usercrawler.crawlers.BaseCrawler.crawlAccount(BaseCrawler.java:175) [classes/:na] at com.fenqiguanjia.usercrawler.webservices.impl.OrderBillCrawlerWebserviceImpl$1.call(OrderBillCrawlerWebserviceImpl.java:92) [classes/:na] at com.fenqiguanjia.usercrawler.webservices.impl.OrderBillCrawlerWebserviceImpl$1.call(OrderBillCrawlerWebserviceImpl.java:85) [classes/:na] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] Caused by: net.sourceforge.htmlunit.corejs.javascript.EvaluatorException: missing name after . operator (https://a.alipayobjects.com:443/authcenter/login/1.2.2/js/login.js#1) at com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter.error(StrictErrorReporter.java:64) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.Parser.addError(Parser.java:188) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.addError(Parser.java:166) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.reportError(Parser.java:223) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.reportError(Parser.java:210) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.reportError(Parser.java:203) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.propertyAccess(Parser.java:2637) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExprTail(Parser.java:2497) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2472) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2334) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2259) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2243) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2224) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2199) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2171) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2160) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2149) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2138) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2127) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2116) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2080) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2051) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.plainProperty(Parser.java:3338) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.objectLiteral(Parser.java:3220) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:2797) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2436) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2334) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2259) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2243) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2224) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2199) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2171) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2160) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2149) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2138) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2127) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2116) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2080) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2051) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2062) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.expr(Parser.java:2039) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.nameOrLabel(Parser.java:1783) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1050) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:919) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.parseFunctionBody(Parser.java:645) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.function(Parser.java:813) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:2791) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2436) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2334) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2259) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2243) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2224) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2199) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2171) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2160) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2149) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2138) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2127) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2116) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2080) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2051) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.argumentList(Parser.java:2403) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExprTail(Parser.java:2553) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2472) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2334) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2259) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2243) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2224) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2199) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2171) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2160) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2149) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2138) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2127) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2116) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2080) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2051) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.expr(Parser.java:2030) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.nameOrLabel(Parser.java:1783) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1050) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:919) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.parse(Parser.java:551) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Parser.parse(Parser.java:478) ~[htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Context.compileImpl(Context.java:2465) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at net.sourceforge.htmlunit.corejs.javascript.Context.compileString(Context.java:1452) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory$TimeoutContext.compileString(HtmlUnitContextFactory.java:201) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at net.sourceforge.htmlunit.corejs.javascript.Context.compileString(Context.java:1441) [htmlunit-core-js-2.16-SNAPSHOT.jar:na] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$2.doRun(JavaScriptEngine.java:622) ~[htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:776) [htmlunit-2.16-SNAPSHOT.jar:2.16-SNAPSHOT] ... 56 common frames omitted 2015-02-27 stand4love |
|
From: Ahmed A. <asa...@ya...> - 2015-02-26 13:48:56
|
Hi, For the time being, please try to test the latest snapshot from https://ci.canoo.com/teamcity/viewLog.html?buildTypeId=Htmlunit_CheckInBuild&buildId=lastSuccessful&tab=artifacts as if you find an issue, a fix can be in 2.16. I guess the development team would reply about the possible date for the next release. Ahmed From: stand4love <sta...@16...> To: htmlunit-user <htm...@li...> Sent: Thursday, February 26, 2015 2:32 PM Subject: [Htmlunit-user] When will the htmlunit 2.16 be released ? Hi, Is there any plan for this ? I'm currently using the htmlunit 2.15 to crawl a specific website, but it fails to parse specific js file (which is important for my crawler code). Thanks. 2015-02-26 stand4love |
|
From: stand4love <sta...@16...> - 2015-02-26 13:32:49
|
Hi, Is there any plan for this ? I'm currently using the htmlunit 2.15 to crawl a specific website, but it fails to parse specific js file (which is important for my crawler code). Thanks. 2015-02-26 stand4love |
189 messages has been excluded from this view by a project administrator.
