htmlunit-user — Discussion of the use of HtmlUnit

[Htmlunit-user] how can I use JsxSetter?

[w. <d8...@si...>]

Dear Sir, 
I have some questions, the screen.java in Htmlunit, the value of Height is 768, 
the value of Width is 1024, but wo want to change it when the WebClient creating, 
I want to set some parameters of the screen, what should I do?
I'm looking forward to your reply soon.
Sincerely yours,
webb

Re: [Htmlunit-user] avoid print(logger) dom element

[Ahmed A. <asa...@ya...>]

Hi,
Have a look at http://htmlunit.sourceforge.net/logging.html
You can also use:
org.apache.log4j.LogManager.getLogger("com").setLevel(org.apache.log4j.Level.OFF);

Ahmed

      From: K Senthil <jks...@gm...>
 To: Htm...@li... 
 Sent: Tuesday, February 28, 2017 5:51 AM
 Subject: [Htmlunit-user] avoid print(logger) dom element
   
Hi Guys,
    I use HTMLUNIT2.23 version.Html page creation is it take more time when run time .That time print all DOM and javascript elements.I think that's why it take more time.Then how to avoid this logger.
   

[Htmlunit-user] avoid print(logger) dom element

[K S. <jks...@gm...>]

Hi Guys,

    I use HTMLUNIT2.23 version.Html page creation is it take more time when
run time .That time print all DOM and javascript elements.I think that's
why it take more time.Then how to avoid this logger.

[Htmlunit-user] [ANN] HtmlUnit 2.25 is released

[Ahmed A. <asa...@ya...>]

Hi all,
It is a pleasure to announce the availability of HtmlUnit 2.25, which is the first release to required Java 8.
The most notable changes are:    - InteractivePage and SvgPage are removed, use HtmlPage    - Clicking a hidden element now does nothing    - PromptHandler: handlePrompt now has a third parameter    - Change return type of WebRequest.getCharset, WebResponse.getContentCharset(), .getContentCharsetOrNull(), and DomNode.getByXPath.    - ScriptPreProcessor was re-introduced    - WebClientOptions: add option to automatically download images.    - HtmlFileInput: have .setFiles() instead of .setValueAttribute()
The fill list of changes can be found in [1]
As always: Happy Coding!
The HtmlUnit team

[1] http://htmlunit.sourceforge.net/changes-report.html#a2.25

[Htmlunit-user] WebDriver Alerts

[Ahmed A. <asa...@ya...>]

Hi all,
Currently there is a branch named "remote" in HtmlUnitDriver, which implements RemoteDriver (client/server) and it passes AlertsTest.
However, HtmlUnit test suite is based on the old Alert implementation, and moving the test cases to the remote HtmlUnitDriver will take some effort.
Having two branches in HtmlUnit and two in HtmlUnitDriver would be inconvenient, since we need to merge new changes to the branches.
So, how about releasing HtmlUnit and HtmlUnitDriver soon, and then for 2.26, we start by excluding all test cases, then gradually migrate them to the new way.
What do you think?Ahmed

Re: [Htmlunit-user] Class Cast Exception

[Ahmed A. <asa...@ya...>]

Hi,
- You must subscribe to the list before posting.- This is now fixed in SVN.
Thanks,Ahmed

     
----- Forwarded Message -----
 From: "htm...@li..." <htm...@li...>
Sent: Wednesday, February 15, 2017 3:10 PM
 Subject: Auto-discard notification
   
----- Forwarded Message -----

The attached message has been automatically discarded.When I am loading a webpage in selenium html driver, there is Javascript execution error, but there is also this in the error logs java.lang.ClassCastException: com.gargoylesoftware.htmlunit.javascript.host.Window cannot be cast to com.gargoylesoftware.htmlunit.javascript.host.Console at com.gargoylesoftware.htmlunit.javascript.host.Console.error(Console.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)I have checked error method in the Console class/**
 * This method performs logging to the console at "error" level.
 * @param cx the JavaScript context
 * @param thisObj the scriptable
 * @param args the arguments passed into the method
 * @param funObj the function
 */
@JsxFunction
public static void error(final Context cx, final Scriptable thisObj,
    final Object[] args, final Function funObj) {
    final WebConsole webConsole = ((Console) thisObj).getWebConsole();
    final Formatter oldFormatter = webConsole.getFormatter();
    webConsole.setFormatter(FORMATTER_);
    webConsole.error(args);
    webConsole.setFormatter(oldFormatter);
}
All Scriptable objects are not subclasses of console, so why is this here ?

   

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[adam a. <aaf...@gm...>]

Hi,

I've also observed higher CPU usage since mouseover event started to clean
computed stylesheet cache (after looking into commits I think it is
connected with :hover pseudoclass handling). I tried to modify code to
clean only some elements from cache, but this simply doesn't worked when
application has complex CSS. Maybe this is same issue (I haven't yet
registered a bug)?

Adam

2017-02-02 12:31 GMT+01:00 manojsakarwal <man...@in...>:

> Hi All,
>
> Yesterday, we upgraded HTML Unit to 2.24 as suggested but we are still
> seeing the same results. Memory usage is higher, near to 95% and around 98%
> of CPU usage is observed after we upgraded from HTML Unit 2.15.
>
> We have also tried to solve it from our side by adding
>  wd.getJobManager().shutdown();
>
> Following is the new change we have made it close method to solve it.
>
> public void closeSession()
>         {
>                 String command = "CloseSession";
>                 try{
>                         if(_webClient != null){
>                                 List<WebWindow> windows =
> _webClient.getWebWindows();
>                                 for (WebWindow wd : windows) {
> //                                  wd.getJobManager().removeAllJobs();
> //removed this to replace
> with shutdown.
>                                     wd.getJobManager().shutdown();
>                                 }
>                                 _webClient.getCookieManager().
> clearCookies();
>                         }
>                 }catch (Exception e) {
>                         Log.writeLogMessage(Log.WARN, ERROR_LOG+command+e.getMessage(),
> e);
>                 }finally{
>                         if(_webClient != null){
>                                 _webClient.close();
>                         }
>                 }
>         }
>
> but this also did not help.
>
> <http://htmlunit.10904.n7.nabble.com/file/n40902/Pasted_
> image_at_2017_02_01_10_441AM.png>
>
> Please help us in troubleshooting.
>
> Thanks,
> Manoj
>
>
>
>
>
>
> --
> View this message in context: http://htmlunit.10904.n7.
> nabble.com/Re-HTMLUnit-2-23-100-CPU-Utilization-and-
> memory-issues-tp40587p40902.html
> Sent from the HtmlUnit - General mailing list archive at Nabble.com.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>

Re: [Htmlunit-user] Cookies not getting set in HTMLUnit 2.23

[Ronald B. <rb...@rb...>]

Now i'm confused...

>   Suppose, I am using HTMLUnit in JS disabled mode and when I interact with a server and the server sends an HTML page
>    which has a Javascript for setting cookies using setcookie method.
>Please note that the response header does not contain
>   Set-Cookie.
>
>    How does HTMLUnit handle the above situation?. It looks like I need to enable JS. Is there a way to solve/work around this issue?

I think i have proposed a solution:

replace 

	mgr.addCookie(new Cookie(".jobdiva.com", "COMPANYLOGOLOCATION", "my_data", "/", -1, false));
by
         	webClient.addCookie("COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/");

This should work.

	RBRi


>
>On 2 February 2017 at 01:59, Ronald Brill <rb...@rb...> wrote:
>
>> Ok Vasu,
>>
>> this was a really tricky one.
>>
>> You are interacting directly with the cookie manager in this way
>>
>>     final CookieManager mgr = webClient.getCookieManager();
>>    mgr.addCookie(new Cookie(".jobdiva.com", "COMPANYLOGOLOCATION",
>> "my_data", "/", -1, false));
>>
>> In this case you are responsible for all the cookie properties.
>> But there is some magic, if a web site sends a comparable cookie like
>> "COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/".
>> Internally there is some cleanup before the cookie is added to the cookie
>> manager, in this case the leading dot will be removed from the domain.
>> When sending th cookies back the cookie matches now, because of the
>> removed dot at the start. This logic is more or less part of HttpClient
>> already.
>>
>> Ok, and what to do now?
>>
>> I guess you like to simulate some cookies sent from the server before you
>> are starting your request. I think the more correct way is to interact with
>> the webClient instead of the CookieManager directly because there is a way
>> to use the same magic as we use
>> when processing cookies from the real response.
>>
>>         webClient.addCookie("COMPANYLOGOLOCATION=my_data; Domain=.
>> jobdiva.com; Path=/");
>>
>> is your friend.
>> Hope that helps.
>>
>>         RBRi

Re: [Htmlunit-user] Cookies not getting set in HTMLUnit 2.23

[Ronald B. <rb...@rb...>]

Ok Vasu,

this was a really tricky one.

You are interacting directly with the cookie manager in this way

    final CookieManager mgr = webClient.getCookieManager();
   mgr.addCookie(new Cookie(".jobdiva.com", "COMPANYLOGOLOCATION", "my_data", "/", -1, false));

In this case you are responsible for all the cookie properties.
But there is some magic, if a web site sends a comparable cookie like "COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/".
Internally there is some cleanup before the cookie is added to the cookie manager, in this case the leading dot will be removed from the domain.
When sending th cookies back the cookie matches now, because of the removed dot at the start. This logic is more or less part of HttpClient already.

Ok, and what to do now?

I guess you like to simulate some cookies sent from the server before you are starting your request. I think the more correct way is to interact with the webClient instead of the CookieManager directly because there is a way to use the same magic as we use 
when processing cookies from the real response.

	webClient.addCookie("COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/");

is your friend.
Hope that helps.

	RBRi

>
>Hi Ronald,
>
>   I am calling http://www2.jobdiva.com/......URL after calling the add
>cookies.
>
>   I have only the clojure snippet which I had copied.
>
>   The clogo-data is the variable which has the Value for the cookie name
>COMPANYLOGOLOCATION.
>
>   The same was working fine with HTMLUnit Version 2.15. I am just
>migrating the app to HTMLUnit 2.23 and found this issue
>
>Regards
> Vasu
>
>On 1 February 2017 at 23:20, Ronald Brill <rb...@rb...> wrote:
>
>> Hi Vasu,
>>
>> it will be really helpfull to provide all the detail you have.
>>
>> Regarding your question
>> * what is clogo-data?
>> * which url you are calling
>>
>> And it will really help to have complete plain java test cases to verify
>> your problems.
>>
>> Have added another test case that tries to simulate/reproduce your problem
>>
>> trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/
>> CookieManager2Test.java
>>     @Test
>>     @Alerts("my_name=my_data")
>>     public void cookie_maxAgeMinusOne() throws Exception {
>>         final WebClient webClient = getWebClient();
>>         final MockWebConnection webConnection = new MockWebConnection();
>>
>>         final URL url = URL_FIRST;
>>         webConnection.setResponse(url, CookieManagerTest.HTML_ALERT_
>> COOKIE);
>>         webClient.setWebConnection(webConnection);
>>
>>         final CookieManager mgr = webClient.getCookieManager();
>>         mgr.addCookie(new Cookie(URL_FIRST.getHost(), "my_name",
>> "my_data", "/", -1, false));
>>
>>         final List<String> collectedAlerts = new ArrayList<>();
>>         webClient.setAlertHandler(new CollectingAlertHandler(
>> collectedAlerts));
>>
>>         webClient.getPage(URL_FIRST);
>>         assertEquals(getExpectedAlerts(), collectedAlerts);
>>     }
>>
>> This test works for me. Please have a look - maybe there is a difference
>> for your case (or maybe the test did not work with your HtmlUnit version).
>>
>>
>> Hope that helps
>>
>>         RBRi
>>
>>
>>
>>
>> On Wed, 1 Feb 2017 18:51:49 +0530 Vasudevan Comandur wrote:
>> >
>> >Hi,
>> >
>> >   I am creating a set of cookies using the constructor of Cookie Object.
>> >   Then I am adding those cookies into the cookieManager of a Webclient
>> >   before sending a request to the site.
>> >
>> >  When I looked into Charles WebProxy tool, I did not see those cookies
>> >  set in the HTTP Cookie header.
>> >
>> >  Whereas the same application was working fine with earlier version of
>> >HTMLUnit
>> >  say 2.15.
>> >
>> >  Am I missing something in HTMLUnit 2.23?
>> >   Help needed urgently.
>> >
>> >   Below is the clojure code snippet for reference:
>> >
>> >   Creating a cookie
>> >   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/"
>> -1
>> >false)
>> >
>> >    Adding cookie to Cookie Manager
>> >   (.addCookie (.getCookieManager *wc*) val1)
>> >
>> >   Please note *wc* is the WebClient Object.
>> >
>> >   Appreciate your help.
>> >
>> >Regards
>> > Vasu
>> >
>> >
>> >
>> >----< Inline text [text-plain-04.txt] >------------------
>> >
>> >-----------------------------------------------------------
>> -------------------
>> >Check out the vibrant tech community on one of the world's most
>> >engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> >
>> >
>> >----< Inline text [text-plain-05.txt] >------------------
>> >
>> >_______________________________________________
>> >Htmlunit-user mailing list
>> >Htm...@li...
>> >https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>> >
>> >
>>
>>
>
>

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[manojsakarwal <man...@in...>]

Hi All,

Yesterday, we upgraded HTML Unit to 2.24 as suggested but we are still
seeing the same results. Memory usage is higher, near to 95% and around 98%
of CPU usage is observed after we upgraded from HTML Unit 2.15. 

We have also tried to solve it from our side by adding 				    
 wd.getJobManager().shutdown();

Following is the new change we have made it close method to solve it.

public void closeSession()
	{
		String command = "CloseSession";
		try{
			if(_webClient != null){
				List<WebWindow> windows = _webClient.getWebWindows();
				for (WebWindow wd : windows) {
//				    wd.getJobManager().removeAllJobs();  //removed this to replace
with shutdown.
				    wd.getJobManager().shutdown();
				}
				_webClient.getCookieManager().clearCookies();
			}
		}catch (Exception e) {
			Log.writeLogMessage(Log.WARN, ERROR_LOG+command+e.getMessage(), e);
		}finally{
			if(_webClient != null){
				_webClient.close();
			}
		}
	}

but this also did not help.

<http://htmlunit.10904.n7.nabble.com/file/n40902/Pasted_image_at_2017_02_01_10_441AM.png> 

Please help us in troubleshooting.

Thanks,
Manoj






--
View this message in context: http://htmlunit.10904.n7.nabble.com/Re-HTMLUnit-2-23-100-CPU-Utilization-and-memory-issues-tp40587p40902.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] Cookies not getting set in HTMLUnit 2.23

[Vasudevan C. <vco...@gm...>]

Hi Ronald,

   I am calling http://www2.jobdiva.com/......URL after calling the add
cookies.

   I have only the clojure snippet which I had copied.

   The clogo-data is the variable which has the Value for the cookie name
COMPANYLOGOLOCATION.

   The same was working fine with HTMLUnit Version 2.15. I am just
migrating the app to HTMLUnit 2.23 and found this issue

Regards
 Vasu

On 1 February 2017 at 23:20, Ronald Brill <rb...@rb...> wrote:

> Hi Vasu,
>
> it will be really helpfull to provide all the detail you have.
>
> Regarding your question
> * what is clogo-data?
> * which url you are calling
>
> And it will really help to have complete plain java test cases to verify
> your problems.
>
> Have added another test case that tries to simulate/reproduce your problem
>
> trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/
> CookieManager2Test.java
>     @Test
>     @Alerts("my_name=my_data")
>     public void cookie_maxAgeMinusOne() throws Exception {
>         final WebClient webClient = getWebClient();
>         final MockWebConnection webConnection = new MockWebConnection();
>
>         final URL url = URL_FIRST;
>         webConnection.setResponse(url, CookieManagerTest.HTML_ALERT_
> COOKIE);
>         webClient.setWebConnection(webConnection);
>
>         final CookieManager mgr = webClient.getCookieManager();
>         mgr.addCookie(new Cookie(URL_FIRST.getHost(), "my_name",
> "my_data", "/", -1, false));
>
>         final List<String> collectedAlerts = new ArrayList<>();
>         webClient.setAlertHandler(new CollectingAlertHandler(
> collectedAlerts));
>
>         webClient.getPage(URL_FIRST);
>         assertEquals(getExpectedAlerts(), collectedAlerts);
>     }
>
> This test works for me. Please have a look - maybe there is a difference
> for your case (or maybe the test did not work with your HtmlUnit version).
>
>
> Hope that helps
>
>         RBRi
>
>
>
>
> On Wed, 1 Feb 2017 18:51:49 +0530 Vasudevan Comandur wrote:
> >
> >Hi,
> >
> >   I am creating a set of cookies using the constructor of Cookie Object.
> >   Then I am adding those cookies into the cookieManager of a Webclient
> >   before sending a request to the site.
> >
> >  When I looked into Charles WebProxy tool, I did not see those cookies
> >  set in the HTTP Cookie header.
> >
> >  Whereas the same application was working fine with earlier version of
> >HTMLUnit
> >  say 2.15.
> >
> >  Am I missing something in HTMLUnit 2.23?
> >   Help needed urgently.
> >
> >   Below is the clojure code snippet for reference:
> >
> >   Creating a cookie
> >   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/"
> -1
> >false)
> >
> >    Adding cookie to Cookie Manager
> >   (.addCookie (.getCookieManager *wc*) val1)
> >
> >   Please note *wc* is the WebClient Object.
> >
> >   Appreciate your help.
> >
> >Regards
> > Vasu
> >
> >
> >
> >----< Inline text [text-plain-04.txt] >------------------
> >
> >-----------------------------------------------------------
> -------------------
> >Check out the vibrant tech community on one of the world's most
> >engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> >
> >
> >----< Inline text [text-plain-05.txt] >------------------
> >
> >_______________________________________________
> >Htmlunit-user mailing list
> >Htm...@li...
> >https://lists.sourceforge.net/lists/listinfo/htmlunit-user
> >
> >
>
>

Re: [Htmlunit-user] Cookies not getting set in HTMLUnit 2.23

[Ronald B. <rb...@rb...>]

Hi Vasu,

it will be really helpfull to provide all the detail you have.

Regarding your question
* what is clogo-data?
* which url you are calling

And it will really help to have complete plain java test cases to verify your problems.

Have added another test case that tries to simulate/reproduce your problem 

trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/CookieManager2Test.java
    @Test
    @Alerts("my_name=my_data")
    public void cookie_maxAgeMinusOne() throws Exception {
        final WebClient webClient = getWebClient();
        final MockWebConnection webConnection = new MockWebConnection();

        final URL url = URL_FIRST;
        webConnection.setResponse(url, CookieManagerTest.HTML_ALERT_COOKIE);
        webClient.setWebConnection(webConnection);

        final CookieManager mgr = webClient.getCookieManager();
        mgr.addCookie(new Cookie(URL_FIRST.getHost(), "my_name", "my_data", "/", -1, false));

        final List<String> collectedAlerts = new ArrayList<>();
        webClient.setAlertHandler(new CollectingAlertHandler(collectedAlerts));

        webClient.getPage(URL_FIRST);
        assertEquals(getExpectedAlerts(), collectedAlerts);
    }

This test works for me. Please have a look - maybe there is a difference for your case (or maybe the test did not work with your HtmlUnit version).


Hope that helps

	RBRi




On Wed, 1 Feb 2017 18:51:49 +0530 Vasudevan Comandur wrote:
>
>Hi,
>
>   I am creating a set of cookies using the constructor of Cookie Object.
>   Then I am adding those cookies into the cookieManager of a Webclient
>   before sending a request to the site.
>
>  When I looked into Charles WebProxy tool, I did not see those cookies
>  set in the HTTP Cookie header.
>
>  Whereas the same application was working fine with earlier version of
>HTMLUnit
>  say 2.15.
>
>  Am I missing something in HTMLUnit 2.23?
>   Help needed urgently.
>
>   Below is the clojure code snippet for reference:
>
>   Creating a cookie
>   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/" -1
>false)
>
>    Adding cookie to Cookie Manager
>   (.addCookie (.getCookieManager *wc*) val1)
>
>   Please note *wc* is the WebClient Object.
>
>   Appreciate your help.
>
>Regards
> Vasu
>
>
>
>----< Inline text [text-plain-04.txt] >------------------
>
>------------------------------------------------------------------------------
>Check out the vibrant tech community on one of the world's most
>engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>----< Inline text [text-plain-05.txt] >------------------
>
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>

[Htmlunit-user] Cookies not getting set in HTMLUnit 2.23

[Vasudevan C. <vco...@gm...>]

Hi,

   I am creating a set of cookies using the constructor of Cookie Object.
   Then I am adding those cookies into the cookieManager of a Webclient
   before sending a request to the site.

  When I looked into Charles WebProxy tool, I did not see those cookies
  set in the HTTP Cookie header.

  Whereas the same application was working fine with earlier version of
HTMLUnit
  say 2.15.

  Am I missing something in HTMLUnit 2.23?
   Help needed urgently.

   Below is the clojure code snippet for reference:

   Creating a cookie
   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/" -1
false)

    Adding cookie to Cookie Manager
   (.addCookie (.getCookieManager *wc*) val1)

   Please note *wc* is the WebClient Object.

   Appreciate your help.

Regards
 Vasu

Re: [Htmlunit-user] Difference in length of a variable computed by browser and HTMLUnit JS Engine

[Vasudevan C. <vco...@gm...>]

Hi Ahmed,

    It is the "th" variable. I did not write any java code. I was using
Clojure code.

    Below is the clojure code snippet

    (let [act-url (str "
http://localhost:8080/vmsconnect/recaptcha-naukri.html")
         ]
           (start-page (do
                      (enable-javascript true)
                      (enable-httpexception false) ; Disable the HTTP and
script exceptions
                      (enable-scriptexception true) ; Disable the script
exceptions
                      (.addRequestHeader *wc* "Referer" (str "
https://resdex.naukri.com/v2/search/pageChange"))
                   act-url)
               (. java.lang.Thread sleep 3000) ; Sleep for 3 seconds
     ;          (hit-count 0)
           (let [src-val (try (with-node (page-root)
                                         (attr (xpath
"//img[@id='recaptcha_challenge_image']" :node) "src")
                                      )
                                      (catch Exception e (println "Inside
Exception in Captcha Image Fetching" e))
                                 )
                 st1-data (println "THE RESPONSE FROM HOST on EXISTING
SESSION AFTER LINKING PROFILE ...." src-val "         ")
                ]
                (.close *wc*)
                   src-val)
       )
     )



Regards
 Vasu

On 26 January 2017 at 22:44, Ahmed Ashour <asa...@ya...> wrote:

> Hi Vasu,
>
> Which variable?
>
> Please provide your Java code.
>
> Ahmed
>
>
> ------------------------------
> *From:* Vasudevan Comandur <vco...@gm...>
> *To:* "htm...@li..." <htmlunit-user@lists.
> sourceforge.net>
> *Sent:* Wednesday, January 25, 2017 4:22 PM
> *Subject:* [Htmlunit-user] Difference in length of a variable computed by
> browser and HTMLUnit JS Engine
>
> Hi All,
>
>    I am using HTMLUnit 2.23 and enabled JS execution. The JS executed by
> the
>    browser is computing value for one variable whose length is always 1230
> on FF
>    browser.
>
>    The same JS when executed through HTMLUnit/Rhino is not producing the
> same
>    length. It is always far less than what I see for the browser.
>
>     Attached is the HTMLPage which connects to google Recaptcha service.
>
>      It would be great if you can tell me the reason for the difference in
> length.
>
>     Appreciate your help on this.
>
> Regards
>  Vasu
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>

Re: [Htmlunit-user] Difference in length of a variable computed by browser and HTMLUnit JS Engine

[Ahmed A. <asa...@ya...>]

Hi Vasu,
Which variable?
Please provide your Java code.
Ahmed

      From: Vasudevan Comandur <vco...@gm...>
 To: "htm...@li..." <htm...@li...> 
 Sent: Wednesday, January 25, 2017 4:22 PM
 Subject: [Htmlunit-user] Difference in length of a variable computed by browser and HTMLUnit JS Engine
   
Hi All,

   I am using HTMLUnit 2.23 and enabled JS execution. The JS executed by the
   browser is computing value for one variable whose length is always 1230 on FF
   browser.

   The same JS when executed through HTMLUnit/Rhino is not producing the same
   length. It is always far less than what I see for the browser.

    Attached is the HTMLPage which connects to google Recaptcha service.

     It would be great if you can tell me the reason for the difference in length.

    Appreciate your help on this.

Regards
 Vasu   

[Htmlunit-user] Difference in length of a variable computed by browser and HTMLUnit JS Engine

[Vasudevan C. <vco...@gm...>]

Hi All,

   I am using HTMLUnit 2.23 and enabled JS execution. The JS executed by the
   browser is computing value for one variable whose length is always 1230
on FF
   browser.

   The same JS when executed through HTMLUnit/Rhino is not producing the
same
   length. It is always far less than what I see for the browser.

    Attached is the HTMLPage which connects to google Recaptcha service.

     It would be great if you can tell me the reason for the difference in
length.

    Appreciate your help on this.

Regards
 Vasu

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[Ahmed A. <asa...@ya...>]

Hi,
- Please sent to user-list, not dev-list.- You must subscribe to the list before posting - Try to use 2.24 - Try to profile the application. - Provide your complete case (with URL).
Ahmed


----- Forwarded Message -----
 From: "htm...@li..." <htm...@li...>
 To: htm...@li... 
 Sent: Tuesday, January 10, 2017 4:19 PM
 Subject: Auto-discard notification
   
----- Forwarded Message -----

The attached message has been automatically discarded.Hi,

We have recently updated HTMLUnit from 2.15 to 2.23 after which we are
facing memory and cpu utilization issues. Below is the code snippet for open
and close connections for webclient. Please review and provide some
feedback:

openSession(){
                        WebClient _webClient = new
WebClient(BrowserVersionFactory.getBrowserVersion(browserVersion));
            _webClient.getOptions().setSSLClientCertificate(url.toURL(), new
String(selfSignedCertificate), "JKS");
            _webClient.getOptions().setUseInsecureSSL(true);
            _webClient.getOptions().setTimeout(Config.getTwbTimeout() * 1000);
            _webClient.getOptions().setCssEnabled(false);
            _webClient.getCookieManager().setCookiesEnabled(true);
            _cookieManager = _webClient.getCookieManager();
            _cookieManager.setCookiesEnabled(true);
            _webClient.getOptions().setJavaScriptEnabled(true);
            _webClient.getOptions().setRedirectEnabled(true);
            _webClient.getOptions().setThrowExceptionOnScriptError(false);
            _webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
            _webClient.setAjaxController(new NicelyResynchronizingAjaxController());
}


closeSession(){
              try{
            if(_webClient != null){
                List<WebWindow> windows = _webClient.getWebWindows();
                for (WebWindow wd : windows) {
                    wd.getJobManager().removeAllJobs();
                }
                _webClient.getCookieManager().clearCookies();
                _webClient.close();
            }
        }catch (Exception e) {
            TPLog.writeLogMessage(Log.WARN, ERROR_LOG+command+e.getMessage(), e);
        }
}

   

Re: [Htmlunit-user] GC warning and stopped executing script

[Ahmed A. <asa...@ya...>]

Hi Nimesh,
Your program should be profiled to see which objects are not collected.
Please try to directly use latest HtmlUnit directly, since it may have already a fix.
Let's know your findings.
Thanks,Ahmed

      From: Nimesh.T.Prajapati <nim...@gm...>
 To: htm...@li... 
 Sent: Sunday, January 8, 2017 10:08 PM
 Subject: [Htmlunit-user] GC warning and stopped executing script
   
Hi Team,
I'm on the project where using HTMLUnit driver-java to crawl data from sites. I'm following a sequence like creating object and closing/quitting. Everything works find under Eclipse environment where I execute program. But when I export my build with jar option and try to execute I can see drastically increase in memory and the time comes where GC fires warnings. from 50 mb usage to 2 gb usage.
Using latest jar that is bundled with Selenium 3.0.1Let me know your views on that.
--
Nimesh T. Prajapati"Quality does matter..."
WebDriver+Testng+Maven+Ant+Sikuli

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user


   

[Htmlunit-user] [ANN] HtmlUnit 2.24 released

[Ahmed A. <asa...@ya...>]

Hi all,

It is a pleasure to announce the release of HtmlUnit 2.24, which would be the last version to support Java 7.
The full list of changes can be found in [1].
Thanks to all the contributors.
Happy testing,The HtmlUnit team
[1] http://htmlunit.sourceforge.net/changes-report.html#a2.24

[Htmlunit-user] GC warning and stopped executing script

[Nimesh.T.Prajapati <nim...@gm...>]

Hi Team,

I'm on the project where using HTMLUnit driver-java to crawl data from
sites. I'm following a sequence like creating object and closing/quitting.
Everything works find under Eclipse environment where I execute program.
But when I export my build with jar option and try to execute I can see
drastically increase in memory and the time comes where GC fires warnings.
from 50 mb usage to 2 gb usage.

Using latest jar that is bundled with Selenium 3.0.1
Let me know your views on that.

--
Nimesh T. Prajapati
"Quality does matter..."

WebDriver+Testng+Maven+Ant+Sikuli

Re: [Htmlunit-user] Multi-part construction through HTMLUnit

[Ronald B. <rb...@rb...>]

Your request body looks like a 'normal' body sent from a file upload defined with the multiple option enabled - give it a try

Re: [Htmlunit-user] Multi-part construction through HTMLUnit

[Vasudevan C. <vco...@gm...>]

Hi Ronald,

   Thanks for the reply. I am copying below the request body as expected by
the server.

*Example request body*



--boundary_string

Content-Disposition: form-data; name="entity_content";

Content-Type: application/json

{

"ContentDocumentId" :  <nullvalue> ,

"ReasonForChange" : "New Profile created from SixthSense",

"PathOnClient" : "<file name with extension>"

}

--boundary_string

Content-Type: application/octet-stream

Content-Disposition: form-data; name="VersionData"; filename="<file name
with extension>"

<Binary data goes here.>

--boundary_string--


 Is the above request body construction is achievable through HTMLUnit?

Regards
 Vasu



On 6 January 2017 at 21:37, Ronald Brill <rb...@rb...> wrote:

> Hi Vasu,
>
> i'm not talking about many input items.
>
> There is an option to switch an file input into multi file mode - the
> 'multiple' attribute.
> >
> >       <input name="filesToUpload[]" id="filesToUpload" type="file"
> multiple="" />
> >
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>

Re: [Htmlunit-user] Multi-part construction through HTMLUnit

[Ronald B. <rb...@rb...>]

Hi Vasu,

i'm not talking about many input items.

There is an option to switch an file input into multi file mode - the 'multiple' attribute.
>
>	<input name="filesToUpload[]" id="filesToUpload" type="file" multiple="" />
>

Re: [Htmlunit-user] Multi-part construction through HTMLUnit

[Vasudevan C. <vco...@gm...>]

Hi Ronald,

    Thanks for the reply.

     I tried creating multiple input type="file" within in a sample form.
But the server side rejects that stating that it should have
     only one filename in the Multi-part.

    In my case, they are expecting one multi-part with body being text
having its own content type and the other is a binary part.

    Looks like this type of multi-part creation through HTMLUnit is not
possible.

    Let me know if there is any alternate way. Perhaps, I might have to use
the HTTPClient interface for constructing the multi-part.


Regards
 Vasu

On 6 January 2017 at 19:17, Ronald Brill <rb...@rb...> wrote:

> Hi Vasu,
>
> sorry for not responding, but i have some problems with the mailings lists
> at the moment.
>
> As a quick guess i think you have to use the multiple option for your file
> upload to be able to select many files (from the same folder) for upload.
> Something like this...
>
>         <input name="filesToUpload[]" id="filesToUpload" type="file"
> multiple="" />
>
> To support this from HtmlUnit the HtmlFileInput offers a setValueAttribute
> method getting an array of file names.
> Hope that helps...
>
> BTW Please use the latest snapshot build instead of the release; we did a
> lot of fixes also for file upload during the last months.
>
>
>     RBRi
>     --------------------------
>     WETATOR
>     Smart web application testing
>     http://www.wetator.org
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>

[Htmlunit-user] Multi-part construction through HTMLUnit

[Ronald B. <rb...@rb...>]

Hi Vasu,

sorry for not responding, but i have some problems with the mailings lists at the moment.

As a quick guess i think you have to use the multiple option for your file upload to be able to select many files (from the same folder) for upload.
Something like this...

	<input name="filesToUpload[]" id="filesToUpload" type="file" multiple="" />

To support this from HtmlUnit the HtmlFileInput offers a setValueAttribute method getting an array of file names.
Hope that helps...

BTW Please use the latest snapshot build instead of the release; we did a lot of fixes also for file upload during the last months.


    RBRi
    --------------------------
    WETATOR
    Smart web application testing
    http://www.wetator.org