htmlunit-user

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[Ahmed A. <asa...@ya...>]

Hi,
- Please sent to user-list, not dev-list.- You must subscribe to the list before posting - Try to use 2.24 - Try to profile the application. - Provide your complete case (with URL).
Ahmed


----- Forwarded Message -----
 From: "htm...@li..." <htm...@li...>
 To: htm...@li... 
 Sent: Tuesday, January 10, 2017 4:19 PM
 Subject: Auto-discard notification
   
----- Forwarded Message -----

The attached message has been automatically discarded.Hi,

We have recently updated HTMLUnit from 2.15 to 2.23 after which we are
facing memory and cpu utilization issues. Below is the code snippet for open
and close connections for webclient. Please review and provide some
feedback:

openSession(){
                        WebClient _webClient = new
WebClient(BrowserVersionFactory.getBrowserVersion(browserVersion));
            _webClient.getOptions().setSSLClientCertificate(url.toURL(), new
String(selfSignedCertificate), "JKS");
            _webClient.getOptions().setUseInsecureSSL(true);
            _webClient.getOptions().setTimeout(Config.getTwbTimeout() * 1000);
            _webClient.getOptions().setCssEnabled(false);
            _webClient.getCookieManager().setCookiesEnabled(true);
            _cookieManager = _webClient.getCookieManager();
            _cookieManager.setCookiesEnabled(true);
            _webClient.getOptions().setJavaScriptEnabled(true);
            _webClient.getOptions().setRedirectEnabled(true);
            _webClient.getOptions().setThrowExceptionOnScriptError(false);
            _webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
            _webClient.setAjaxController(new NicelyResynchronizingAjaxController());
}


closeSession(){
              try{
            if(_webClient != null){
                List<WebWindow> windows = _webClient.getWebWindows();
                for (WebWindow wd : windows) {
                    wd.getJobManager().removeAllJobs();
                }
                _webClient.getCookieManager().clearCookies();
                _webClient.close();
            }
        }catch (Exception e) {
            TPLog.writeLogMessage(Log.WARN, ERROR_LOG+command+e.getMessage(), e);
        }
}

   

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[manojsakarwal <man...@in...>]

Hi All,

Yesterday, we upgraded HTML Unit to 2.24 as suggested but we are still
seeing the same results. Memory usage is higher, near to 95% and around 98%
of CPU usage is observed after we upgraded from HTML Unit 2.15. 

We have also tried to solve it from our side by adding 				    
 wd.getJobManager().shutdown();

Following is the new change we have made it close method to solve it.

public void closeSession()
	{
		String command = "CloseSession";
		try{
			if(_webClient != null){
				List<WebWindow> windows = _webClient.getWebWindows();
				for (WebWindow wd : windows) {
//				    wd.getJobManager().removeAllJobs();  //removed this to replace
with shutdown.
				    wd.getJobManager().shutdown();
				}
				_webClient.getCookieManager().clearCookies();
			}
		}catch (Exception e) {
			Log.writeLogMessage(Log.WARN, ERROR_LOG+command+e.getMessage(), e);
		}finally{
			if(_webClient != null){
				_webClient.close();
			}
		}
	}

but this also did not help.

<http://htmlunit.10904.n7.nabble.com/file/n40902/Pasted_image_at_2017_02_01_10_441AM.png> 

Please help us in troubleshooting.

Thanks,
Manoj






--
View this message in context: http://htmlunit.10904.n7.nabble.com/Re-HTMLUnit-2-23-100-CPU-Utilization-and-memory-issues-tp40587p40902.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[adam a. <aaf...@gm...>]

Hi,

I've also observed higher CPU usage since mouseover event started to clean
computed stylesheet cache (after looking into commits I think it is
connected with :hover pseudoclass handling). I tried to modify code to
clean only some elements from cache, but this simply doesn't worked when
application has complex CSS. Maybe this is same issue (I haven't yet
registered a bug)?

Adam

2017-02-02 12:31 GMT+01:00 manojsakarwal <man...@in...>:

> Hi All,
>
> Yesterday, we upgraded HTML Unit to 2.24 as suggested but we are still
> seeing the same results. Memory usage is higher, near to 95% and around 98%
> of CPU usage is observed after we upgraded from HTML Unit 2.15.
>
> We have also tried to solve it from our side by adding
>  wd.getJobManager().shutdown();
>
> Following is the new change we have made it close method to solve it.
>
> public void closeSession()
>         {
>                 String command = "CloseSession";
>                 try{
>                         if(_webClient != null){
>                                 List<WebWindow> windows =
> _webClient.getWebWindows();
>                                 for (WebWindow wd : windows) {
> //                                  wd.getJobManager().removeAllJobs();
> //removed this to replace
> with shutdown.
>                                     wd.getJobManager().shutdown();
>                                 }
>                                 _webClient.getCookieManager().
> clearCookies();
>                         }
>                 }catch (Exception e) {
>                         Log.writeLogMessage(Log.WARN, ERROR_LOG+command+e.getMessage(),
> e);
>                 }finally{
>                         if(_webClient != null){
>                                 _webClient.close();
>                         }
>                 }
>         }
>
> but this also did not help.
>
> <http://htmlunit.10904.n7.nabble.com/file/n40902/Pasted_
> image_at_2017_02_01_10_441AM.png>
>
> Please help us in troubleshooting.
>
> Thanks,
> Manoj
>
>
>
>
>
>
> --
> View this message in context: http://htmlunit.10904.n7.
> nabble.com/Re-HTMLUnit-2-23-100-CPU-Utilization-and-
> memory-issues-tp40587p40902.html
> Sent from the HtmlUnit - General mailing list archive at Nabble.com.
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[manojsakarwal <man...@in...>]

Hi,

We did profiling of our systems to investigate the memory leaks with HTML
Unit(2.24). We found that one of the object of HTML Page is using 2 GB of
memory. Following are the screenshots. Can you help us to find out why it is
eating that much of memory.

<http://htmlunit.10904.n7.nabble.com/file/n41146/image_%281%29.png> 
<http://htmlunit.10904.n7.nabble.com/file/n41146/image_%282%29.png> 
<http://htmlunit.10904.n7.nabble.com/file/n41146/image273.png> 

Thanks,
Manoj Sakarwal




--
View this message in context: http://htmlunit.10904.n7.nabble.com/Re-HTMLUnit-2-23-100-CPU-Utilization-and-memory-issues-tp40587p41146.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] HTMLUnit-2.23: 100% CPU Utilization and memory issues.

[Ronald B. <rb...@rb...>]

Hi Manjo,

i know that hunting for memory leaks is not that simple. But believe me it is nearly impossible to do based on screen shots.
HtmlUnit is maintained in our spare time - this implies that you have to help us a bit if you like to have any support.

1. please try to use the latest SNAPSHOT build. The quality of the snapshot builds is the same as the release build because they are done only if the whole test suite is green.
There is no time for us to support older versions

2. We need more details. In your case you have to figure out, which collection is your problem. And we have to know, which entries are inside and why this is not correct. And we need the page 
source code to be able to understand your problem.

3. In general there is a good chance to get a fix if you are able to nail down the problem to the root cause.

Hope that helps - waiting for your details to make HtmlUnit better


    RBRi
    --------------------------
    WETATOR
    Smart web application testing
    http://www.wetator.org