Re: [Htmlunit-user] Exception when using Page.submit()

[Gian F. C. <gia...@to...>]

Hi Dave,

I haven't tried to increase the heap size, because it's
not really an option in my situation.

The java code in the jsp is just limited to the call of
the "screen scraper" method in a java class.

Do you have any clue why the same java class works
fine in isolation as a java application?

Cheers,

Gian

----- Original Message ----- 
From: "David D.Kilzer" <ddk...@ki...>
To: <htm...@li...>
Sent: Tuesday, October 26, 2004 3:46 PM
Subject: Re: [Htmlunit-user] Exception when using Page.submit()


It looks like WebSphere is running out of memory.  Have you tried  
profiling your application using either WSAD or the WAS Admin Console  
utilities?  Have you tried increasing the heap size for your  
application?

Have you considered running the Htmlunit code in an Action (if you're  
using Struts) or a servlet before you get to the JSP so that you simply  
output data once you're in the JSP?  (Typically, it's not considered  
good design to write Java code inside a JSP.)

I've written a "screen scraping" application using Htmlunt recently,  
and found that after creating some 200-300 HtmlPage objects, the JVM  
runs out of memory using a heap size of 128MB.  If I increase the heap  
size to 256MB, I can load 600-700 HtmlPage objects before running out  
of memory.  I suspect that the loop between the WebClient and the  
WebConnection may be causing the garbage collector issues (although I  
haven't had time to try to profile the code or testing the "breaking"  
of the circular dependency):

    
HtmlPage.getWebClient().getWebConnection().getWebClient().getWebConnecti 
on().adNaseum();

Is this a known issue?

Dave


On Oct 26, 2004, at 5:41 AM, Gian Franco Casula wrote:

> Hi,
>
> I've written some code in a jsp page that
> calls some java code that uses HtmlUnit to
> connect to a web site.
>
> The java code works correctly when I use it
> in isolation as a java application.
>
> Problems arise with the above mentioned jsp
> page. When the code is running I get a long
> list with the same message:
> ---------------list begin---------------
>
> ...
> [26-10-04 8:54:20:914 CEST] 5dc773e1 HttpMethodBas W
> org.apache.commons.httpclient.HttpMethodBase
> Response content length is not known
> [26-10-04 8:54:30:398 CEST] 5dc773e1 HttpMethodBas W
> org.apache.commons.httpclient.HttpMethodBase
> Response content length is not known
> ...
> ---------------list end---------------
> 
> This continues for a long time, until I get the message:
> 
> ---------------list begin---------------
> JVMDG315: JVM Requesting Heap dump file
> ...JVMDG318: Heap dump file written to E:\Program  
> Files\IBM\WebSphere...
> JVMDG303: JVM Requesting Java core file
> JVMDG304: Java core file written to E:\Program Files\IBM\WebSphere\...
> JVMDG215: Dump Handler has Processed Outofmemory Signal -1.
> ---------------list end---------------
> 
> A printStackTrace() shows the following:
>
> ---------------list begin---------------
> com.gargoylesoftware.htmlunit.ObjectInstantiationException:
> Exception when calling constructor
> [com.gargoylesoftware.htmlunit.html.HtmlFrame
>   
> (com.gargoylesoftware.htmlunit.html.HtmlPage,org.w3c.dom.Element)]
> com.gargoylesoftware.htmlunit.html.SimpleHtmlElementCreator.create
> (SimpleHtmlElementCreator.java:91)
> com.gargoylesoftware.htmlunit.html.HtmlPage.getHtmlElement
> (HtmlPage.java:464)
>   
> com.gargoylesoftware.htmlunit.html.HtmlElement.getHtmlElementsByTagName 
> s
> (HtmlElement.java:721)
>   
> com.gargoylesoftware.htmlunit.html.HtmlPage.initializeFramesIfNeeded
> (HtmlPage.java:1108)
>   
> com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java: 
> 172)
> com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:356)
> com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:276)
>   
> com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:169)
>   
> com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:120)
> ---------------list end---------------
> 
> 
> Can anyone help me with this?
> 
> Cheers,
> 
> Gian Franco Casula



-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user