Re: [Htmlunit-user] Exception when using Page.submit()

[David D.K. <ddk...@ki...>]

It looks like WebSphere is running out of memory.  Have you tried =20
profiling your application using either WSAD or the WAS Admin Console =20=

utilities?  Have you tried increasing the heap size for your =20
application?

Have you considered running the Htmlunit code in an Action (if you're =20=

using Struts) or a servlet before you get to the JSP so that you simply =20=

output data once you're in the JSP?  (Typically, it's not considered =20
good design to write Java code inside a JSP.)

I've written a "screen scraping" application using Htmlunt recently, =20
and found that after creating some 200-300 HtmlPage objects, the JVM =20
runs out of memory using a heap size of 128MB.  If I increase the heap =20=

size to 256MB, I can load 600-700 HtmlPage objects before running out =20=

of memory.  I suspect that the loop between the WebClient and the =20
WebConnection may be causing the garbage collector issues (although I =20=

haven't had time to try to profile the code or testing the "breaking" =20=

of the circular dependency):

   =20
HtmlPage.getWebClient().getWebConnection().getWebClient().getWebConnecti=20=

on().adNaseum();

Is this a known issue?

Dave


On Oct 26, 2004, at 5:41 AM, Gian Franco Casula wrote:

> Hi,
>
> I've written some code in a jsp page that
> calls some java code that uses HtmlUnit to
> connect to a web site.
>
> The java code works correctly when I use it
> in isolation as a java application.
>
> Problems arise with the above mentioned jsp
> page. When the code is running I get a long
> list with the same message:
> ---------------list begin---------------
>
> =A0=A0=A0 ...
> =A0=A0=A0 [26-10-04 8:54:20:914 CEST] 5dc773e1 HttpMethodBas W
> =A0=A0=A0 org.apache.commons.httpclient.HttpMethodBase
> =A0=A0=A0 Response content length is not known
> =A0=A0=A0 [26-10-04 8:54:30:398 CEST] 5dc773e1 HttpMethodBas W
> =A0=A0=A0 org.apache.commons.httpclient.HttpMethodBase
> =A0=A0=A0 Response content length is not known
> =A0=A0=A0 ...
> ---------------list end---------------
> =A0
> This continues for a long time, until I get the message:
> =A0
> ---------------list begin---------------
> JVMDG315: JVM Requesting Heap dump file
> ...JVMDG318: Heap dump file written to E:\Program =20
> Files\IBM\WebSphere...
> JVMDG303: JVM Requesting Java core file
> JVMDG304: Java core file written to E:\Program Files\IBM\WebSphere\...
> JVMDG215: Dump Handler has Processed Outofmemory Signal -1.
> ---------------list end---------------
> =A0
> A printStackTrace() shows the following:
>
> ---------------list begin---------------
> =A0=A0=A0 com.gargoylesoftware.htmlunit.ObjectInstantiationException:
> =A0=A0=A0 =A0=A0=A0=A0 Exception when calling constructor
> =A0=A0=A0 =A0=A0=A0=A0 [com.gargoylesoftware.htmlunit.html.HtmlFrame
> =A0=A0=A0=A0=A0=A0=A0=A0 =20
> (com.gargoylesoftware.htmlunit.html.HtmlPage,org.w3c.dom.Element)]
> =A0=A0=A0 =
com.gargoylesoftware.htmlunit.html.SimpleHtmlElementCreator.create
> =A0=A0=A0 =A0=A0=A0=A0 (SimpleHtmlElementCreator.java:91)
> =A0=A0=A0 com.gargoylesoftware.htmlunit.html.HtmlPage.getHtmlElement
> =A0=A0=A0 =A0=A0=A0=A0 (HtmlPage.java:464)
> =A0=A0=A0 =20
> =
com.gargoylesoftware.htmlunit.html.HtmlElement.getHtmlElementsByTagName=20=

> s
> =A0=A0=A0 =A0=A0=A0=A0 (HtmlElement.java:721)
> =A0=A0=A0 =20
> com.gargoylesoftware.htmlunit.html.HtmlPage.initializeFramesIfNeeded
> =A0=A0=A0 =A0=A0=A0=A0 (HtmlPage.java:1108)
> =A0=A0=A0 =20
> com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:=20=

> 172)
> =A0=A0=A0 =
com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:356)
> =A0=A0=A0 =
com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:276)
> =A0=A0=A0 =20
> com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:169)
> =A0=A0=A0 =20
> com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:120)
> ---------------list end---------------
> =A0
> =A0
> Can anyone help me with this?
> =A0
> Cheers,
> =A0
> Gian Franco Casula