htmlunit-user — Discussion of the use of HtmlUnit

Re: [Htmlunit-user] GargoyleWrapper0()

[Mike B. <mb...@Ga...>]

Vinay Murthy wrote:
> Do onClick attributes get wrapped by GargoyleWrapper0() ? I have a
> button whose onclick attribute corresponds to calling a javascript
> function. As I click the button, the ScriptPreProcessor  traps a js
> corresponding to "function GargoyleWrapper0() { Submitform("XYZ")} "
> Is this normal ? Could you please help me figure out what might be
> happening here ? I would normally expect the ScriptPreProcessor to
> trap all js snippets appearing in a page. And if it has trapped the
> one mentioned above, was the wrapper added into the response somewhere
> by HtnlUnit ?

Any time there is javascript specified in an event handler (ie onload) 
then we wrap it in a generated javascript function and execute it. 
These functions are named "GargoyleWrapper" + some unique number.

We have to do this so that we can trap return values from the event 
handlers.

-- 
Mike Bowler
Principal, Gargoyle Software Inc.
Voice: (416) 822-0973 | Email  : mb...@Ga...
Fax  : (416) 822-0975 | Website: http://www.GargoyleSoftware.com

[Htmlunit-user] Exception when using Page.submit()

[Gian F. C. <gia...@to...>]

Hi,

I've written some code in a jsp page that
calls some java code that uses HtmlUnit to
connect to a web site.

The java code works correctly when I use it
in isolation as a java application.

Problems arise with the above mentioned jsp
page. When the code is running I get a long
list with the same message:

---------------list begin---------------
    ...
    [26-10-04 8:54:20:914 CEST] 5dc773e1 HttpMethodBas W
    org.apache.commons.httpclient.HttpMethodBase
    Response content length is not known
    [26-10-04 8:54:30:398 CEST] 5dc773e1 HttpMethodBas W
    org.apache.commons.httpclient.HttpMethodBase
    Response content length is not known
    ...
---------------list end---------------

This continues for a long time, until I get the message:

---------------list begin---------------
JVMDG315: JVM Requesting Heap dump file
...JVMDG318: Heap dump file written to E:\Program Files\IBM\WebSphere...
JVMDG303: JVM Requesting Java core file
JVMDG304: Java core file written to E:\Program Files\IBM\WebSphere\...
JVMDG215: Dump Handler has Processed Outofmemory Signal -1.
---------------list end---------------

A printStackTrace() shows the following:

---------------list begin---------------
    com.gargoylesoftware.htmlunit.ObjectInstantiationException:
         Exception when calling constructor
         [com.gargoylesoftware.htmlunit.html.HtmlFrame
         =
(com.gargoylesoftware.htmlunit.html.HtmlPage,org.w3c.dom.Element)]
    com.gargoylesoftware.htmlunit.html.SimpleHtmlElementCreator.create
         (SimpleHtmlElementCreator.java:91)
    com.gargoylesoftware.htmlunit.html.HtmlPage.getHtmlElement
         (HtmlPage.java:464)
    =
com.gargoylesoftware.htmlunit.html.HtmlElement.getHtmlElementsByTagNames
         (HtmlElement.java:721)
    com.gargoylesoftware.htmlunit.html.HtmlPage.initializeFramesIfNeeded
         (HtmlPage.java:1108)
    =
com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:172)=

    com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:356)
    com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:276)
    =
com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:169)
    =
com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:120)
---------------list end---------------


Can anyone help me with this?

Cheers,

Gian Franco Casula

[Htmlunit-user] GargoyleWrapper0()

[Vinay M. <vin...@gm...>]

Hi,
Do onClick attributes get wrapped by GargoyleWrapper0() ? I have a
button whose onclick attribute corresponds to calling a javascript
function. As I click the button, the ScriptPreProcessor  traps a js
corresponding to "function GargoyleWrapper0() { Submitform("XYZ")} "
Is this normal ? Could you please help me figure out what might be
happening here ? I would normally expect the ScriptPreProcessor to
trap all js snippets appearing in a page. And if it has trapped the
one mentioned above, was the wrapper added into the response somewhere
by HtnlUnit ?

Best Regards
Vinay

[Htmlunit-user] wiki

[Brad C. <yo...@br...>]

Seems I spoke too soon :)

https://sourceforge.net/tracker/index.php?func=detail&aid=1052417&group_id=1&atid=350001

The important part is that sometime in the not-so-distant future sf.net will be
upgrading the web servers with python 2.3.3. Once this is complete I will get us a
copy of the MoinMoin wiki clone installed, primarily for examples but available for
anything HtmlUnit related.

Brad C

[Htmlunit-user] re-direction

[Vinay M. <vin...@gm...>]

Hi,

Is there a way through which I could see the intermediate urls when I
get redirected from one page to another ? Say, I submit some form data
to page A, where a decision is taken(based on form data) to redirect
me to page B. Is there a way to figure out the url of B ?

Best Regards
Vinay

Re: [Htmlunit-user] Cookie handling

[Mike B. <mb...@Ga...>]

Vinay Murthy wrote:
> Is there a way that I can actually get notified whenever I receive a
> cookie ? Something like an eventhandler ?

Not at present.


-- 
Mike Bowler
Principal, Gargoyle Software Inc.
Voice: (416) 822-0973 | Email  : mb...@Ga...
Fax  : (416) 822-0975 | Website: http://www.GargoyleSoftware.com

[Htmlunit-user] Cookie handling

[Vinay M. <vin...@gm...>]

Hi,
Firstly, the ScriptPreProcessor is cool. I have been able to navigate
through a large portion of my testsite, by replacing all faulty
javascript with correct ones.
Is there a way that I can actually get notified whenever I receive a
cookie ? Something like an eventhandler ?

Best Regards
Vinay

[Htmlunit-user] Re: Re: htmlunit ignores some pages

[Niklas V. <nik...@ja...>]

OK, so maybe the hyphen is not the issue. The problem is that the documents 
don't have a complete ending tag. In my example, 
i) <! DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" >

is skipped altogether. I suppose the reason is that the space at the end of 
the tag (before the >) confuses the parser, making it expect 
a url such as "http://www.w3.org/TR/html4/loose.dtd" at the end. 

But - simply removing the last space, which gives us:
ii) <! DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

works just fine with htmlunit. This is very strange behaviour, wouldn't you 
say? If it is ok to exclude the ending url, then it should of course be ok to 
have spaces at the end of the tag (before the >). In particular since a HTML 
parser is not supposed to care too much about spaces. It just seems a bit too 
picky :)

/ Niklas

Re: [Htmlunit-user] htmlunit ignores some pages

[Brad C. <yo...@br...>]

I'd guess that putting the hyphen outside the quotes makes in invaild so nekohtml
just throws it out. Our doctype looks like this and works just fine:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

Could you provide a small test case for your error?

Brad C


--- Niklas Vargensten <nik...@ja...> wrote:

> I am trying to parse various downloaded html documents with htmlunit, using a 
> mock web-connection. But - all pages starting with the following line (before 
> <HTML>):
> 
> <! DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
> 
> are completely ignored by the parser. However, if I change the line to 
> 
> <! DOCTYPE HTML PUBLIC - "//W3C//DTD HTML 4.01 Transitional//EN">
> 
> (the only difference being the hyphen placed outside the double quotes) then 
> it works just fine. How is this possible?
> 
> / Niklas
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
> 

Re: [Htmlunit-user] problem in accessing a specific window

[Brad C. <yo...@br...>]

You might also try registering a WebWindowListener:

http://httpunit.sourceforge.net/doc/api/com/meterware/httpunit/WebWindowListener.html

Brad C

--- Mike Bowler <mb...@Ga...> wrote:

> Vinay Murthy wrote:
> > I need to be able to access window #2. I know window #2 exists. I
> > checked it out from webClient.getWebWindows() and sequentially dumped
> > all the pages in the list.
> > 
> > How do I access window #2 ?
> 
> WebClient.getWebWindows() will return a list of all the windows 
> currently accessible.  If you don't have a name for the window you want 
> then you'd have to look for some specific content.
> 
> -- 
> Mike Bowler

[Htmlunit-user] htmlunit ignores some pages

[Niklas V. <nik...@ja...>]

I am trying to parse various downloaded html documents with htmlunit, using a 
mock web-connection. But - all pages starting with the following line (before 
<HTML>):

<! DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

are completely ignored by the parser. However, if I change the line to 

<! DOCTYPE HTML PUBLIC - "//W3C//DTD HTML 4.01 Transitional//EN">

(the only difference being the hyphen placed outside the double quotes) then 
it works just fine. How is this possible?

/ Niklas

Re: [Htmlunit-user] problem in accessing a specific window

[Mike B. <mb...@Ga...>]

Vinay Murthy wrote:
> I need to be able to access window #2. I know window #2 exists. I
> checked it out from webClient.getWebWindows() and sequentially dumped
> all the pages in the list.
> 
> How do I access window #2 ?

WebClient.getWebWindows() will return a list of all the windows 
currently accessible.  If you don't have a name for the window you want 
then you'd have to look for some specific content.

-- 
Mike Bowler
Principal, Gargoyle Software Inc.
Voice: (416) 822-0973 | Email  : mb...@Ga...
Fax  : (416) 822-0975 | Website: http://www.GargoyleSoftware.com

[Htmlunit-user] problem in accessing a specific window

[Vinay M. <vin...@gm...>]

Hi,
I needed a way to specifically access a particular window .
Unfortunately this window doesn't have a name and I guess, its not
even the current window. Here's the sequence of events happening
1. Window # 1 contains two frames, say window #1.1 and window #1.2  I
click on a button in window #1.1 which results in a page getting
loaded in #1.2. This page comprises chiefly of a javascript snippet
which launches(using window.open) a new window(nameless) #2

2. After window #2 is populated with its respective page, window #1.2
changes its content to what it was in the beginning of 1.

I need to be able to access window #2. I know window #2 exists. I
checked it out from webClient.getWebWindows() and sequentially dumped
all the pages in the list.

How do I access window #2 ?

Regards
Vinay

Re: [Htmlunit-user] ScriptPreProcessor

[Brad C. <yo...@br...>]

The example test case I mentioned before points out that a second <script> tag is a
separate calling of the processor.


--- Vinay Murthy <vin...@gm...> wrote:

> Hi,
> I had a couple questions about ScriptPreProcessor:
> 
> 1. is the preProcess() function called in an event-based manner, say
> whenever the preprocessor encounter a <script> tag ?
> 
> 2. Will I get the entire js present in the page (the ones defined
> within <head></head> and the ones present in the body) as one single
> string which I'll have to parse to get to the script which I might
> want to change/replace?
> 
> Best Regards
> Vinay

Re: [Htmlunit-user] com.gargoylesoftware.base ?

[Brad C. <yo...@br...>]

--- Mike Bowler <mb...@Ga...> wrote:

> Vinay Murthy wrote:
> > I figured out that the nightly build available for download from the
> > htmlunit website doesnt contain the appropriate version of WebClient.
> 
> That's odd.  It appears that the last nightly build happened on 
> September 29.

sf.net turned off cron a while back which killed our nightly builds:

https://sourceforge.net/tracker/?func=detail&aid=1000129&group_id=1&atid=200001

I looked at trying to do it in the compile farm but you can't connect back to the
shell/web servers from there so we'd still need cron on the shell servers to make it
automatic. I've got 3 CruiseControl machines here at work I might could use unless
someone has a better idea.

Brad C

Re: [Htmlunit-user] com.gargoylesoftware.base ?

[Mike B. <mb...@Ga...>]

Vinay Murthy wrote:
> I figured out that the nightly build available for download from the
> htmlunit website doesnt contain the appropriate version of WebClient.

That's odd.  It appears that the last nightly build happened on 
September 29.

> The files in the folder src/test/java
> import from a package com.gargoylesoftware.base . Where is this
> package located ? Which dev path ?

http://gsbase.sourceforge.net

This page on the website lists all the dependencies with links to the 
homepages for those projects.

http://htmlunit.sourceforge.net/dependencies.html

If you use maven to build the source (the recommended way) then it will 
automatically download any required dependencies.  Maven can be found at 
http://maven.apache.org

-- 
Mike Bowler
Principal, Gargoyle Software Inc.
Voice: (416) 822-0973 | Email  : mb...@Ga...
Fax  : (416) 822-0975 | Website: http://www.GargoyleSoftware.com

[Htmlunit-user] com.gargoylesoftware.base ?

[Vinay M. <vin...@gm...>]

Hi,
I figured out that the nightly build available for download from the
htmlunit website doesnt contain the appropriate version of WebClient.
I guess it points to sept 26th. I was looking out for the
setScriptPreProcessor() method. I checked out code from the CVS
repository to build it myself. The files in the folder src/test/java
import from a package com.gargoylesoftware.base . Where is this
package located ? Which dev path ?

As a result of this, I wasnt able to build the source. Could somebody
help me out here please ?

Regards
Vinay

[Htmlunit-user] ScriptPreProcessor

[Vinay M. <vin...@gm...>]

Hi,
I had a couple questions about ScriptPreProcessor:

1. is the preProcess() function called in an event-based manner, say
whenever the preprocessor encounter a <script> tag ?

2. Will I get the entire js present in the page (the ones defined
within <head></head> and the ones present in the body) as one single
string which I'll have to parse to get to the script which I might
want to change/replace?

Best Regards
Vinay

[Htmlunit-user] wiki

[Brad C. <br...@br...>]

After a bit of testing it seems MoinMoin will not run on sf.net's server because
they run an age-old python version (1.5.2 vs 2.3.4, current MoinMoin requires 2.2).
People have been asking sf.net to get it upgraded for 2 years so I wouldn't hold
your breath...too bad too because the new MoinMoin 1.3 that's in beta has nice java
code coloring support :(

I guess this means any wiki we pick must be either perl or php.

Brad C

[Htmlunit-user] Wiki

[Brad C. <br...@br...>]

http://twikiplugins.sourceforge.net/cgi-bin/twiki/view/Sandbox/JavaTest

It seems to use red for the code and brown for the highlights but the default of red
for the <pre> comes from the skin, not the plugin. Once that's changed I think it
would look pretty good. If we could get it to link the class names back to
webcvs.... :)

Any other suggestions out there? Maybe something that uses mysql?

Brad C

Re: [Htmlunit-user] how to get around a faulty javascript code

[Brad C. <pr...@ya...>]

Actually, the ScriptPreProcessor that was committed for the ActiveX object support
gives the ability to hack out any javascript before it executes...it's pretty ugly
but it'll do the job in a pinch.

I've committed an "example" test case as
WebClientTest.testScriptPreProcessor_UnimplementedJavascript()

Brad C

--- Mike Bowler <mb...@Ga...> wrote:

> Vinay Murthy wrote:
> > Is there a way to get around a faulty javascript code, say a function
> > without disabling javascript execution ? Can I override this faulty
> > function on the page in some way through a correct function in my code
> > ?
> 
> Not that I'm aware of.  Disabling javascript is the only way I know.
> 

RE: [Htmlunit-user] Re:Re: how do I handle pop-up windows ?

[Erskine, C. <chr...@ed...>]

If frmSubmit does not factor into the pages that you are working on, it may
be an error from one of the earlier pages.  Does frmSubmit have any
Microsoft only code in it?  Not many of the Microsoft functions have been
coded yet.

Clicking on an image does work in the latest release.

Chris Erskine
 
EDS Consulting Services
F5-EDS-001
2424 Garden of the Gods Rd
Colorado Springs, CO  80919
 
Phone: 719-535-6064
 
Phone:
mailto:chr...@ed...

-----Original Message-----
From: Vinay Murthy [mailto:vin...@gm...] 
Sent: Monday, October 18, 2004 7:03 AM
To: htm...@li...
Subject: [Htmlunit-user] Re:Re: how do I handle pop-up windows ?

Hi,


  > EcmaError: lineNumber=[1] column=[6] lineSource=[return frmSubmit()]
  > name=[SyntaxError] sourceName=[body.onLoad] message=[invalid return]
  > errorObject=[SyntaxError: invalid return]
  > com.gargoylesoftware.htmlunit.ScriptException: invalid return
 
 The problem is in the onload handler of the new page that has just been 
 loaded.  What does frmSubmit() return?

 The page I am trying to access through a link happens to be a .asp .
 Interestingly, frmSubmit() doesn't figure anywhere in the page that
loads up freshly or the page where the link to this new page lies.
Actually, when I click on the link, I get a new browser window(minus
the tabs). And I noticed some kind of an activity additionally on the
previous window where the link lies.(a frame blanking out and then
getting loaded with the same previous contents)

Also, has the "clicking on an image" feature included in the latest
stable release ?

Best Regards,
Vinay


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] Re:Re: how do I handle pop-up windows ?

[Mike B. <mb...@Ga...>]

Vinay Murthy wrote:
>  The page I am trying to access through a link happens to be a .asp .
>  Interestingly, frmSubmit() doesn't figure anywhere in the page that
> loads up freshly or the page where the link to this new page lies.

It's possible that a redirect has happened or an external javascript 
file was loaded.

> Also, has the "clicking on an image" feature included in the latest
> stable release ?

Off hand, I'm not sure (and I'm in a rush so I don't have time to 
check).  The easiest way to find out would be to see if it has a click() 
method.  All changes are also tracked on this page

http://htmlunit.sourceforge.net/changes-report.html



-- 
Mike Bowler
Principal, Gargoyle Software Inc.
Voice: (416) 822-0973 | Email  : mb...@Ga...
Fax  : (416) 822-0975 | Website: http://www.GargoyleSoftware.com

Re: [Htmlunit-user] how to get around a faulty javascript code

[Mike B. <mb...@Ga...>]

Vinay Murthy wrote:
> Is there a way to get around a faulty javascript code, say a function
> without disabling javascript execution ? Can I override this faulty
> function on the page in some way through a correct function in my code
> ?

Not that I'm aware of.  Disabling javascript is the only way I know.

-- 
Mike Bowler
Principal, Gargoyle Software Inc.
Voice: (416) 822-0973 | Email  : mb...@Ga...
Fax  : (416) 822-0975 | Website: http://www.GargoyleSoftware.com

[Htmlunit-user] how to get around a faulty javascript code

[Vinay M. <vin...@gm...>]

Hi,
Is there a way to get around a faulty javascript code, say a function
without disabling javascript execution ? Can I override this faulty
function on the page in some way through a correct function in my code
?

Regards
Vinay