htmlunit-user — Discussion of the use of HtmlUnit

Re: [Htmlunit-user] How to change a form background color

[Jim X. <jx...@cl...>]

George,

         Here is what I want to do:
     1. display url in JWebBrowser (JWebBrowser.navigate(url))
     2. final HtmlPage page = WebClient.getPage(url);
     3. parse HtmlPage to get a list of HtmlElement.
     4. change some style on HtmlElemnt using first email of code.
     5. New change of style should be seen in JWebBrowser.

   At this point, I cannot see new change in JWebBrowser.

Jim

----- Original Message -----
From: "George Dinwiddie" <li...@iD...>
To: htm...@li...
Sent: Monday, January 20, 2014 3:13:40 PM
Subject: Re: [Htmlunit-user] How to change a form background color

Jim,

On 1/20/14, 2:44 PM, Jim Xie wrote:
> I have the following code that changes a form style
>
>                  HtmlForm htmlForm = getForm();  //supposed to get a HtmlForm object.
>                  ScriptableObject scriptObject = htmlForm.getScriptObject();
>                  HTMLFormElement formElement = (HTMLFormElement)scriptObject;
>                  ComputedCSSStyleDeclaration style = formElement.getCurrentStyle();
>                  style.setBackgroundColor("#FF0000");
>                  style.setBorderColor("blue");
>
> The above code doesn't work. Please help me out.

What were you expecting it to do?

  - George

-- 
  ----------------------------------------------------------------------
   * George Dinwiddie *                      http://blog.gdinwiddie.com
   Software Development                    http://www.idiacomputing.com
   Consultant and Coach                    http://www.agilemaryland.org
  ----------------------------------------------------------------------


------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] How to change a form background color

[George D. <li...@iD...>]

Jim,

On 1/20/14, 2:44 PM, Jim Xie wrote:
> I have the following code that changes a form style
>
>                  HtmlForm htmlForm = getForm();  //supposed to get a HtmlForm object.
>                  ScriptableObject scriptObject = htmlForm.getScriptObject();
>                  HTMLFormElement formElement = (HTMLFormElement)scriptObject;
>                  ComputedCSSStyleDeclaration style = formElement.getCurrentStyle();
>                  style.setBackgroundColor("#FF0000");
>                  style.setBorderColor("blue");
>
> The above code doesn't work. Please help me out.

What were you expecting it to do?

  - George

-- 
  ----------------------------------------------------------------------
   * George Dinwiddie *                      http://blog.gdinwiddie.com
   Software Development                    http://www.idiacomputing.com
   Consultant and Coach                    http://www.agilemaryland.org
  ----------------------------------------------------------------------

[Htmlunit-user] How to change a form background color

[Jim X. <jx...@cl...>]

I have the following code that changes a form style

                HtmlForm htmlForm = getForm();  //supposed to get a HtmlForm object.
                ScriptableObject scriptObject = htmlForm.getScriptObject();
                HTMLFormElement formElement = (HTMLFormElement)scriptObject;
                ComputedCSSStyleDeclaration style = formElement.getCurrentStyle();
                style.setBackgroundColor("#FF0000");
                style.setBorderColor("blue");

The above code doesn't work. Please help me out.

Jim

Re: [Htmlunit-user] HtmlPage returning form instead of content

[Ahmed A. <asa...@ya...>]

Hi Arya,

Please provide more details, so others can reproduce the issue.

Ahmed

________________________________
 From: Arya Farzan <ary...@gm...>
To: Ahmed Ashour <asa...@ya...>; htm...@li... 
Sent: Sunday, January 19, 2014 11:16 PM
Subject: Re: [Htmlunit-user] HtmlPage returning form instead of content
 


This problem could also be because of a server error



On Sun, Jan 19, 2014 at 11:02 AM, Arya Farzan <ary...@gm...> wrote:

I tried it with 
>
>
>webClient.setAjaxController(new NicelyResynchronizingAjaxController()); 
>
>
>but I am still facing the same issue. I have also put Thread.sleep(5000) after
>
>
>HtmlPage page2 = form.getInputByValue("Submit").click();
>
>
>
>but it's not making any difference. Also I don't know if this is relevant but this is a custom constructed html page which I'm generating by this code
>
>
>URL url = new URL("ThePAGEURL");
>
>
>StringWebResponse response = new StringWebResponse("<html><head><title>attach</title></head><body><form method='post' action='POSTURL'><input type='file'   name='file0'><input type='text'   name='requests'   value='" + request + "'><input type='text'   name='automatic' value='false'><input type='submit' value='Submit'></form></body></html>", url);
>HtmlPage page = HTMLParser.parseHtml(response, webClient.getCurrentWindow());
>
>
>
>
>When the server's response is not returned but the html form is returned, there is one head tag printed and it looks like this: <head/>
>
>
>On Sun, Jan 19, 2014 at 2:58 AM, Ahmed Ashour <asa...@ya...> wrote:
>
>Hi Arya,
>>
>>
>>Please read http://htmlunit.sourceforge.net/faq.html#AJAXDoesNotWork
>>
>>
>>Ahmed
>>
>>________________________________
>> From: Arya Farzan <ary...@gm...>
>>To: htm...@li... 
>>Sent: Sunday, January 19, 2014 6:22 AM
>>Subject: [Htmlunit-user] HtmlPage returning form instead of content
>> 
>>
>>
>>I'm using the following code to submit a form
>>
>>
>>HtmlForm form = page.getForms().get(0);
>>form.getInputByName("file0").setValueAttribute("1.jpg");
>>HtmlPage page2 = form.getInputByValue("Submit").click();
>>
>>
>>
>>This "sometimes" returns the server's response, but sometimes returns the form itself when I print page2's text or xml.
>>
>>
>>Am I doing something wrong here?
>>
>>------------------------------------------------------------------------------
>>CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>>Learn Why More Businesses Are Choosing CenturyLink Cloud For
>>Critical Workloads, Development Environments & Everything In Between.
>>Get a Quote or Start a Free Trial Today.
>>http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>>_______________________________________________
>>Htmlunit-user mailing list
>>Htm...@li...
>>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>

Re: [Htmlunit-user] WebConnectionWrapper in 2.13

[Ahmed A. <asa...@ya...>]

Hi Arya,

It should work, please provide your complete case.

Ahmed

________________________________
 From: Arya Farzan <ary...@gm...>
To: htm...@li... 
Sent: Sunday, January 19, 2014 11:17 PM
Subject: [Htmlunit-user] WebConnectionWrapper in 2.13
 


Has the way of implementing WebConnectionWrapper changed in 2.13?

It used to work like this
new WebConnectionWrapper(webClient)

{

}

but it's no longer working.
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

[Htmlunit-user] WebConnectionWrapper in 2.13

[Arya F. <ary...@gm...>]

Has the way of implementing WebConnectionWrapper changed in 2.13?

It used to work like this
new WebConnectionWrapper(webClient)
{

}

but it's no longer working.

Re: [Htmlunit-user] HtmlPage returning form instead of content

[Arya F. <ary...@gm...>]

This problem could also be because of a server error


On Sun, Jan 19, 2014 at 11:02 AM, Arya Farzan <ary...@gm...> wrote:

> I tried it with
>
> webClient.setAjaxController(new NicelyResynchronizingAjaxController());
>
> but I am still facing the same issue. I have also put Thread.sleep(5000)
> after
>
> HtmlPage page2 = form.getInputByValue("Submit").click();
>
> but it's not making any difference. Also I don't know if this is relevant
> but this is a custom constructed html page which I'm generating by this code
>
> URL url = new URL("ThePAGEURL");
>
> StringWebResponse response = new
> StringWebResponse("<html><head><title>attach</title></head><body><form
> method='post' action='POSTURL'><input type='file'   name='file0'><input
> type='text'   name='requests'   value='" + request + "'><input type='text'
>   name='automatic' value='false'><input type='submit'
> value='Submit'></form></body></html>", url);
>  HtmlPage page = HTMLParser.parseHtml(response,
> webClient.getCurrentWindow());
>
>
> When the server's response is not returned but the html form is returned,
> there is one head tag printed and it looks like this: <head/>
>
> On Sun, Jan 19, 2014 at 2:58 AM, Ahmed Ashour <asa...@ya...> wrote:
>
>> Hi Arya,
>>
>> Please read http://htmlunit.sourceforge.net/faq.html#AJAXDoesNotWork
>>
>> Ahmed
>>   ------------------------------
>>  *From:* Arya Farzan <ary...@gm...>
>> *To:* htm...@li...
>> *Sent:* Sunday, January 19, 2014 6:22 AM
>> *Subject:* [Htmlunit-user] HtmlPage returning form instead of content
>>
>> I'm using the following code to submit a form
>>
>> HtmlForm form = page.getForms().get(0);
>> form.getInputByName("file0").setValueAttribute("1.jpg");
>> HtmlPage page2 = form.getInputByValue("Submit").click();
>>
>> This "sometimes" returns the server's response, but sometimes returns the
>> form itself when I print page2's text or xml.
>>
>> Am I doing something wrong here?
>>
>>
>>
>> ------------------------------------------------------------------------------
>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> Critical Workloads, Development Environments & Everything In Between.
>> Get a Quote or Start a Free Trial Today.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>

Re: [Htmlunit-user] HtmlPage returning form instead of content

[Arya F. <ary...@gm...>]

I tried it with

webClient.setAjaxController(new NicelyResynchronizingAjaxController());

but I am still facing the same issue. I have also put Thread.sleep(5000)
after

HtmlPage page2 = form.getInputByValue("Submit").click();

but it's not making any difference. Also I don't know if this is relevant
but this is a custom constructed html page which I'm generating by this code

URL url = new URL("ThePAGEURL");

StringWebResponse response = new
StringWebResponse("<html><head><title>attach</title></head><body><form
method='post' action='POSTURL'><input type='file'   name='file0'><input
type='text'   name='requests'   value='" + request + "'><input type='text'
  name='automatic' value='false'><input type='submit'
value='Submit'></form></body></html>", url);
HtmlPage page = HTMLParser.parseHtml(response,
webClient.getCurrentWindow());


When the server's response is not returned but the html form is returned,
there is one head tag printed and it looks like this: <head/>

On Sun, Jan 19, 2014 at 2:58 AM, Ahmed Ashour <asa...@ya...> wrote:

> Hi Arya,
>
> Please read http://htmlunit.sourceforge.net/faq.html#AJAXDoesNotWork
>
> Ahmed
>   ------------------------------
>  *From:* Arya Farzan <ary...@gm...>
> *To:* htm...@li...
> *Sent:* Sunday, January 19, 2014 6:22 AM
> *Subject:* [Htmlunit-user] HtmlPage returning form instead of content
>
> I'm using the following code to submit a form
>
> HtmlForm form = page.getForms().get(0);
> form.getInputByName("file0").setValueAttribute("1.jpg");
> HtmlPage page2 = form.getInputByValue("Submit").click();
>
> This "sometimes" returns the server's response, but sometimes returns the
> form itself when I print page2's text or xml.
>
> Am I doing something wrong here?
>
>
>
> ------------------------------------------------------------------------------
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>

Re: [Htmlunit-user] HtmlPage returning form instead of content

[Ahmed A. <asa...@ya...>]

Hi Arya,

Please read http://htmlunit.sourceforge.net/faq.html#AJAXDoesNotWork

Ahmed

________________________________
 From: Arya Farzan <ary...@gm...>
To: htm...@li... 
Sent: Sunday, January 19, 2014 6:22 AM
Subject: [Htmlunit-user] HtmlPage returning form instead of content
 


I'm using the following code to submit a form


HtmlForm form = page.getForms().get(0);
form.getInputByName("file0").setValueAttribute("1.jpg");
HtmlPage page2 = form.getInputByValue("Submit").click();


This "sometimes" returns the server's response, but sometimes returns the form itself when I print page2's text or xml.

Am I doing something wrong here?

[Htmlunit-user] HtmlPage returning form instead of content

[Arya F. <ary...@gm...>]

I'm using the following code to submit a form

HtmlForm form = page.getForms().get(0);
form.getInputByName("file0").setValueAttribute("1.jpg");
HtmlPage page2 = form.getInputByValue("Submit").click();

This "sometimes" returns the server's response, but sometimes returns the
form itself when I print page2's text or xml.

Am I doing something wrong here?

Re: [Htmlunit-user] change text of readonly html input

[Ahmed A. <asa...@ya...>]

Hi David,

Yes, you can change the value of readonly input in HtmlUnit, using .setValueAttribute();

Ahmed

________________________________
 From: David Michael Gang <mic...@gm...>
To: htm...@li... 
Sent: Thursday, January 16, 2014 3:55 PM
Subject: [Htmlunit-user] change text of readonly html input
 


Hi all,

Is it possible to change or clear the text of a readonly text box with htmlunit?
In selenium it throws me a org.openqa.selenium.InvalidElementStateException.
Is the behavior the same in htmlunit or is it possible?

Thanks,
David

[Htmlunit-user] change text of readonly html input

[David M. G. <mic...@gm...>]

Hi all,

Is it possible to change or clear the text of a readonly text box with
htmlunit?
In selenium it throws me a org.openqa.selenium.InvalidElementStateException.
Is the behavior the same in htmlunit or is it possible?

Thanks,
David

[Htmlunit-user] IE supported versions

[Ahmed A. <asa...@ya...>]

Hi all,

The team wants to get your opinion about the supported browsers.

As you know, we focus on latest FF ESR and Chrome, but for IE we give more time since it doesn't get updated as frequent as the others.

And as the team man power is very limited, the focus on specific version(s) would be much helpful.

What do you think about IE 8, 9, 10 and 11? Specifically in 6 months down the road (see Deprecation Policy [1]).

Ahmed

[1] http://htmlunit.sourceforge.net/codingConventions.html#Deprecation

Re: [Htmlunit-user] how to ignore javascript error in htmlunit 2.13

[Ahmed A. <asa...@ya...>]

Hi,

client.getOptions().setThrowExceptionOnScriptError()

Ahmed

________________________________
 From: Li Li <fan...@gm...>
To: htm...@li... 
Sent: Tuesday, January 14, 2014 2:38 PM
Subject: [Htmlunit-user] how to ignore javascript error in htmlunit 2.13
 

I found the question in stackoverflow but it seems out dated
http://stackoverflow.com/questions/8745061/is-it-possible-to-ignore-javascript-exceptions-when-working-with-webdriver-html

[Htmlunit-user] how to ignore javascript error in htmlunit 2.13

[Li Li <fan...@gm...>]

I found the question in stackoverflow but it seems out dated
http://stackoverflow.com/questions/8745061/is-it-possible-to-ignore-javascript-exceptions-when-working-with-webdriver-html

Re: [Htmlunit-user] htmlunit and (keep alive - incoorect) handling frames

[David M. G. <mic...@gm...>]

Hi all,

I had an error in the analysis.
There is another issue here
The html contains a frame

<frame src="http://xn--archivoespaoldearte-53b.revistas.csic.es/index.php/aea/article/viewDownloadInterstitial/557/554
<view-source:http://archivoespa%C3%B1oldearte.revistas.csic.es/index.php/aea/article/viewDownloadInterstitial/557/554>"
frameborder="0"/>

which is loaded and this frame performs the force download.

Working with the tutorial
*http://htmlunit.sourceforge.net/frame-howto.html
<http://htmlunit.sourceforge.net/frame-howto.html>*

solved the issue.


Thanks,

David




On Mon, Jan 6, 2014 at 10:19 AM, David Michael Gang
<mic...@gm...>wrote:

> Hi all,
>
> I have the following problem:
> When  fetching  the page with a browser:
>
> http://xn--archivoespaoldearte-53b.revistas.csic.es/index.php/aea/article/view/557/554
>
> I get a pdf.
>
> Through htmlunit i just get a html page.
>
>
> Here is the program
> package test;
>
> import java.io.File;
> import java.io.IOException;
> import java.io.InputStream;
> import java.net.MalformedURLException;
>
> import org.apache.commons.io.FileUtils;
> import org.apache.pdfbox.exceptions.COSVisitorException;
>
> import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
> import com.gargoylesoftware.htmlunit.ImmediateRefreshHandler;
> import com.gargoylesoftware.htmlunit.Page;
> import com.gargoylesoftware.htmlunit.WebClient;
>
>
> public class ForceDownload {
>
>     public static void main(String[] args) throws
> FailingHttpStatusCodeException, MalformedURLException, IOException,
> COSVisitorException {
>         WebClient client = new WebClient();
>         client.setRefreshHandler(new ImmediateRefreshHandler());
>
>         final String downloadUrl = "
> http://xn--archivoespaoldearte-53b.revistas.csic.es/index.php/aea/article/view/557/554
> ";
>         final Page page = client.getPage(downloadUrl);
>
>         System.out.println(page.getWebResponse().getContentType());
>
>         final InputStream is = page.getWebResponse().getContentAsStream();
>         FileUtils.copyInputStreamToFile(is, new File("file.pdf"));
>
>
>     }
>
> }
>
> I get as output the html file.
> I already tried to set ImmediateRefreshHandler but it did not help.
> I tried to understand why and saw through the firefox web developer that
> it sends keep alive signals.
> How can i refresh the page to wait for keep alive till i get the pdf page?
>
> Thanks,
> David
>

Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs

[Andrew P. <ape...@42...>]

I'm using HtmlUnit 2.13, by the way.


On Mon, Jan 6, 2014 at 9:57 AM, Andrew Pennebaker <ape...@42...>wrote:

> Can you reproduce in JDK 6?
>
>
> On Mon, Jan 6, 2014 at 9:53 AM, Ahmed Ashour <asa...@ya...> wrote:
>
>> Hi Andrew,
>>
>> I use Java 7 on Windows 7.
>>
>> Ahmed
>>   ------------------------------
>>  *From:* Andrew Pennebaker <ape...@42...>
>> *To:* Ahmed Ashour <asa...@ya...>; HtmlUnit Users <
>> htm...@li...>
>> *Sent:* Monday, January 6, 2014 5:43 PM
>> *Subject:* Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in
>> file:/... URIs
>>
>> I'm using Apple JDK 6 on Mac OS X 10.9.1 Mavericks. I see you're using
>> Windows, which OS and JDK version? Behavior may be different in Windows.
>>
>>
>> On Sun, Jan 5, 2014 at 2:19 AM, Ahmed Ashour <asa...@ya...> wrote:
>>
>> Hi Andrew,
>>
>> Can you provide your complete case, as the below works with me.
>>
>>
>>         HtmlPage page = webClient.getPage("file://c://use//=.html");
>>          System.out.println(page.asText());
>>
>>
>> Ahmed
>>   ------------------------------
>>  *From:* Andrew Pennebaker <ape...@42...>
>> *To:* HtmlUnit Users <htm...@li...>
>> *Sent:* Friday, January 3, 2014 9:39 PM
>> *Subject:* [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/...
>> URIs
>>
>> I'm trying to use local test resource .html pages in my unit tests,
>> rather than reaching out to the live, fluctuating Internet. Some of my test
>> pages have equals (=) in their filenames, which
>> ClassLoader.getSystemResource().toString() encodes as "%3d" in the
>> resulting URI string.
>>
>> Trouble is, HtmlUnit WebClient can't retrieve encoded URIs, so my unit
>> tests are giving false negatives.
>>
>> For now, I'm manually decoding the URIs before passing them to WebClient.
>>
>> In the future, could WebClient please automatically decode URIs?
>>
>> --
>> Cheers,
>>
>> Andrew Pennebaker
>> ape...@42...
>>
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most IT
>> organizations don't have a clear picture of how application performance
>> affects their revenue. With AppDynamics, you get 100% visibility into
>> your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
>> Pro!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most IT
>> organizations don't have a clear picture of how application performance
>> affects their revenue. With AppDynamics, you get 100% visibility into your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
>> Pro!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>>
>>
>> --
>> Cheers,
>>
>> Andrew Pennebaker
>> ape...@42...
>>
>>
>>
>
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>



-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs

[Andrew P. <ape...@42...>]

Can you reproduce in JDK 6?


On Mon, Jan 6, 2014 at 9:53 AM, Ahmed Ashour <asa...@ya...> wrote:

> Hi Andrew,
>
> I use Java 7 on Windows 7.
>
> Ahmed
>   ------------------------------
>  *From:* Andrew Pennebaker <ape...@42...>
> *To:* Ahmed Ashour <asa...@ya...>; HtmlUnit Users <
> htm...@li...>
> *Sent:* Monday, January 6, 2014 5:43 PM
> *Subject:* Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in
> file:/... URIs
>
> I'm using Apple JDK 6 on Mac OS X 10.9.1 Mavericks. I see you're using
> Windows, which OS and JDK version? Behavior may be different in Windows.
>
>
> On Sun, Jan 5, 2014 at 2:19 AM, Ahmed Ashour <asa...@ya...> wrote:
>
> Hi Andrew,
>
> Can you provide your complete case, as the below works with me.
>
>
>         HtmlPage page = webClient.getPage("file://c://use//=.html");
>         System.out.println(page.asText());
>
>
> Ahmed
>   ------------------------------
>  *From:* Andrew Pennebaker <ape...@42...>
> *To:* HtmlUnit Users <htm...@li...>
> *Sent:* Friday, January 3, 2014 9:39 PM
> *Subject:* [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/...
> URIs
>
> I'm trying to use local test resource .html pages in my unit tests, rather
> than reaching out to the live, fluctuating Internet. Some of my test pages
> have equals (=) in their filenames, which
> ClassLoader.getSystemResource().toString() encodes as "%3d" in the
> resulting URI string.
>
> Trouble is, HtmlUnit WebClient can't retrieve encoded URIs, so my unit
> tests are giving false negatives.
>
> For now, I'm manually decoding the URIs before passing them to WebClient.
>
> In the future, could WebClient please automatically decode URIs?
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>
>
>


-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs

[Ahmed A. <asa...@ya...>]

Hi Andrew,

I use Java 7 on Windows 7.

Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: Ahmed Ashour <asa...@ya...>; HtmlUnit Users <htm...@li...> 
Sent: Monday, January 6, 2014 5:43 PM
Subject: Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs
 


I'm using Apple JDK 6 on Mac OS X 10.9.1 Mavericks. I see you're using Windows, which OS and JDK version? Behavior may be different in Windows.



On Sun, Jan 5, 2014 at 2:19 AM, Ahmed Ashour <asa...@ya...> wrote:

Hi Andrew,
>
>
>Can you provide your complete case, as the below works with me.
>
>
>
>
>        HtmlPage page = webClient.getPage("file://c://use//=.html");
>        System.out.println(page.asText());
>
>
>
>
>Ahmed
>
>________________________________
> From: Andrew Pennebaker <ape...@42...>
>To: HtmlUnit Users <htm...@li...> 
>Sent: Friday, January 3, 2014 9:39 PM
>Subject: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs
> 
>
>
>I'm trying to use local test resource .html pages in my unit tests, rather than reaching out to the live, fluctuating Internet. Some of my test pages have equals (=) in their filenames, which ClassLoader.getSystemResource().toString() encodes as "%3d" in the resulting URI string.
>
>
>Trouble is, HtmlUnit WebClient can't retrieve encoded URIs, so my unit tests are giving false negatives.
>
>
>For now, I'm manually decoding the URIs before passing them to WebClient.
>
>
>In the future, could WebClient please automatically decode URIs?
>
>
>-- 
>
>Cheers,
>
>
>Andrew Pennebaker
>ape...@42...
>------------------------------------------------------------------------------
>Rapidly troubleshoot problems before they affect your business. Most IT 
>organizations don't have a clear picture of how application performance 
>affects their revenue. With AppDynamics, you get 100% visibility into your 
>Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>------------------------------------------------------------------------------
>Rapidly troubleshoot problems before they affect your business. Most IT
>organizations don't have a clear picture of how application performance
>affects their revenue. With AppDynamics, you get 100% visibility into your
>Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 

Cheers,


Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs

[Andrew P. <ape...@42...>]

I'm using Apple JDK 6 on Mac OS X 10.9.1 Mavericks. I see you're using
Windows, which OS and JDK version? Behavior may be different in Windows.


On Sun, Jan 5, 2014 at 2:19 AM, Ahmed Ashour <asa...@ya...> wrote:

> Hi Andrew,
>
> Can you provide your complete case, as the below works with me.
>
>
>         HtmlPage page = webClient.getPage("file://c://use//=.html");
>         System.out.println(page.asText());
>
>
> Ahmed
>   ------------------------------
>  *From:* Andrew Pennebaker <ape...@42...>
> *To:* HtmlUnit Users <htm...@li...>
> *Sent:* Friday, January 3, 2014 9:39 PM
> *Subject:* [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/...
> URIs
>
> I'm trying to use local test resource .html pages in my unit tests, rather
> than reaching out to the live, fluctuating Internet. Some of my test pages
> have equals (=) in their filenames, which
> ClassLoader.getSystemResource().toString() encodes as "%3d" in the
> resulting URI string.
>
> Trouble is, HtmlUnit WebClient can't retrieve encoded URIs, so my unit
> tests are giving false negatives.
>
> For now, I'm manually decoding the URIs before passing them to WebClient.
>
> In the future, could WebClient please automatically decode URIs?
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
Cheers,

Andrew Pennebaker
ape...@42...

[Htmlunit-user] htmlunit and keep alive

[David M. G. <mic...@gm...>]

Hi all,

I have the following problem:
When  fetching  the page with a browser:
http://xn--archivoespaoldearte-53b.revistas.csic.es/index.php/aea/article/view/557/554

I get a pdf.

Through htmlunit i just get a html page.


Here is the program
package test;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;

import org.apache.commons.io.FileUtils;
import org.apache.pdfbox.exceptions.COSVisitorException;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.ImmediateRefreshHandler;
import com.gargoylesoftware.htmlunit.Page;
import com.gargoylesoftware.htmlunit.WebClient;


public class ForceDownload {

    public static void main(String[] args) throws
FailingHttpStatusCodeException, MalformedURLException, IOException,
COSVisitorException {
        WebClient client = new WebClient();
        client.setRefreshHandler(new ImmediateRefreshHandler());

        final String downloadUrl = "
http://xn--archivoespaoldearte-53b.revistas.csic.es/index.php/aea/article/view/557/554
";
        final Page page = client.getPage(downloadUrl);

        System.out.println(page.getWebResponse().getContentType());

        final InputStream is = page.getWebResponse().getContentAsStream();
        FileUtils.copyInputStreamToFile(is, new File("file.pdf"));


    }

}

I get as output the html file.
I already tried to set ImmediateRefreshHandler but it did not help.
I tried to understand why and saw through the firefox web developer that it
sends keep alive signals.
How can i refresh the page to wait for keep alive till i get the pdf page?

Thanks,
David

Re: [Htmlunit-user] deal with application/force-download

[David M. G. <mic...@gm...>]

Hi,

This helped .

Thanks,
David


>
> On Thu, Jan 2, 2014 at 4:21 PM, <
> htm...@li...> wrote:
>
>> Send Htmlunit-user mailing list submissions to
>>         htm...@li...
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>> or, via email, send a message with subject or body 'help' to
>>         htm...@li...
>>
>> You can reach the person managing the list at
>>         htm...@li...
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Htmlunit-user digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Re: Htmlunit-user Digest, Vol 92, Issue 1 (David Michael Gang)
>>    2. Re: deal with application/force-download (Ahmed Ashour)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Thu, 2 Jan 2014 14:56:57 +0200
>> From: David Michael Gang <mic...@gm...>
>> Subject: Re: [Htmlunit-user] Htmlunit-user Digest, Vol 92, Issue 1
>> To: htm...@li...
>> Message-ID:
>>         <
>> CAJ...@ma...>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Hi Ahmed,
>>
>> Thanks for your reply.
>> This solves halve of the problem.
>> The immediate refresh handler redirects me automatically to the page which
>> sends me the header "application/force-download".
>> The question is how to emulate the browser behavior so that i get the pdf
>> page automatically.
>> Here is the code:
>>
>> package test;
>>
>> import java.io.IOException;
>> import java.net.MalformedURLException;
>>
>> import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
>> import com.gargoylesoftware.htmlunit.ImmediateRefreshHandler;
>> import com.gargoylesoftware.htmlunit.Page;
>> import com.gargoylesoftware.htmlunit.WebClient;
>>
>>
>> public class ForceDownload {
>>
>>     public static void main(String[] args) throws
>> FailingHttpStatusCodeException, MalformedURLException, IOException {
>>         WebClient client = new WebClient();
>>         client.setRefreshHandler(new ImmediateRefreshHandler());
>>
>>         final String downloadUrl = "
>>
>> http://archivoespa?oldearte.revistas.csic.es/index.php/aea/article/download/552/549
>> <
>> http://xn--archivoespaoldearte-53b.revistas.csic.es/index.php/aea/article/download/552/549
>> >
>> ";
>>         final Page page = client.getPage(downloadUrl);
>>
>>         System.out.println(page.getWebResponse().getContentType());
>>
>>     }
>>
>> }
>>
>> I get the output:
>> application/force-download
>>
>> How can i get to the pdf?
>>
>> Thanks,
>> David
>>
>>

[Htmlunit-user] ImmediateRefreshHandler vs NiceRefreshHandler

[David M. G. <mic...@gm...>]

Hi,

The current default of htmlunit is NiceRefreshHandler with 2 seconds wait
time.
I use htmlunit primarily for data scraping.
Sometimes there is a refresh after a waiting time which is more than 2
seconds and then i don't get the data.
The solution suggested here is to use ImmediateRefreshHandler.
Is there any possible downside to use for the purpose of data scraping the
ImmediateRefreshHandler or could i use it safely for any web page i scrape?


Thanks,
David

Re: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs

[Ahmed A. <asa...@ya...>]

Hi Andrew,

Can you provide your complete case, as the below works with me.


        HtmlPage page = webClient.getPage("file://c://use//=.html");
        System.out.println(page.asText());


Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: HtmlUnit Users <htm...@li...> 
Sent: Friday, January 3, 2014 9:39 PM
Subject: [Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs
 


I'm trying to use local test resource .html pages in my unit tests, rather than reaching out to the live, fluctuating Internet. Some of my test pages have equals (=) in their filenames, which ClassLoader.getSystemResource().toString() encodes as "%3d" in the resulting URI string.

Trouble is, HtmlUnit WebClient can't retrieve encoded URIs, so my unit tests are giving false negatives.

For now, I'm manually decoding the URIs before passing them to WebClient.

In the future, could WebClient please automatically decode URIs?

-- 

Cheers,


Andrew Pennebaker
ape...@42...
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

[Htmlunit-user] HtmlUnit can't load "%3d" ("=") in file:/... URIs

[Andrew P. <ape...@42...>]

I'm trying to use local test resource .html pages in my unit tests, rather
than reaching out to the live, fluctuating Internet. Some of my test pages
have equals (=) in their filenames, which
ClassLoader.getSystemResource().toString() encodes as "%3d" in the
resulting URI string.

Trouble is, HtmlUnit WebClient can't retrieve encoded URIs, so my unit
tests are giving false negatives.

For now, I'm manually decoding the URIs before passing them to WebClient.

In the future, could WebClient please automatically decode URIs?

-- 
Cheers,

Andrew Pennebaker
ape...@42...