htmlunit-user — Discussion of the use of HtmlUnit

Re: [Htmlunit-user] How do I configure WebClient for different kinds of HTTP authentications?

[Ahmed A. <asa...@ya...>]

Hi Andrew,

It can be used for both, but you can fine tune it by:

    public void addCredentials(final String username, final String password, final String host,
            final int port, final String realm)


Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: Ahmed Ashour <asa...@ya...>; HtmlUnit Users <htm...@li...> 
Sent: Thursday, October 17, 2013 4:54 PM
Subject: Re: [Htmlunit-user] How do I configure WebClient for different kinds of HTTP authentications?
 


Yes. I want to clarify: Are the username/password shown there for authenticating with the proxy, or with the target website? Whichever one the username/password are for, how do I authenticate for the other one?



On Wed, Oct 16, 2013 at 5:03 PM, Ahmed Ashour <asa...@ya...> wrote:

Hi Andrew,
>
>
>It is included inside "Using a proxy server"
>
>
>    final DefaultCredentialsProvider credentialsProvider = (DefaultCredentialsProvider) webClient.getCredentialsProvider();
>    credentialsProvider.addCredentials("username", "password");
>
>
>Ahmed 
>
>________________________________
> From: Andrew Pennebaker <ape...@42...>
>To: HtmlUnit Users <htm...@li...> 
>Sent: Wednesday, October 16, 2013 9:21 PM
>Subject: [Htmlunit-user] How do I configure WebClient for different kinds of	HTTP authentications?
> 
>
>
>I read the snippet in the Getting Started guide, but I'm still a little confused. How can I:
>
>
>* Specify a username/password for accessing a proxy server?
>
>* Specify a username/password for reaching a protected web page?
>
>
>The Getting Started guide seems to only answer the first one question, but I'm also interested in the second.
>
>-- 
>
>Cheers,
>
>
>Andrew Pennebaker
>ape...@42...
>------------------------------------------------------------------------------
>October Webinars: Code for Performance
>Free Intel webinars can help you accelerate application performance.
>Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
>the latest Intel processors and coprocessors. See abstracts and register >
>http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>------------------------------------------------------------------------------
>October Webinars: Code for Performance
>Free Intel webinars can help you accelerate application performance.
>Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
>the latest Intel processors and coprocessors. See abstracts and register >
>http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 

Cheers,


Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] Upload more files in <input type="file" multiple/>

[Ahmed A. <asa...@ya...>]

Hi Zmirc,

Please open a feature-request.

Yours,
Ahmed

________________________________
 From: Zmirc <zmi...@gm...>
To: htm...@li... 
Sent: Thursday, October 17, 2013 4:54 PM
Subject: [Htmlunit-user] Upload more files in <input type="file" multiple/>
 


Hi!


I searched a lot about how to upload more files in HTML <input type="file" multiple/>tag with HtmlUnit in Java, but I didn't find out yet.
Can anyone help? I can't modify the web pages, because they're not
      mine.
I am currently using for single file <input/>:
HtmlFileInput#setValueAttribute(path);HtmlFileInput#setContentType(contenType);
Thank you so much!
There is also on Stackoverflow: http://stackoverflow.com/questions/19389426/how-to-upload-multiple-files-in-input-multiple-with-htmlunit-java 
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] Upload more files in <input type="file" multiple/>

[Zmirc <zmi...@gm...>]

Hi!

Any help with this one, please?
If not possible, is there any other alternative I could achieve this 
functionality?

Thanks a lot!
Mircea

On 10/17/2013 3:54 PM, Zmirc wrote:
> Hi!
>
> I searched a lot about how to upload more files in HTML|<input 
> type="file" multiple/>|tag with HtmlUnit in Java, but I didn't find 
> out yet.
> Can anyone help? I can't modify the web pages, because they're not mine.
>
> I am currently using for single file <input/>:
>
> |HtmlFileInput#setValueAttribute(path);
> HtmlFileInput#setContentType(contenType);|
>
> Thank you so much!
>
> There is also on Stackoverflow: 
> http://stackoverflow.com/questions/19389426/how-to-upload-multiple-files-in-input-multiple-with-htmlunit-java 

Re: [Htmlunit-user] How can I set a transport timeout?

[Ahmed A. <asa...@ya...>]

Hi Andrew,

webClient.getOptions().setTimeout().

Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: HtmlUnit Users <htm...@li...> 
Sent: Thursday, October 17, 2013 5:59 PM
Subject: [Htmlunit-user] How can I set a transport timeout?
 


I like that WebClient lets me set a JavaScript execution timeout. Can I also set a transport layer timeout, so that HTTP requests longer than X ms trigger an exception?

-- 

Cheers,


Andrew Pennebaker
ape...@42...
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] Feature request: ability to configure proxy settings after initializing a WebClient

[Ahmed A. <asa...@ya...>]

Hi Andrew,

webClient.getOptions().setProxyConfig()


Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: HtmlUnit Users <htm...@li...> 
Sent: Thursday, October 17, 2013 6:05 PM
Subject: [Htmlunit-user] Feature request: ability to configure proxy settings after initializing a WebClient
 


I'd like to be able to do WebClient webClient = new WebClient(); and then later, configure proxy settings. This would make my code a lot easier to refactor.

-- 

Cheers,


Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] Upload javadocs for com.gargoylesoftware.htmlunit.javascript.*

[Ahmed A. <asa...@ya...>]

Hi Andrew,

Nice catch!

JavaDocs are automatically generated, and it wasn't included as we are explicitly excluding all *.javascript packages.

For get/setJavaScriptEngine, they are marked as "Use at your own risk".

@others:
I guess JavaScriptErrorListener  should be moved to *.htmlunit package, what do you think?

Yours,
Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: HtmlUnit Users <htm...@li...> 
Sent: Thursday, October 17, 2013 7:29 PM
Subject: [Htmlunit-user] Upload javadocs for	com.gargoylesoftware.htmlunit.javascript.*
 


The API docs for HtmlUnit are excellent! I just noticed that the docs for the javasript engines, error listeners, etc. mentioned in the WebClient docs are missing.

Could we please upload the javascript API docs?

-- 

Cheers,


Andrew Pennebaker
ape...@42...
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] Fw: Bug 1344: is this fix in 2.12?

[Ahmed A. <asa...@ya...>]

Hi David,

Also, are you calling .closeAllWindows()?

Ahmed

________________________________
 From: Ahmed Ashour <asa...@ya...>
To: "htm...@li..." <htm...@li...> 
Sent: Friday, October 18, 2013 11:32 AM
Subject: Re: [Htmlunit-user] Fw: Bug 1344: is this fix in 2.12?
 


Hi David,

There are two cleanUp(), one in InMemory and another in OnFile.

Attached is a patch (against SVN), with which the test case below prints "Cleaning..." :)

What is the output of the test case below on your machine, the second Assertion fails?

Yours,
Ahmed

Re: [Htmlunit-user] Fw: Bug 1344: is this fix in 2.12?

[Ahmed A. <asa...@ya...>]

Index: src/main/java/com/gargoylesoftware/htmlunit/DownloadedContent.java
===================================================================
--- src/main/java/com/gargoylesoftware/htmlunit/DownloadedContent.java	(revision 8656)
+++ src/main/java/com/gargoylesoftware/htmlunit/DownloadedContent.java	(working copy)
@@ -82,6 +82,7 @@
         }
 
         public void cleanUp() {
+            System.out.println("Cleaning...");
             if (temporary_) {
                 FileUtils.deleteQuietly(file_);
             }
Index: src/main/java/com/gargoylesoftware/htmlunit/HttpWebConnection.java
===================================================================
--- src/main/java/com/gargoylesoftware/htmlunit/HttpWebConnection.java	(revision 8656)
+++ src/main/java/com/gargoylesoftware/htmlunit/HttpWebConnection.java	(working copy)
@@ -640,7 +640,7 @@
         return newWebResponseInstance(responseData, loadTime, request);
     }
 
-    private static final long MAX_IN_MEMORY = 500 * 1024;
+    private static final long MAX_IN_MEMORY = 1;
 
     /**
      * Downloads the response body.

Re: [Htmlunit-user] No such method: WebClient.setJavaScriptEnabled

[Ahmed A. <asa...@ya...>]

Hi Andrew,

Thanks for pointing it out, fixed in SVN.

Ahmed

________________________________
 From: Andrew Pennebaker <ape...@42...>
To: HtmlUnit Users <htm...@li...> 
Sent: Thursday, October 17, 2013 9:34 PM
Subject: Re: [Htmlunit-user] No such method: WebClient.setJavaScriptEnabled
 


Ah, WebClient.setJavaScriptEnabled(boolean) was deprecated in v2.11, but the official documentation was never updated to account for the new style of configuration. Could we please update that page?



On Thu, Oct 17, 2013 at 12:38 PM, Andrew Pennebaker <ape...@42...> wrote:

I'm trying to disable JavaScript, but it looks like it's no longer part of the API. Is there another way to disable it? I'm using HtmlUnit v2.13.
> 
>To disable JavaScript processing for one WebClient, call WebClient.setJavaScriptEnabled(false).
>
>
>--http://htmlunit.sourceforge.net/#JavaScript_Support
>
>
>Enable/disable JavaScript support. By default, this property is enabled.
>
>
>--http://www.netlikon.de/docs/javadoc-htmlunit/htmlunit-2.1/com/gargoylesoftware/htmlunit/WebClient.html#setJavaScriptEnabled(boolean)
>
>
>-- 
>
>Cheers,
>
>
>Andrew Pennebaker
>ape...@42...


-- 

Cheers,


Andrew Pennebaker
ape...@42...

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk

_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] Treatment of link with href="#"

[Ronald B. <rb...@rb...>]

Hi Satya,

we definitive need more details to understand what is going on.
Maybe a good starting point is to install Charles Web Proxy or maybe Fiddler and than have a look at the communication done by the browsers an HtmlUnit. If there is any difference this might be a 
good starting point to analyze the problem.


	RBRi


On Thu, 17 Oct 2013 07:28:07 -0700 (PDT) Satyanarayana K wrote:
>
>Hi Ahmed,
>
>The issue seems to be about Facebook Cross-domain Messaging. Is HtmlUnit handling this library?
>
>Thanks,
>Satya
>
>
>________________________________
> From: Ahmed Ashour <asa...@ya...>
>To: Satyanarayana K <kri...@ya...> 
>Sent: Thursday, October 17, 2013 1:30 PM
>Subject: Re: [Htmlunit-user] Treatment of link with href="#"
> 
>
>
>Hi Satya,
>
>- Please always use the user-list
>- The below is not a minimal case, please read http://htmlunit.sourceforge.net/submittingJSBugs.html
>
>Ahmed
>
>________________________________
> From: Satyanarayana K <kri...@ya...>
>To: Ahmed Ashour <asa...@ya...> 
>Sent: Thursday, October 17, 2013 9:58 AM
>Subject: Re: [Htmlunit-user] Treatment of link with href="#"
> 
>
>
>Hi Ahmed,
>
>I append below the test-case-code.
>
>Thanks,
>Satya
>
>*************
>
>System.out.println("***** ADHOC - BEGIN");
>WebClient wClnt = new WebClient();
>wClnt.getOptions().setCssEnabled(true);
>HtmlPage hPage = null;
>try{
> hPage = wClnt.getPage("http://www.skillpages.com/");
>System.out.println("Web-page-received: ".concat(((HtmlPage)hPage).getTitleText()));
>String linkDivPath = "//div[@class='inputContainer']";
>HtmlElement hElem = (HtmlElement)hPage.getByXPath(linkDivPath).get(0);
>HtmlInput hInput = (HtmlInput)hElem.getByXPath("./input[1]").get(0);
>hInput.setValueAttribute("Painter");
>HtmlElement clickElem = (HtmlElement)hElem.getByXPath("./a[1]").get(0);
>HtmlPage rPage = clickElem.click();
>wClnt.waitForBackgroundJavaScript(30000);
>wClnt.waitForBackgroundJavaScriptStartingBefore(30000);
>rPage = (HtmlPage)wClnt.getCurrentWindow().getEnclosedPage();
>System.out.println("Click-result-page: ".concat(rPage.getTitleText()));
>//Expected: //results/skills/Painter/0/Bangalore-KA-India/0/0/0/0/1?PromptOpp=true#
>}catch(Exception ex){
>System.out.println("Exception: ".concat(ex.getMessage() != null ? ex.getMessage() : "No Message!"));
>}
>System..out.println("ADHOC - END *****");
>
>*************
>
>
>________________________________
> From: Ahmed Ashour <asa...@ya...>
>To: Satyanarayana K <kri...@ya...>; "htm...@li..." <htm...@li...>; "htm...@li..." 
<htm...@li...> 
>Sent: Thursday, October 17, 2013 11:39 AM
>Subject: Re: [Htmlunit-user] Treatment of link with href="#"
> 
>
>
>Hi Satya,
>
>As always, tiny test cases are very helpful.
>
>I am not sure I understand "JavaScript that may be (indirectly) associated with a link"
>
>Yours,
>Ahmed
>
>________________________________
> From: Satyanarayana K <kri...@ya...>
>To: "htm...@li..." <htm...@li...> 
>Sent: Thursday, October 17, 2013 9:05 AM
>Subject: [Htmlunit-user] Treatment of link with href="#"
> 
>
>
>Hi,
>
>I wish to know if HtmlUnit - HtmlElement / HtmlAnchor.click() method includes code to execute any JavaScript that may be (indirectly) associated with a link with href="#", but without any 
value for onclick attribute.
>
>My experience / experiments tell me that no JavaScript is executed in such a case:
>
>1. HtmlUnit-based application fails to fetch/load the page that a browser does for such links.
>2. I have observed the net-work activity in the Windows TaskManager: a. when I click the link in a browser, the graph shows net-work activity, b. when the click() is called in the 
HtmlUnit-based app, the graph does not show any net-work 
>activity.
>
>If the click() method code does include JavaScript execution, what may be the reason for page not being fetched / loaded? I have already tried waiting upto even two minutes for JavaScript 
completion.
>
>If the click() method code  does not include JavaScript execution, any work-around that I can immediately try?
>
>Thanks,
>Satya
>------------------------------------------------------------------------------
>October Webinars: Code for Performance
>Free Intel webinars can help you accelerate application performance.
>Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
>the latest Intel processors and coprocessors. See abstracts and register >
>http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>

Re: [Htmlunit-user] Invalid selector: *#advIdProgIdCombo option:selected, jquery 1.3.2?

[Ronald B. <rb...@rb...>]

Hi Eric,

some more information might helpful.

HtmlUnit version?
Is the page you are accessing is public so we can try to reproduce the problem?

	RBRi

On Mon, 14 Oct 2013 11:03:54 -0700 Eric Peters wrote:
>
>10:56:04.924 [main] ERROR c.g.h.javascript.StrictErrorReporter -
>runtimeError: message=[An invalid or illegal selector was specified
>(selector: '#advIdProgIdCombo option:selected' error: Invalid selector:
>*#advIdProgIdCombo option:selected).] sourceName=[
>https://publisher.ebaypartnernetwork.com/common/js-16449593/min/jquery-1.3.2-min.js]
>line=[19] lineSource=[null] lineOffset=[0]
>
>
>Anyone know what browser feature might fix this?
>
>Thanks,
>
>Eric
>

Re: [Htmlunit-user] Feature request: parse a page from a string

[Andrew P. <ape...@42...>]

Updated my snippet for exception handling.

package com.gargoylesoftware.htmlunit;
import com.gargoylesoftware.htmlunit.html.HTMLParser;

import java.net.URL;
import java.io.IOException;
import java.net.MalformedURLException;

public class WebClient {
  // ...

  public static Page parse(String html) {
    try {
      return parse(html, new URL("http://fakeurl"), new WebClient());
    }
    catch(MalformedURLException e) {
      return null;
    }
  }

  public static Page parse(String html, URL url) {
    return parse(html, url, new WebClient());
  }

  public static Page parse(String html, URL url, WebClient webClient) {
    try {
      return HTMLParser.parseHtml(
        new StringWebResponse(
          html,
          url
        ),
        webClient.getCurrentWindow()
      );
    }
    catch(IOException e) {
      return null;
    }
  }
}


On Thu, Oct 17, 2013 at 11:41 AM, Andrew Pennebaker
<ape...@42...>wrote:

> Better snippets are on Clean Coder:
>
>
> https://sites.google.com/site/unclebobconsultingllc/dependency-management-htmlunit
>
>
>
>
> Could the HtmlUnit API please offer a static method like:
>
>
> public class WebClient {
>   public static Page parseFromHTMLString(String html) {
>     return parse(html, new WebClient());
>   }
>
>   public static Page parseFromHTMLString(String html, WebClient webClient)
> {
>     return HTMLParser.parseHtml(
>       new StringWebResponse(
>         html,
>         new URL("http://fakeurl")
>       ),
>       webClient.getCurrentWindow()
>     );
>   }
>
> That way, parsing pages from html would be as easy as Page page =
> WebClient.parseFromHTMLString("<html>...</html");
>
>
> On Thu, Oct 17, 2013 at 11:34 AM, Zmirc <zmi...@gm...> wrote:
>
>>  That would be really useful.
>>
>> Cheers!
>> Mircea
>>
>>
>> On 10/17/2013 5:28 PM, Andrew Pennebaker wrote:
>>
>> Ronald Brill came up with a snippet<http://htmlunit.10904.n7.nabble.com/Load-HtmlPage-from-a-file-td27667.html#a27670>for parsing a page from a string. Could HtmlUnit provide this nifty feature
>> by default? Jsoup<http://jsoup.org/cookbook/input/parse-document-from-string>provides this, and it's really helpful for testing.
>>
>>  --
>> Cheers,
>>
>> Andrew Pennebaker
>> ape...@42...
>>
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
>> the latest Intel processors and coprocessors. See abstracts and register >http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>>
>>
>>
>> _______________________________________________
>> Htmlunit-user mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from
>> the latest Intel processors and coprocessors. See abstracts and register >
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Htmlunit-user mailing list
>> Htm...@li...
>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>>
>>
>
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>



-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] No such method: WebClient.setJavaScriptEnabled

[Andrew P. <ape...@42...>]

Ah, WebClient.setJavaScriptEnabled(boolean) was deprecated in
v2.11<http://svn.code.sf.net/p/htmlunit/code/tags/HtmlUnit-2.11/src/main/java/com/gargoylesoftware/htmlunit/WebClient.java>,
but the official
documentation<http://htmlunit.sourceforge.net/#JavaScript_Support>was
never updated to account for the new style of configuration. Could we
please update that page?


On Thu, Oct 17, 2013 at 12:38 PM, Andrew Pennebaker
<ape...@42...>wrote:

> I'm trying to disable JavaScript, but it looks like it's no longer part of
> the API<http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/WebClient.html>.
> Is there another way to disable it? I'm using HtmlUnit v2.13.
>
>
>> To disable JavaScript processing for one WebClient, call
>> WebClient.setJavaScriptEnabled(false).
>
>
> --http://htmlunit.sourceforge.net/#JavaScript_Support
>
> Enable/disable JavaScript support. By default, this property is enabled.
>
>
> --
> http://www.netlikon.de/docs/javadoc-htmlunit/htmlunit-2.1/com/gargoylesoftware/htmlunit/WebClient.html#setJavaScriptEnabled(boolean)
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>



-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] Fw: Bug 1344: is this fix in 2.12?

[David T. <da...@de...>]

Ahmed,

I think that I am saying test case #2.

What I have done is add your code for test case #2 to my code temporarily
and run it.

It is clear that the code executes "final File file =
File.createTempFile("htmlunit", ".tmp");" at line 679 in HttpWebConnection.
If I run in debug mode it visits there a lot.

It is equally clear that it never visits:
        public void cleanUp() {
            if (temporary_) {
                FileUtils.deleteQuietly(file_);
            }
        }
 in DownloadedContent line 85 where I have a breakpoint.

Do you have other advice?

Thanks.


David
--
David Taylor, Detayls


On Wed, Oct 16, 2013 at 1:55 PM, Ahmed Ashour <asa...@ya...> wrote:

> Hi David,
>
> You are saying two different things:
>
>   1- DownloadedContent.InMemory is used even for files larger than
> HttpWebConnection.MAX_IN_MEMORY
>   2- DownloadedContent.OnFile is not deleted when calling
> webClient.closeAllWindows().
>
> To be on the same page, let's specify test cases.
>
> Please check the below test case for #2 (I have manually
> changed HttpWebConnection.MAX_IN_MEMORY to 1 (to make it use OnFile for
> that image).
>
>
>         WebClient webClient = new WebClient();
>         Page page = webClient.getPage("
> https://www.google.com/images/srpr/logo11w.png");
>
>         field = WebResponse.class.getDeclaredField("responseData_");
>         field.setAccessible(true);
>         WebResponseData responseData = (WebResponseData)
> field.get(page.getWebResponse());
>
>         field =
> WebResponseData.class.getDeclaredField("downloadedContent_");
>         field.setAccessible(true);
>         DownloadedContent downloadedContent = (DownloadedContent)
> field.get(responseData);
>
>         field = DownloadedContent.OnFile.class.getDeclaredField("file_");
>         field.setAccessible(true);
>
>         File file = (File) field.get(downloadedContent);
>         Assert.assertTrue(file.exists());
>         webClient.closeAllWindows();
>         Assert.assertFalse(file.exists());
>
>
> Ahmed Ashour
>   ------------------------------
>  *From:* David Taylor <da...@de...>
> *To:* Ahmed Ashour <asa...@ya...>;
> htm...@li...
> *Sent:* Wednesday, October 16, 2013 8:52 PM
> *Subject:* Re: [Htmlunit-user] Fw: Bug 1344: is this fix in 2.12?
>
> Ahmed,
>
> I have transitioned to 2.13 and I am still seeing lots of html*.tmp files
> being left around.
>
> It seems to me that at the cleanup() phase a lot of these WebResponses are
> typed DownloadedContent.InMemory despite being over the size limit so the
> cleanup() does nothing.
>
> Is there some special way that I am supposed to create the HTMLPage to
> ensure that the logic creates a DownloadedContent.OnFile when the response
> size is over the limit ?
>
> Thanks, David
>
> David
> --
> David Taylor, Detayls
>
>
> On Wed, Oct 9, 2013 at 3:00 PM, Ahmed Ashour <asa...@ya...> wrote:
>
> Hi David,
>
> I manually tested, and the file is deleted on
> calling webClient.closeAllWindows().
>
> Please use snapshot from http://build.canoo.com/htmlunit/artifacts
>
> Yours,
> Ahmed
>   ------------------------------
>  *From:* Ahmed Ashour <asa...@ya...>
> *To:* "htm...@li..." <
> htm...@li...>
> *Sent:* Thursday, October 10, 2013 12:47 AM
> *Subject:* [Htmlunit-user] Fw: Bug 1344: is this fix in 2.12?
>
> Hi David,
>
> I have replied back, but the email may not have gone through :((
>
> I will try to make a test case to verify the findings, hope this can be
> done before the eminent release.
>
> Yours,
> Ahmed
>
>   ----- Forwarded Message -----
>  *From:* Ahmed Ashour <asa...@ya...>
> *To:* "htm...@li..." <
> htm...@li...>
> *Sent:* Sunday, October 6, 2013 11:03 AM
> *Subject:* Re: [Htmlunit-user] Bug 1344: is this fix in 2.12?
>
> Hi David,
>
> Are you seeing any issue now?
>
> The proper fix was done in https://sourceforge.net/p/htmlunit/code/7832
>
> Yours,
> Ahmed
>   ------------------------------
>  *From:* David Taylor <da...@de...>
> *To:* htm...@li...
> *Sent:* Sunday, October 6, 2013 12:56 AM
> *Subject:* [Htmlunit-user] Bug 1344: is this fix in 2.12?
>
> Hi! First post.
>
> I noticed a pair of fixes for a problem "WebClient.closeAllWindows() to
> delete all temporary created big files." Those revisions were 7813 and 7816.
>
> Have these fixes made it into version 2.12?
>
> It seems to me the fixes are not present. Was there a regression perhaps?
>
> David
> --
> David W. Taylor, Detayls
> www.detayls.com
>
>
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
>
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
>

[Htmlunit-user] No such method: WebClient.setJavaScriptEnabled

[Andrew P. <ape...@42...>]

I'm trying to disable JavaScript, but it looks like it's no longer part of
the API<http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/WebClient.html>.
Is there another way to disable it? I'm using HtmlUnit v2.13.


> To disable JavaScript processing for one WebClient, call
> WebClient.setJavaScriptEnabled(false).


--http://htmlunit.sourceforge.net/#JavaScript_Support

Enable/disable JavaScript support. By default, this property is enabled.


--
http://www.netlikon.de/docs/javadoc-htmlunit/htmlunit-2.1/com/gargoylesoftware/htmlunit/WebClient.html#setJavaScriptEnabled(boolean)

-- 
Cheers,

Andrew Pennebaker
ape...@42...

[Htmlunit-user] Upload javadocs for com.gargoylesoftware.htmlunit.javascript.*

[Andrew P. <ape...@42...>]

The API docs for HtmlUnit are excellent! I just noticed that the docs for
the javasript engines, error listeners, etc. mentioned in the
WebClient docs<http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/WebClient.html#setJavaScriptErrorListener(com.gargoylesoftware.htmlunit.javascript.JavaScriptErrorListener)>are
missing <http://htmlunit.sourceforge.net/apidocs/index.html>.

Could we please upload the javascript API docs?

-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] Feature request: parse a page from a string

[Andrew P. <ape...@42...>]

Better snippets are on Clean Coder:

https://sites.google.com/site/unclebobconsultingllc/dependency-management-htmlunit




Could the HtmlUnit API please offer a static method like:


public class WebClient {
  public static Page parseFromHTMLString(String html) {
    return parse(html, new WebClient());
  }

  public static Page parseFromHTMLString(String html, WebClient webClient) {
    return HTMLParser.parseHtml(
      new StringWebResponse(
        html,
        new URL("http://fakeurl")
      ),
      webClient.getCurrentWindow()
    );
  }

That way, parsing pages from html would be as easy as Page page =
WebClient.parseFromHTMLString("<html>...</html");


On Thu, Oct 17, 2013 at 11:34 AM, Zmirc <zmi...@gm...> wrote:

>  That would be really useful.
>
> Cheers!
> Mircea
>
>
> On 10/17/2013 5:28 PM, Andrew Pennebaker wrote:
>
> Ronald Brill came up with a snippet<http://htmlunit.10904.n7.nabble.com/Load-HtmlPage-from-a-file-td27667.html#a27670>for parsing a page from a string. Could HtmlUnit provide this nifty feature
> by default? Jsoup<http://jsoup.org/cookbook/input/parse-document-from-string>provides this, and it's really helpful for testing.
>
>  --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Htmlunit-user mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] Feature request: parse a page from a string

[Zmirc <zmi...@gm...>]

That would be really useful.

Cheers!
Mircea

On 10/17/2013 5:28 PM, Andrew Pennebaker wrote:
> Ronald Brill came up with a snippet 
> <http://htmlunit.10904.n7.nabble.com/Load-HtmlPage-from-a-file-td27667.html#a27670> 
> for parsing a page from a string. Could HtmlUnit provide this nifty 
> feature by default? Jsoup 
> <http://jsoup.org/cookbook/input/parse-document-from-string> provides 
> this, and it's really helpful for testing.
>
> -- 
> Cheers,
>
> Andrew Pennebaker
> ape...@42... <mailto:ape...@42...>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user

[Htmlunit-user] Feature request: parse a page from a string

[Andrew P. <ape...@42...>]

Ronald Brill came up with a
snippet<http://htmlunit.10904.n7.nabble.com/Load-HtmlPage-from-a-file-td27667.html#a27670>for
parsing a page from a string. Could HtmlUnit provide this nifty
feature
by default? Jsoup<http://jsoup.org/cookbook/input/parse-document-from-string>provides
this, and it's really helpful for testing.

-- 
Cheers,

Andrew Pennebaker
ape...@42...

[Htmlunit-user] Feature request: ability to configure proxy settings after initializing a WebClient

[Andrew P. <ape...@42...>]

I'd like to be able to do WebClient webClient = new WebClient(); and then
later, configure proxy settings. This would make my code a lot easier to
refactor.

-- 
Cheers,

Andrew Pennebaker
ape...@42...

[Htmlunit-user] How can I set a transport timeout?

[Andrew P. <ape...@42...>]

I like that WebClient lets me set a JavaScript execution timeout. Can I
also set a transport layer timeout, so that HTTP requests longer than X ms
trigger an exception?

-- 
Cheers,

Andrew Pennebaker
ape...@42...

Re: [Htmlunit-user] Treatment of link with href="#"

[Satyanarayana K <kri...@ya...>]

Hi Ahmed,

The issue seems to be about Facebook Cross-domain Messaging. Is HtmlUnit handling this library?

Thanks,
Satya


________________________________
 From: Ahmed Ashour <asa...@ya...>
To: Satyanarayana K <kri...@ya...> 
Sent: Thursday, October 17, 2013 1:30 PM
Subject: Re: [Htmlunit-user] Treatment of link with href="#"
 


Hi Satya,

- Please always use the user-list
- The below is not a minimal case, please read http://htmlunit.sourceforge.net/submittingJSBugs.html

Ahmed

________________________________
 From: Satyanarayana K <kri...@ya...>
To: Ahmed Ashour <asa...@ya...> 
Sent: Thursday, October 17, 2013 9:58 AM
Subject: Re: [Htmlunit-user] Treatment of link with href="#"
 


Hi Ahmed,

I append below the test-case-code.

Thanks,
Satya

*************

System.out.println("***** ADHOC - BEGIN");
WebClient wClnt = new WebClient();
wClnt.getOptions().setCssEnabled(true);
HtmlPage hPage = null;
try{
 hPage = wClnt.getPage("http://www.skillpages.com/");
System.out.println("Web-page-received: ".concat(((HtmlPage)hPage).getTitleText()));
String linkDivPath = "//div[@class='inputContainer']";
HtmlElement hElem = (HtmlElement)hPage.getByXPath(linkDivPath).get(0);
HtmlInput hInput = (HtmlInput)hElem.getByXPath("./input[1]").get(0);
hInput.setValueAttribute("Painter");
HtmlElement clickElem = (HtmlElement)hElem.getByXPath("./a[1]").get(0);
HtmlPage rPage = clickElem.click();
wClnt.waitForBackgroundJavaScript(30000);
wClnt.waitForBackgroundJavaScriptStartingBefore(30000);
rPage = (HtmlPage)wClnt.getCurrentWindow().getEnclosedPage();
System.out.println("Click-result-page: ".concat(rPage.getTitleText()));
//Expected: //results/skills/Painter/0/Bangalore-KA-India/0/0/0/0/1?PromptOpp=true#
}catch(Exception ex){
System.out.println("Exception: ".concat(ex.getMessage() != null ? ex.getMessage() : "No Message!"));
}
System.out.println("ADHOC - END *****");

*************


________________________________
 From: Ahmed Ashour <asa...@ya...>
To: Satyanarayana K <kri...@ya...>; "htm...@li..." <htm...@li...>; "htm...@li..." <htm...@li...> 
Sent: Thursday, October 17, 2013 11:39 AM
Subject: Re: [Htmlunit-user] Treatment of link with href="#"
 


Hi Satya,

As always, tiny test cases are very helpful.

I am not sure I understand "JavaScript that may be (indirectly) associated with a link"

Yours,
Ahmed

________________________________
 From: Satyanarayana K <kri...@ya...>
To: "htm...@li..." <htm...@li...> 
Sent: Thursday, October 17, 2013 9:05 AM
Subject: [Htmlunit-user] Treatment of link with href="#"
 


Hi,

I wish to know if HtmlUnit - HtmlElement / HtmlAnchor.click() method includes code to execute any JavaScript that may be (indirectly) associated with a link with href="#", but without any value for onclick attribute.

My experience / experiments tell me that no JavaScript is executed in such a case:

1. HtmlUnit-based application fails to fetch/load the page that a browser does for such links.
2. I have observed the net-work activity in the Windows TaskManager: a. when I click the link in a browser, the graph shows net-work activity, b. when the click() is called in the HtmlUnit-based app, the graph does not show any net-work activity.

If the click() method code does include JavaScript execution, what may be the reason for page not being fetched / loaded? I have already tried waiting upto even two minutes for JavaScript completion.

If the click() method code  does not include JavaScript execution, any work-around that I can immediately try?

Thanks,
Satya
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: [Htmlunit-user] How do I configure WebClient for different kinds of HTTP authentications?

[Andrew P. <ape...@42...>]

*Yes*. I want to clarify: Are the username/password shown there for
authenticating with the proxy, or with the target website? Whichever one
the username/password are for, how do I authenticate for the other one?


On Wed, Oct 16, 2013 at 5:03 PM, Ahmed Ashour <asa...@ya...> wrote:

> Hi Andrew,
>
> It is included inside "Using a proxy server"
>
>     final DefaultCredentialsProvider credentialsProvider =
> (DefaultCredentialsProvider) webClient.getCredentialsProvider();
>     credentialsProvider.addCredentials("username", "password");
>
> Ahmed
>   ------------------------------
>  *From:* Andrew Pennebaker <ape...@42...>
> *To:* HtmlUnit Users <htm...@li...>
> *Sent:* Wednesday, October 16, 2013 9:21 PM
> *Subject:* [Htmlunit-user] How do I configure WebClient for different
> kinds of HTTP authentications?
>
> I read the snippet in the Getting Started<http://htmlunit.sourceforge.net/gettingStarted.html>guide, but I'm still a little confused. How can I:
>
> * Specify a username/password for accessing a proxy server?
> * Specify a username/password for reaching a protected web page?
>
> The Getting Started guide seems to only answer the first one question, but
> I'm also interested in the second.
>
> --
> Cheers,
>
> Andrew Pennebaker
> ape...@42...
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
Cheers,

Andrew Pennebaker
ape...@42...

[Htmlunit-user] Upload more files in <input type="file" multiple/>

[Zmirc <zmi...@gm...>]

Hi!

I searched a lot about how to upload more files in HTML|<input 
type="file" multiple/>|tag with HtmlUnit in Java, but I didn't find out yet.
Can anyone help? I can't modify the web pages, because they're not mine.

I am currently using for single file <input/>:

|HtmlFileInput#setValueAttribute(path);
HtmlFileInput#setContentType(contenType);|

Thank you so much!

There is also on Stackoverflow: 
http://stackoverflow.com/questions/19389426/how-to-upload-multiple-files-in-input-multiple-with-htmlunit-java 

Re: [Htmlunit-user] Treatment of link with href="#"

[Satyanarayana K <kri...@ya...>]

Hi Ahmed,

Thank you fro your prompt response.

I read through instructions in the page:  http://htmlunit.sourceforge.net/submittingJSBugs.html

To be able to comply with these instructions, one needs to be good in javaScript as well, which, unfortunately, I am not.

The expected procedure is somewhat demanding. Isn't there an easier way?

Thanks,
Satya


________________________________
 From: Ahmed Ashour <asa...@ya...>
To: Satyanarayana K <kri...@ya...> 
Sent: Thursday, October 17, 2013 1:30 PM
Subject: Re: [Htmlunit-user] Treatment of link with href="#"
 


Hi Satya,

- Please always use the user-list
- The below is not a minimal case, please read http://htmlunit.sourceforge.net/submittingJSBugs.html

Ahmed

________________________________
 From: Satyanarayana K <kri...@ya...>
To: Ahmed Ashour <asa...@ya...> 
Sent: Thursday, October 17, 2013 9:58 AM
Subject: Re: [Htmlunit-user] Treatment of link with href="#"
 


Hi Ahmed,

I append below the test-case-code.

Thanks,
Satya

*************

System.out.println("***** ADHOC - BEGIN");
WebClient wClnt = new WebClient();
wClnt.getOptions().setCssEnabled(true);
HtmlPage hPage = null;
try{
 hPage = wClnt.getPage("http://www.skillpages.com/");
System.out.println("Web-page-received: ".concat(((HtmlPage)hPage).getTitleText()));
String linkDivPath = "//div[@class='inputContainer']";
HtmlElement hElem = (HtmlElement)hPage.getByXPath(linkDivPath).get(0);
HtmlInput hInput = (HtmlInput)hElem.getByXPath("./input[1]").get(0);
hInput.setValueAttribute("Painter");
HtmlElement clickElem = (HtmlElement)hElem.getByXPath("./a[1]").get(0);
HtmlPage rPage = clickElem.click();
wClnt.waitForBackgroundJavaScript(30000);
wClnt.waitForBackgroundJavaScriptStartingBefore(30000);
rPage = (HtmlPage)wClnt.getCurrentWindow().getEnclosedPage();
System.out.println("Click-result-page: ".concat(rPage.getTitleText()));
//Expected: //results/skills/Painter/0/Bangalore-KA-India/0/0/0/0/1?PromptOpp=true#
}catch(Exception ex){
System.out.println("Exception: ".concat(ex.getMessage() != null ? ex.getMessage() : "No Message!"));
}
System.out.println("ADHOC - END *****");

*************


________________________________
 From: Ahmed Ashour <asa...@ya...>
To: Satyanarayana K <kri...@ya...>; "htm...@li..." <htm...@li...>; "htm...@li..." <htm...@li...> 
Sent: Thursday, October 17, 2013 11:39 AM
Subject: Re: [Htmlunit-user] Treatment of link with href="#"
 


Hi Satya,

As always, tiny test cases are very helpful.

I am not sure I understand "JavaScript that may be (indirectly) associated with a link"

Yours,
Ahmed

________________________________
 From: Satyanarayana K <kri...@ya...>
To: "htm...@li..." <htm...@li...> 
Sent: Thursday, October 17, 2013 9:05 AM
Subject: [Htmlunit-user] Treatment of link with href="#"
 


Hi,

I wish to know if HtmlUnit - HtmlElement / HtmlAnchor.click() method includes code to execute any JavaScript that may be (indirectly) associated with a link with href="#", but without any value for onclick attribute.

My experience / experiments tell me that no JavaScript is executed in such a case:

1. HtmlUnit-based application fails to fetch/load the page that a browser does for such links.
2. I have observed the net-work activity in the Windows TaskManager: a. when I click the link in a browser, the graph shows net-work activity, b. when the click() is called in the HtmlUnit-based app, the graph does not show any net-work activity.

If the click() method code does include JavaScript execution, what may be the reason for page not being fetched / loaded? I have already tried waiting upto even two minutes for JavaScript completion.

If the click() method code  does not include JavaScript execution, any work-around that I can immediately try?

Thanks,
Satya
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user