htmlunit-user — Discussion of the use of HtmlUnit

Re: [Htmlunit-user] Fwd: [crawler-common] How can I scrape google suggestions?

[Marc G. <mgu...@ya...>]

Hi,

be sure to use type() and not setValueQttribute or set Attribute* in
order to have JS event handlers executed.
Then use webClient.waitForBackgroundJavaScriptStartingBefore to give
time for suggestions to come.
Finally dump the DOM to the console with HtmlPage.asXml() to see if the
suggestion are there or not. If yes, you should see how to retrieve them
and if not, hmm, perhaps is it a problem in HtmlUnit (try with latest
snapshot, simulating FF24).

Cheers,
Marc.
-- 
HtmlUnit support & consulting from the source
Blog: http://mguillem.wordpress.com

Le 26/04/2014 19:55, אברהם חיון a écrit :
> Hi,
> 
> 
> 
> I want to scrape the Ajax suggestions of google (or other sites).
> 
> 
> Can anyone give me a hand on how to address it?
> 
> What I already tried?
> I used HtmlUnit to objectify the page then add to the search box the
> word I need suggestions for.
> 
> It works well, and when I download the current page using page.asText(),
> I do get the page with my word in the search field, but I don't get the
> Ajax suggestions which I can see on the screen but can't grab using
> HtmlUnit.
> 
> 
> 
> 
> Any tip on how to do it?
> 

Re: [Htmlunit-user] Problem with getPage() function

[asashour <asa...@ya...>]

Hello,

- You must subscribe to the list before posting.
- The method is now supported for Chrome in SVN version. You can change your
code to use Firefox or IE11. If you need Chrome support, then get latest
snapshot, or wait for HtmlUnit 2.15 which should be released soon.

Ahmed


Can Oncel wrote
> Hi everyone,
> 
> I tried to get data from a website which is named as vatanbilgisayar.com. 
> I could not get the data by using any HTTP queries or getting HTML codes,
> because the data I need is not placed in source code, so my aim is to get
> the data by using HTML Unit.
> 
> So, for this I use the code below, after the code, there is error logs.  I
> will be very thankful, if anyone can help me to solve this problem.
> 
> Best Regards,
> Can Öncel
> 
> -----CODE START------
> 
> import java.util.List;
> 
> import com.gargoylesoftware.htmlunit.BrowserVersion;
> import com.gargoylesoftware.htmlunit.WebClient;
> import com.gargoylesoftware.htmlunit.html.FrameWindow;
> import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
> import com.gargoylesoftware.htmlunit.html.HtmlPage;
> 
> 
> public class HTMLUnitTest {
> 
> 	/**
> 	 * @param args
> 	 * @throws Exception 
> 	 */
> 	public static void main(String[] args) throws Exception {
> 		// TODO Auto-generated method stub
> 		find();
> 	}
> 
> 	
> 	 private static void find() throws Exception {
> 	       
> java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(java.util.logging.Level.OFF);
> 
> 	        HtmlPage nextPage;
> 	        String url =
> "http://www.vatanbilgisayar.com/hp-14-n201st-core-i5-4200u-16ghz-4gb-500gb-14-2gb-w81-notebook.html";
> 
> 	        WebClient webclient = new WebClient(BrowserVersion.CHROME);
> 	        webclient.getOptions().setUseInsecureSSL(true);
> 	        HtmlPage page = webclient.getPage(url);
> 
> 	        List
> <HtmlAnchor>
>  links = (List
> <HtmlAnchor>
> ) page.getByXPath("//td[@class='gridUrunOzellikDetayIcerik']");
> 
> 	        for(int x=0; x&lt;links.size(); x++) {
> 	            System.out.println(links.get(x).asText());
> 	        }
> 	    }
> 
> }
> 
> -----CODE FINISH-----
> 
> 
> Exception in thread &quot;main&quot; ======= EXCEPTION START ========
> EcmaError: lineNumber=[2] column=[0] lineSource=[&lt;no source&gt;]
> name=[TypeError]
> sourceName=[http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js]
> message=[TypeError: Cannot find function getPropertyValue in object
> [object ComputedCSSStyleDeclaration].
> (http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js#2)]
> com.gargoylesoftware.htmlunit.ScriptException: TypeError: Cannot find
> function getPropertyValue in object [object ComputedCSSStyleDeclaration].
> (http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js#2)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:689)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:620)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:513)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:575)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:1074)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:391)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:272)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:286)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:702)
> 	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:662)
> 	at
> org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
> 	at
> org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
> 	at
> org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
> 	at
> org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
> 	at
> org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3126)
> 	at
> org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2093)
> 	at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:920)
> 	at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
> 	at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
> 	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:926)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:245)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:191)
> 	at
> com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268)
> 	at
> com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156)
> 	at
> com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:455)
> 	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:329)
> 	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:394)
> 	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:379)
> 	at HTMLUnitTest.find(HTMLUnitTest.java:30)
> 	at HTMLUnitTest.main(HTMLUnitTest.java:18)
> Caused by: net.sourceforge.htmlunit.corejs.javascript.EcmaError:
> TypeError: Cannot find function getPropertyValue in object [object
> ComputedCSSStyleDeclaration].
> (http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js#2)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3629)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3613)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3634)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError2(ScriptRuntime.java:3650)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.notFunctionError(ScriptRuntime.java:3714)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.getPropFunctionAndThisHelper(ScriptRuntime.java:2233)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.getPropFunctionAndThis(ScriptRuntime.java:2215)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1333)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:411)
> 	at
> com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3057)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:566)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:674)
> 	... 31 more
> Enclosed exception: 
> net.sourceforge.htmlunit.corejs.javascript.EcmaError: TypeError: Cannot
> find function getPropertyValue in object [object
> ComputedCSSStyleDeclaration].
> (http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js#2)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3629)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3613)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3634)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError2(ScriptRuntime.java:3650)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.notFunctionError(ScriptRuntime.java:3714)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.getPropFunctionAndThisHelper(ScriptRuntime.java:2233)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.getPropFunctionAndThis(ScriptRuntime.java:2215)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1333)
> 	at script(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script.en(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script(http://www.vatanbilgisayar.com/Scripts/jquery-base.min.js:2)
> 	at script(http://www.vatanbilgisayar.com/upload/js/jquery-ui.js:174)
> 	at script(http://www.vatanbilgisayar.com/upload/js/jquery-ui.js:5)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:411)
> 	at
> com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3057)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:566)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:674)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:620)
> 	at
> net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:513)
> 	at
> com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:575)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:1074)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:391)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:272)
> 	at
> com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:286)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:702)
> 	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:662)
> 	at
> org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
> 	at
> org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
> 	at
> org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
> 	at
> org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
> 	at
> org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3126)
> 	at
> org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2093)
> 	at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:920)
> 	at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
> 	at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
> 	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:926)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:245)
> 	at
> com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:191)
> 	at
> com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268)
> 	at
> com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156)
> 	at
> com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:455)
> 	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:329)
> 	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:394)
> 	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:379)
> 	at HTMLUnitTest.find(HTMLUnitTest.java:30)
> 	at HTMLUnitTest.main(HTMLUnitTest.java:18)
> ======= EXCEPTION END ========





--
View this message in context: http://htmlunit.10904.n7.nabble.com/Problem-with-getPage-function-tp33725p33745.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] Populating a Dynamic Webpage

[Alain B. <alb...@gm...>]

I don't have too much experience with it But I will try something like that:

WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24);
            HtmlPage page = webClient.getPage("http://yourpage.com");

HtmlDivision div = page.getHtmlElementById("introductionDate");

String content = div.getTextContent();

Regards
Alain

2014-05-03 1:59 GMT+02:00 Rich Goldman <ri...@um...>:

> Hi.  I'm new to HtmlUnit and am hoping this is an easy newbie question.
>
> I'm trying to parse out information about legislation from the DC City
> Council website.  They recently changed their system from a basic HTML page
> of data to a dynamically loaded page.
>
> An example bill page is:
> http://lims.dccouncil.us/Legislation/PR20-0624
>
> When I retrieve the Page, I get the html, but without the fields
> populated.
>
> I've tried without success to identify a javascript function to call via
> HtmlUnit that would populate the html.
>
> What might I be missing in order to get a Page object that has the html
> with populated fields?
>
> For example, in the provided link, the html has the string/row header
> "Introduction Date" hard-coded, but I need to get the populated value
> of Jan 22, 2014.
>
> Any advise, code, or reading material would be greatly appreciated.
>
> Thank you.
> -Rich
>
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>

[Htmlunit-user] Populating a Dynamic Webpage

[Rich G. <ri...@um...>]

Hi.  I'm new to HtmlUnit and am hoping this is an easy newbie question.

I'm trying to parse out information about legislation from the DC City Council website.  They recently changed their system from a basic HTML page of data to a dynamically loaded page.

An example bill page is:
http://lims.dccouncil.us/Legislation/PR20-0624

When I retrieve the Page, I get the html, but without the fields populated.  

I've tried without success to identify a javascript function to call via HtmlUnit that would populate the html.

What might I be missing in order to get a Page object that has the html with populated fields?  

For example, in the provided link, the html has the string/row header "Introduction Date" hard-coded, but I need to get the populated value of Jan 22, 2014.

Any advise, code, or reading material would be greatly appreciated.

Thank you.
	-Rich

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[David M. G. <mic...@gm...>]

Hi,

I opened defect #1598 for it.

Thanks,
David

On Wed, Apr 30, 2014 at 9:46 AM, David Michael Gang
<mic...@gm...>wrote:

> Hi,
>
> I am already using the functions to wait for the javascript.
>
> I used both
> waitForBackgroundJavaScript(10000);
>
> and
> waitForBackgroundJavaScriptStartingBefore(10000)
>
> and it did not help.
>
>
> Besides this I need a generic solution, which can be achieved.
>
> For example jsoup knows how to cope with this html
>
> package test;
>
> import java.io.File;
>
> import java.io.IOException;
>
> import org.jsoup.nodes.Document;
> import org.jsoup.nodes.Element;
> import org.jsoup.select.Elements;
> import org.jsoup.Jsoup;
> public class JsoupTest1 {
>
> 	public static void main(String[] args) throws IOException {
>
> 		File in = new File("l.html");
> 		Document doc = Jsoup.parse(in, null);
> 		Elements elems = doc.select("table");
> 		
> 		for (Element elem:elems) {
> 			System.out.println(elem.text());
> 		}
>
> 		
> 	}
>
> }
>
> Maybe i should file a bug, but i don't think that there is a reason for executing the special javascript command.
>
> Thanks,
>
> David
>
>
>
> >In the source page i could see   body tag appended with:
>
>  >onload="hideDiv(true);initBoxes('listview');callSubScroll('frm_tagged_documents',0,1);updateResultsNav();reloadClassification('false');scrollToHitPos('false');"
> >onUnload="storeScrollToHitPos('false');
>
> >Execute this js functions then try extracting the page.
>
> >Once more thing what is your desired output
>
>

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[David M. G. <mic...@gm...>]

Hi,

I am already using the functions to wait for the javascript.

I used both
waitForBackgroundJavaScript(10000);

and
waitForBackgroundJavaScriptStartingBefore(10000)

and it did not help.


Besides this I need a generic solution, which can be achieved.

For example jsoup knows how to cope with this html

package test;

import java.io.File;
import java.io.IOException;

import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.jsoup.Jsoup;
public class JsoupTest1 {

	public static void main(String[] args) throws IOException {
		File in = new File("l.html");
		Document doc = Jsoup.parse(in, null);
		Elements elems = doc.select("table");
		
		for (Element elem:elems) {
			System.out.println(elem.text());
		}
		
	}

}

Maybe i should file a bug, but i don't think that there is a reason
for executing the special javascript command.

Thanks,

David



>In the source page i could see   body tag appended with:

 >onload="hideDiv(true);initBoxes('listview');callSubScroll('frm_tagged_documents',0,1);updateResultsNav();reloadClassification('false');scrollToHitPos('false');"
>onUnload="storeScrollToHitPos('false');

>Execute this js functions then try extracting the page.

>Once more thing what is your desired output

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[chris d. <ach...@gm...>]

In the source page i could see   body tag appended with:

 onload="hideDiv(true);initBoxes('listview');callSubScroll('frm_tagged_documents',0,1);updateResultsNav();reloadClassification('false');scrollToHitPos('false');"
onUnload="storeScrollToHitPos('false');

Execute this js functions then try extracting the page.

Once more thing what is your desired output


On Tue, Apr 29, 2014 at 7:21 PM, David Michael Gang
<mic...@gm...> wrote:
> Hi,
>
> Here is the page attached.
>
> BR,
> David
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>



-- 
" What we sow we will reap."

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[chris d. <ach...@gm...>]

Save the page in HTML and attach it. which help in finding the tags and
other detail easier


On Tue, Apr 29, 2014 at 5:51 PM, David Michael Gang
<mic...@gm...>wrote:

> Hi,
>
> I agree and normally i give an exact example with a program and the url,
> but this is a private url.
>
> Thanks,
> David
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
*" What we sow we will reap."*

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[David M. G. <mic...@gm...>]

Hi,

When looking on the function which creates the row iterator:

 private void setNextRow(final DomNode node) {
            nextRow_ = null;
            for (DomNode next = node; next != null; next =
next.getNextSibling()) {
                if (next instanceof HtmlTableRow) {
                    nextRow_ = (HtmlTableRow) next;
                    return;
                }
                else if (currentGroup_ == null && next instanceof
TableRowGroup) {
                    currentGroup_ = (TableRowGroup) next;
                    setNextRow(next.getFirstChild());
                    return;
                }
            }
            if (currentGroup_ != null) {
                final DomNode group = currentGroup_;
                currentGroup_ = null;
                setNextRow(group.getNextSibling());
            }
        }

        public Iterator<HtmlTableRow> iterator() {
            return this;
        }
    }

As we see it just descends into the lower tags, if next is an instance of a
TableRowGroup, like tbody, but not if there is a custom tag.
I don't know if it is a root cause because maybe the html parser should
ignore these tags. When inspecting the element with firefox i see that he
does it (inspect element).

BR,
David


On Tue, Apr 29, 2014 at 2:50 PM, David Michael Gang
<mic...@gm...>wrote:

> Hi all,
>
> I crawled a page and the HtmlPage.asText function did not return the
> desired result:
> I tracked it down that somehow that the table rows were ignored.
> The reason is that in the crawled page the table trs were wrapped into a
> doc tag.
>
> <table width="100%" border="0" cellspacing="0" cellpadding="0">
>                   <tbody>
>                     <tr class="nopadding">
>                       <td>
>                         <a name="DOCNO_1">
>                         </a>
>                       </td>
>                     </tr>
>                     <doc>
>                       <tr valign="baseline" height="8" class="toprow">
>                         <th width="6%" align="left" valign="center"
> nowrap="nowrap">
>                           <span>
>                             <input type="checkbox" id="frm_control_box"
> title="Click here to select or de-select all" name="frm_control_box"
> value="checkbox" onclick="javascript:subSetAllSelectionStatus()"/>
>                           </span>
>                         </th>
>                         <th width="94%" align="left" valign="center"
> nowrap="nowrap">
>                           <span>
>                             Results
>                           </span>
>                         </th>
>                       </tr>
>                       <tr class="noshaderow1st" style="padding-bottom:
> 8px;" height="8" valign="baseline">
>                         <td width="6%" align="left" nowrap="nowrap"
> valign="top">
>                           <input onclick="javascript:manageBox('1')"
> type="checkbox" value="1" name="frm_tagged_documents" title="Click here to
> deliver or to view tagged documents" id="frm_tagged_documents1"/>
>                           <label style="{cursor: pointer; cursor: hand;}"
> for="frm_tagged_documents1">
>                             1.
>                           </label>
>                         </td>
>                         <td width="94%" align="left" valign="top">
>                           <a href="aaa" target="_parent">
> aaa
>                           </a>
>                           <br class="br"/>
>                           <span class="notranslate">
>      bbb
>                           </span>
>                           , November 19, 2011, Pg. 7, 758 words
>                         </td>
>                       </tr>
>                     </doc>
>                     <tr class="nopadding">
>                       <td>
>                         <a name="DOCNO_2">
>                         </a>
>                       </td>
>                     </tr>
>                     <doc>
>                       <tr class="shaderow1st" style="padding-bottom: 8px;"
> height="8" valign="baseline">
>                         <td width="6%" align="left" nowrap="nowrap"
> valign="top">
>                           <input onclick="javascript:manageBox('2')"
> type="checkbox" value="2" name="frm_tagged_documents" title="Click here to
> deliver or to view tagged documents" id="frm_tagged_documents2"/>
>                           <label style="{cursor: pointer; cursor: hand;}"
> for="frm_tagged_documents2">
>                             2.
>                           </label>
>                         </td>
>                         <td width="94%" align="left" valign="top">
>                           <a href="ccc" target="_parent">
>                             ddd
>                           </a>
>                           <br class="br"/>
>                           <span class="notranslate">
>                            eee
>                           </span>
>                           , November 19, 2011, Pg. 18, 1216 words, MICHAEL
> HENDERSON
>                         </td>
>                       </tr>
>                     </doc>
>
>                   </tbody>
>  In firefox the page is displayed nice.
> Is it somehow possible to tell htmlunit to ignore the doc tag and recurse
> into it to find the tr tag?
>
>
> Thanks,
> David
>

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[David M. G. <mic...@gm...>]

Hi,

I agree and normally i give an exact example with a program and the url,
but this is a private url.

Thanks,
David

Re: [Htmlunit-user] problems with crawling a table when html contains custom tags

[chris d. <ach...@gm...>]

If you could share the URL its easy to view content in the browser


On Tue, Apr 29, 2014 at 5:20 PM, David Michael Gang
<mic...@gm...>wrote:

> Hi all,
>
> I crawled a page and the HtmlPage.asText function did not return the
> desired result:
> I tracked it down that somehow that the table rows were ignored.
> The reason is that in the crawled page the table trs were wrapped into a
> doc tag.
>
> <table width="100%" border="0" cellspacing="0" cellpadding="0">
>                   <tbody>
>                     <tr class="nopadding">
>                       <td>
>                         <a name="DOCNO_1">
>                         </a>
>                       </td>
>                     </tr>
>                     <doc>
>                       <tr valign="baseline" height="8" class="toprow">
>                         <th width="6%" align="left" valign="center"
> nowrap="nowrap">
>                           <span>
>                             <input type="checkbox" id="frm_control_box"
> title="Click here to select or de-select all" name="frm_control_box"
> value="checkbox" onclick="javascript:subSetAllSelectionStatus()"/>
>                           </span>
>                         </th>
>                         <th width="94%" align="left" valign="center"
> nowrap="nowrap">
>                           <span>
>                             Results
>                           </span>
>                         </th>
>                       </tr>
>                       <tr class="noshaderow1st" style="padding-bottom:
> 8px;" height="8" valign="baseline">
>                         <td width="6%" align="left" nowrap="nowrap"
> valign="top">
>                           <input onclick="javascript:manageBox('1')"
> type="checkbox" value="1" name="frm_tagged_documents" title="Click here to
> deliver or to view tagged documents" id="frm_tagged_documents1"/>
>                           <label style="{cursor: pointer; cursor: hand;}"
> for="frm_tagged_documents1">
>                             1.
>                           </label>
>                         </td>
>                         <td width="94%" align="left" valign="top">
>                           <a href="aaa" target="_parent">
> aaa
>                           </a>
>                           <br class="br"/>
>                           <span class="notranslate">
>      bbb
>                           </span>
>                           , November 19, 2011, Pg. 7, 758 words
>                         </td>
>                       </tr>
>                     </doc>
>                     <tr class="nopadding">
>                       <td>
>                         <a name="DOCNO_2">
>                         </a>
>                       </td>
>                     </tr>
>                     <doc>
>                       <tr class="shaderow1st" style="padding-bottom: 8px;"
> height="8" valign="baseline">
>                         <td width="6%" align="left" nowrap="nowrap"
> valign="top">
>                           <input onclick="javascript:manageBox('2')"
> type="checkbox" value="2" name="frm_tagged_documents" title="Click here to
> deliver or to view tagged documents" id="frm_tagged_documents2"/>
>                           <label style="{cursor: pointer; cursor: hand;}"
> for="frm_tagged_documents2">
>                             2.
>                           </label>
>                         </td>
>                         <td width="94%" align="left" valign="top">
>                           <a href="ccc" target="_parent">
>                             ddd
>                           </a>
>                           <br class="br"/>
>                           <span class="notranslate">
>                            eee
>                           </span>
>                           , November 19, 2011, Pg. 18, 1216 words, MICHAEL
> HENDERSON
>                         </td>
>                       </tr>
>                     </doc>
>
>                   </tbody>
>  In firefox the page is displayed nice.
> Is it somehow possible to tell htmlunit to ignore the doc tag and recurse
> into it to find the tr tag?
>
>
> Thanks,
> David
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
*" What we sow we will reap."*

Re: [Htmlunit-user] currentPage.asText() not work how should work

[chris d. <ach...@gm...>]

Check the body tag whether any onload js functions present in it. If so
execute the functions before getting elements


On Tue, Apr 29, 2014 at 5:23 PM, David Michael Gang
<mic...@gm...>wrote:

> Hi Valentin,
>
> This is a real coincidence.
> I have the same problem with a different tag.
>
> BR,
> David
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
*" What we sow we will reap."*

Re: [Htmlunit-user] currentPage.asText() not work how should work

[David M. G. <mic...@gm...>]

Hi Valentin,

This is a real coincidence.
I have the same problem with a different tag.

BR,
David

[Htmlunit-user] problems with crawling a table when html contains custom tags

[David M. G. <mic...@gm...>]

Hi all,

I crawled a page and the HtmlPage.asText function did not return the
desired result:
I tracked it down that somehow that the table rows were ignored.
The reason is that in the crawled page the table trs were wrapped into a
doc tag.

<table width="100%" border="0" cellspacing="0" cellpadding="0">
                  <tbody>
                    <tr class="nopadding">
                      <td>
                        <a name="DOCNO_1">
                        </a>
                      </td>
                    </tr>
                    <doc>
                      <tr valign="baseline" height="8" class="toprow">
                        <th width="6%" align="left" valign="center"
nowrap="nowrap">
                          <span>
                            <input type="checkbox" id="frm_control_box"
title="Click here to select or de-select all" name="frm_control_box"
value="checkbox" onclick="javascript:subSetAllSelectionStatus()"/>
                          </span>
                        </th>
                        <th width="94%" align="left" valign="center"
nowrap="nowrap">
                          <span>
                            Results
                          </span>
                        </th>
                      </tr>
                      <tr class="noshaderow1st" style="padding-bottom:
8px;" height="8" valign="baseline">
                        <td width="6%" align="left" nowrap="nowrap"
valign="top">
                          <input onclick="javascript:manageBox('1')"
type="checkbox" value="1" name="frm_tagged_documents" title="Click here to
deliver or to view tagged documents" id="frm_tagged_documents1"/>
                          <label style="{cursor: pointer; cursor: hand;}"
for="frm_tagged_documents1">
                            1.
                          </label>
                        </td>
                        <td width="94%" align="left" valign="top">
                          <a href="aaa" target="_parent">
aaa
                          </a>
                          <br class="br"/>
                          <span class="notranslate">
     bbb
                          </span>
                          , November 19, 2011, Pg. 7, 758 words
                        </td>
                      </tr>
                    </doc>
                    <tr class="nopadding">
                      <td>
                        <a name="DOCNO_2">
                        </a>
                      </td>
                    </tr>
                    <doc>
                      <tr class="shaderow1st" style="padding-bottom: 8px;"
height="8" valign="baseline">
                        <td width="6%" align="left" nowrap="nowrap"
valign="top">
                          <input onclick="javascript:manageBox('2')"
type="checkbox" value="2" name="frm_tagged_documents" title="Click here to
deliver or to view tagged documents" id="frm_tagged_documents2"/>
                          <label style="{cursor: pointer; cursor: hand;}"
for="frm_tagged_documents2">
                            2.
                          </label>
                        </td>
                        <td width="94%" align="left" valign="top">
                          <a href="ccc" target="_parent">
                            ddd
                          </a>
                          <br class="br"/>
                          <span class="notranslate">
                           eee
                          </span>
                          , November 19, 2011, Pg. 18, 1216 words, MICHAEL
HENDERSON
                        </td>
                      </tr>
                    </doc>

                  </tbody>
 In firefox the page is displayed nice.
Is it somehow possible to tell htmlunit to ignore the doc tag and recurse
into it to find the tr tag?


Thanks,
David

Re: [Htmlunit-user] currentPage.asText() not work how should work

[chris d. <ach...@gm...>]

Check whether any javascript functions are involved to generate . If so
execute the js
if (webdriver instanceof JavascriptExecutor) {
 ((JavascriptExecutor)
webdriver).executeScript("init();TabInit();Accordinit();");
 }


On Mon, Apr 28, 2014 at 1:07 PM, Valentin Popa <v.v...@gm...>wrote:

> Hello
>
> I want to get visible text from this page:
> http://zeitarbeit-online-datenbank.de/index.php?art_id=53466&load=10,1,1&search=such_taetigkeit|;such_kategorie|;sort_order|erstellt_am;sort_dir|desc;such_erst|;such_zeitraum|;such_bl|;such_ort|;such_plz|;such_term|;nmbr|4;ap|0
>
> For this I use the bellow simple code:
>
> final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);
> HtmlPage page = webClient.getPage("
> http://zeitarbeit-online-datenbank.de/index.php?art_id=53466&load=10,1,1&search=such_taetigkeit|;such_kategorie|;sort_order|erstellt_am;sort_dir|desc;such_erst|;such_zeitraum|;such_bl|;such_ort|;such_plz|;such_term|;nmbr|4;ap|0
> ");
> System.out.println(page.asText());
>
> The output is not what it should be.
>
> Looking in html I can see this:
>
> <table cellpadding="0" cellspacing="0" align="right" border="0">   <form name="suchformular" method="get" action="index.php"></form>      <input name="load" value="7" type="hidden">      <input name="sid" value="c23371c7273e7a23add375c01fe35183" type="hidden">      <input name="abgesendet" value="yes" type="hidden">
>            <tbody><tr>
>                      <td align="right" height="32" valign="bottom">
>
>
> As you can see <tr> is in form instead to be after <table> but in browser this is working. Looking in HtmlSerializer class from htmlunit I can see that for tables is a special method like this:
>
> else if (node instanceof HtmlTable) {
>     appendHtmlTable((HtmlTable) node);
> }
>
> private void appendHtmlTable(final HtmlTable htmlTable) {
>         doAppendBlockSeparator();
>         final String caption = htmlTable.getCaptionText();
>         if (caption != null) {
>             doAppend(caption);
>             doAppendBlockSeparator();
>         }
>
>         boolean first = true;
>
>         // first thead has to be displayed first and first tfoot has to be displayed last
>         final HtmlTableHeader tableHeader = htmlTable.getHeader();
>         if (tableHeader != null) {
>             first = appendHtmlTableRows(tableHeader.getRows(), true, null, null);
>         }
>         final HtmlTableFooter tableFooter = htmlTable.getFooter();
>
>         first = appendHtmlTableRows(htmlTable.getRows(), first, tableHeader, tableFooter);
>
>         if (tableFooter != null) {
>             first = appendHtmlTableRows(tableFooter.getRows(), first, null, null);
>         }
>
>         doAppendBlockSeparator();
> }
>
> When this: htmlTable.getRows() is called it is not able to return the rows because the rows are in form.
>
> Any idea how to fix this? Or do you know why htmlunit use speial routine to print tables?
>
> Best Regards
>
> Valentin
>
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


-- 
*" What we sow we will reap."*

[Htmlunit-user] currentPage.asText() not work how should work

[Valentin P. <v.v...@gm...>]

Hello

I want to get visible text from this page:
http://zeitarbeit-online-datenbank.de/index.php?art_id=53466&load=10,1,1&search=such_taetigkeit|;such_kategorie|;sort_order|erstellt_am;sort_dir|desc;such_erst|;such_zeitraum|;such_bl|;such_ort|;such_plz|;such_term|;nmbr|4;ap|0

For this I use the bellow simple code:

final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);
HtmlPage page = webClient.getPage("
http://zeitarbeit-online-datenbank.de/index.php?art_id=53466&load=10,1,1&search=such_taetigkeit|;such_kategorie|;sort_order|erstellt_am;sort_dir|desc;such_erst|;such_zeitraum|;such_bl|;such_ort|;such_plz|;such_term|;nmbr|4;ap|0
");
System.out.println(page.asText());

The output is not what it should be.

Looking in html I can see this:

<table cellpadding="0" cellspacing="0" align="right" border="0">
<form name="suchformular" method="get" action="index.php"></form>
<input name="load" value="7" type="hidden">      <input name="sid"
value="c23371c7273e7a23add375c01fe35183" type="hidden">      <input
name="abgesendet" value="yes" type="hidden">
           <tbody><tr>
                     <td align="right" height="32" valign="bottom">


As you can see <tr> is in form instead to be after <table> but in
browser this is working. Looking in HtmlSerializer class from htmlunit
I can see that for tables is a special method like this:

else if (node instanceof HtmlTable) {
    appendHtmlTable((HtmlTable) node);
}

private void appendHtmlTable(final HtmlTable htmlTable) {
        doAppendBlockSeparator();
        final String caption = htmlTable.getCaptionText();
        if (caption != null) {
            doAppend(caption);
            doAppendBlockSeparator();
        }

        boolean first = true;

        // first thead has to be displayed first and first tfoot has
to be displayed last
        final HtmlTableHeader tableHeader = htmlTable.getHeader();
        if (tableHeader != null) {
            first = appendHtmlTableRows(tableHeader.getRows(), true,
null, null);
        }
        final HtmlTableFooter tableFooter = htmlTable.getFooter();

        first = appendHtmlTableRows(htmlTable.getRows(), first,
tableHeader, tableFooter);

        if (tableFooter != null) {
            first = appendHtmlTableRows(tableFooter.getRows(), first,
null, null);
        }

        doAppendBlockSeparator();
}

When this: htmlTable.getRows() is called it is not able to return the
rows because the rows are in form.

Any idea how to fix this? Or do you know why htmlunit use speial
routine to print tables?

Best Regards

Valentin

Re: [Htmlunit-user] Does not get file when InputButton clicked

[UliScherrer <uli...@gm...>]

Hi Ronald,

thanks for Help. I had success.
I just took an anchor (does not matter which one) and changed href to
setAttribute("href",
"https://......../showpdf.do?requestor=Shipper&filetype=pdf");
WebResponse was now a pdf file.

Uli


UliScherrer wrote
> Hi Ronald,
> 
> yes, there is no form.
> Live HTTP headers (Firefox add on) shows me after the click following url:
> https://......../showpdf.do?requestor=Shipper&filetype=pdf
> 
> I suppose, the following window reads a cookie to get the information
> which pdf is shown.
> 
> I tried page.getWebClient().getPage(url_as_shown_above); but without the
> desired result.
> Is there another way to execute this url in my pagecontext?
> Or should I dig into javascript and deal with ScriptableObject?
> I am not experienced enough to see the right way.
> 
> thanks
> Uli





--
View this message in context: http://htmlunit.10904.n7.nabble.com/Does-not-get-file-when-InputButton-clicked-tp33583p33601.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] Does not get file when InputButton clicked

[UliScherrer <uli...@gm...>]

Hi Ronald,

yes, there is no form.
Live HTTP headers (Firefox add on) shows me after the click following url:
https://......../showpdf.do?requestor=Shipper&filetype=pdf

I suppose, the following window reads a cookie to get the information which
pdf is shown.

I tried page.getWebClient().getPage(url_as_shown_above); but without the
desired result.
Is there another way to execute this url in my pagecontext?
Or should I dig into javascript and deal with ScriptableObject?
I am not experienced enough to see the right way.

thanks
Uli
 



--
View this message in context: http://htmlunit.10904.n7.nabble.com/Does-not-get-file-when-InputButton-clicked-tp33583p33594.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] Does not get file when InputButton clicked

[Ronald B. <rb...@rb...>]

Hi Uli,

i can only guess. I see no form and the button you press is of type button (not submit) that usually only triggers  javascript code.
There is a good chance, that this js code at least opens a new window and does some more magic like Ajax.
So you first have to understand what is really going on when clicking the button.
With this info we can check what the reason for your problem migth be.

	RBRi

On Tue, 15 Apr 2014 04:52:12 -0700 (PDT) UliScherrer wrote:
>
>Hi,
>when I click in a GUI Browser on InputButton (Submit) another Windows opens
>and shows a Pdf.
>
><div id="viewAndDownloadIntlInvoiceDialogLiteCsv"
>class="dialogWithTitleAndButtons">
>  <div class="dialogTitle" >
>    View/Download Invoice Data
>  </div>
>  <div class="dialogContent">
>    <p>Please select a file format to view/download your invoice.</p>
>    <input type="radio" name="format" value="pdf"  checked="checked"/>PDF
>    <input type="radio" name="format" value="csv" />UPS Billing Data (.csv
>format, 250 data elements)
>  </div>
>  <div class="dialogButtons">	
>    <input type="button"
>id="viewAndDownloadIntlInvoiceDialogLiteCsvCancelBtn" class="bSwan"
>value="Cancel"/>
>    <input type="button"
>id="viewAndDownloadIntlInvoiceDialogLiteCsvSubmitBtn" class="bSwan"
>value="Submit"/>	
>  </div>
></div>
>
>
>
>In HtmlUnit (2.14) I dont get the Pdf.
>HtmlButtonInput submit =
>myPage.getHtmlElementById("viewAndDownloadIntlInvoiceDialogLiteCsvSubmitBtn");
>Page rslt = submit.click();
>WebResponse response = rslt.getWebResponse();
>
>response shows the loaded html-page.
>
>What is going wrong?
>
>Cheers
>Uli
>
>
>
>
>--
>View this message in context: http://htmlunit.10904.n7.nabble.com/Does-not-get-file-when-InputButton-clicked-tp33583.html
>Sent from the HtmlUnit - General mailing list archive at Nabble.com.
>
>------------------------------------------------------------------------------
>Learn Graph Databases - Download FREE O'Reilly Book
>"Graph Databases" is the definitive new guide to graph databases and their
>applications. Written by three acclaimed leaders in the field,
>this first edition is now available. Download your free book today!
>http://p.sf.net/sfu/NeoTech
>_______________________________________________
>Htmlunit-user mailing list
>Htm...@li...
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>

[Htmlunit-user] Does not get file when InputButton clicked

[UliScherrer <uli...@gm...>]

Hi,
when I click in a GUI Browser on InputButton (Submit) another Windows opens
and shows a Pdf.

<div id="viewAndDownloadIntlInvoiceDialogLiteCsv"
class="dialogWithTitleAndButtons">
  <div class="dialogTitle" >
    View/Download Invoice Data
  </div>
  <div class="dialogContent">
    <p>Please select a file format to view/download your invoice.</p>
    <input type="radio" name="format" value="pdf"  checked="checked"/>PDF
    <input type="radio" name="format" value="csv" />UPS Billing Data (.csv
format, 250 data elements)
  </div>
  <div class="dialogButtons">	
    <input type="button"
id="viewAndDownloadIntlInvoiceDialogLiteCsvCancelBtn" class="bSwan"
value="Cancel"/>
    <input type="button"
id="viewAndDownloadIntlInvoiceDialogLiteCsvSubmitBtn" class="bSwan"
value="Submit"/>	
  </div>
</div>



In HtmlUnit (2.14) I dont get the Pdf.
HtmlButtonInput submit =
myPage.getHtmlElementById("viewAndDownloadIntlInvoiceDialogLiteCsvSubmitBtn");
Page rslt = submit.click();
WebResponse response = rslt.getWebResponse();

response shows the loaded html-page.

What is going wrong?

Cheers
Uli




--
View this message in context: http://htmlunit.10904.n7.nabble.com/Does-not-get-file-when-InputButton-clicked-tp33583.html
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

Re: [Htmlunit-user] java hangs with 100 %

[David M. G. <mic...@gm...>]

Hi,

The problem looks to be solved.

Thanks,
David

Re: [Htmlunit-user] RowContainer.insertRow(int) doesn't allow insert row before the last row

[Ronald B. <rb...@rb...>]

Fixed in SVN, please try with the latest build.

	RBRi

On Fri, 28 Mar 2014 11:19:38 +0100 Marc Guillemot wrote:
>
>Hi,
>
>can you provide a minimal Html + Js code allowing to reproduce the 
>problem and open an issue with it?
>
>Cheers,
>Marc.

Re: [Htmlunit-user] RowContainer.insertRow(int) doesn't allow insert row before the last row

[Amer Al-A. <ame...@ya...>]

Here's a simple code

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 
<html> 
<head> 
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> 
<title>Test</title> 
<script> 
        function insertMyRow(){ 
                var mytable = document.getElementById("MyTable");
                var row = mytable.insertRow(1); 
                var cell = row.insertCell(0); 
                cell.innerHTML =  "<h2>New SecondRow :) </h2>"; 
        } 
</script> 
</head> 
<body> 
<table id="MyTable"> 
        <tr><td><h3>First Row</h3></td></tr> 
        <tr><td><h3>Second Row</h3></td></tr> 
</table> 
<a href="javascript:void(insertMyRow())" id="AddRowLink">Add new Second Row</a> 
</body> 
</html> 

final HtmlPage enclosedPage = (HtmlPage) webClient.getCurrentWindow().getEnclosedPage();
final DomElement elementById = enclosedPage.getElementById("MyTable");
System.out.println("Before insert"); 
System.out.println(elementById.asXml());
                
final HtmlElement link = (HtmlElement) enclosedPage.getElementById("AddRowLink");
link.click(); 
                
System.out.println("\nAfter insert"); 
System.out.println(elementById.asXml());

Before insert 
<table id="MyTable"> 
  <tbody> 
    <tr> 
      <td> 
        <h3> 
          First Row 
        </h3> 
      </td> 
    </tr> 
    <tr> 
      <td> 
        <h3> 
          Second Row 
        </h3> 
      </td> 
    </tr> 
  </tbody> 
</table> 

After insert 
<table id="MyTable"> 
  <tbody> 
    <tr> 
      <td> 
        <h3> 
          First Row 
        </h3> 
      </td> 
    </tr> 
    <tr> 
      <td> 
        <h3> 
          Second Row 
        </h3> 
      </td> 
    </tr> 
    <tr> 
      <td> 
        <h2> 
          New SecondRow :) 
        </h2> 
      </td> 
    </tr> 
  </tbody> 
</table> 

Sent from Yahoo Mail on Android

Re: [Htmlunit-user] RowContainer.insertRow(int) doesn't allow insert row before the last row

[Marc G. <mgu...@ya...>]

Hi,

can you provide a minimal Html + Js code allowing to reproduce the 
problem and open an issue with it?

Cheers,
Marc.
-- 
HtmlUnit support & consulting from the source
Blog: http://mguillem.wordpress.com

On 26/03/2014 04:49, Amer Al-Amayreh wrote:
> Hello
> I have found that RowContainer.insertRow(int) doesn't allow insert row
> before the last row
> The problem is in this line:
>
> if(index >= rowCount - 1) {
>
> I think the -1 should be removed.
>
> http://sourceforge.net/p/htmlunit/code/HEAD/tree/tags/HtmlUnit-2.14/src/main/java/com/gargoylesoftware/htmlunit/javascript/host/RowContainer.java#l150
>
> Thanks
>

Re: [Htmlunit-user] HtmlUnit misses few set of request than real browser

[Marc G. <mgu...@ya...>]

Hi,

can you investigate on the type of the requests and where they come from?

HtmlUnit doesn't load pictures when it is not needed.

Cheers,
Marc.
-- 
HtmlUnit support & consulting from the source
Blog: http://mguillem.wordpress.com


On 28/03/2014 06:01, Sreenath wrote:
> Hello,
>
> I have done some navigation with both htmlunit and real browser,
> compared request invocations in both cases, found few request
> invocations are missing htmlunit navigation. Please advice on this.
>
> Thanks
> Sreenadh