Re: [Htmlunit-user] Sending Cookies in HTTP header

[Marvin D. <mar...@gm...>]

Ahmed,
My goal is to use htmlunit to login into my account and complete the Oauth2
process and receive the authorization code. This process consists of a GET
(receiving login page), a POST(submitting the form from the login page) and
another POST (submitting the form granting access to my account) and
returning the Authorization code in the response. When I complete the
process on Chrome the second POST returns a response (Status Code: 302
Found )with a location header that contains the Authorization Code.

I'm having trouble submitting the second post as I receive:

java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException:
Connect to 127.0.0.1:80 [/127.0.0.1] failed: Connection refused: connect
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2018)
at com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:140)
at
com.gargoylesoftware.htmlunit.html.HtmlButton.doClickStateUpdate(HtmlButton.java:87)
at
com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1321)
at
com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1268)
at
com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1216)
at com.ricoh.box.App.main(App.java:244)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to
127.0.0.1:80 [/127.0.0.1] failed: Connection refused: connect
at
org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:140)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:314)
at
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at
com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:178)
at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1313)
at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1371)
at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1230)
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2014)
... 6 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at
org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:72)
at
org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:117)
... 19 more



Hi Marvin,
You usually put log4j.properties in your project path, and configure it
accordingly.
Anyhow, your case fails because of 404, I am not sure if you get a
different result.
Also, by enabling HtmlUnit to throw JavaScript errors, there is error
because NativeError.stack is not yet implements [1], and this is a known
issue.
Yours,Ahmed
[1]
https://sourceforge.net/p/htmlunit/code/HEAD/tree/trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/javascript/NativeErrorTest.java#l34


      From: Marvin Desrosiers <marvin.desrosiers@...>
 To: htmlunit-user@...
 Sent: Friday, March 18, 2016 9:44 PM
 Subject: [Htmlunit-user] Fwd: Emailing: HTML-Email.html

Ahmed,I tried enabling the logger by adding:Logger.
getLogger("org.apache.http.headers").setLevel(Level.ALL);
But I didn't see any change. I've provided the credentials below.Thank
you,-Marvin
USER_EMAIL = ricohusa23@... = htmlunit AUTHORIZE_LOCATION =
https://app.box.com/api/oauth2/authorize?response_type=code&client_id=tr0z0beizpf2zutquoy0heainecgq7s0&redirect_uri=http://127.0.0.1&state=unused
Begin forwarded message:


To: marvin.desrosiers@...
Subject: Emailing: HTML-Email.html



Hi Marvin,

You can enable the logger of "org.apache.http.headers"in log4j.properties
[1] so see what HtmlUnit is sending.

Otherwise you need to provide more details,including the URL and
credentials, for others to look reproduce the issue.

Ahmed

[1]
https://sourceforge.net/p/htmlunit/code/HEAD/tree/trunk/htmlunit/src/test/resources/log4j.properties



From: Marvin Desrosiers <marvin.desrosiers@...>
To: htmlunit-user@...
Sent: Friday, March 18, 2016 6:43 PM
Subject: [Htmlunit-user] Sending Cookies in HTTP header

I'm trying to understand why the cookies are not being sent back to the
server. htmlunit is set with setUseInsecureSSLby default to false for the
secure tag.

try {
                WebClient webClient= newWebClient(BrowserVersion.CHROME);

                /*webClient.getOptions().setUseInsecureSSL(false);*/

/*webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);*/
                webClient.getOptions().setCssEnabled(false);
                webClient.getOptions().setJavaScriptEnabled(false);

                CookieManager cookieManager=webClient.getCookieManager();

                out.println(cookieManager.getCookies().toString());

            out.println("start");

            final HtmlPage loginPage= webClient.getPage(AUTHORIZE_URL);
            String requestToken =
loginPage.getElementByName("request_token").getAttribute("value");
            out.println("requesttoken :" + requestToken);


            out.println("afterloginPage");



            out.println(cookieManager.getCookies().toString());
            Set<Cookie> boxCookies= newHashSet<Cookie>();
            boxCookies.addAll(webClient.getCookieManager().getCookies());
            StringBuilder cookieHeader =newStringBuilder();

            /*Iterator<Cookie> ite= boxCookies.iterator();
            while (ite.hasNext()){
                    Cookiecookie = ite.next();
                    cookie.getDomain().substring(1);
                    Stringname = cookie.getName();
                    Stringvalue = cookie.getValue();

                    System.out.println("Cookie:"+ name + "=" +value);
                    if()){
                           cookieHeader.append(name + "=" +value);
                    }
                    cookieHeader.append(";"+ name + "=" +value);
            }
            */
            webClient.addRequestHeader("Accept-Encoding","gzip, deflate,
sdch");
            webClient.addRequestHeader("Accept-Language","en-US,en;q=0.8");
            webClient.addRequestHeader("Cache-Control","no-cache");
            webClient.addRequestHeader("Upgrade-Insecure-Requests","1");
            webClient.addRequestHeader("DNT","1");
            webClient.addRequestHeader("Host","app.company.com");

            for (int index= 0; index < boxCookies.size(); index++){
                    Stringcookie = boxCookies.toArray()[index].toString();
                    StringcookieNameValue=cookie.substring(0,
cookie.indexOf(";"));
                String name =
cookieNameValue.substring(0,cookieNameValue.indexOf("="));
                String value
=cookieNameValue.substring(cookieNameValue.indexOf("=") + 1);

                    if(index == 0){
                          cookieHeader.append(name + "=" +value);
                    }else {
                          cookieHeader.append("; "+ name + "="+value);
                    }

            }
            WebRequest secondLoginPage =newWebRequest(AUTHORIZE_URL);

secondLoginPage.setAdditionalHeader("Cookie",cookieHeader.toString());
            HtmlPage loginPage2 = webClient.getPage(secondLoginPage);


out.println(loginPage2.getWebResponse().getWebRequest().getRequestParameters().toString());
            out.println("\n");
            Map<?, ?> additionalRequest1=
loginPage2.getWebResponse().getWebRequest().getAdditionalHeaders();

            Iterator<?> ite0 = additionalRequest1.entrySet().iterator();
             while(ite0.hasNext()){
                    out.println(ite0.next());
             }
             out.println("\n");


            final HtmlTextInput login= (HtmlTextInput)
loginPage2.getElementById("login");
            login.setValueAttribute(USER_EMAIL);

            final HtmlPasswordInputpassword= (HtmlPasswordInput)
loginPage2.getElementById("password");
            password.setValueAttribute(USER_PASS);

            final HtmlSubmitInputbutton_submit =
loginPage2.getElementByName("login_submit");

            final HtmlPage accessGrantingPage= button_submit.click();

            final HtmlForm  requestForm=
(HtmlForm)accessGrantingPage.getElementById("consent_form");

            Map<?, ?> additionalRequest=
accessGrantingPage.getWebResponse().getWebRequest().getAdditionalHeaders();
            out.println("\n");
            Iterator<?> ite2 = additionalRequest.entrySet().iterator();
             while(ite2.hasNext()){
                    out.println(ite2.next());
             }
            out.println("\n");

            out.println("afteraccessGrantingPage");

            out.println(cookieManager.getCookies().toString());
            final HtmlButton consent_accept_button=
accessGrantingPage.getElementByName("consent_accept");

            try {
                    finalHtmlPage authorizationPage =
consent_accept_button.click();
                    out.println("afterauthorizationPage");
                    out.println(authorizationPage.getUrl().toString());

out.println(authorizationPage.getWebResponse().getStatusMessage());


out.println(authorizationPage.getWebResponse().getResponseHeaders());

            } catch (RuntimeExceptionre){
                    re.printStackTrace();
            }
            webClient.closeAllWindows();
        } catch (IOException ioe){
                ioe.printStackTrace();
        } finally {

        }
Your message is ready to be sent with the following file or link
attachments:
HTML-Email.html

Note: To protect against computer viruses, e-mail programs may
preventsending or receiving certain types of file attachments.  Check
youre-mail security settings to determine how attachments are handled.


------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.

_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user