Re: [Htmlunit-user] HTMLUnit with Recaptcha service of Google

[Vasudevan C. <vco...@gm...>]

Hi Ahmed,

    I can rerun the application & the real browser after clearing the
cache. Accept header position was different which
    I also noticed. Does it have an impact?.

   My suggestion is if you can support the position in request header for
addRequestHeader method, then we can compare
    the behaviour exactly.

    I am storing the cookie value in my app for the designated browser and
sending the same to www.google.com for
    subsequent requests raised from the browser.

    I shall send you the header info after clearing the cache/cookies etc.

Regards
 Vasu



On Thu, Aug 21, 2014 at 12:10 PM, Ahmed Ashour <asa...@ya...> wrote:

> Hi Vasu,
>
> There are two differences:
>     - Cookies in the second request of real browser, they seem to be
> stored from previous requests. Please clear cache and rerun the comparison.
> I also wonder from where did HtmlUnit send the cookies in the second
> request.
>     - 'Accept' header location: this could be a reason, but Cookie are the
> much more the possibility.
>
> Ahmed
>   ------------------------------
>  *From:* Vasudevan Comandur <vco...@gm...>
> *To:* Ahmed Ashour <asa...@ya...>; "
> htm...@li..." <htm...@li...>
>
> *Sent:* Wednesday, August 20, 2014 9:06 PM
> *Subject:* Re: [Htmlunit-user] HTMLUnit with Recaptcha service of Google
>
> Hi Ahmed,
>
>     I am giving you the request details for REAL Browser & HTMLUnit below.
>
> REal Browser (First Request)
>
>
> GET /challenge?k=6Lf1-AQAAAAAAL9EskceOVja5Uh5_yo1FRtb1Q83 HTTP/1.1Host
> api.recaptcha.netUser-AgentMozilla/5.0 (Windows NT 5.1; rv:31.0)
> Gecko/20100101 Firefox/31.0Accept*/*Accept-Languageen-US,en;q=0.5
> Accept-Encodinggzip, deflateReferer
> http://www.patcurtis.com/captcha/recaptcha_demo.phpConnectionkeep-alive
> Cache-Controlmax-age=0
> WebServer(Response)
>
> HTTP/1.1 302 Moved TemporarilyCache-Controlno-cache, no-store, max-age=0,
> must-revalidatePragmano-cacheExpiresFri, 01 Jan 1990 00:00:00 GMTDateWed,
> 20 Aug 2014 11:40:18 GMTLocation
> http://www.google.com/recaptcha/api/challenge?k=6Lf1-AQAAAAAAL9EskceOVja5Uh5_yo1FRtb1Q83
> Content-Typetext/html; charset=UTF-8Content-Encodinggzip
> X-Content-Type-OptionsnosniffX-XSS-Protection1; mode=blockContent-Length
> 227ServerGSEAlternate-Protocol80:quic
> RB(Request)
>
> GET /recaptcha/api/challenge?k=6Lf1-AQAAAAAAL9EskceOVja5Uh5_yo1FRtb1Q83
> HTTP/1.1Hostwww.google.comUser-AgentMozilla/5.0 (Windows NT 5.1; rv:31.0)
> Gecko/20100101 Firefox/31.0Accept*/*Accept-Languageen-US,en;q=0.5
> Accept-Encodinggzip, deflateReferer
> http://www.patcurtis.com/captcha/recaptcha_demo.phpCookiePREF=ID=27454a65a243cd70:U=508d4e3f9fa20406:FF=0:LD=en:TM=1406649235:LM=1407436763:GM=1:S=aCHb3CgmGRRcedMi;
> _ga=GA1.1.1138220839.1407426218; OGP=-4061129:;
> NID=67=pLb98eXX7XshplSC25O1hUrM6VPe1bG9epaYJREF4EBGoV-TPIQ0MynwuZdb1ra4i_j8jfzeKgHyJRL917tqVvJ1drL2BTn2QE4HaRblzxd0lpITPiswWjYNWO3YMn8g0aV7bw3zFqdW9HrdT3h2iJxykpn7QTCJm7fFHZoSYVRR1zWL_jNpSHvs7u912M5K1skdqY-lGtU;
> SID=DQAAAMkAAAD1nK0mZZoRhWbUjiHM6nCXAWouJuka0s3NW3l9AO0Tp7vLrszmHqYEeOI2mbDn6Ew2DuOvg0DxTEDPsjQ_-lMPWoXn_8TQYCEpFyvdZUdqdYT1qb2jNjv7cb6192vCkFAjY4LPJo5tKDUvm8_UDsJGrrILWzwGEf9TMkyo8uMP_rZcL3JFbVWfsN7W8j2MZWRLlSoZ-wcpW8PiQc_se5XSVXoZKUdGd67OemZdbEBmX_VRZ66mJln6EtJeJdOWruXp_I72ldDaz9UZMvZJ7ySH;
> HSID=AbDchAUPI4DtlOcyV; APISID=xLgJV19gOJC9cCD_/ARvW0PuATO71HDPNc
> Connectionkeep-alive
> WebServer(Response)
>
> HTTP/1.1 200 OKCache-Controlno-cache, no-store, max-age=0, must-revalidate
> Pragmano-cacheExpiresFri, 01 Jan 1990 00:00:00 GMTDateWed, 20 Aug 2014
> 11:39:22 GMTContent-Typetext/javascriptContent-Encodinggzip
> X-Content-Type-OptionsnosniffX-XSS-Protection1; mode=blockContent-Length
> 7090ServerGSEAlternate-Protocol80:quic
> HTMLUnit (First Request)
>
> GET /challenge?k=6Lf1-AQAAAAAAL9EskceOVja5Uh5_yo1FRtb1Q83 HTTP/1.1Host
> api.recaptcha.netUser-AgentMozilla/5.0 (Windows NT 5.1; rv:31.0)
> Gecko/20100101 Firefox/31.0Accept-Languageen-US,en;q=0.5Accept-Encodinggzip,
> deflateRefererhttp://www.patcurtis.com/captcha/recaptcha_demo.php
> Connectionkeep-aliveAccept*/*Cache-Controlmax-age=0
> WebServer(Response)
>
>
> HTTP/1.1 302 Moved TemporarilyCache-Controlno-cache, no-store, max-age=0,
> must-revalidatePragmano-cacheExpiresFri, 01 Jan 1990 00:00:00 GMTDateWed,
> 20 Aug 2014 18:22:58 GMTLocation
> http://www.google.com/recaptcha/api/challenge?k=6Lf1-AQAAAAAAL9EskceOVja5Uh5_yo1FRtb1Q83
> Content-Typetext/html; charset=UTF-8Content-Encodinggzip
> X-Content-Type-OptionsnosniffX-XSS-Protection1; mode=blockContent-Length
> 227ServerGSEAlternate-Protocol80:quic
> HTMLUnit(Request)
>
> GET /recaptcha/api/challenge?k=6Lf1-AQAAAAAAL9EskceOVja5Uh5_yo1FRtb1Q83
> HTTP/1.1Hostwww.google.comUser-AgentMozilla/5.0 (Windows NT 5.1; rv:31.0)
> Gecko/20100101 Firefox/31.0Accept-Languageen-US,en;q=0.5Cookie
> NID=67=cA1Qro6B-PbS07zWxNtD3iJ0hHfuc0itJYr1CVnyDCL4PbJNELomp3zFICR6NGdC0C3rQ440_1Y743dl87W5cTtWbAadzMGOZ0-1hbYTqYOSqZrSeN922y0XQr5IcKGE
> Accept-Encodinggzip, deflateReferer
> http://www.patcurtis.com/captcha/recaptcha_demo.phpConnectionkeep-alive
> Accept*/*Cache-Controlmax-age=0
> WebServer(Response)
>
> HTTP/1.1 200 OKCache-Controlno-cache, no-store, max-age=0, must-revalidate
> Pragmano-cacheExpiresFri, 01 Jan 1990 00:00:00 GMTDateWed, 20 Aug 2014
> 18:22:58 GMTContent-Typetext/javascriptContent-Encodinggzip
> X-Content-Type-OptionsnosniffX-XSS-Protection1; mode=blockContent-Length
> 6994ServerGSEAlternate-Protocol80:quic
> *******************************************************************
>
> Kindly Let me know if you need any additional information. Please note
> that there are many request/response exchanges
> after the above steps till we get the actual CAPTCHA image
>
> Thanks for your help in advance.
>
> Regards
>  Vasu
>
>
>
> ------------------------------------------------------------------------------
> Slashdot TV.
> Video for Nerds.  Stuff that matters.
> http://tv.slashdot.org/
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>