esniper: a lightweight eBay sniping tool / Bugs / #783 Login fails on 'Security measure page"

Andreas Schwarz - 2019-10-13

Same here (with 2.35.1).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ABK - 2019-10-14

Getting the same error. 2.35.1

Last edit: ABK 2019-10-14

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-14

The reason, why you get a "Security Measure" error is, that several login attempts fails within a short time interval.
I can't analyse the problem at the moment, because the internet connection at home is damaged and the german telekom will do a repair of the failed cable not until Oct. 16th.
Please be patient ...

Last edit: Michael S. 2019-10-14

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-14

A short look from my pc at my office:

HTTP/1.1 405 Method Not Allowed

It seems, that the strategy with the "fake login" no longer works.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- ABK - 2019-10-14
  
  Sorry to hear about the cut cable. Yes, multiple failed attempts are unlikely becuase web log out/log in works just fine.
  
  Last edit: ABK 2019-10-14
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

I've had a look to what happens and it seems, that the parameters passed through ebay has been changed:

i1=
pageType=-1
returnUrl=https://www.ebay.com
srt=01000400000050c15dbd9a1cc23523c37f248251e16933930756f768de67d338355c0bfac368df22d463ee82d8165c68ae911fe1edc84099526ef20e645d2da7d3a3542749999c1849e4bea135516de9d52bd04d8c6e1c
tagInfo=ht5%3DAQAAAWYTLNkMAAVlN2U4ZmNmZTE2NTBhYjZiNzBlMWQyMzZmZjcwYjFmZgAAhf0SMLvi1ruq%252BQ%252FZ%252BgYjskepFk4*%7Cht5new%3Dfalse%26usid%3D<usid>
mid=AQAAAWwcIKjuAAU2NDNlYTc4MzE2YjBhNGU4Yzg0MjUzNGRlZGNmMmQ5YwAAWbi6SZzKCFKN4sn6KjkBT+lltbk*
usid=cfb9246616d0a9c475effcccfffd54dd
htmid=
fypReset=
ICurl=
src=
AppName=
srcAppId=
errmsg=
rtmData=PS=T.0
rqid=cfb9246716d0a9c475e0df8fffffffff
lkdhjebhsjdhejdshdjchquwekguid=cfb9246716d0a9c475e0df8fffffffff
distilReqId=b8c4d7f9-00bb-42bd-a749-13c871d90e29
isRecgUser=false
recgUser=
userid=USERNAME
pass=PASSWORD
kmsi-unchecked=1
kmsi=1
rdrlog=<data>

Some data is easy to find in the html source, but the content of 'tagInfo' is not quite clear at the moment.

Maybe it works again, if a correct parameter set is passed.

Also important: The first 'fake login' must use 'httpGet' to avoid error 405.

Greetings

Michael S. - 2019-10-17

tagInfo will be populated by https://secureir.ebaystatic.com/f/0vk0rkyoky1ltm32dhy0hthnxyx.js

Must we built a copy of the javascript in C language ?

Maybe we can pass a simple version of 'tagInfo'.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- ABK - 2019-10-17
  
  Would it be easier to use a transpiler of some sort? https://github.com/andrei-markeev/ts2c
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-17

Tried on https://andrei-markeev.github.io/ts2c/

Transpilation failed. TypeError: Cannot read property '0' of undefined

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-19

I've committed my latest changes, but it's still not working.
The passed parameter-set is exact the same, that the browser passes to ebay.
Using the developer tools within chromium shows different headers an cookies compared to esniper. As a result esniper fails with "HTTP/1.1 405 Method Not Allowed" and "Security Measure" accordingly.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-20

How to reproduce the behavior within a webbrowser (e.g. Chromium):

Switch off (block) java script for www.ebay.com

Delete all cookies from ebay.com

Enter www.ebay.com/signin/s

Within the developer tools go to 'elements'

Search for <button disabled="" id="user-content-sgnBt" ...=""></button>

Edit attribute and change from 'disabled' to 'enabled'

Enter user and password and submit the form

You will get a response 405 and will be redirected to a captcha security page.

I would say, it's a cookie problem. With java script disabled some cookies are not available and they are probably needed for the login process.

The following cookies are missed:

DG_IID

DG_UID

DG_ZID

DG_ZUID

DG_HID

DG_SID

I currently havent found, where they are transfered (js, ajax, ...). Maybe someone can help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-20

Seems to come from https://www.ebay.com/nkfytkqtoxtljvzbxhr.js via XMLHttpRequest.send (async) and ajax. One of the parameters - response header "X-UID" - isn't accessable to esniper.

Maybe someone can help solving this issue. My time isn't unlimited.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

em1 - 2019-10-21

Perhaps some data is stored in IndexedDB. If I disable IndexedDB in Firefox (by setting dom.indexedDB.enabled to false), eBay lets me solve a captcha over and over (apparently forever).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-21

The reason are the cookies I mentioned above.

To reproduce:
1. Call https://www.ebay.com/signin/s (with java enabled)
2. Disable java for all ebay.com pages
3. Remove the cookies starting with "DG"
4. Submit the login form

These cookies seems to be installed by js and ajax technique by a background/parallel process.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-22

Does anyone have experience in js and ajax technique to help solving this issue ?
We need a way to get the "DG" cookies without using js or ajax.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-23

Store the tiny html example and open it in your browser:

<!DOCTYPE html> <html> <body> <form action = "https://www.ebay.com/nkfytkqtoxtljvzb.js?PID=AFB3D8E3-006A-3087-97C7-5DE536335AAC" method = "POST"> Press the submit button to create the DG cookies ...<br> <input type = "submit" name = "submit" value = "Submit" /> </form> </body> </html>

Correction:
The local storage and indexed DB come from this js.

Last edit: Michael S. 2019-10-23
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

NickT - 2019-10-25

Michael, Even in java with a GUI connection, I would struggle to get the cookies. I think the Async task with its associated callback would cause the most difficulty, especially on my always-on headless Raspberry Pi Zero W, accessed via ssh.

I have what, for me might be a possible route, using my junk laptop running Ubuntu and firefox in collaboration with the RPi. I see that from first use the DG cookies have an expiry date of about a month in the future. I can find the value of all those DG cookies on the laptop (after sqlite was installed) by the command:

sqlite> select * from moz_cookies where Name like 'DG_%';
37208|ebay.com||DG_IID|B727316E-2021-3965-B6EB-AE0301AE9B2E|.ebay.com|/|1574204125|1571996863262641|1571576125311795|0|1|0|0
37209|ebay.com||DG_UID|F9831628-E3BF-39D0-9B95-43D811C03A60|.ebay.com|/|1574204125|1571996863262641|1571576125313967|0|1|0|0
37210|ebay.com||DG_ZID|7E5A23EC-6868-3C2B-9BBA-471146E92AF5|.ebay.com|/|1574204125|1571996863262641|1571576125315617|0|1|0|0
37211|ebay.com||DG_ZUID|24226B9B-CE99-3B05-9823-CB5C84F5E5FA|.ebay.com|/|1574204125|1571996863262641|1571576125317342|0|1|0|0
37212|ebay.com||DG_HID|AB32A424-A6CC-3AF8-BEE2-98BF17B3E79A|.ebay.com|/|1574204125|1571996863262641|1571576125318895|0|1|0|0
37213|ebay.com||DG_SID|82.0.110.61:i8Ebt/nFcOPocwMp9dDYBYfRoCzrnXqeXOSCbn1nG2k|.ebay.com|/|1603112125|1571996863262641|1571576125320504|0|1|0|0
sqlite>

The titles of the columns are also available with a .schema command

If a bash script copied a smilar text file over to the RPi would this be of use for a little while to the sniper program?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-25

I'm afraid, that exporting this cookies and load them on startup into esniper would not help to solve this issue, because the js uses a kind of fingerprinting.
An asynchronous task isn't very challenging. You just have to create a 2nd thread which is listening on a particular port. The difficulty would be the analysis about the underlying communication.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Now I know, how to get the 6 DG cookies:

$ curl -v -H 'Content-type: text/xml' --data "p=%7B%22proof..." https://www.ebay.com/nkfytkqtoxtljvzb.js?PID=8C0B2488-68DA-3A7B-BE9F-9A8B123D07E7

The data content you can get from the "request payload" of the XHR network debugger.

eBay answer:

*   Trying 23.211.2.124...
* TCP_NODELAY set
* Connected to www.ebay.com (23.211.2.124) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Jose; O=eBay, Inc.; OU=Slot9428 v2; CN=www.ebay.com
*  start date: Jul 17 00:00:00 2019 GMT
*  expire date: Aug 18 12:00:00 2020 GMT
*  subjectAltName: host "www.ebay.com" matched cert's "www.ebay.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f7ea8)
> POST /nkfytkqtoxtljvzb.js?PID=8C0B2488-68DA-3A7B-BE9F-9A8B123D07E7 HTTP/1.1
> Host: www.ebay.com
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: text/xml
> Content-Length: 6740
> 
* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< content-type: application/x-javascript
< x-uid: 01B2BCDC-3C65-367C-AC20-B1206F4AD8CA
< server: envoy
< x-ah: yyabeafzbaaz
< x-ju: /nkfytkqtoxtljvzb.js
< expires: Thu, 01 Jan 1970 00:00:01 GMT
< cache-control: private, no-cache, no-store, must-revalidate
< surrogate-control: no-store, bypass-cache
< strict-transport-security: max-age=31536000
< x-envoy-upstream-service-time: 150
< x-ebay-pop-id: UFES2-DUS-dweb-1
< x-edgeconnect-midmile-rtt: 6
< x-edgeconnect-origin-mex-latency: 151
< date: Sun, 27 Oct 2019 16:11:07 GMT
< content-length: 0
< set-cookie: DG_IID=2356DC46-...;Max-Age=2628000;HttpOnly;Path=/;Domain=.ebay.com
< set-cookie: DG_UID=3A1C0119-...;Max-Age=2628000;HttpOnly;Path=/;Domain=.ebay.com
< set-cookie: DG_ZID=ADA54D45-...;Max-Age=2628000;HttpOnly;Path=/;Domain=.ebay.com
< set-cookie: DG_ZUID=01B2BCDC-...;Max-Age=2628000;HttpOnly;Path=/;Domain=.ebay.com
< set-cookie: DG_HID=C5699527-...;Max-Age=2628000;HttpOnly;Path=/;Domain=.ebay.com
< set-cookie: DG_SID=...;Max-Age=31536000;HttpOnly;Path=/;Domain=.ebay.com
< 
* Curl_http_done: called premature == 0
* Connection #0 to host www.ebay.com left intact

Now we need to find out, which is a minimal parameter set to this request.

Update I

I've got it in the esniper code and I get the DG cookies, but the "request payload" does not match with the user-agent, language, .... esniper uses for the http-requests.
So the login request fails with code 405.

Update II

You only get the DG cookies, if you leave the "request payload" from your browser untouched. Editing something will break the fingerprint.

Last edit: Michael S. 2019-10-27

Hey Michael. Du scheinst auch deutsch zu sprechen richtig? Ich verstehe nicht genau, wo das Problem liegt.

Ich bekomme die gleiche Fehlermeldung:

Automated esniper bug report.
    esniper version 2.35.0
    libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3
    Error encountered in function ebayLogin in auction.c line 672
    auction = 323960659055, price = 19.99, remain = 0
    latency = 0, result = -1, error = 19
    buf = 0x16802b8, size = 85923, read = 0x16802b8
    time = 1572285709, offset = 0
    pagename = "Sign in or Register | eBay", pageid = "(null)", srcid = "(null)"
    specified options or config values:
      1 x username(u) = ***
      1 x password() = ***
      1 x seconds(s) = 6
      1 x (f) = "magic1"
      1 x reduce() = 1
      1 x bid() = 1
      1 x debug(d) = 1
      1 x batch(b) = 0
      1 x logdir(l) = "/home/pi/esniperlog"
    unknown pageinfo

Was muss ich genau machen, damit es wieder funktioniert? habe was von Cookies, Fingerprint und Java im Browser gelesen. Ich bin gerade etwas überfordert. Vielleicht kannst du mir kurz helfen. Vielen Dank!

Michael S. - 2019-10-28

You couldn't do anything to solve this isssue. esniper uses the curllib, which does not have a java script engine, which is required to run the scripts included in the html code.
The developers must find a way to transmit a "fake fingerprint" and get the mentioned cookies and make esniper work again.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Leo Lu - 2019-10-28
  
  Oh okay :) Thx. Hope, they find a way.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Fred Cox - 2019-10-30

Curiously, JBidWatcher is still working. Do they have an instance of a browser to run JS?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Matteo Neviani - 2019-12-07
  
  Curiously, JBidWatcher is still working. Do they have an instance of a browser to run JS?
  
  JBidWatcher stopped working as well a few days ago.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael S. - 2019-10-31

JBidWatcher based on java. esniper is pure C-code.

A parameter set is build by nkfytkqtoxtljvzb.js an is transmitted to https://www.ebay.com. The response is a DG-cookie bundle und some stuff in a local DB. A minimum of that have to be created by C-code within esniper. Currently I have no idea, what ist the "minimal parameter set". Maybe someone can assist.

You do not need to have C-experience. You just have to playing with the payload of the

$ curl -v -H 'Content-type: text/xml' --data "p=%7B%22proof..." https://www.ebay.com/nkfytkqtoxtljvzb.js?PID=8C0B2488-68DA-3A7B-BE9F-9A8B123D07E7

request.

The data can be copied out out of the browser network debugger. Decoding and encoding of the payload can be done e.g. by https://www.urlencoder.org

If you transmit a valid payload, you get the DG-cookies (see above).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Login fails on 'Security measure page"

Group

Searches

Help

#783 Login fails on 'Security measure page"

Related

Discussion