esniper: a lightweight eBay sniping tool / Bugs / #649 Automated esniper bug report. esniper version 2.28.0

Bodo - 2014-03-21

It looks as if ebay added another bid confirmation page or modified it in a way that esniper can no longer handle.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Frank Löffler - 2014-03-21

Same just now here:

Auction 111301583006: Bidding...
esniper encountered a bug. Please go to:
http://sourceforge.net/tracker/?func=add&group_id=45285&atid=442436
paste this into "Detailed Description":
Automated esniper bug report.
esniper version 2.28.0
libcurl/7.26.0 GnuTLS/2.12.20 zlib/1.2.7 libidn/1.25 libssh2/1.4.2 librtmp/2.3
Error encountered in function parseBid in auction.c line 584
auction = 111301583006, price = 65.65, remain = 127
latency = 0, result = -1, error = 0
buf = 0x2822130, size = 13220, read = 0x2822130
time = 1395425701, offset = 0
pagename = "Review and confirm your bid", pageid = "(null)", srcid = "(null)"
specified options or config values:
1 x username(u) =
1 x password() =
1 x seconds(s) = 5
1 x (f) = "lego_sets"
1 x bid() = 1
1 x delay(D) = 10
unknown pagename
then upload and attach esniper.19382.1.bug.html and click submit.
Cannot determine result of bid
Auction 111301583006: Waiting 6 seconds for auction to complete...

Auction 111301583006: Post-bid info:
Auction 111301583006: Lego Kits
Time remaining: -- (0 seconds)
Currently: 49.99 (your maximum bid: HIGHER)

of bids: 2

High bidder: n***s (NOT ME)

won 0 item(s)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Frank Löffler - 2014-03-21

log file attached

esniper.19382.1.bug.html

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Frank Löffler - 2014-03-21

Looking through the code I cannot find a place where esniper would handle a confirmation page by actially confirming a bid. acceptBid() looked like the right place at first, but all it seems to do is to look whether a bid was accepted - not to handle a confirmation.

Is esniper not supposed to also handle these?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Bodo - 2014-03-21

I'm not the original author of this part,so I have to analyze the code as well.
The attached page looks different from what I remember to get when I last bid manually. Bidding as you do it manually is (used to be) a two-step process, assuming you are already logged in. Of course the same is implemented in esniper. The first step is called preBid and I think it is done 2 minutes before the end of the auction. At this step esniper will get a hidden value (bid uiid) which is needed for the next step. The second step, which is in fact the confirmation, is done at the calculated bid time.
So ebay must have changed something. Maybe it is no longer possible to wait 2 minutes bitween preparing and confirming the bid, or something might have changed in the mechanism of bidding that makes ebay present the/another confirmation page after what should be the final step.
I guess this needs a detailed analysis of the HTML forms used for bidding.

Last edit: Bodo 2014-03-21

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Frank Löffler - 2014-03-22

The attached page is the confirmation page, the one that comes up after first entering the bid amount. I also see that what happens next depends on what you bid on. Toys seem to get the page that I attached (which opens as completely new page), while other items I tried only open a pop-up window. Yet other items get the page the original poster attached - a completely new page but without the hint about toys and safety recalls.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Frank Löffler - 2014-03-22

The confirmation page seems to point to itself, with some additional arguments in the html form. So I tried naively to add them to the original request to avoid that page altogether, but that also did not seem to work. Most likely I did a mistake in my attempt, but maybe another get request is necessary at this point which would be unfortunate. More insight is needed, but I don't have it right now.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

DavidM - 2014-03-25

I love esniper and use it constantly, so this issue is hurting. Is someone taking ownership of the fix? If so, is there anything I can do to help? If not, I'm happy to try to take a look.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

easy_john - 2014-03-25

This sofware is opensource, so if you can fix it - please upload a fix.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

adlerweb - 2014-03-31

For reference: This also happens with "seconds=now" so problem is most likely not the 2 minute timeout.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Bodo - 2014-03-31
  
  This also happens with "seconds=now" so problem is most likely not the 2 minute timeout.
  
  That means ebay must have changed the forms or mechanism used for bidding.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Christian Burger - 2014-04-01

I think the page we are seeing is a result of an incomplete bid and actually the second time esniper sees that page. Might be, that the following is happening: First time esniper sees the confirmation page is during parsePreBid() in "auction.c". esniper parses it and tries to confirm with the parsed information. eBay is not satisfied with the answer (i. e. the confirmation) and displays the same page again. The page now gets handled by parseBid() in "auction.c", which actually should handle the bid result.

So, in my opinion the bid confirmation process was slightly altered.

At this moment there are two potential issues, I can see, with the way esniper is confirming a bid.

eBay uses "POST", esniper uses "GET" for the bid confirmation

esniper submits the following values:
MfcISAPICommand, maxbid, quant, mode, uiid, co_partnerid, user, fb, item
eBay form actually submits:
MfcISAPICommand, maxbid, quant, mode, stok, srt, uiid, co_partnerid, user, fb, item

Issue "1." might be no issue at all, could be that eBay does not care if the values are send by POST or GET.

Issue "2." might be the culprit. Both variables "stok" and "srt" seem to hold values which could be some kind of access token. "stok" is constant on every reload of the bid-confirmation page. "srt" changes every time and it's a very long number -- has 174 characters in hexadecimal notation. You actually can see both values in the attached bug reports.

I think the first thing to do is parse "stok" and "srt" during prebid: Should be probably implemented in "auction.c" around line 314 in function "parsePreBid()".

@Bodo, judging from the recently closed tickets, it seems like you're in charge of the project. Any thoughts?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

adlerweb - 2014-04-01

1) eBay does not care (or esniper is already using POST, see below).
2) The content should not be a problem since uiid is a similar kind of access token which is already implemented - afair per auction so should also be fine for auctionfiles with multiple items.

I already modified my local build and added stok and srt. First try placed the bid just fine, however i tested only a single auction before i went off to work and result parsing seems to be still broken.

Edit:
OK, i can not see any way aip->won is anything else than -1 in auction.c:924… Patch of my (barely tested) bid-fix attached…

Last edit: adlerweb 2014-04-01

esniper-2-28-0-bidfix.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

adlerweb - 2014-04-01

Nevermind my comment about result parsing, guess that was just because i used time = now for testing. Last snipe exited with "won 1 item(s)" as intended. So if someone still got items queued please test the patch attached in the previous comment, my products where mostly computer related.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

easy_john - 2014-04-01

Not work:
./start_esniper: line 2: 16701 Segmentation fault esniper -b ./sniper.conf

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

adlerweb - 2014-04-01

Hmm… Patched a fresh source here, just this + CURL_CA_BUNDLE-Patch, no segfault on x86_64… Mind to give some more info? What is start_esniper? Architecture? What action did you try to perform? Type and state of item/auction?

./esniper -b /home/adlerweb/testfiles/test1 Auction **********: ********* Time remaining: -- (0 seconds) Currently: **.** (your maximum bid: **) # of bids: 1 High bidder: *****!!! You have already won 1 item(s). Quantity reduced to 0 item(s).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

./start_esniper is:
#!/bin/bash
esniper -b ./sniper.conf

System:
$ uname -a
Linux john 3.6.11-gentoo #1 SMP Wed Aug 21 00:20:31 MSK 2013 x86_64 Intel(R) Core(TM) i5-3550 CPU @ 3.30GHz GenuineIntel GNU/Linux

All ok until 2 minutes before end:

Auction 111310148099: Yamaha sound Module FB-01
Time remaining: 10 mins 11 secs (611 seconds)
End time: 01/04/2014 22:28:13
# of bids: 0
Currently: --  (your maximum bid: 61)
High bidder: -- (NOT easy_john)
Latency: 0 seconds
Tue Apr  1 22:18:03 2014: Sleeping for 8 minutes 0 seconds

Auction 111310148099: Yamaha sound Module FB-01
Time remaining: 2 mins 9 secs (129 seconds)
End time: 01/04/2014 22:28:12
# of bids: 0
Currently: --  (your maximum bid: 61)
High bidder: -- (NOT easy_john)
Latency: 2 seconds

./start_esniper: line 2: 15692 Segmentation fault      esniper -b ./sniper.conf

then i start it again and:

$ ./start_esniper
Auction 111310148099: Yamaha sound Module FB-01
Time remaining: 1 min 28 secs (88 seconds)
End time: 01/04/2014 22:28:13
# of bids: 0
Currently: --  (your maximum bid: 61)
High bidder: -- (NOT easy_john)

Auction 111310148099: Yamaha sound Module FB-01
Time remaining: 1 min 27 secs (87 seconds)
End time: 01/04/2014 22:28:13
# of bids: 0
Currently: --  (your maximum bid: 61)
High bidder: -- (NOT easy_john)
Latency: 0 seconds

./start_esniper: line 2: 16701 Segmentation fault      esniper -b ./sniper.conf

easy_john - 2014-04-01

Ok, may be it's my bug. Recompile from clean sources. But don't have items to test rught now.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

adlerweb - 2014-04-01

Hmmm… This would be somewhere in preBid()… I found some missing free()-calls there, but I (as not-really-C-adept) would think this just leaks memory and does not cause the crash. I also could hopefully recreate the problem (inserted a forced call of preBid during discovery), according to the log its related to the pointer-foo during stok-search - did i mention how much i hate C and its memory management? ;)

(Attached the missing free()-calls)

Last edit: adlerweb 2014-04-01

esniper-2-28-0-bidfix.2.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Frank Löffler - 2014-04-01

After looking through the code: is there any specific reason esniper is written in C, instead of some scripting language? Maybe only the original author knows...

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

adlerweb - 2014-04-01

Gnah - C for startes page 1… Missed the variable initialization, these do happen in a separate file (aka grep missed it) and i am cursed using using weakly typed scripting language on a daily basis (aka i missed it)… Attached version should no longer segfault…

((OT
Well… Its looks not quite as esoteric as perl, got still more consistent function calls than PHP, can - unlike python - use do-while, does (usually) not break using newer compilers like ruby and needs several gigabytes less RAM than java - and you don't need to install additional interpreters or waste time/memory loading these first ;)
))

esniper-2-28-0-bidfix.3.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Bodo - 2014-04-01

Thanks for the analysis and the patch.
I hope to find some time to integrate it into the source code and to build a new version.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Christian Burger - 2014-04-01

Awesome. Hope you don't mind an additional patch. I made a quick code review and I think there is a small problem with binary literal handling in the last patch:

0x00000010 is actually 16
0b00000010 is 2

But the "0b0..." notation is not standard C, though it works in GCC and probably in Clang. I would prefer the C language extension, but I don't know if people use Visual Studio to compile esniper.

I have adlerweb's patch running now. I will drop a line as soon as I can tell if it worked or not.

0002-esniper-2-28-0-bidfix.part2.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Bodo - 2014-04-02

Thanks for checking and improving the fix.
I also intended to replace += with |=. Additionally I will use #defines instead of magic numbers (1, 2, 4).
The proposed error handling from esniper-2-28-0-bidfix.3.patch may print up to three bug reports if more than one token cannot be found. I will replace this with a single call to bugReport, e.g.

bugReport("preBid", __FILE__, __LINE__, aip, mp, optiontab, "cannot find bid token (found=%d)", found);

Last edit: Bodo 2014-04-02
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Christian Burger - 2014-04-03
  
  Additionally I will use #defines instead of magic numbers (1, 2, 4).
  
  Yeah, I thought a bit about code readibility while I was looking for the problem. Probably a good idea. :)
  
  A bid just got through on my end. Thanks everybody.
  
  BTW: Ever thought about switching the version control system? To something like Git or Mercurial? Just a thought.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Automated esniper bug report. esniper version 2.28.0

Group

Searches

Help

#649 Automated esniper bug report. esniper version 2.28.0

of bids: 1

Discussion

of bids: 2