#1210 HTTP PUT not receiving peer cgi's error response

[Fraxinas]

as reported on the mailing list
see http://curl.haxx.se/mail/lib-2013-03/0225.html

Here's a step-to-step tutorial to simulate the problem:

• save attachments to /tmp & cd /tmp
  $ gcc -g -o /srv/http/cgi-bin/ret_400.cgi ret_400.c
  (or output wherever your httpd's cgi-bin directory is)
  $ gcc -g pkg-config libcurl --cflags --libs curlputtest.c -o curlputtest
  $ ./curlputtest
• now you get a prompt where you have to enter an http get request.
  if you enter the absolute path of an existing file, it will upload that, e.g.:
  GET /tmp/curlputtest.c HTTP/1.0
• the second line is the upload-destination, so the path where on your
  webserver the ret_400 cgi-bin resides, for example:
  X-DESTINATION: http://localhost/cgi-bin/ret_400.cgi
• the testprog will then output a debug line with the aquired paths:
  curl put mode on!! source path '/dream/devel/dnstreamproxy/curlputtest.c'
  destination url 'http://localhost/cgi-bin/ret_400.cgi'
• after another newline, curl will try to connect to the given url and try to
  upload the file which fails with error 400 because the cgi-bin expects a
  parameter with a length of exactly 10 characters. this is expected!
• if you enter a X-DESTINATION url with a 10-digit parameter, it succeeds with
  status code 201:
  X-DESTINATION: http://localhost/cgi-bin/ret_400.cgi?1234567890
• now if instead of an accessible file, you put some bogus stuff after in the
  get request, it'll generate dummy packets and simulate a continuous stream
  without determined file size:
  GET /thisisnotafilename HTTP/1.0
• as X-DESTINATION your specify the cgi-bins uri but with wrong parameter,
  then the expected behaviour is that it should not even once call the READFUNC
  and error out with the 404 status right away like it does with the determined
  file size.
  instead the READFUNC is called over and over again and the upload never ends
  (even though the cgi-bin on the server has set the 404 status and terminated
  already) i've wiresharked this and it shows, that the apache doesn't actually
  send out the http error code, until the curlputtest is aborted. i'm guessing
  that the testcase for some reason doesn't even give the webserver a chance to
  send its answer in this upload mode without determined content-size.
  when uploading an actual file, it does work just fine, so this is already kind
  of a minimal example and can't really be more simplified i guess :(
  cheers,
  fraxinas

version info
libcurl/7.29.0 OpenSSL/1.0.1e zlib/1.2.7 libssh2/1.4.3

[Fraxinas]

testcases.log

[Fraxinas]

ret_400.c

[Daniel Stenberg]

We have 32 test cases doing PUT and they all work. Can you please modify one of them to repeat the problem? This way to repeat the problem is way to complicated for me to ever bother with.

[Fraxinas]

here we go, i hope i did it right

[Daniel Stenberg]

The problem with that approach is that the test server isn't very good at handling an incoming chunked-encoded PUT and responding before the PUT is done.

I created a variation of your test that makes a length-limited PUT that gets aborted by a 400 response, and it verifies that the write callback can indeed read the HTTP response code as soon as it has been received - which is before the full PUT has been sent. The fact that it is length-limited instead of chunked really is of minor difference to libcurl internals.

My patch adds test1513 and should be applied on top of your patch - I hope. If not, let me know and I'll clean it up.

[Fraxinas]

i've put up the example on my server:
http://frax.ath.cx/cgi-bin/ret_400.cgi
it instantly returns error 400

[Daniel Stenberg]

No it doesn't. It first returns a 100. What exactly is the test file supposed to prove?

[Fraxinas]

the problem was that the internal test case http server doesn't support this scenario, so you asked me to put up my script on a public server

it returns 400 in this case:

$ curl -XPUT -T - http://frax.ath.cx/cgi-bin/ret_400.cgi?bla_argument -vvv < ret_400.c

• Adding handle: conn: 0xd77450
• Adding handle: send: 0
• Adding handle: recv: 0
• Curl_addHandleToPipeline: length: 1
• • Conn 0 (0xd77450) send_pipe: 1, recv_pipe: 0
• About to connect() to frax.ath.cx port 80 (#0)
• Trying 217.83.34.172...
• Connected to frax.ath.cx (217.83.34.172) port 80 (#0)

  PUT /cgi-bin/ret_400.cgi?bla_argument HTTP/1.1
  User-Agent: curl/7.30.0
  Host: frax.ath.cx
  Accept: /
  Transfer-Encoding: chunked
  Expect: 100-continue

  < HTTP/1.1 100 Continue
  < HTTP/1.1 400 Bad Request
  < Date: Mon, 27 May 2013 18:07:22 GMT

• Server Apache/2.2.22 (Ubuntu) is not blacklisted
  < Server: Apache/2.2.22 (Ubuntu)
  < Connection: Close
  < Vary: Accept-Encoding
  < Content-Length: 37
  < Content-Type: text/html
  <
  Invalid argument data: bla_argument
• Closing connection 0

[Fraxinas]

$ curl -XPUT -H Expect: -T - "http://frax.ath.cx/cgi-bin/ret_400.cgi?12345678900" -vvv < ret_400.c

• Adding handle: conn: 0x1be6490
• Adding handle: send: 0
• Adding handle: recv: 0
• Curl_addHandleToPipeline: length: 1
• • Conn 0 (0x1be6490) send_pipe: 1, recv_pipe: 0
• About to connect() to frax.ath.cx port 80 (#0)
• Trying 217.83.34.172...
• Connected to frax.ath.cx (217.83.34.172) port 80 (#0)

  PUT /cgi-bin/ret_400.cgi?12345678900 HTTP/1.1
  User-Agent: curl/7.30.0
  Host: frax.ath.cx
  Accept: /
  Transfer-Encoding: chunked

  < HTTP/1.1 400 Bad Request
  < Date: Mon, 27 May 2013 18:11:49 GMT

• Server Apache/2.2.22 (Ubuntu) is not blacklisted
  < Server: Apache/2.2.22 (Ubuntu)
  < Connection: Close
  < Vary: Accept-Encoding
  < Transfer-Encoding: chunked
  < Content-Type: text/html
  <
  Invalid argument data: 12345678900
• Closing connection 0

[Fraxinas]

$ curl -XPUT -H Expect: -T - "http://frax.ath.cx/cgi-bin/ret_400.cgi?12345678900" -vvv < ret_400.c

• Adding handle: conn: 0x1be6490
• Adding handle: send: 0
• Adding handle: recv: 0
• Curl_addHandleToPipeline: length: 1
• • Conn 0 (0x1be6490) send_pipe: 1, recv_pipe: 0
• About to connect() to frax.ath.cx port 80 (#0)
• Trying 217.83.34.172...
• Connected to frax.ath.cx (217.83.34.172) port 80 (#0)

  PUT /cgi-bin/ret_400.cgi?12345678900 HTTP/1.1
  User-Agent: curl/7.30.0
  Host: frax.ath.cx
  Accept: /
  Transfer-Encoding: chunked

  < HTTP/1.1 400 Bad Request
  < Date: Mon, 27 May 2013 18:11:49 GMT

• Server Apache/2.2.22 (Ubuntu) is not blacklisted
  < Server: Apache/2.2.22 (Ubuntu)
  < Connection: Close
  < Vary: Accept-Encoding
  < Transfer-Encoding: chunked
  < Content-Type: text/html
  <
  Invalid argument data: 12345678900
• Closing connection 0

[Daniel Stenberg]

(as was discussed on IRC a few hours ago)

The server waits for the full request before it responds, as this command line shows:

"curl http://frax.ath.cx/cgi-bin/ret_400.cgi?bla_argument --trace-ascii dump -H Expect: -T README"

... so the given URL isn't a valid test case to prove the claims in this bug report.

[Daniel Stenberg]

No problem found, closing.