#1494 Cookies Not Following Path Rules

[Steve Corriere]

I do not believe that Curl is correctly following the path rules for cookies based on http://curl.haxx.se/rfc/cookie_spec.html. Specifically, I do not think that it is following:
"The path "/foo" would match "/foobar" and "/foo/bar.html". The path "/" is the most general path."

In the following example, I would expect that we send the "partial_path" cookie in all requests. Note that for the request to "http://www.example.com/foobar," we are missing this cookie, but we have the cookie for "/foo/bar.html."

I have seen the same behavior in version curl-7.37.0. A really old data point, but I believe that the behavior followed the cookie spec back in curl 7.19.7.

If curl is not supposed to be following the cookie_spec page, my apologies.

[scorriere@outrage src]$ cat /tmp/broken_cookie 
# Netscape HTTP Cookie File
# http://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.

.example.com    TRUE    /foo/bar.html   FALSE   2426364040      full_path 1
.example.com    TRUE    /foo            FALSE   2426364040      partial_path 1
.example.com    TRUE    /               FALSE   2426364040      general_path 1
[scorriere@outrage src]$ ./curl -vvv -o /dev/null -b /tmp/broken_cookie "http://www.example.com/foo"
*   Trying 93.184.216.34...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to www.example.com (93.184.216.34) port 80 (#0)
> GET /foo HTTP/1.1
> User-Agent: curl/7.41.0
> Host: www.example.com
> Accept: */*
> Cookie: partial_path 1=; general_path 1=
> 
< HTTP/1.1 404 Not Found
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/html
< Date: Fri, 13 Mar 2015 12:27:53 GMT
< Etag: "359670651"
< Expires: Fri, 20 Mar 2015 12:27:53 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (ewr/144C)
< X-Cache: 404-HIT
< x-ec-custom-error: 1
< Content-Length: 1270
< 
{ [1270 bytes data]
100  1270  100  1270    0     0  78696      0 --:--:-- --:--:-- --:--:-- 84666
* Connection #0 to host www.example.com left intact
[scorriere@outrage src]$ ./curl -vvv -o /dev/null -b /tmp/broken_cookie "http://www.example.com/foobar"
*   Trying 93.184.216.34...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to www.example.com (93.184.216.34) port 80 (#0)
> GET /foobar HTTP/1.1
> User-Agent: curl/7.41.0
> Host: www.example.com
> Accept: */*
> Cookie: general_path 1=
> 
< HTTP/1.1 404 Not Found
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/html
< Date: Fri, 13 Mar 2015 12:27:59 GMT
< Etag: "359670651"
< Expires: Fri, 20 Mar 2015 12:27:59 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (ewr/15AB)
< X-Cache: 404-HIT
< x-ec-custom-error: 1
< Content-Length: 1270
< 
{ [1270 bytes data]
100  1270  100  1270    0     0  78657      0 --:--:-- --:--:-- --:--:-- 84666
* Connection #0 to host www.example.com left intact
[scorriere@outrage src]$ ./curl -vvv -o /dev/null -b /tmp/broken_cookie "http://www.example.com/foo/bar.html"
*   Trying 93.184.216.34...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to www.example.com (93.184.216.34) port 80 (#0)
> GET /foo/bar.html HTTP/1.1
> User-Agent: curl/7.41.0
> Host: www.example.com
> Accept: */*
> Cookie: full_path 1=; partial_path 1=; general_path 1=
> 
< HTTP/1.1 404 Not Found
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/html
< Date: Fri, 13 Mar 2015 12:28:03 GMT
< Etag: "359670651"
< Expires: Fri, 20 Mar 2015 12:28:03 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (ewr/15CC)
< X-Cache: 404-HIT
< x-ec-custom-error: 1
< Content-Length: 1270
< 
{ [1270 bytes data]
100  1270  100  1270    0     0  80242      0 --:--:-- --:--:-- --:--:-- 84666
* Connection #0 to host www.example.com left intact
[scorriere@outrage src]$

OS Version
cat /etc/redhat-release
CentOS Linux release 6.0 (Final)
[scorriere@outrage src]$ uname -a
Linux outrage.verivue.com 2.6.32-71.el6.x86_64 #1 SMP Fri May 20 03:51:51 BST 2011 x86_64 x86_64 x86_64 GNU/Linux

Curl Version
./curl --version
curl 7.41.0 (x86_64-unknown-linux-gnu) libcurl/7.41.0 OpenSSL/1.0.2 zlib/1.2.5 libidn/1.18 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

[Daniel Stenberg]

Cookies are defined in RFC 6265 since a while back:

http://tools.ietf.org/html/rfc6265

I believe curl acts correctly, but I've been wrong before...

[Steve Corriere]

Don't worry, I am wrong consistently. I changed the cookie around to have better syntax (Tabs between the fields). I am still seeing the same behavior, however. Based on http://tools.ietf.org/html/rfc6265#section-5.1.4, the partial path should have a '/' suffix. I have tried with this, but I don't believe that it changed anything.

cat /tmp/broken_cookie
# Netscape HTTP Cookie File
# http://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.

.example.com    TRUE    /foo/bar.html   FALSE   2426364040      full_path   1=true
.example.com    TRUE    /foo/           FALSE   2426364040      partial_path1   1=true
.example.com    TRUE    /foo            FALSE   2426364040      partial_path2   1=true
.example.com    TRUE    /               FALSE   2426364040      general_path    1=true
[scorriere@outrage src]$ ./curl -vvv -o /dev/null -b /tmp/broken_cookie "http://www.example.com/foobar"
*   Trying 93.184.216.34...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to www.example.com (93.184.216.34) port 80 (#0)
> GET /foobar HTTP/1.1
> User-Agent: curl/7.41.0
> Host: www.example.com
> Accept: */*
> Cookie: general_path    1=true=
>
< HTTP/1.1 404 Not Found
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/html
< Date: Fri, 13 Mar 2015 15:10:31 GMT
< Etag: "359670651"
< Expires: Fri, 20 Mar 2015 15:10:31 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (ewr/15AB)
< X-Cache: 404-HIT
< x-ec-custom-error: 1
< Content-Length: 1270
<
{ [1270 bytes data]
100  1270  100  1270    0     0  81020      0 --:--:-- --:--:-- --:--:-- 84666
* Connection #0 to host www.example.com left intact

[Daniel Stenberg]

Can you show us a full recipe where curl does wrong against a site where for example a browser works differently?

I believe the change you object to is this: https://github.com/bagder/curl/commit/04f52e9b4db01bcbf672

[Steve Corriere]

Alright, so I crafted some cookies in Firefox that I believe are acting like they did back in curl 7.19.7.

sqlite> select * from moz_cookies where baseDomain="example.com";
5|example.com|0|0|fullpath|1=1|.example.com|/foo/bar.html|1426596548|1426510303124730|1426510184149128|0|1
6|example.com|0|0|partialpath1|1=1|.example.com|/foo|1426596621|1426510303124730|1426510239285097|0|1
7|example.com|0|0|general|1=1|.example.com|/|1426596646|1426510262173271|1426510262173271|0|1

When I made a request for "http://example.com/foo/bar,html," Firefox sent all 3 cookies (See attached screenshot from Firefox developer tools).

If you think that I am wrong, feel free to close this out. My reason behind opening this was based on the link to http://curl.haxx.se/rfc/cookie_spec.html. If curl should not be following that, I am completely okay with that. Perhaps Firefox is still based off of that old "spec."

[Daniel Stenberg]

Thanks,

The cookie_spec is there for historical purposes to teach us where we come from, Firefox and curl both should adhere to the cookie RFC.

I'm still not happy with peeking into nor relying on what is stored locally about the headers but more exactly what the behavior is regards to cookies sent from a server and then sent back by curl. Can you show us a HTTP header sequence of what to receive that then leads to a different request (cookie wise) sent back when comparing curl vs for example Firefox ?

[Daniel Stenberg]

I'm attempting to repeat what you say here:

http://daniel.haxx.se/test/foo/

But I cannot make Firefox send cookies on partial path names. What am I doing wrong?

[Steve Corriere]

So I now think that curl is behaving the same as Firefox when I set up a server to send the cookies. I am rather confused as to what I have done differently, but I need to take a peek into that.

curl -vvv -o /dev/null -b /tmp/cookiejar  "http://127.0.0.1/cookie/cookie.html"
*   Trying 127.0.0.1...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET /cookie/cookie.html HTTP/1.1
> User-Agent: curl/7.41.0
> Host: 127.0.0.1
> Accept: */*
> Cookie: full=1; general=1
>
< HTTP/1.1 200 OK
< Server: nginx/1.0.15
< Date: Mon, 16 Mar 2015 19:20:28 GMT
< Content-Type: text/plain
< Content-Length: 20
< Last-Modified: Mon, 16 Mar 2015 19:19:46 GMT
< Connection: keep-alive
* Replaced cookie general="1" for domain 127.0.0.1, path /, expire 1458069628
< Set-Cookie: general=1;Domain=127.0.0.1;Path=/;Max-Age=31536000
* Replaced cookie partial="1" for domain 127.0.0.1, path /coo, expire 1458069628
< Set-Cookie: partial=1;Domain=127.0.0.1;Path=/coo;Max-Age=31536000
* Replaced cookie full="1" for domain 127.0.0.1, path /cookie/cookie.html, expire 1458069628
< Set-Cookie: full=1;Domain=127.0.0.1;Path=/cookie/cookie.html;Max-Age=31536000
< Accept-Ranges: bytes
<
{ [20 bytes data]
100    20  100    20    0     0  37878      0 --:--:-- --:--:-- --:--:-- 20000
* Connection #0 to host 127.0.0.1 left intact

[Daniel Stenberg]

Marked as pending.

[Daniel Stenberg]

I'm closing this as we haven't been able to show an actual bug. Please feel free to open a new bug if you can spot an actual problem.