Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#851 URL with IPv6 address can't be accessed

version 3.0.19
pending
Fabian Keil
5
2014-07-06
2012-01-30
zooloo
No

My test set has eth0/1 interfaces configured on different IPv6 subnets
Test client uses curl to access IPv6 in different from it's own subnet trough proxy.

When trying to access URL like https:://[fc00:1:2::5:6:b]:8443/testsrv/index.php

All addresses and URLs - are my local internal use services, but it seems like problem is not related to exact service or privoxy configuration

[root@sh5 ~]# curl -v -g -k "https://[fc00:1:2::5:6:b]:8443/enterprise/control/agent.php"
* About to connect() to proxy 105.168.1.1 port 8008
* Trying 105.168.1.1... connected
* Connected to 105.168.1.1 (105.168.1.1) port 8008
* Establish HTTP proxy tunnel to fc00:1:2::5:6:b:8443
> CONNECT fc00:1:2::5:6:b:8443 HTTP/1.0
> Host: fc00:1:2::5:6:b:8443
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 404 No such domain
< Content-Length: 7377
< Content-Type: text/html
< Cache-Control: no-cache
< Date: Mon, 30 Jan 2012 11:06:06 GMT
< Last-Modified: Wed, 08 Jun 1955 12:00:00 GMT
< Expires: Sat, 17 Jun 2000 12:00:00 GMT
< Pragma: no-cache
< Connection: close
<
* Received HTTP code 404 from proxy after CONNECT
* Closing connection #0
curl: (56) Received HTTP code 404 from proxy after CONNECT

I'm using:

curl --version
curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

Discussion

  • zooloo
    zooloo
    2012-01-30

    privoxy config

     
    Attachments
  • zooloo
    zooloo
    2012-01-30

     
    Attachments
  • zooloo
    zooloo
    2012-01-30

    • labels: 414490 --> debian specific
     
  • zooloo
    zooloo
    2012-01-30

    Looks privoxy resolve IPv6 address like it was domain name. Anyway it looks weird.

     
  • zooloo
    zooloo
    2012-01-30

    Forgotten log file

     
    Attachments
  • Fabian Keil
    Fabian Keil
    2012-01-30

    • assigned_to: nobody --> fabiankeil
    • status: open --> pending
     
  • Fabian Keil
    Fabian Keil
    2012-01-30

    Thanks a lot for the report.

    I confirmed that Privoxy doesn't properly deal with the request, but with FEATURE_IPV6_SUPPORT the error message I get is "HTTP/1.1 503 Connect failed". This difference might be due to GNU libc vs. FreeBSD's libc.

    While testing this, I also found a related bug with CONNECT requests to ::1.

    Potential fixes (only briefly tested) for both issues are attached, please given them a try.

     
  • zooloo
    zooloo
    2012-01-31

    I don't know if this is a false alarm but seems curl is the one who makes troubles
    I've added bug to curl branch
    https://sourceforge.net/tracker/?func=detail&aid=3482093&group_id=976&atid=100976

    And I wander if there is need to make any fixes for case when CONNECT request is in wrong format (e.g. check for multiple ':' symbols in host name without any '[ ]' symbols in case of wrong IPv6 address)

     
  • zooloo
    zooloo
    2012-01-31

    • status: pending --> open
     
  • zooloo
    zooloo
    2012-01-31

    Turned into feature request

     
  • zooloo
    zooloo
    2012-01-31

    According to log of 'CONNECT' request from my Mozilla privoxy still has a problem with numeric IPv6 handling

    2012-01-31 21:44:06.105 b774a6c0 Info: Privoxy version 3.0.19
    2012-01-31 21:44:06.207 b774a6c0 Info: Program name: privoxy
    2012-01-31 21:44:06.207 b774a6c0 Info: Loading filter file: /etc/privoxy/default.filter
    2012-01-31 21:44:06.804 b774a6c0 Info: Loading actions file: /etc/privoxy/match-all.action
    2012-01-31 21:44:07.186 b774a6c0 Info: Loading actions file: /etc/privoxy/default.action
    2012-01-31 21:44:07.201 b774a6c0 Info: Loading actions file: /etc/privoxy/user.action
    2012-01-31 21:44:07.411 b774a6c0 Info: Listening on port 8008 on IP address 105.168.1.1
    2012-01-31 21:44:07.411 b774a6c0 Info: Listening on port 8008 on IP address fc00:1:2:2::2
    2012-01-31 21:44:07.411 b774a6c0 Info: Listening on port 8118 on IP address 10.27.145.104
    2012-01-31 21:45:44.325 b7749b90 Header: scan: CONNECT fc00:1:2::5:6:b:8443 HTTP/1.1
    2012-01-31 21:45:44.337 b7749b90 Header: scan: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.24) Gecko/20111109 CentOS/3.6.24-3.el6.centos Firefox/3.6.24
    2012-01-31 21:45:44.337 b7749b90 Header: scan: Proxy-Connection: keep-alive
    2012-01-31 21:45:44.337 b7749b90 Header: scan: Host: [fc00:1:2::5:6:b]:8443
    2012-01-31 21:45:44.337 b7749b90 Header: crumble crunched: Proxy-Connection: keep-alive!
    2012-01-31 21:45:44.337 b7749b90 Header: Adding: Connection: close
    2012-01-31 21:45:44.337 b7749b90 Request: fc00:1:2::5:6:b:8443/
    2012-01-31 21:45:45.032 b7749b90 Info: Can not resolve fc00: Name or service not known
    2012-01-31 21:45:45.035 b7749b90 Crunch: DNS failure: fc00:1:2::5:6:b:8443

     
  • Fabian Keil
    Fabian Keil
    2012-02-08

    • status: open --> pending
     
  • Fabian Keil
    Fabian Keil
    2012-02-08

    Did you try the patch I attached? To me it looks like you didn't.

    More recent Firefox versions don't seem to be affected anyway. At least my Firefox 10 on FreeBSD is using brackets around the IPv6 address: CONNECT [fc00:1:2::5:6:b]:8443 HTTP/1.1

     
  • zooloo
    zooloo
    2012-02-08

    Actually I did.
    According to log in my previous post Parser shows that it parses host well enough
    '2012-01-31 21:45:44.337 b7749b90 Header: scan: Host: [fc00:1:2::5:6:b]:8443'
    But still privoxy eager to connect to host fc00:1:2::5:6:b:8443...
    '2012-01-31 21:45:44.337 b7749b90 Request: fc00:1:2::5:6:b:8443/'

    Did I miss something from the patch?

     
  • zooloo
    zooloo
    2012-02-08

    • status: pending --> open
     
  • Fabian Keil
    Fabian Keil
    2012-02-08

    The Host header isn't that important here, as proxies are supposed to take the host from the request line.

    Also the patch should have no effect on the "scan:" lines as they are always taken from the client request as-is.

    The patch should, however, let Privoxy split "fc00:1:2::5:6:b:8443" (from the request line) at the last colon and from your log it looks like it may still be splitting at the first one instead.

    The patch seems to work for me:

    2012-02-08 15:13:08.972 801807400 Connect: Listening for new connections ...
    2012-02-08 15:13:08.973 801808800 Header: scan: CONNECT fc00:1:2::5:6:b:8443 HTTP/1.1
    2012-02-08 15:13:08.974 801808800 Header: Tagger 'client-ip-address' added tag 'IP-ADDRESS: 10.0.0.1'. No action bits update necessary.
    2012-02-08 15:13:08.974 801808800 Header: Tagger 'http-method' added tag 'CONNECT'. Action bits updated accordingly.
    2012-02-08 15:13:08.974 801808800 Header: scan: Host: [fc00:1:2::5:6:b]:8443
    2012-02-08 15:13:08.974 801808800 Header: Adding: Connection: close
    2012-02-08 15:13:08.974 801808800 Connect: Overriding forwarding settings based on 'forward .'
    2012-02-08 15:13:08.974 801808800 Request: fc00:1:2::5:6:b:8443/
    2012-02-08 15:13:08.974 801808800 Connect: to fc00:1:2::5:6:b:8443
    2012-02-08 15:13:08.974 801808800 Connect: Connected to fc00:1:2::5:6:b[fc00:1:2::5:6:b]:8443.
    2012-02-08 15:13:08.975 801808800 Connect: Created new connection to fc00:1:2::5:6:b:8443 on socket 5.
    2012-02-08 15:13:08.975 801808800 Connect: to fc00:1:2::5:6:b:8443 successful
    10.0.0.1 - - [08/Feb/2012:15:13:08 +0100] "CONNECT fc00:1:2::5:6:b:8443 HTTP/1.1" 200 0
    2012-02-08 15:13:08.975 801808800 Connect: The connection on server socket 5 to fc00:1:2::5:6:b isn't reusable. Closing. Server connection: keep-alive 0, tainted: 1, socket alive 1. Client connection: socket alive: 0. Server timeout: 300. Configuration file change detected: 0

    Can you please double-check that the patch has been properly applied?

    If you grep for strrchr, you should get two hits:

    fk@r500 ~/git/privoxy $grep strrchr *.c
    parsers.c: p = strrchr(header, ':');
    urlmatch.c: port = strrchr(host, ':');

    If you get the same result, please additionally enable "debug 2" and post a new log excerpt.

     
  • Fabian Keil
    Fabian Keil
    2012-02-08

    • status: open --> pending
     
  • zooloo
    zooloo
    2012-02-08

    Now I see that patch wasn't applied. Well now the initial request (https://[fc00:1:2::5:6:b]:8443/enterprise/control/agent.php) seems working.
    According to source... What will happen if URL like "https:://[fc00:1:2::5:6:b]/testsrv/index.php" being used.
    There is no ":port" signature and strrchr(header, ':') will find something wrong.

     
  • zooloo
    zooloo
    2012-02-08

    • status: pending --> open
     
  • Fabian Keil
    Fabian Keil
    2012-02-13

    • status: open --> pending
     
  • Fabian Keil
    Fabian Keil
    2012-02-13

    The URL "https://[fc00:1:2::5:6:b]/testsrv/index.php" doesn't contain any white-space before a colon so technically it shouldn't really matter which colon is found (or if none is found).

    Having said that, unlike a request header a request line is supposed to only contain single spaces, so it's probably preferable to let Privoxy reject invalid ones instead of trying to fix them up.

    Additionally I just realized that normalize_lws() isn't special-casing the colon because it might be separating a host from a port, but to convert headers like "name : value" into "name: value". The intention is unfortunately not clear from the code (yet another issue), but I have regression tests that fail when the check is removed:

    2012-02-13 20:20:25: Ooops. Got: 'No matching header' while expecting: 'X-LWS-Test: superfluous white space in the house'
    2012-02-13 20:20:25: Failure for test 733/771/54/155. Header 'X-LWS-Test : superfluous white space in the house' and tag 'LWS Tests'
    2012-02-13 20:20:25: Ooops. Got: 'No matching header' while expecting: 'X-LWS-Test: superfluous white tabs in the house'
    2012-02-13 20:20:25: Failure for test 737/775/54/159. Header 'X-LWS-Test : superfluous white tabs in the house' and tag 'LWS Tests'

    As no white-space is allowed between the header name and the colon, trying to fix those invalid headers (now) seems questionable to me as well.

    A better fix might be to remove the whole block without replacement and to make the header parser more strict instead.

     
  • Fabian Keil
    Fabian Keil
    2014-07-06

    This was fixed before the 3.0.21 release.