#560 socks5 message / error passthrough

pending
Fabian Keil
None
5
2014-02-02
2013-02-09
grarpamp
No

Privoxy 3.0.19 on FreeBSD 8.3 i386

With polipo I'd see helpful diagnostic messages from Tor's socks5 implementation
passed through polipo and eventually reaching wget's log file. Though I'm not sure
if it was the exact socks5 error code number, the text was still useful. Example...

1300 504 Connect to onion failed: SOCKS error: host unreachable.
80 504 Connect to onion failed: SOCKS error: TTL expired.
30 504 Connect to onion failed: SOCKS connection not allowed.
15 504 Connect to onion failed: General SOCKS server failure.
10 504 Connect to onion failed: SOCKS error: connection refused.

With Privoxy, I get vague 500 level messages that don't really tell me anything
about the far end. (I don't have relative counts for this example right now).

ERROR 500: Internal Privoxy Error.
ERROR 500: Internal Server Error.
ERROR 502: Server or forwarder response invalid.
ERROR 503: Service unavailable.
ERROR 504: Gateway Time-out.
Proxy tunneling failed: Internal Privoxy ErrorUnable to establish SSL connection.

So I'm hereby suggesting that...
1) more detailed messages be passed through wherever available from the far end.
2) these 'internal' errors receive more verbose or coded treatment as to what function
is kicking them out. (that's probably the bug-ish part of this feature-ish report).

Thx.

Discussion

1 2 > >> (Page 1 of 2)
  • Fabian Keil
    Fabian Keil
    2013-02-09

    Adding the socks error description to the response line seems useful to me. Thanks for the suggestion.

    Note that Privoxy already provides the details as part of the response body and properly logs them when configured to do so.

    Internal Privoxy Errors usually aren't bugs but configuration problems and I don't think discarding the problem description in the body and only looking at the response line is a reasonable debugging strategy, even when doing it from the "far end".

    BTW, at least some of your example error messages don't actually come from Privoxy and checking Privoxy's log would make their origin more obvious. Unfortunately correlation socks errors to Tor circuits currently isn't trivial.

     
  • Fabian Keil
    Fabian Keil
    2013-02-09

    • assigned_to: nobody --> fabiankeil
    • status: open --> pending
     
  • grarpamp
    grarpamp
    2013-02-09

    • status: pending --> open
     
  • grarpamp
    grarpamp
    2013-02-09

    > Adding the socks error description to the response line seems useful to me.

    Ok, cool. Section 6 would be apropriate: https://tools.ietf.org/html/rfc1928

    > BTW, at least some of your example error messages don't actually come from Privoxy

    Ahh, so maybe there is ability to indicate in the status response header from privoxy to
    the client (wget), in general, if it was privoxy that generated the error. A formatted
    privoxy: label perhaps.

    > Note that Privoxy already provides the details as part of the response body
    > I don't think discarding the problem description in the body ... [is] reasonable

    Hmm, I think wget deems any such reply body received along with an error header
    as erroneous to the intended body and thus does not save them to disk. I will look
    at this and possibly ticket or replace wget. Thanks.

    > Privoxy already provides the details ... and properly logs them when configured.
    > checking Privoxy's log would make their origin more obvious.
    > correlation socks errors to Tor circuits currently isn't trivial.

    Yes, for reproducing the error as well. And matching up thousands of
    lines in wget, privoxy, and Tor logs, from parallel fetches, is tough too.
    Divide and conquer may work here for whatever is not done by this ticket.
    In the meantime I'll bump 'debug' to see what privoxy currently thinks
    about 'privoxy <--> Tor' regarding socks.

    wget <--> privoxy <--> Tor <--> webserver
    'far end' - I mean reply from the webserver (the far end) back through to wget.

    Thanks.

     
  • grarpamp
    grarpamp
    2013-02-10

    Did more work on this...

    https://tools.ietf.org/html/rfc2616

    Per HTTP rfc2616 sec:10.5, user agents SHOULD display any entity...
    ok, displayed - firefox, curl, elinks, lynx, telnet, netcat
    broken, not displayed - wget, fetch, socat[!]

    This entity, 'forwarding-failed', cannot know any more
    info than the socks5 rfc1928 sec:6 reply itself.
    We can pick an HTTP rfc2616 sec:10.5 error code to ride on top of.

    Unusuitable since the error is already specifically defined...
    502 Bad Gateway - invalid response from webserver (or auxiliary: Tor)
    504 Gateway Timeout - timeout on webserver (or auxiliary: Tor)

    The most sensible HTTP error code seems to be...
    500 Internal Server Error - unexpected condition prevented fulfillment

    Privoxy can pass that through from webserver when not using socks,
    or when the socks5 reply code is 00h.

    And when using socks5, if reply code of 01h through FFh occur,
    Privoxy could then reissue the 500 in that case as...

    500 Internal Server Error - Privoxy: SOCKS<ver>: <nn> <msg>

    Where <ver> is socks version. If version 5, <nn> is 01~FF, and
    <msg> is as shown in socks5 rfc1928 or highly abbreviated.

    socks4 and socks4a follow similarly.

    There is merit in optionally including the attempted URL in the
    'forwarding-failed' entity. I removed that file, so it emits this
    header instead: 500 Internal Privoxy Error

    Privoxy should probably not do that, or have a 'support fileless
    normally' mode. That is for another ticket.

     
  • Fabian Keil
    Fabian Keil
    2013-02-11

    In my opinion using status code 500 for socks errors would be worse than 502 or 504, even though I agree that they don't fit perfectly either.

    The templates aren't considered optional and deleting them is supposed to result in error messages.

    You are welcome to empty or edit the templates, though.

    In my opinion socks4(a) improvements would be a waste of time given that all the relevant gateways support the mostly-superior socks5(t).

    I don't see the point of mentioning Privoxy in all its response lines. It's already mentioned in some, though.

    BTW, you can already do the opposite and use a server-header filter to mark error-indicating response lines that aren't coming from Privoxy.

    I think the correlation issues can only be solved with another Tor-specific socks extension but haven't had time to write a proposal or implement it yet.

     
  • Fabian Keil
    Fabian Keil
    2013-02-11

    • status: open --> pending
     
  • grarpamp
    grarpamp
    2013-02-14

    > In my opinion using status code 500 for socks errors would be
    > worse than 502 or 504, even though I agree that they don't fit
    > perfectly either.

    502 and 504 were the only ones that defined any 'gateway' context.
    For socks, privoxy currently utilizes 503 which is not defined to
    be a gateway message, and is defined for a fairly specific problem,
    just like 502 and 504 are.

    503 Service Unavailable
    The server is currently unable to handle the request due to a
    temporary overloading or maintenance of the server.

    So I figured 500 was a generic enough place to append the specific
    socks codes. 503 is reasonable if considering its short definition
    'Service Unavailable'. Choose whatever works. However I would put
    all the SOCKS reply codes/msgs under the same extended HTTP code.

    With temporary templates, here's wget's vision of 8000 fetches...
    8 ERROR 502: No data received from server or forwarder.
    12 ERROR 502: Server or forwarder response invalid.
    3421 ERROR 503: Forwarding failure.
    1 ERROR 503: Service Temporarily Unavailable.
    1 ERROR 503: Service Unavailable.
    3385 Proxy tunneling failed: Forwarding failureUnable to establish SSL connection.

    > The templates

    I'm doing template handling in a separate ticket when I'm able to
    present it.

    > In my opinion socks4(a) improvements would be a waste of time
    > given that all the relevant gateways support the mostly-superior
    > socks5(t).

    I have no use case for socks4/4a either and wish people would not
    deploy those servers. I just mentioned it to be complete :) Though
    if 4/4a are still in Privoxy, it's probably better to fix them up
    too.

    > I don't see the point of mentioning Privoxy in all its response
    > lines. It's already mentioned in some, though.

    Not for all lines, only lines where it would help distinguish who
    is actually emitting the error. Another example, chained proxies.
    Who is to know if the local privoxy got the error, or was passed
    it from farther away.

    Though for the socks error case, if the literal 'Privoxy: ' were
    trimmed out, the strict format of the remaining line would give a
    clue to whoever looks up that string in the Privoxy docs.

    Whichever way, at least the 'SOCKS<ver>: <nn> <msg>' needs added.

    The '<ver>' could be trimmed because the socks4/4a/5 '<nn> <msg>'
    codes do not overlap.

    > BTW, you can already do the opposite and use a server-header
    > filter to mark error-indicating response lines that aren't coming
    > from Privoxy.

    In some projects I wish to preserve the original headers from the
    far side. In others I need to translate. Privoxy is amazing to do
    that.

    > I think the correlation issues can only be solved with another
    > Tor-specific socks extension but haven't had time to write a
    > proposal or implement it

    Too, if I serialized things for debugging I could line up multiple
    logs by time or content, but serial just isn't a production option.
    I do some testing by hand as needed. Polipo locks up under heavy
    parallel use, and seems a dead project, so I switched back to Privoxy
    for everything :)

     
  • grarpamp
    grarpamp
    2013-02-14

    • status: pending --> open
     
  • grarpamp
    grarpamp
    2014-02-01

    Would like to see these concepts patched for testing and committed :) After a reread are there any remaining questions/discuss needed for that? Thx :)

     
1 2 > >> (Page 1 of 2)