#780 The "Connection: close" header breaks NTLM authentication

version 3.0.10
closed-fixed
None
5
2009-10-04
2009-03-28
No

I am a programmer. I recently added NTLM authentication support to the
Chromium browser, and I'm working with a Privoxy 3.0.10 user on an NTLM
authentication problem to his company proxy.

Note: although I found old bugs on Privoxy's NTLM authentication problems:
https://sourceforge.net/tracker/index.php?func=detail&aid=620124&group_id=11118&atid=361118
https://sourceforge.net/tracker/index.php?func=detail&aid=1760018&group_id=11118&atid=211118
I have to open a new bug because I can't add a comment to those old bugs.

My investigation found pretty much what was reported in bug ID: 1760018,
that Privoxy adds the "Connection: close" header to the proxy's 407 response
that contains the proxy's NTLM challenge.

The Chromium browser respects the "Connection: close" header, so it
closes the old connection and opens a new one to send its response to
the NTLM challenge to the proxy. Since NTLM challenge-response must
proceed on a keep-alive connection, the authentication fails.

I made an experimental change to Chromium so that it would ignore the
"Connection: close" header that Privoxy added and leave the connection
open, and NTLM authentication to the proxy succeeded. This experiment
shows that when Privoxy adds the "Connection: close" header, it doesn't
mean it -- Privoxy won't close the connection if the browser leaves the
connection open.

I'd really like to find a way to fix this bug while respecting the HTTP
"Connection: close" header.

Can Privoxy detect that NTLM authentication is in progress and not
add the "Connection: close" header?

Alternatively, is there a configuration option to prevent Privoxy from
adding the "Connection: close" header?

Thank you for your help.

Discussion

1 2 > >> (Page 1 of 2)
  • Fabian Keil

    Fabian Keil - 2009-03-28
    • assigned_to: nobody --> fabiankeil
    • status: open --> pending
     
  • Fabian Keil

    Fabian Keil - 2009-03-28

    Privoxy 3.0.10 is out of date and more recent versions really
    "mean" the "Connection: close" header and will terminate the
    connection to the client no matter what the client does.

    That's because Privoxy 3.0.11 and later support
    "Connection: keep-alive" between Privoxy and the server,
    but not between the Privoxy and the client (yet).

    Supporting "Connection: keep-alive" between Privoxy and the
    client is on the TODO list, but from a performance point of view
    it's unlikely to matter (unless Privoxy and the client run on different
    hosts), so it hasn't been implemented yet.

    Once that's done, NTLM through Privoxy work as a side-effect and
    I think it's the only clean solution.

    While NTLM-only hacks could work as well, the code that would have
    to be modified is complex enough already so I don't think it's a good
    idea to add additional hacks unless it's really necessary.

    Given that one can work around the problem by putting a proxy
    like like cntlm between Privoxy and the company proxy I don't think
    that's the case here.

    I intend to implement client-side "Connection: keep-alive" support
    this year, but the priority for the next release is IPv6 support.

    Patches for client-side "Connection: keep-alive" support are
    welcome of course ...

     
  • Francois Botha

    Francois Botha - 2009-03-29

    Hi,

    I'm the user who reported the bug to the Chromium developer.

    With Privoxy 3.0.10, authentication failed totally. I upgraded to v3.0.11 and later to 3.0.12 and found that it worked better but still not as expected.

    For 3.0.10, authentication would fail completely and the authentication dialog would pop up over and over again.

    For 3.0.11 and 3.0.12, the request would pop up the dialog, I'd enter my details and the authentication would work. But some subsequent requests (not all of them) would not auto-authenticate after that and the popup would appear again.

    So typically, if a page has a lot of images, I'd have to enter my details about 10 times before all images and other resources are loaded. Some load fine though.

    Does it perhaps have something to do with the new timeout settings introduced in v3.0.11 ?

     
  • Fabian Keil

    Fabian Keil - 2009-03-29

    Not with the timeout option directly, but with the fact that
    connections between Privoxy and the server or proxy can
    be kept alive in the first place.

    I assume auto-authenticate works for requests for which
    already open connections are reused, while the authentication
    popup appears if Privoxy had to open a new connection for
    the request.

     
  • Wan-Teh Chang

    Wan-Teh Chang - 2009-03-31

    In addition to not adding a "Connection: close" header, Privoxy may
    also want to add a "Proxy-Support: Session-Based-Authentication"
    header, as specified in RFC 4559 Section 6, to indicate its support
    of NTLM/Negotiate authentication to the client.

    The Chromium browser doesn't support the Negotiate authentication
    scheme specified in RFC 4559 yet, so it's not checking for the
    "Proxy-Support: Session-Based-Authentication" header in a
    response received through a proxy. When we implement RFC 4559,
    we may need to check that header when a proxy is used.

     
  • Wan-Teh Chang

    Wan-Teh Chang - 2009-03-31
    • status: pending --> open
     
  • Nobody/Anonymous

    Be cautious with NTLM and the Negotiate for Kerberos because they are used to authenticate a TCP Connection. They do not authenticate individual HTTP requests, but rather the TCP connection over which HTTP request(s) are made.

    Once the process of NTLM authentication is initiated then the particular connections of the client and the server must be locked to each other by the proxy. Any HTTP request from the client connection must use the particular server connection exclusively. The server TCP connection involved in NTLM must not be pooled or used for anything initiated by other client TCP connections.

    Privoxy 3.0.11 introduced keep-alive of server connections for reuse which are not locked to the original client connection. It will cause unpredictable things to happen like failed, invalid, and error-condition authentications.

     
  • Fabian Keil

    Fabian Keil - 2009-04-02

    nobody, note that Privoxy currently doesn't advertise keep-alive
    support to the client and also closes the client connection after each
    request.

    Unless I'm mistaken, Chromium has been temporarily changed for
    debugging purposes to "answer" the challenge through a newly created
    connection to Privoxy. This could create problems in multi-user setups
    like you suggested, but normally browsers shouldn't do that.

    According to the Chromium bug tracker IE8b2 doesn't support
    NTLM through Privoxy 3.0.11 and later at all, which I consider
    the expected behavior.

    Once Privoxy gets complete keep-alive support, sharing outgoing connections
    between different clients will be off by default (if it will be supported at all).

     
  • Lee

    Lee - 2009-04-03

    > Once Privoxy gets complete keep-alive support, sharing outgoing
    > connections
    > between different clients will be off by default (if it will be supported
    > at all).

    Shared outgoing connections between different clients could be a real security problem. Please, don't support it at all.

    Thanks,
    Lee

     
  • Nobody/Anonymous

    Fabian, your point is clear about it not being for different users. But I was referring to the multiple connections from the same user and browser that are each in stages of authentication with NTLM tokens.

    Privoxy is valid to provide the 'close' to the client, which in effect denies NTLM exchanges. When the 'close' is not honored, for experimental investigation as was done with Chromium, then that is what opens the door to invalid assumptions. Wtchang could see occasional success doing that but it would not be consistent. That would be no fault of Privoxy.

     
  • Fabian Keil

    Fabian Keil - 2009-04-06

    This patch (against the CVS HEAD, apply with patch -p1)
    adds client-side connection keep-alive support:
    http://www.fabiankeil.de/sourcecode/privoxy/proxy-connection-keep-alive.diff
    http://www.fabiankeil.de/sourcecode/privoxy/proxy-connection-keep-alive.diff.asc

    It doesn't add the "Proxy-Support: Session-Based-Authentication" header
    because I didn't look up the exact meaning yet, but I think that NTLM
    authentication should work with it. I don't have a NTLM setup, so I didn't
    test that.

    The connections aren't shared (for now) and connections will only be kept
    alive if the client asked for it and only as long as the client keeps the connection
    to Privoxy open. Depending on the client the patch may cost a bit of performance,
    but for modern clients it may not matter.

     
  • Fabian Keil

    Fabian Keil - 2009-05-10

    An improved version of the patch has been committed to CVS.
    There's a fair chance that NTLM authentication will work now.
    Testers welcome.

     
  • Fabian Keil

    Fabian Keil - 2009-05-10
    • status: open --> pending
     
  • Nobody/Anonymous

    fabiankeil: thanks a lot for adding client-side connection keep-alive support.

    igitur: do you know how to check out the source code of Privoxy from CVS and build it?

     
  • Francois Botha

    Francois Botha - 2009-05-13

    I'm comfortable with CVS, but not with building. Are the build instructions somewhere? Otherwise, private message me with the details.

     
  • SourceForge Robot

    • status: pending --> closed
     
  • SourceForge Robot

    This Tracker item was closed automatically by the system. It was
    previously set to a Pending status, and the original submitter
    did not respond within 14 days (the time period specified by
    the administrator of this Tracker).

     
  • Fabian Keil

    Fabian Keil - 2009-05-25
    • status: closed --> pending
     
  • SourceForge Robot

    This Tracker item was closed automatically by the system. It was
    previously set to a Pending status, and the original submitter
    did not respond within 14 days (the time period specified by
    the administrator of this Tracker).

     
  • SourceForge Robot

    • status: pending --> closed
     
  • Fabian Keil

    Fabian Keil - 2009-06-15
    • status: closed --> pending
     
  • Francois Botha

    Francois Botha - 2009-06-17

    No, sorry, I still experience the problem in 3.0.13 beta. I get a proxy authentication dialog over and over.

     
1 2 > >> (Page 1 of 2)

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks