Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#79 Incorrect proxy env parsing

closed-fixed
9
2008-02-06
2007-07-27
Björn Voigt
No

mount.davfs looks for the environment variable "http_proxy" (davfs2-1.2.2/src/mount_davfs.c:1687).

Unfortunately the parsing of this proxy variable is too simple, inconsistent and therefore incorrect.

Proxy variables are defined like this (taken from curl-7.16.1/lib/uri.c):
* http_proxy=http://some.server.dom:port/
* https_proxy=http://some.server.dom:port/
* ftp_proxy=http://some.server.dom:port/
* gopher_proxy=http://some.server.dom:port/
* no_proxy=domain1.dom,host.domain2.dom
* (a comma-separated list of hosts which should
* not be proxied, or an asterisk to override
* all proxy variables)
* all_proxy=http://some.server.dom:port/
* (seems to exist for the CERN www lib. Probably
* the first to check for.)

mount.davfs only knows one incompatible format:
* http_proxy=some.server.dom:port

If the "http_proxy" variable starts with "http://" it tries to connect to server "http".

Idea:
- neon has a library function "ne_uri_parse(const char *uri, ne_uri *parsed)"
- this function could be used to parse the proxy environment variables
- additional logic could be found in "wget", "curl" and other software, which uses the proxy variables

Discussion

  • Werner Baumann
    Werner Baumann
    2007-07-28

    Logged In: YES
    user_id=1260327
    Originator: NO

    Hello Björn,

    I didn't even think of a client-proxy-protocol other than HTTP. So at first I wondered what http_proxy=*http://*some.server.dom:port/ would be good for. Neon only accepts proxy-host and proxy-port and probably only http as client-proxy-protocol (there is no way to configure something like a proxy-scheme).

    I added function proxy_from_env in mount_davfs.c. It only looks for https_proxy (if the server scheme is https), http_proxy and all_proxy, and only accepts client-proxy-protocol http, or no scheme. Unfortunately ne_uri_parse fails to parse uris like "server:port", so it became a little bit ugly.

    Support for no_proxy will be added later.

    Please have a look at the CVS repository and maybe do some testing.

    I did not yet look at wget, curl and others. Do you know whether the xxx_proxy environment variables are really *defined* somewhere, or is it just common usage?

    Cheers
    Werner

     
  • Werner Baumann
    Werner Baumann
    2007-07-28

    • assigned_to: nobody --> wbaumann
    • status: open --> open-accepted
     
  • Werner Baumann
    Werner Baumann
    2007-07-28

    Logged In: YES
    user_id=1260327
    Originator: NO

    Support for environment variable no_proxy is added in CVS.

     
  • Werner Baumann
    Werner Baumann
    2007-07-28

    • status: open-accepted --> open-fixed
     
  • Björn Voigt
    Björn Voigt
    2007-07-29

    Patch fixes some little issues (see description)

     
  • Björn Voigt
    Björn Voigt
    2007-07-29

    Logged In: YES
    user_id=41637
    Originator: YES

    Thanks for adding a complete proxy env support. I looked at the new CVS proxy env code and tested it a bit.

    There are some little things which are still incorrect as far as I know (looked at curl, wget):

    1) no_proxy="*" could be used to ignore proxies completely
    2) the form http_proxy="*.some.domain" is not supported in common tools (wget, lynx, ...). Instead http_proxy=".some.domain" should be used to ignore hosts ending with ".some.domain" e.g. "host1.some.domain"
    3) the no_proxy variable can contain port numbers, e.g. no_proxy="www.some.domain:80"
    4) spaces are allowed around ","

    I think, the form http_proxy="host.some.domain:80" (without http://\) should not be used. Should we write out a warning (see my patch)?

    I fixed all the above issues with my attached patch. I tested the patch but feel free to test it again and maybe change it.
    File Added: davfs2_proxy_env.patch

     
  • Werner Baumann
    Werner Baumann
    2007-07-29

    Logged In: YES
    user_id=1260327
    Originator: NO

    Thanks for the patch.

    I included *most* of it in CVS:
    - '*'
    - portnumbers
    - ".some.domain"
    - spaces in the list

    I do not like the idea of allowing partial match in domain-name-components, like "oo.bar" matching "foo.bar". This will not work, if both are valid domain names, and proxying shall only be disabled for "oo.bar". So I believe it's a bug in lynx. I also assume that matches of this kind will usually be caused by typing errors and rarely by intention.

    I also did not deprecate the form "host:port". I looked around, and could not find anything that looks like a standard. According to the curl man page, the scheme is otional. Most browsers don't need or even understand this form (but I rarely see something else but my old iceape). As for many users proxies like squid are the only kind of proxy they are concerned with, I can't see any harm in allowing this form and defaulting to "http:".
    But I will not hesitate to deprecate this form, if there is any standard or at least a published agreement between major free software groups.

    Cheers
    Werner

     
  • Björn Voigt
    Björn Voigt
    2007-07-29

    Logged In: YES
    user_id=41637
    Originator: YES

    I also did not found a written standard document for the proxy variables. But many console programs like web browsers (lynx, links, w3m, KDE konqueror, ...), tools (wget, (lib)curl) and libraries (Perl libwww, ...) use them.

    The most detailed description I found in Lynx documentation:
    http://lynx.isc.org/current/lynx2-8-7/lynx_help/keystrokes/environments.html

    I think the form no_proxy="host.domain.dom, domain1.dom, domain2" (from the Lynx documentation) should be supported. If course this value also filters the host "xdomain1.dom". This could be considered as a bug. But we should "davfs" fix this "long-living bug"?

     
  • Werner Baumann
    Werner Baumann
    2007-07-29

    Logged In: YES
    user_id=1260327
    Originator: NO

    > But we should "davfs" fix this "long-living bug"?

    davfs2 will not fix this bug in lynx. But as it really is bad, if you cannot disable proxying for foo.bar without disabling proxying for completely-different-foo.bar and antifoo.bar, davfs2 now does this:
    - ".foo.bar" matches all subdomains of foo.bar (and of course sub-sub-sub-..-domains).
    - "foo.bar" only matches "foo.bar".
    I did not proof it, but I assume, not all applications have this lynx-bug.

    Standards:
    It is necessary for davfs2 to support xxx-proxy values with scheme. But additionally supporting values without scheme and defaulting to http, does not interfere with anything and does not break any other application and may be convenient for many users. It would be different, if there would be a standard (maybe POSIX), that deprecates this form. Curious enough: the only proxy-protocol in all the examples in the lynx documentation is http.

    Cheers
    Werner

    P.S.: Maybe I can change the code so that "foo.bar" matches "foo.bar" and subdomains of "foo.bar", but not other domains, that end in "foo.bar", like "antifoo.bar". But this will take some time, as there are other things I should do within the next four weeks.

     
  • Werner Baumann
    Werner Baumann
    2007-07-29

    Logged In: YES
    user_id=1260327
    Originator: NO

    Sorry,

    what I wrote in P.S. is nonsense. The user would not be able to disable proxying for one host, without disabling it for all subdomains. Nearly as bad as the lynx-bug.

     
  • Björn Voigt
    Björn Voigt
    2007-07-29

    Logged In: YES
    user_id=41637
    Originator: YES

    For very special network setups (especially if a special host should be connected through proxy, but all other hosts should be contacted directly) it's still possible to write hacks like this:

    (env) http_proxy="http://my.proxy:8080/" no_proxy="" mount -t davfs ...

    But in most cases, only the local DNS domain and maybe some IP adresses (subdomains, e.g. 192.168.0 do not work here) and short names ("localhost") are used for no_proxy. Example: no_proxy="127.0.0.1, localhost, (.?)my.local.network"

    I see some special problems, if davfs is used with the "noauto,user" options in /etc/fstab. The user can fake a "special" proxy and thus can mount all hosts not only the intended ones in /etc/fstab. Also, if the filesystem is mounted automatically on boot ("auto" in /etc/fstab) there may no proxy env variables be present).

     
  • Werner Baumann
    Werner Baumann
    2007-07-30

    Logged In: YES
    user_id=1260327
    Originator: NO

    For complicated proxy configurations, there is a much simpler solution: use the 'proxy' option in the configuration file. It can be set differently for every mount point. proxy settings in the configuration file will have precedence over the environment variables anyway.

    > I see some special problems, if davfs is used with the "noauto,user"
    > options in /etc/fstab. The user can fake a "special" proxy and thus can
    > mount all hosts not only the intended ones in /etc/fstab.
    This indeed is a bug, a security hole. Users may use the proxy settings to circumvent restrictions set by the administrator. The davfs-aproach (using fstab) is, to give the administrator full control over what resources she will allow to mount.
    When a nonpriveleged user mounts, mount.davfs starts in the environment of this user, but with root privileges (setuid). As the environment is controled by the user (this includes xxx_proxy=), davfs2 must not use environment variables that might affect security. There is also a 'proxy'-option in the user configuration file, that could be misused this way.
    So what I have to do:
    - disable the 'proxy'-option in the user configuration file
    - not use xxx_proxy environment variables, if the mounting user is not root.

    > Also, if the filesystem is mounted automatically on boot ("auto" in /etc/fstab)
    > there may no proxy env variables be present).
    I don't see a problem here. When no environment variables are available, the proxy must be set in the system wide configuration file. This file *must* be available, even when the davfs filesystem is mounted at boot time. Note: the davfs filesystem is not suited to hold essential system files (some people try, but davfs2 is not a replacement for a full featured network wile system). So /etc, /usr and /var will be available when a davfs2 file system is mounted.

    Cheers
    Werner

     
  • Werner Baumann
    Werner Baumann
    2007-07-30

    • priority: 5 --> 9
    • status: open-fixed --> open-accepted
     
  • Werner Baumann
    Werner Baumann
    2008-02-06

    • status: open-accepted --> closed-fixed