From: Jochen H. <Joc...@Ha...> - 2007-03-08 19:17:31
|
A while ago I started experiments with socks5 (-> dante) and created a /etc/socks.conf on my local host and I set up a socks5 proxy server on a neighbour box. I did not instruct my local fetchmail to make use of my socks5 proxy and I also wasn't aware, that my local fetchmail actually does us it, but in a situation, when that proxy server on the neighbour box temporarily did not run, I finally noticed (using "strace"), that fetchmail contacted my socks5 proxy. I did some rtfm-ing, but I did not find a way to instruct fetchmail to *not* attempt contacting my socks5 proxy. I had to get rid of my local /etc/socks.conf , so that fetchmail talked straight to the IMAP instead of through the socks5 proxy. I would actually prefer a command line option and an entry in the RC file. Is there already a feature of fetchmail, that this could be made similar to? Does the community have any comments on this? J. |
From: Rob M. <rob...@gm...> - 2007-03-08 21:30:56
|
On 3/8/07, Jochen Hayek <Joc...@ha...> wrote: > A while ago I started experiments with socks5 (-> dante) > and created a /etc/socks.conf on my local host and I set up a socks5 proxy server on a neighbour box. > > I did not instruct my local fetchmail to make use of my socks5 proxy > and I also wasn't aware, that my local fetchmail actually does us it, > but in a situation, when that proxy server on the neighbour box temporarily did not run, > I finally noticed (using "strace"), that fetchmail contacted my socks5 proxy. > > > I did some rtfm-ing, > but I did not find a way to instruct fetchmail to *not* attempt contacting my socks5 proxy. Fetchmail has *no* native socks support. You're either running it with a socks wrapper (runsock/socksify) or have added this yourself (eg via ld.so.preload). It's hard to say though as you've provided no real information (version numbers, contents of configuration files, command lines used, anything really). I'd suggest you consult the dante documentation or mailing list, for help on how to configure dante :) -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche |
From: <Joc...@Ha...> - 2007-03-15 14:40:53
|
>>>>> "RMG" == Rob MacGregor writes: On 3/8/07, Jochen Hayek wrote: >> A while ago I started experiments with socks5 (-> dante) >> and created a /etc/socks.conf on my local host and I set up a socks5 proxy server on a neighbour box. >> >> I did not instruct my local fetchmail to make use of my socks5 proxy >> and I also wasn't aware, that my local fetchmail actually does us it, >> but in a situation, when that proxy server on the neighbour box temporarily did not run, >> I finally noticed (using "strace"), that fetchmail contacted my socks5 proxy. >> >> >> I did some rtfm-ing, >> but I did not find a way to instruct fetchmail to *not* attempt contacting my socks5 proxy. RMG> Fetchmail has *no* native socks support. Currently I am convinced, your statement is incorrect. But I rather have some sympathy for your suspicion, as this magical and undescribed feature of fetchmail is certainly rather weird, and fetchmail shouldn't behave like that. Have you ever run fetchmail in a socks5 ready environment? Give yourself a try and let's continue discussing the matter than! RMG> You're either running it RMG> with a socks wrapper (runsock/socksify) RMG> or have added this yourself (eg via ld.so.preload). No, I don't, nothing like that. (No socks wrapper, no LD_PRELOAD, no /etc/ld.so.preload, no ...) I told you above: "I did not instruct my local fetchmail to make use of my socks5 proxy" I mean, what I write. And you read my statement, didn't you?!? RMG> It's hard to say though as you've provided no RMG> real information (version numbers, contents of configuration files, RMG> command lines used, anything really). Alright, alright, yet another doubting Thomas ;-) Can you see "SOCKS" here: [2007-03-15 14:11:49] johayek@HayekJ $ fetchmail --version This is fetchmail release 6.3.2+POP2+IMAP-GSS+RPA+NTLM+SDPS+SSL+OPIE+SOCKS+NLS. Copyright (C) 2002, 2003 Eric S. Raymond Copyright (C) 2004 Matthias Andree, Eric S. Raymond, Rob F. Funk, Graham Wilson Copyright (C) 2005 Matthias Andree, Sunil Shetye Copyright (C) 2006 Matthias Andree Fetchmail comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. For details, please see the file COPYING in the source or documentation directory. Fallback MDA: (none) Linux HayekJ 2.6.16.21-0.13-default #1 Mon Jul 17 17:22:44 UTC 2006 i686 i686 i386 GNU/Linux Taking options from command line and /home/jochen_hayek/.fetchmailrc Idfile is /home/jochen_hayek/.fetchids Fetchmail will show progress dots even in logfiles. Fetchmail will forward misaddressed multidrop messages to johayek. Fetchmail will direct error mail to the postmaster. close(3) = 0 [...] open("/etc/socks.conf", O_RDONLY) = 4 [...] And here we have some config file extract: skip shuttle.de via "blablabla.shuttle.de" with proto IMAP and no dns tracepolls user "jh9999" there with password "PASSWORD" is johayek here options ssl sslproto tls1 sslfingerprint "D9:83:30:EA:3B:A5:02:A2:D6:72:1D:B7:AB:C3:30:CB" fetchall stripcr dropstatus dropdelivered warnings 3600 expunge 20 folders imap/folder-misc/_to-be-downloaded,imap/folder/prio-4 properties "'mailbox_from':'INBOX','mailbox_dir':'imap','mailbox_dir_separator':'/'" antispam 571 no rewrite pass8bits And here we have some "strace" output: [2007-03-15 13:30:59] johayek@HayekJ $ strace fetchmail shuttle.de execve("/usr/bin/fetchmail", ["fetchmail", "shuttle.de"], [/* 112 vars */]) = 0 brk(0) = 0x808d000 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fb3000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/usr/local/lib/tls/i686/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/tls/i686/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/tls/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/tls/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/i686/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/i686/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/lib/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/i686/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/i686/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/i686/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/i686/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/sse2/libsocks.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/libsocks.so.0", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 O\0\000"..., 512) = 512 [...] open("/etc/socks.conf", O_RDONLY) = 4 fstat64(4, {st_mode=S_IFREG|0644, st_size=4374, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7bb3000 read(4, "# $Id: socks.conf,v 1.28 2001/12"..., 8192) = 4374 read(4, "", 4096) = 0 [...] |
From: Jochen H. <Joc...@Ha...> - 2007-03-16 14:29:44
|
>>>>> "MA" == Matthias Andree <mat...@gm...> writes: MA> Well - Dante doesn't have run-time switches, MA> it redirects all the network-related functions to itself, MA> fetchmail has not means to MA> circumvent SOCKS if linked against it. MA> Such run-time configuration would have to happen by means of socks.conf. MA> I am not sure if that's possible with "via: direct" statements MA> somehow for your configuration. I asked the dante people for help (-> news.gmane.org:gmane.network.socks.dante.general : "how to tell socks enabled application to not go via the socks server"), and I will come back to you, if there will be a viable solution.. Cheers, Jochen |
From: Matthias A. <mat...@gm...> - 2007-03-16 15:17:54
|
Jochen Hayek schrieb: >>>>>> "MA" == Matthias Andree <mat...@gm...> writes: > > MA> Well - Dante doesn't have run-time switches, > MA> it redirects all the network-related functions to itself, > MA> fetchmail has not means to > MA> circumvent SOCKS if linked against it. > > MA> Such run-time configuration would have to happen by means of socks.conf. > > MA> I am not sure if that's possible with "via: direct" statements > MA> somehow for your configuration. > > I asked the dante people for help (-> news.gmane.org:gmane.network.socks.dante.general : "how to tell socks enabled application to not go via the socks server"), > and I will come back to you, if there will be a viable solution.. Thank you. |
From: Jochen H. <Joc...@Ha...> - 2007-03-17 15:57:27
|
>>>>> "MA" == Matthias Andree writes: MA> Well - Dante doesn't have run-time switches, it redirects all MA> the network-related functions to itself, MA> fetchmail has not means to circumvent SOCKS if linked against it. MA> Such run-time configuration would have to happen by means of socks.conf. >>>>> On Sat, 17 Mar 2007 15:18:02 +0100, >>>>> Michael Shuldman >>>>> (whose comments are cited below with " MS> "), >>>>> had this to say in article <200...@ba...> >>>>> in newsgroups gmane.network.socks.dante.general >>>>> concerning the subject of "Re: 2007-03-16 / how to tell socks enabled application to not go via the socks server" > Jochen Hayek wrote, >> But how can a socks enabled (compiled, linked, ...) application >> instruct the socks5 library to not go via the socks5 server >> other than by removing /etc/socks.conf ? MS> Maybe you could set the environmentvariable SOCKS_CONF. MS> What happens if you do "SOCKS_CONF=/dev/null application"? That does exactly what you and I expect. Thanks a lot! Now that we (the fetchmail community) know this, fetchmail can also get an internal putenv("SOCKS_CONF=/dev/null") somewhere around its SOCKSinit("fetchmail"); and than a command line switch (e.g. --socks_conf=/etc/socks.conf ) in order to change it back, so that somebody, who really wants to employ socks, actually makes use of it: putenv("SOCKS_CONF=/etc/socks.conf") But there are dozens of alternatives ... |
From: Jochen H. <Joc...@Ha...> - 2007-03-18 15:27:51
|
>>>>> "MA" == Matthias Andree writes: > Jochen Hayek writes: MA> But why would fetchmail then need its own command line switch at all? Because that feature needs to get locked out, until it gets explicitly invited. And the fact that ready-made fetchmails in distros come with not so obvious linked-in socks support is not really a contradiction, to the fact, that people don't want that feature until they say "I want it". Because a fetchmail switch "--use-socks" defaulting to false would have helped not costing me and others quite a couple of hours. And maybe far more hours to others, that were not as sick as me, to go here to tell you, there *is* a weird thing with fetchmail. Pls don't get upset! I certainly don't consider *you* responsible for that problem. Who is responsible then? Hard to tell ... When people attempt socksification of fetchmail (likewise other applications), they are faced with the suggestion to do it via CPP macros. If they are lucky, they don't even need to touch their code. Apparently as long, as no /etc/socks.conf is created, the application continues working as before, so nobody considers introducing a "--use-socks" switch. Only at a far later stage impacts and problems appear and get discussed. Meanwhile the recentyl introduced has long gotten an established feature, and changing it would break compatibility ... A good example for the downsides of the bazaar approach ... (Sorry for this, but I couldn't resist.) MA> I mean - if the SOCKS library reads the environment, MA> we can just document this. MA> I'll do that now. A couple of words in the man page and in the FAQ, that will get found by search machines, so that googling for "fetchmail socks" will immediately reveal the problem. But I guess this thread here will get picked up the search machines anyway and will also actually help already. But still: when *I* couldn't connect to my ISP's IMAP server any longer I was ***far*** away from googling for "fetchmail socks", and it took me pretty long to find out, that I had gone through my socks server for quite a while (without being aware of it, as I never told fetchmail to do so) and that I could not connect to the IMAP server, only because my socks server was down. Now that I know that, it's obviously easy to make use of that env. var. "SOCKS_CONF". This already found its place in my ~/.profile -- maybe this should even go to /etc/profile : export SOCKS_CONF=/dev/null And wherever I do want to go through the socks server, I will "prepend" this to a command line: env SOCKS_CONF=/etc/socks.conf e.g. env SOCKS_CONF=/etc/socks.conf socksify ftp ... env SOCKS_CONF=/etc/socks.conf socksify wget ... env SOCKS_CONF=/etc/socks.conf socksify curl ... Nothing easier than that. I hated it anyway, that "socksify" does not take command line switches, and so you are not able to try different socks.conf on the the socksified command lines -- of course *after* the word socksify. But as we do know by now: you can get this effect through that env. var. -- see above! MA> I'm not going to change the default behaviour in 6.3.X releases anyways: MA> incompatibilities do not belong in patchlevel update releases. MA> So no putenvs without options. MA> Adding even more code to handle yet another option MA> however doesn't seem justified in this case MA> -- this new option would just be one different MA> formulation for exactly the same purpose MA> - and we already have env(1) and shells to handle this. You are very right. I cannot blame you for this position. MA> It's not the putenv, MA> but the dozens of code lines in the fetchmail dump, option MA> parser, rcfile parser, fetchmailconf MA> and everywhere else I've forgotten. MA> I hope that's acceptable. It is. MA> If it's not, please find some good excuses for me for duplicating the MA> environment modification code that is already in env(1) MA> - and please one that keeps compatibility with 6.3.7. :-) My idea was to *break* compatibility here, as this feature has always been a very bad feature. Cheers, Jochen |
From: Jochen H. <Joc...@Ha...> - 2007-03-19 12:48:03
|
>>>>> Rob MacGregor writes: > On 3/18/07, Matthias Andree <mat...@gm...> wrote: RM> if fetchmail is linked with the socks libraries RM> *and* the required socks config file RM> is in place then, by default, use socks. Pls remember: readymade distro fetchmails may come linked with the socks library, but its user is still quite likely not aware of it. Socksification is really not an easy peace of cake. I recently spent quite some hours (to be honest: far too many) on evaluating, whether and with which socks server certain applications like "wget" and "curl" work resp. don't work. Trust me (pls!), neither explicit nor implicit sockisification *usually* work out of the box, ie. w/o changing the code, so somebody ("a user"), who "moves" into an enforced socks server environment, will have to try / verify socksification on every single application on its own, and not all at once by toggling the one and only global switch. RM> As long as there is a command line argument RM> to either disable socks or use an alternative socks config file RM> (which implicitly enables socks) RM> then that should be fine. I seriously plea for: the default shall be: "off". I am sorry, but I cannot spend any more time on this issue. The topic socksification and all that has already been far too expensive for me during the last couple of weeks. I presented all the facts I could present to you. Take it or leave it. Oh, just a last "word": After all I honestly do appreciate the abilities of socks servers and the capabilities of explicit and implicit socksification. If small applications get developed (or carefully migrated) with explicit socksification in mind, the code doesn't even get spoilt, your TCP-whatever programming still looks like in the RFC, Stevens or whatever, and amazingly enough -- if your IT dept. socks server admin allows the connections, your application needs -- your application transparently gets through a firewall and does its job, just as if there even wasn't any disturbing firewall and just as if you were in the "real nature Internet", not within an Intranet. Kind regards, JH |
From: Jochen H. <Joc...@Ha...> - 2007-03-20 14:52:33
|
>>>>> "MA" == Matthias Andree <mat...@gm...> writes: > Jochen Hayek schrieb: >>>>>>> Rob MacGregor writes: >>> On 3/18/07, Matthias Andree <mat...@gm...> wrote: RM> if fetchmail is linked with the socks libraries RM> *and* the required socks config file RM> is in place then, by default, use socks. >> Pls remember: readymade distro fetchmails may come linked with the socks library, >> but its user is still quite likely not aware of it. MA> That's the distributor's business, MA> not that of the fetchmail maintainers. Well I think, at the end of the day it's "the user's" problem, and I guess 99% of the fetchmail users employ a ready-made installation, and fetchmail is giving them grief, if it assumes uncomfortable defaults. Obviously everybody may have his/her own position on what is (un)comfortable in this respect. But maybe "relevance of positions" should somehow relate to "competence and experience in *the* area and *not* *just* in IT". -- Pls leave responses strictly on this mailing list! |
From: Matthias A. <mat...@gm...> - 2007-03-21 00:40:33
|
Jochen Hayek schrieb: >> On 3/18/07, Matthias Andree <mat...@gm...> wrote: > > MA> That's the distributor's business, > MA> not that of the fetchmail maintainers. > > Well I think, at the end of the day it's "the user's" problem, > and I guess 99% of the fetchmail users employ a ready-made installation, > and fetchmail is giving them grief, if it assumes uncomfortable defaults. Sorry to those who are bored by this discussion, but as this is still boiling, I must conclude the participants are unhappy with the state of the discussion. Let's try to break this down to objective criteria and understand the actual cause of these emotions. Up front, maintainers are loathe to change system behavior, because that causes users pain, causes complaints, support work and all else we don't want. Trying to satisfy contradictory requirements leads either to a majority vote, a maintainer vote, or a configuration option. We know the configuration option at run-time isn't feasible beyond SOCKS_CONF -- which itself invalidates the need for a command-line/rcfile option in fetchmail. The questions I'm having: 1. What good is SOCKS, anyways? Why use SOCKS if I can have IPFW, IPFW2, PF, netfilter? 2. What was the original problem or surprise? You configured socks, which implies assigning socks proxies to certain destination addresses/networks. Your proxy went down, so you could not connect. So what? In SOCKS scenarios, the SOCKS proxy is the interface to the outside world, and indispensable in such configurations. Not a surprise to me. If there is a relevant standard (IETF, LSB, IEEE...) that states how socks-enabled applications should behave, I'll reconsider. 3. What is the scenario in production where each application had its own socks configuration? In what way is the SOCKS_CONF configuration mechanism insufficient? Why do you think fetchmail should try to be smarter than the user and stomp over his environment variables or configuration files? 4. Would not it be more fruitful if the end user talked to the distributor who packaged fetchmail with the *nondefault* --with-socks[5] option rather than offering socksify or similar? The upstream default - at compile time - already is "no socks". > Obviously everybody may have his/her own position on what is (un)comfortable in this respect. > But maybe "relevance of positions" should somehow relate to "competence and experience in *the* area and *not* *just* in IT". I understand your double frustration, first the socks irritations as such, and then that the upstream maintainers we don't see reasons to change existing code soon in spite of your desperate attempts to convince them. Note that invoking experience in a certain area isn't a valid reason to support your suggestions. I'll rather look at objective arguments. I hope to ground the unhealthy emotions from this discussion and get to the core arguments. Best, Matthias |
From: Rob M. <rob...@gm...> - 2007-03-15 15:58:11
|
On 3/15/07, Joc...@ha... <Joc...@ha...> wrote: > >>>>> "RMG" == Rob MacGregor writes: > > Currently I am convinced, your statement is incorrect. (And eventually so am I - it's nice to learn something new :>) > But I rather have some sympathy for your suspicion, > as this magical and undescribed feature of fetchmail is certainly rather weird, > and fetchmail shouldn't behave like that. > > Have you ever run fetchmail in a socks5 ready environment? > Give yourself a try and let's continue discussing the matter than! I have, and have never seen this before. > RMG> You're either running it > RMG> with a socks wrapper (runsock/socksify) > RMG> or have added this yourself (eg via ld.so.preload). > > No, I don't, nothing like that. > (No socks wrapper, no LD_PRELOAD, no /etc/ld.so.preload, no ...) > I told you above: "I did not instruct my local fetchmail to make use of my socks5 proxy" > I mean, what I write. > And you read my statement, didn't you?!? If you're going to take that attitude I'll simply stop trying to help you :/ That you didn't (mean to) explicitly configure fetchmail itself to use socks doesn't it isn't. And, frankly, your original email contained no data at all, just your words. Without data showing what's actually going on I have to make assumptions. > Can you see "SOCKS" here: > > [2007-03-15 14:11:49] johayek@HayekJ $ fetchmail --version > This is fetchmail release 6.3.2+POP2+IMAP-GSS+RPA+NTLM+SDPS+SSL+OPIE+SOCKS+NLS. Yeah that looks like it was configured with socks support. Having myself dug through the documentation and FAQ it appears that there is socks support after all (it's not documented anywhere except the FAQ): http://www.fetchmail.info/fetchmail-FAQ.html#K1 Rebuild fetchmail without the socks support. It looks like fetchmail will automatically use socks if you build it with socks support - there's no runtime switch. (At least that's what a read of the source tells me) So, in short, while you didn't intend to configure fetchmail to use your socks server, you did by configuring socks support into fetchmail (and I'm guessing defining some environment variables or putting entries in socks.conf). -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche |
From: Matthias A. <mat...@gm...> - 2007-03-15 23:17:21
|
Joc...@Ha... schrieb: >>>>>> "RMG" == Rob MacGregor writes: > > On 3/8/07, Jochen Hayek wrote: > > >> A while ago I started experiments with socks5 (-> dante) > >> and created a /etc/socks.conf on my local host and I set up a socks5 proxy server on a neighbour box. > >> > >> I did not instruct my local fetchmail to make use of my socks5 proxy > >> and I also wasn't aware, that my local fetchmail actually does us it, > >> but in a situation, when that proxy server on the neighbour box temporarily did not run, > >> I finally noticed (using "strace"), that fetchmail contacted my socks5 proxy. > >> > >> > >> I did some rtfm-ing, > >> but I did not find a way to instruct fetchmail to *not* attempt contacting my socks5 proxy. > > RMG> Fetchmail has *no* native socks support. > > Currently I am convinced, your statement is incorrect. > > But I rather have some sympathy for your suspicion, > as this magical and undescribed feature of fetchmail is certainly rather weird, > and fetchmail shouldn't behave like that. Well - Dante doesn't have run-time switches, it redirects all the network-related functions to itself, fetchmail has not means to circumvent SOCKS if linked against it. Such run-time configuration would have to happen by means of socks.conf. I am not sure if that's possible with "via: direct" statements somehow for your configuration. HTH MA |
From: Jochen H. <Joc...@Ha...> - 2007-03-16 14:48:25
|
>>>>> "RMG2" == Rob MacGregor writes: >> [2007-03-15 14:11:49] johayek@HayekJ $ fetchmail --version >> This is fetchmail release 6.3.2+POP2+IMAP-GSS+RPA+NTLM+SDPS+SSL+OPIE+SOCKS+NLS. RMG2> Yeah that looks like it was configured with socks support. Having RMG2> myself dug through the documentation and FAQ it appears that there is RMG2> socks support after all (it's not documented anywhere except the FAQ): RMG2> http://www.fetchmail.info/fetchmail-FAQ.html#K1 RMG2> Rebuild fetchmail without the socks support. RMG2> It looks like fetchmail will automatically use socks RMG2> if you build it with socks support ... resp. e.g. if your ready-made Linux distribution provides you with a fetchmail configured like that. RMG2> - there's no runtime switch. RMG2> (At least that's what a read of the source tells me) RMG2> So, in short, while you didn't intend to configure fetchmail RMG2> to use your socks server, RMG2> you did by configuring socks support into fetchmail I don't want to appear picky (although I probably am), but if a usual DAU (and I myself have grown out of that state during some 25 years in IT), who certainly has the right to use a Linux, BSD or whatever distro instead of compiling everything himself from scratch and who esp. has the right to use a fetchmail, that comes with that distro, and such a distro may provide him with a socks enabled fetchmail, and if he experiments in one corner (socks), then he does not necessarily need to expect impact in a slightly different corner (fetchmail). Although addmittedly both corners are "a little" associated "through" TCP/IP resp. a substitute aka socks library. So ... it would be nice to have a socks enabled fetchmail defaulting to "do *not* use socks until I tell you via command line or via RC option". And that's where I started "a while ago": JH> I would actually prefer a command line option and an entry in the RC file. JH> Is there already a feature of fetchmail, that this could be made similar to? Having a look at the sources myself, too, I hoped, that the call to SOCKSinit would be an appropriate location to include such a switch, but that hope sadly did not fulfill. Alright, I think, I just admit, that my idea of a fetchmail has not yet been implemented and is also not so easy to implement. I asked the dante people for the help (-> news.gmane.org:gmane.network.socks.dante.general : "how to tell socks enabled application to not go via the socks server"), and I will come back to you, if there will be a viable solution.. RMG2> (and I'm guessing defining some RMG2> environment variables or putting entries in socks.conf). I "just" made my socks.conf to point to a real socks5 server. Before that, that socks.conf just waited to be taken care of ... Cheers, Jochen |
From: Rob M. <rob...@gm...> - 2007-03-16 16:25:49
|
On 3/16/07, Jochen Hayek <Joc...@ha...> wrote: > > I don't want to appear picky (although I probably am), > but if a usual DAU (and I myself have grown out of that state during some 25 years in IT), > who certainly has the right to use a Linux, BSD or whatever distro instead of compiling everything himself from scratch > and who esp. has the right to use a fetchmail, that comes with that distro, > and such a distro may provide him with a socks enabled fetchmail, > and if he experiments in one corner (socks), > then he does not necessarily need to expect impact in a slightly different corner (fetchmail). I'm with you on that one. Sadly the problem is that whoever created the binary package didn't understand the impact of what they were doing when they compiled in SOCKS support. > So ... it would be nice to have a socks enabled fetchmail > defaulting to "do *not* use socks until I tell you via command line or via RC option". If you're able to supply patches to make this happen then I suspect the developers would be interested (not a dig, just a suggestion). I've also submitted a patch for the man page (which should make it into 6.3.8 with luck) to document this behaviour. -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche |
From: Matthias A. <mat...@gm...> - 2007-03-17 20:42:40
|
Jochen Hayek schrieb: > Now that we (the fetchmail community) know this, fetchmail can also get an internal > > putenv("SOCKS_CONF=/dev/null") > > somewhere around its > > SOCKSinit("fetchmail"); > > and than a command line switch (e.g. --socks_conf=/etc/socks.conf ) > in order to change it back, > so that somebody, who really wants to employ socks, actually makes use of it: > > putenv("SOCKS_CONF=/etc/socks.conf") > > But there are dozens of alternatives ... But why would fetchmail then need its own command line switch at all? I mean - if the SOCKS library reads the environment, we can just document this. I'll do that now. I'm not going to change the default behaviour in 6.3.X releases anyways: incompatibilities do not belong in patchlevel update releases. So no putenvs without options. Adding even more code to handle yet another option however doesn't seem justified in this case -- this new option would just be one different formulation for exactly the same purpose - and we already have env(1) and shells to handle this. It's not the putenv, but the dozens of code lines in the fetchmail dump, option parser, rcfile parser, fetchmailconf and everywhere else I've forgotten. I hope that's acceptable. If it's not, please find some good excuses for me for duplicating the environment modification code that is already in env(1) - and please one that keeps compatibility with 6.3.7. :-) Best, MA |
From: Matthias A. <mat...@gm...> - 2007-03-18 23:44:52
|
Jochen Hayek schrieb: >>>>>> "MA" == Matthias Andree writes: > >> Jochen Hayek writes: > > MA> But why would fetchmail then need its own command line switch at all? > > Because that feature needs to get locked out, until it gets explicitly invited. > > And the fact that ready-made fetchmails in distros come with not so obvious linked-in socks support > is not really a contradiction, > to the fact, that people don't want that feature until they say "I want it". Well, this feature has been automatically enabled in fetchmail since at least 6.2.5, and I think we should keep breaking compatibility in this respect until the next minor or major release -- I'll certainly consider changing the default for fetchmail 6.4 or 7.0 or whatever version, and I'm very much inclined to choose defaults that do not surprise the user. I wonder if there is really a common expectation. Other users might expect that *if* they have a socks.conf, *then* socks-enabled applications should use it. This isn't clear to me, and it's a bet the maintainer will always lose - one group will be happy and quiet, and the other group will shout. :-) > Because a fetchmail switch "--use-socks" defaulting to false > would have helped not costing me and others quite a couple of hours. Sorry for that - it's documented now <http://www.fetchmail.info/fetchmail-FAQ.html#K1> and will also be documented in the manpage of fetchmail 6.3.8. Feel free to suggest further changes, the FAQ is surely not cast in concrete. > A good example for the downsides of the bazaar approach ... > (Sorry for this, but I couldn't resist.) Not my essay, and I'm not enthusiastic about several of the assumptions fetchmail makes, and I'm still pondering whether a complete redesign is more of an effort than rewriting existing code... I sympathize with you on the bazaar "appreciation", too. The "many eyeballs" argument that is invoked so often doesn't hold water IMO - those eyeballs are only looking after someone has encountered a *local* problem (or in the rare case some distributor pays for the audit or some company tries their automated code validator and shares the results). The accumulation of dozens of patches, with uncorrected programming styles, which are really just patches that have often not properly been integrated - this is a constant headache, and the security fixes that went into 6.3.6 had to be large as a result of these inconsistencies and ad-hoc hacking -- and the fallout caused regression fixes in 6.3.7 and 6.3.8. This is the thing I feared most, I did several 6.3.6 release candidates, but still only field deployment revealed these issues. :-( Getting distracted a bit, I'd also say beta and rc versions don't work without contracted beta testers, since users will just wait for the final release out of convenience - and that's understandable, consumers don't want unripe garbage... > A couple of words in the man page and in the FAQ, > that will get found by search machines, > so that googling for "fetchmail socks" > will immediately reveal the problem. Search engines aren't under my control -- let me know if you want anything else or suggest changes to the FAQ URL I've given above. > But still: > when *I* couldn't connect to my ISP's IMAP server any longer > I was ***far*** away from googling for "fetchmail socks", > and it took me pretty long to find out, > that I had gone through my socks server for quite a while > (without being aware of it, as I never told fetchmail to do so) > and that I could not connect to the IMAP server, only because my socks server was down. If fetchmail were actively enabling socks, then we might just log additional information to the connecting... ("via socks server XYZ port N") but this isn't the case. I haven't looked too deep into SOCKS yet and I'm short on time. I plan to do 6.3.8-rc2 this week and 6.3.8 this month, so that I can move on to actually opening the 6.4 branch. -- Matthias Andree |
From: Rob M. <rob...@gm...> - 2007-03-19 00:14:53
|
On 3/18/07, Matthias Andree <mat...@gm...> wrote: > > Well, this feature has been automatically enabled in fetchmail since at > least 6.2.5, and I think we should keep breaking compatibility in this > respect until the next minor or major release -- I'll certainly consider > changing the default for fetchmail 6.4 or 7.0 or whatever version, and > I'm very much inclined to choose defaults that do not surprise the user. > > I wonder if there is really a common expectation. Other users might > expect that *if* they have a socks.conf, *then* socks-enabled > applications should use it. > > This isn't clear to me, and it's a bet the maintainer will always lose - > one group will be happy and quiet, and the other group will shout. :-) As long as the default is clearly documented it shouldn't matter what it is. I think the real problem here has been the lack of clear documentation. I'd be tempted to go with your second paragraph - if fetchmail is linked with the socks libraries *and* the required socks config file is in place then, by default, use socks. As long as there is a command line argument to either disable socks or use an alternative socks config file (which implicitly enables socks) then that should be fine. > Not my essay, and I'm not enthusiastic about several of the assumptions > fetchmail makes, and I'm still pondering whether a complete redesign is > more of an effort than rewriting existing code... Which is more effort to maintain :) > If fetchmail were actively enabling socks, then we might just log > additional information to the connecting... ("via socks server XYZ port > N") but this isn't the case. I haven't looked too deep into SOCKS yet > and I'm short on time. I plan to do 6.3.8-rc2 this week and 6.3.8 this > month, so that I can move on to actually opening the 6.4 branch. It probably wouldn't hurt for the --configdump option to include the fact that socks is compiled in - which, in theory, is little more than: #ifdef HAVE_SOCKS "'socks'," #endif after line 186 of conf.c -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche |
From: Matthias A. <mat...@gm...> - 2007-03-25 17:03:40
|
"Rob MacGregor" <rob...@gm...> writes: > I'd be tempted to go with your second paragraph - if fetchmail is > linked with the socks libraries *and* the required socks config file > is in place then, by default, use socks. As long as there is a > command line argument to either disable socks or use an alternative > socks config file (which implicitly enables socks) then that should be > fine. I have decided to do exactly that, even if it causes pain to those who experiment with socks -- IOW, if you leave shards lying around, wear tough shoes. >> Not my essay, and I'm not enthusiastic about several of the assumptions >> fetchmail makes, and I'm still pondering whether a complete redesign is >> more of an effort than rewriting existing code... > > Which is more effort to maintain :) Need not be your concern though. 8-) > It probably wouldn't hurt for the --configdump option to include the > fact that socks is compiled in - which, in theory, is little more > than: > > #ifdef HAVE_SOCKS > "'socks'," > #endif > > after line 186 of conf.c Done for 6.3.8. Thanks for the report. -- Matthias Andree |
From: Rob F. <rf...@fu...> - 2007-03-19 17:16:08
|
Jochen Hayek wrote: > Trust me (pls!), neither explicit nor implicit sockisification > *usually* work out of the box, ie. w/o changing the code, > so somebody ("a user"), who "moves" into an enforced socks server > environment, will have to try / verify socksification on every single > application on its own, and not all at once by toggling the one and > only global switch. IMHO, the goal should be that configuring socks in one place makes it work everywhere. No, that's not the current situation, but if I need socks then that's what I want. Therefore, if I'm a user needing socks, I want to be able to set up socks.conf and have as much work with socks as possible. I don't want to have to look for --enable-socks runtime switches everywhere. So if fetchmail currently tries to use socks if you've set up socks.conf, then I think that's the right thing to do. If you don't want fetchmail to use socks, don't set up socks.conf. -- ==============================| "A microscope locked in on one point Rob Funk <rf...@fu...> |Never sees what kind of room that it's in" http://www.funknet.net/rfunk | -- Chris Mars, "Stuck in Rewind" |
From: Matthias A. <mat...@gm...> - 2007-03-19 21:57:37
|
Jochen Hayek schrieb: >>>>>> Rob MacGregor writes: > >> On 3/18/07, Matthias Andree <mat...@gm...> wrote: > > RM> if fetchmail is linked with the socks libraries > RM> *and* the required socks config file > RM> is in place then, by default, use socks. > > Pls remember: readymade distro fetchmails may come linked with the socks library, > but its user is still quite likely not aware of it. That's the distributor's business, not that of the fetchmail maintainers. See - if they change the defaults, they need to support those changes. And the default fetchmail compilation options DO NOT include socks support. We cannot afford - and I personally do not even want - to support every compilation or perhaps patch variant there is in practice. > I recently spent quite some hours (to be honest: far too many) on evaluating, > whether and with which socks server certain applications like "wget" and "curl" work resp. don't work. I understand your frustration, but what has this got to do with fetchmail? I'm very much with Rob Funk here, if there is a system-wide configuration, we should just use it -- the possibility for local override is documented now. > > Trust me (pls!), neither explicit nor implicit sockisification *usually* work out of the box, > ie. w/o changing the code, > so somebody ("a user"), who "moves" into an enforced socks server environment, > will have to try / verify socksification on every single application on its own, > and not all at once by toggling the one and only global switch. echo >>$HOME/.profile 'SOCKS_CONF=/dev/null ; export SOCKS_CONF' . .profile Csh left as an exercise to the reader. > I seriously plea for: the default shall be: "off". Noted for consideration for the fetchmail 6.4 branch. |
From: Jochen H. <Joc...@Ha...> - 2007-03-20 14:42:12
|
>>>>> "RF" == Rob Funk writes: > Jochen Hayek wrote: >> Trust me (pls!), neither explicit nor implicit sockisification >> *usually* work out of the box, ie. w/o changing the code, >> so somebody ("a user"), who "moves" into an enforced socks server >> environment, will have to try / verify socksification on every single >> application on its own, and not all at once by toggling the one and >> only global switch. RF> IMHO, the goal should be that configuring socks in one place RF> makes it work everywhere. () Have you already been in a situation, where you were forced to use socks, in order to overcome firewall restrictions? () Have you ever (really) used a socks server? () Did you experience trouble when doing so? () Have you only just read an article on socks and are you presenting your ideas on that basis? () Are you so much convinced of socks, that you recommended setting up a socks server to the IT dept. manager of the last bank you worked for? RF> [...] |
From: Rob M. <rob...@gm...> - 2007-03-20 16:00:39
|
On 3/20/07, Jochen Hayek <Joc...@ha...> wrote: > () Have you already been in a situation, where you were forced to use socks, > in order to overcome firewall restrictions? And that's the scary part. What you're saying (though I doubt it's what you mean to say) is that you've used socks so you can completely bypass your organisations IT security policy regarding Internet traffic. > Well I think, at the end of the day it's "the user's" problem, > and I guess 99% of the fetchmail users employ a ready-made installation, > and fetchmail is giving them grief, if it assumes uncomfortable defaults. Sorry, but there is nothing either the author's or supporters of fetchmail can do if distributors want to make their own non-standard changes to fetchmail's configuration or compile in all the possible libraries. There is enough time taken up with helping people as it is (not to mention the amount of time this thread has taken up). I do agree that the socks situation could have been better documented, both in the output of fetchmail's configdump and in the man page. That however is within "our" control and is being resolved. > Obviously everybody may have his/her own position on what is (un)comfortable in this respect. > But maybe "relevance of positions" should somehow relate to "competence and experience in *the* area and *not* *just* in IT". And an understanding that you're not always right, no matter how much pain a problem may have caused you :) -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche |