From: David G. <da...@dg...> - 2005-03-18 19:59:40
|
Hi I'm upgrading from 6.2.5 to 6.2.6 and I've encountered a possible bug where 6.2.6 resends a CAPA challenge even when the server doesn't understand it. The ISP is a major UK one. Matthias asked me to use the code from here to check another bug (this was an embarrisingly long time ago) http://home.pages.de/~mandree/tmp/fetchmail-6.2.6.tar.bz2 I was going to hack on the code but thought I'd check first so... Is this a known issue? Is this code up-to-date? config: 1. OS: Linux RedHat 7.3 kernel 2.6.6 2. gcc version 2.96 20000731 (Red Hat Linux 7.3 2.96-110) 3. below 4. forwarding to lmtp listener on cyrus 2.2.3 (bounce goes to SMTP exim 4.2) here's a working 6.2.5 behaviour - notice that when it gets an ERR to the CAPA challenge it skips it (the CAPA) on the repoll: fetchmail: 6.2.5 querying pop.freeserve.net (protocol POP3) at Fri 18 Mar 2005 06:44:29 PM GMT: poll started fetchmail: POP3< +OK connected to pop3 on 3212 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3212 fetchmail: POP3> USER da...@in... fetchmail: POP3< +OK name is a valid mailbox fetchmail: POP3> PASS * fetchmail: POP3< +OK user exist with that password fetchmail: selecting or re-polling default folder fetchmail: POP3> STAT fetchmail: POP3< +OK 1 2518 1 message for da...@in... at pop.freeserve.net (2518 octets). fetchmail: POP3> LIST 1 fetchmail: POP3< +OK 1 2518 fetchmail: POP3> RETR 1 fetchmail: POP3< +OK Message follows reading message da...@in...@pop.freeserve.com:1 of 1 (2518 octets) About to rewrite Return-Path: <ka...@em...> Rewritten version is Return-Path: <ka...@em...> About to rewrite From: "Scotty Akers" <ka...@em...> Rewritten version is From: "Scotty Akers" <ka...@em...> About to rewrite To: da...@dg... Rewritten version is To: da...@dg... fetchmail: LMTP< 220 willow LMTP Cyrus v2.2.3 ready fetchmail: LMTP> LHLO localhost fetchmail: SMTP< 250-willow fetchmail: SMTP< 250-8BITMIME fetchmail: SMTP< 250-ENHANCEDSTATUSCODES fetchmail: SMTP< 250-PIPELINING fetchmail: SMTP< 250-SIZE fetchmail: SMTP< 250-AUTH EXTERNAL fetchmail: SMTP< 250 IGNOREQUOTA fetchmail: forwarding to /var/imap/socket/lmtp fetchmail: LMTP> MAIL FROM:<ka...@em...> SIZE=2518 fetchmail: LMTP< 250 2.1.0 ok fetchmail: LMTP> RCPT TO:<da...@dg...> fetchmail: LMTP< 250 2.1.5 ok fetchmail: LMTP> DATA fetchmail: LMTP< 354 go ahead #***************************fetchmail: LMTP>. (EOM) fetchmail: LMTP< 250 2.1.5 Ok flushed fetchmail: POP3> DELE 1 fetchmail: POP3< +OK message deleted fetchmail: POP3> QUIT fetchmail: POP3< +OK fetchmail: 6.2.5 querying pop.freeserve.net (protocol POP3) at Fri 18 Mar 2005 06:44:48 PM GMT: poll completed on 6.2.6, same config: fetchmail: 6.2.6 querying pop.freeserve.net (protocol POP3) at Fri 18 Mar 2005 06:42:42 PM GMT: poll started fetchmail: POP3< +OK connected to pop3 on 3013 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3212 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3114 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3013 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3114 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com this hangs forever. David More config: set syslog set postmaster "da...@dg..." set nobouncemail set properties "" set daemon 180 set idfile /var/run/fetchmail.ids # tracepolls: Insert headers with pop3 acc info... # fetchall: Sometimes messages are marked 'seen' if e.g. a fetchmail fails due # to network dropping. This makes sure they're all retrieved: # The ukfsn accounts poll pop3.ukfsn.org with proto POP3 tracepolls user 'dgreaves' there with password 'xxxxxxxx' is da...@dg... here options fetchal l lmtp smtp /var/imap/socket/lmtp expunge 30 antispam 571 550 501 554 user 'de...@at...' there with password 'xxxxxxxxxxx' is de...@dg... here options fetchall lmtp smtp /var/imap/socket/lmtp expunge 30 antispam 571 550 501 554 user 'de...@dg...' there with password 'xxxxxxxxxxxx' is de...@dg... here opt ions fetchall lmtp smtp /var/imap/socket/lmtp expunge 30 antispam 571 550 501 554 # Old freeserve account - mainly spam now poll pop.freeserve.net with proto POP3 tracepolls user 'da...@in...' there with password 'xxxxxxxxxx' is da...@dg... h ere options fetchall lmtp smtp /var/imap/socket/lmtp expunge 30 antispam 571 550 501 554 user 'de...@in...' there with password 'xxxxxxxxx' is de...@dg... here options fetchall lmtp smtp /var/imap/socket/lmtp expunge 30 antispam 571 550 501 554 # /usr/bin/fetchmail -V -v -v -f /etc/fetchmailrc This is fetchmail release 6.2.6+NLS Fallback MDA: (none) Linux willow 2.6.6 #1 Wed Jun 2 12:15:21 BST 2004 i586 unknown Taking options from command line and /etc/fetchmailrc Poll interval is 180 seconds Idfile is /var/run/fetchmail.ids Progress messages will be logged via syslog Fetchmail will forward misaddressed multidrop messages to da...@dg.... Fetchmail will direct error mail to the postmaster. Options for retrieving from dgr...@po...: True name of server is pop3.ukfsn.org. This host will be queried when no host is specified. Password = "xxxxxxxxxx". Protocol is POP3 (using default port). All available authentication methods will be tried. Server nonresponse timeout is 300 seconds (default). Default mailbox selected. All messages will be retrieved (--all on). Fetched messages will not be kept on the server (--keep off). Old messages will not be flushed before message retrieval (--flush off). Rewrite of server-local addresses is enabled (--norewrite off). Carriage-return stripping is disabled (stripcr off). Carriage-return forcing is disabled (forcecr off). Interpretation of Content-Transfer-Encoding is enabled (pass8bits off). MIME decoding is disabled (mimedecode off). Idle after poll is disabled (idle off). Nonempty Status lines will be kept (dropstatus off) Delivered-To lines will be kept (dropdelivered off) No received-message limit (--fetchlimit 0). Fetch message size limit is 100 (--fetchsizelimit 100). Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10). No SMTP message batch limit (--batchlimit 0). Deletion interval between expunges forced to 30 (--expunge 30). Messages will be LMTP-forwarded to: /var/imap/socket/lmtp Recognized listener spam block responses are: 571 550 501 554 No pre-connection command. No post-connection command. Single-drop mode: 1 local name(s) recognized. da...@dg... No interface requirement specified. No monitor interface specified. No plugin command specified. No plugout command specified. No UIDs saved from this host. Poll trace information will be added to the Received header. Options for retrieving from de...@at...@pop3.ukfsn.org: True name of server is pop3.ukfsn.org. This host will be queried when no host is specified. Password = "xxxxxxxx". Protocol is POP3 (using default port). All available authentication methods will be tried. Server nonresponse timeout is 300 seconds (default). Default mailbox selected. All messages will be retrieved (--all on). Fetched messages will not be kept on the server (--keep off). Old messages will not be flushed before message retrieval (--flush off). Rewrite of server-local addresses is enabled (--norewrite off). Carriage-return stripping is disabled (stripcr off). Carriage-return forcing is disabled (forcecr off). Interpretation of Content-Transfer-Encoding is enabled (pass8bits off). MIME decoding is disabled (mimedecode off). Idle after poll is disabled (idle off). Nonempty Status lines will be kept (dropstatus off) Delivered-To lines will be kept (dropdelivered off) No received-message limit (--fetchlimit 0). Fetch message size limit is 100 (--fetchsizelimit 100). Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10). No SMTP message batch limit (--batchlimit 0). Deletion interval between expunges forced to 30 (--expunge 30). Messages will be LMTP-forwarded to: /var/imap/socket/lmtp Recognized listener spam block responses are: 571 550 501 554 No pre-connection command. No post-connection command. Single-drop mode: 1 local name(s) recognized. de...@dg... No interface requirement specified. No monitor interface specified. No plugin command specified. No plugout command specified. No UIDs saved from this host. No poll trace information will be added to the Received header. . Options for retrieving from de...@dg...@pop3.ukfsn.org: True name of server is pop3.ukfsn.org. This host will be queried when no host is specified. Password = "xxxxxxxxx". Protocol is POP3 (using default port). All available authentication methods will be tried. Server nonresponse timeout is 300 seconds (default). Default mailbox selected. All messages will be retrieved (--all on). Fetched messages will not be kept on the server (--keep off). Old messages will not be flushed before message retrieval (--flush off). Rewrite of server-local addresses is enabled (--norewrite off). Carriage-return stripping is disabled (stripcr off). Carriage-return forcing is disabled (forcecr off). Interpretation of Content-Transfer-Encoding is enabled (pass8bits off). MIME decoding is disabled (mimedecode off). Idle after poll is disabled (idle off). Nonempty Status lines will be kept (dropstatus off) Delivered-To lines will be kept (dropdelivered off) No received-message limit (--fetchlimit 0). Fetch message size limit is 100 (--fetchsizelimit 100). Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10). No SMTP message batch limit (--batchlimit 0). Deletion interval between expunges forced to 30 (--expunge 30). Messages will be LMTP-forwarded to: /var/imap/socket/lmtp Recognized listener spam block responses are: 571 550 501 554 No pre-connection command. No post-connection command. Single-drop mode: 1 local name(s) recognized. de...@dg... No interface requirement specified. No monitor interface specified. No plugin command specified. No plugout command specified. No UIDs saved from this host. No poll trace information will be added to the Received header. . Options for retrieving from da...@in...@pop.freeserve.net: True name of server is pop.freeserve.net. This host will be queried when no host is specified. Password = "xxxxxxxx". Protocol is POP3 (using default port). All available authentication methods will be tried. Server nonresponse timeout is 300 seconds (default). Default mailbox selected. All messages will be retrieved (--all on). Fetched messages will not be kept on the server (--keep off). Old messages will not be flushed before message retrieval (--flush off). Rewrite of server-local addresses is enabled (--norewrite off). Carriage-return stripping is disabled (stripcr off). Carriage-return forcing is disabled (forcecr off). Interpretation of Content-Transfer-Encoding is enabled (pass8bits off). MIME decoding is disabled (mimedecode off). Idle after poll is disabled (idle off). Nonempty Status lines will be kept (dropstatus off) Delivered-To lines will be kept (dropdelivered off) No received-message limit (--fetchlimit 0). Fetch message size limit is 100 (--fetchsizelimit 100). Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10). No SMTP message batch limit (--batchlimit 0). Deletion interval between expunges forced to 30 (--expunge 30). Messages will be LMTP-forwarded to: /var/imap/socket/lmtp Recognized listener spam block responses are: 571 550 501 554 No pre-connection command. No post-connection command. Single-drop mode: 1 local name(s) recognized. da...@dg... No interface requirement specified. No monitor interface specified. No plugin command specified. No plugout command specified. No UIDs saved from this host. Poll trace information will be added to the Received header. Options for retrieving from de...@in...@pop.freeserve.net: True name of server is pop.freeserve.net. This host will be queried when no host is specified. Password = "xxxxxxxxxx". Protocol is POP3 (using default port). All available authentication methods will be tried. Server nonresponse timeout is 300 seconds (default). Default mailbox selected. All messages will be retrieved (--all on). Fetched messages will not be kept on the server (--keep off). Old messages will not be flushed before message retrieval (--flush off). Rewrite of server-local addresses is enabled (--norewrite off). Carriage-return stripping is disabled (stripcr off). Carriage-return forcing is disabled (forcecr off). Interpretation of Content-Transfer-Encoding is enabled (pass8bits off). MIME decoding is disabled (mimedecode off). Idle after poll is disabled (idle off). Nonempty Status lines will be kept (dropstatus off) Delivered-To lines will be kept (dropdelivered off) No received-message limit (--fetchlimit 0). Fetch message size limit is 100 (--fetchsizelimit 100). Do binary search of UIDs during 9 out of 10 polls (--fastuidl 10). No SMTP message batch limit (--batchlimit 0). Deletion interval between expunges forced to 30 (--expunge 30). Messages will be LMTP-forwarded to: /var/imap/socket/lmtp Recognized listener spam block responses are: 571 550 501 554 No pre-connection command. No post-connection command. Single-drop mode: 1 local name(s) recognized. de...@dg... No interface requirement specified. No monitor interface specified. No plugin command specified. No plugout command specified. No UIDs saved from this host. No poll trace information will be added to the Received header. . |
From: Matthias A. <mat...@gm...> - 2005-03-19 00:40:04
|
On Fri, 18 Mar 2005, David Greaves wrote: > I'm upgrading from 6.2.5 to 6.2.6 and I've encountered a possible bug > where 6.2.6 resends a CAPA challenge even when the server doesn't > understand it. The ISP is a major UK one. Note that the 6.2.6 address given below... > Matthias asked me to use the code from here to check another bug (this > was an embarrisingly long time ago) > http://home.pages.de/~mandree/tmp/fetchmail-6.2.6.tar.bz2 ...was test code, not the official 6.2.6 code. Which is - we can share embarrassment here - still not released yet. > I was going to hack on the code but thought I'd check first so... > Is this a known issue? Is this code up-to-date? The code is not up-to-date, and I don't see this problem when using my version against a server that does not support CAPA. May I ask you to try http://home.pages.de/~mandree/tmp/fetchmail-6.2.5.991.tar.bz2 first and report here if your problem persists in 6.2.5.991? That is newer code (2005-03-01) in spite of the lower version number (the "6.2.6" snapshot is from 2004-11-10). If the problem persists in that code, I'll have a closer look. HTH, -- Matthias Andree |
From: David G. <da...@dg...> - 2005-03-19 18:57:32
|
Matthias Andree wrote: >The code is not up-to-date, and I don't see this problem when using my >version against a server that does not support CAPA. > >May I ask you to try >http://home.pages.de/~mandree/tmp/fetchmail-6.2.5.991.tar.bz2 >first and report here if your problem persists in 6.2.5.991? > > it does... >That is newer code (2005-03-01) in spite of the lower version number >(the "6.2.6" snapshot is from 2004-11-10). > > I didn't configure --with-ssl :) I think this is the correct fix. --- fetchmail-6.2.5.991-dg/pop3.c Sat Mar 19 11:35:59 2005 +++ fetchmail-6.2.5.991/pop3.c Sun Feb 27 20:19:35 2005 @@ -361,10 +361,7 @@ static int pop3_getauth(int sock, struct * These authentication methods are blessed by RFC1734, * describing the POP3 AUTHentication command. */ - if ( -#ifdef SSL_ENABLE - (ctl->use_ssl != FLAG_FALSE) || -#endif /* SSL_ENABLE (*/ + if ((ctl->use_ssl != FLAG_FALSE) || (ctl->server.authenticate == A_ANY) || (ctl->server.authenticate == A_GSSAPI) || (ctl->server.authenticate == A_KERBEROS_V4) || |
From: Matthias A. <mat...@gm...> - 2005-03-19 18:13:33
|
On Sat, 19 Mar 2005, David Greaves wrote: > >That is newer code (2005-03-01) in spite of the lower version number > >(the "6.2.6" snapshot is from 2004-11-10). > > I didn't configure --with-ssl :) > > I think this is the correct fix. > > --- fetchmail-6.2.5.991-dg/pop3.c Sat Mar 19 11:35:59 2005 > +++ fetchmail-6.2.5.991/pop3.c Sun Feb 27 20:19:35 2005 > @@ -361,10 +361,7 @@ static int pop3_getauth(int sock, struct > * These authentication methods are blessed by RFC1734, > * describing the POP3 AUTHentication command. > */ > - if ( > -#ifdef SSL_ENABLE > - (ctl->use_ssl != FLAG_FALSE) || > -#endif /* SSL_ENABLE (*/ > + if ((ctl->use_ssl != FLAG_FALSE) || > (ctl->server.authenticate == A_ANY) || > (ctl->server.authenticate == A_GSSAPI) || > (ctl->server.authenticate == A_KERBEROS_V4) || You've got the patch backwards. Use "diff -ur OLDER NEWER". Even reading it in the right direction, I don't see what this does - and how it could possibly fix your problem. use_ssl defaults to false, and the bug also shows with SSL enabled at compile time and unused in the configuration. I can reproduce this on my computer even with the latest SVN code, so I have debugger fodder. This is a genuine regression from 6.2.5 indeed. I'll debug this a bit. -- Matthias Andree |
From: David G. <da...@dg...> - 2005-03-19 19:14:12
|
Matthias Andree wrote: > You've got the patch backwards. Use "diff -ur OLDER NEWER". sorry. >Even reading it in the right direction, I don't see what this does - and >how it could possibly fix your problem. use_ssl defaults to false, > use_ssl *should* default to false. I didn't follow the logic through - it was an eyeballing of the use of an _ssl variable outside an SSL_ENABLE block. It does actually fix the problem for me. Once an 'obvious' bug fixed it for me I passed it on to you... Anyway, I'm glad it's helped find a regression. This code snippet was my debug test... I didn't know the insides of fetchmail well enough to assume anything else :) if (ctl->use_ssl == FLAG_FALSE) fprintf(stderr, "is FLAG_false\n"); else fprintf(stderr, "use_ssl isn't FLAG_false\n"); if ( #ifdef SSL_ENABLE (ctl->use_ssl != FLAG_FALSE) || #endif /* SSL_ENABLE (*/ fetchmail: 6.2.5.991 querying pop.freeserve.net (protocol POP3) at Sat 19 Mar 2005 01:07:16 PMGMT: poll started fetchmail: POP3< +OK connected to pop3 on 3002 use_ssl isn't FLAG_false fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3212 use_ssl isn't FLAG_false fetchmail: POP3> USER da...@in... fetchmail: POP3< +OK name is a valid mailbox fetchmail: POP3> PASS * fetchmail: POP3< +OK user exist with that password ################################################ anyway.... I couldn't leave it at that :) a context grep of use_ssl gives an interesting result in fetchmail.c: fetchmail.c-#ifdef SSL_ENABLE fetchmail.c: DEFAULT(ctl->use_ssl, FALSE); fetchmail.c- DEFAULT(ctl->sslcertck, FALSE); fetchmail.c-#endif so if SSL_ENABLE is not defined it's undefined. So, I'm thinking my patch was right after all ;) :-P nyah! David |
From: David G. <da...@dg...> - 2005-03-19 19:14:12
|
Matthias Andree wrote: >The code is not up-to-date, and I don't see this problem when using my >version against a server that does not support CAPA. > > thanks :) well, since the problem occurs before the user/pass - feel free to use that bit of my fetchmailrc to see if you have the problem. >May I ask you to try >http://home.pages.de/~mandree/tmp/fetchmail-6.2.5.991.tar.bz2 >first and report here if your problem persists in 6.2.5.991? > > sure :- fetchmail: 6.2.5.991 querying pop.freeserve.net (protocol POP3) at Sat 19 Mar 2005 08:24:38 AMGMT: poll started fetchmail: POP3< +OK connected to pop3 on 3107 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command fetchmail: Repoll immediately on da...@in...@pop.freeserve.com fetchmail: POP3< +OK connected to pop3 on 3006 fetchmail: POP3> CAPA fetchmail: POP3< -ERR unknown command fetchmail: unknown command and now I have the up to date source, I'll take a quick look too... David |
From: Matthias A. <mat...@gm...> - 2005-03-19 18:13:33
|
[[ CC: Nalin Dahyabai Nalin, as the list archives are down currently: David Greaves originally reported that your patch that Eric S. Raymond merged as "Nalin Dahyabai's fix for POP3 strong authentication" after fetchmail-6.2.5 caused fetchmail to go into an unterminated loop when the server rejected the CAPA command. If you can shed any light on your patch that changed pop3.c like this, please do -- if this wasn't your patch, ESR would take the blame for coalescing several patches under a bogus comment. --- pop3.c (Revision 3872) +++ pop3.c (Revision 3873) @@ -365,7 +365,12 @@ * These authentication methods are blessed by RFC1734, * describing the POP3 AUTHentication command. */ - if (ctl->server.authenticate == A_ANY) + if ((ctl->use_ssl != FLAG_FALSE) || + (ctl->server.authenticate == A_ANY) || + (ctl->server.authenticate == A_GSSAPI) || + (ctl->server.authenticate == A_KERBEROS_V4) || + (ctl->server.authenticate == A_OTP) || + (ctl->server.authenticate == A_CRAM_MD5)) ... ]] [[ Rob, I made inofficial snapshots from Subversion for testing on 2004-11-10, named fetchmail 6.2.6, and 2005-03-01, named fetchmail 6.2.5.991, for the translation project These revealed the regression. BTW, I have not yet heard back from the translation project :-(( ]] David Greaves <da...@dg...> writes: > I'm upgrading from 6.2.5 to 6.2.6 and I've encountered a possible bug > where 6.2.6 resends a CAPA challenge even when the server doesn't > understand it. The ISP is a major UK one. The problem is really that fetchmail replaces the server.authenticate member, but the use_ssl flag overrides this. Besides that, FLAG_FALSE is not a state that we can be in at this time, as the DEFAULT() macro in fetchmail.c will have replaced it by FALSE at this time. I don't currently see why an SSL-wrapped connection alone would require CAPA, so I'll just remove this bogus check. OTOH, this (code with patch) may still not be flawed because falling back to USER/PASS when the user has configured a more secure authentication mechanism is bogus and a security risk. If the user configures something strong and it fails, so be it. I don't see why we would use CAPA for anything but the _ANY, and I don't find Nalin's post that got sent to ESR somewhen before January 2004, so I presume the patch was sent privately. Please try this band-aid patch: Index: pop3.c =================================================================== --- pop3.c (Revision 4024) +++ pop3.c (Arbeitskopie) @@ -361,8 +361,7 @@ * These authentication methods are blessed by RFC1734, * describing the POP3 AUTHentication command. */ - if ((ctl->use_ssl != FLAG_FALSE) || - (ctl->server.authenticate == A_ANY) || + if ((ctl->server.authenticate == A_ANY) || (ctl->server.authenticate == A_GSSAPI) || (ctl->server.authenticate == A_KERBEROS_V4) || (ctl->server.authenticate == A_OTP) || -- Matthias Andree |