From: Matthias A. <mat...@gm...> - 2011-05-23 21:00:29
|
Greetings, Sunil has worked on handling large IMAP mailboxes. I have been working on the STARTTLS hang problem reported by Thomas Jarosch, which should be fixed now, along with assorted minor protocol nits that were picked. I am not sure if I want/need a CVE for a denial of service that is OS dependent. Opinions solicited. Please test fetchmail 6.3.20-pre1 on your operating system. To do that, please: 1. download and unpack the fetchmail tarball (URLs below) 2. cd to to the unpacked directory 3. ./configure and install as usual 4. run fetchmail with these additional options: --auth any -vvvd0 --nodetach --nosyslog 5. report success or failure to the list or me personally. PLEASE HELP: If you can offer access to test servers that I can send a short test mail to and then log into to retrieve that test message - particularly Exchange 2007 or Exchange 2010 is desired, but others besides Cyrus IMAP and Dovecot are also welcome - please let me know. PLEASE HELP: fetchmail needs translators for the program strings. Some languages (such as those shown below) are in quite good shape, but others are lacking a bit. Translation information can be found at <http://translationproject.org/domain/fetchmail.html> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ DOWNLOAD this beta software from: <http://home.pages.de/~mandree/fetchmail/> The repository can be browsed at and cloned from: <http://gitorious.org/fetchmail/fetchmail> - the branch is "legacy_63". Git (the software used to keep the fetchmail source code version controlled) information is at: <http://git-scm.com/> CHANGES since the previous formal release of fetchmail listed below. Unless otherwise noted, the changes were made by Matthias Andree: # SECURITY FIXES * Fetchmail's socket timeout handling was incomplete. Network outages in the wrong phase of a communication, combined with unlucky operating systems and their defaults, could cause fetchmail to hang for extended amounts of time. Freezes for beyond a week were reported by Thomas Jarosch. Fetchmail sets UNIX- and Internet-domain socket send and receive timeouts now. This fixes a hang during STARTTLS negotiation reported by Thomas Jarosch. # CHANGES * fetchmail now always uses its own MD5 implementation. The library and header variants are too diverse, and we've been bitten before -- and configure complains noisily on Cyrus-SASL's RFC1321 md5.h. * fetchmail now supports an environment variable to suppress marking deleted messages as seen at the same time, FETCHMAIL_IMAP_DELETED_REMAINS_UNSEEN. See the manual page for details. Requested by Jonathan Buschmann. * fetchmail sets Internet domain sockets to "keepalive" mode now. Note that there is no portable way to configure actual timeouts for this mode, and some systems only support a system-wide timeout setting. # BUG FIXES * Call strlen() only once when removing CRLF from a line. (Sunil Shetye) * Do not search for UNSEEN messages in ranges. Usually, there are very few new messages and most of the range searches result in nothing. Instead, split the long response to make the IMAP driver think that there are multiple lines of response. (Sunil Shetye) * Do not print "skipping message" for old messages even in verbose mode. If there are too many old messages, the logs just get filled without any real activity. (Sunil Shetye) (suggested by Yunfan Jiang) # TRANSLATION UPDATES [de] German (Matthias Andree) [ja] Japanese (Takeshi Hamasaki) -- Matthias Andree |
From: Thomas J. <tho...@in...> - 2011-05-24 17:22:44
|
Hello Matthias, On Monday, 23. May 2011 21:00:27 Matthias Andree wrote: > DOWNLOAD this beta software from: > <http://home.pages.de/~mandree/fetchmail/> Small side note: I had trouble downloading the .tar.bz2 version, the tar.xz version downloaded just fine. > # SECURITY FIXES > * Fetchmail's socket timeout handling was incomplete. Network outages in > the wrong phase of a communication, combined with unlucky operating > systems and their defaults, could cause fetchmail to hang for extended > amounts of time. Freezes for beyond a week were reported by Thomas > Jarosch. Fetchmail sets UNIX- and Internet-domain socket send and > receive timeouts now. This fixes a hang during STARTTLS negotiation > reported by Thomas Jarosch. The timeout works fine, I've tested every step of the POP3 protocol communication until we triggered the bug the last time. I think I've seen a kind of unrelated bug: If you let the TLS negotation time out on the "server side", fetchmail will proceed to send the user name before shutting down: "USER xyz". Bug in the state machine? > # CHANGES > * fetchmail now supports an environment variable to suppress marking Out of curiosity, why is this an environment variable instead of a configuration option? Cheers, Thomas |
From: Matthias A. <mat...@gm...> - 2011-05-24 22:30:46
|
Am 24.05.2011 17:22, schrieb Thomas Jarosch: > On Monday, 23. May 2011 21:00:27 Matthias Andree wrote: >> DOWNLOAD this beta software from: >> <http://home.pages.de/~mandree/fetchmail/> > > Small side note: I had trouble downloading the .tar.bz2 version, > the tar.xz version downloaded just fine. Hi Thomas, just tried downloading the .tar.bz2 from a different computer, no troubles here. What were the symptoms of the problem? >> # SECURITY FIXES >> * Fetchmail's socket timeout handling was incomplete. Network outages in >> the wrong phase of a communication, combined with unlucky operating >> systems and their defaults, could cause fetchmail to hang for extended >> amounts of time. Freezes for beyond a week were reported by Thomas >> Jarosch. Fetchmail sets UNIX- and Internet-domain socket send and >> receive timeouts now. This fixes a hang during STARTTLS negotiation >> reported by Thomas Jarosch. > > The timeout works fine, I've tested every step of the POP3 protocol > communication until we triggered the bug the last time. Thanks. > I think I've seen a kind of unrelated bug: If you let the TLS negotation > time out on the "server side", fetchmail will proceed to send the user name > before shutting down: "USER xyz". Bug in the state machine? Probably. I suspect now that we get socket-level errors in a few places, we're ferreting up such protocol bugs that used to be concealed behind a freeze or possibly an alarm timer followed by siglongjmp. I've now started wondering, considering Jonathan's trouble that the setsockopt() isn't working on his kernel, if the fixes in 6.3.20-pre1 are heading the right way, or if I need to use setitimer()-based timeouts (providing that OpenSSL doesn't mask SIGALRM), and possibly avoid set_timeout(0) (which in itself may open up yet another can of worms). >> # CHANGES >> * fetchmail now supports an environment variable to suppress marking > > Out of curiosity, why is this an environment variable > instead of a configuration option? It was sort-of experimental and is much less effort that way -- and turned out to be non-functional. Perhaps I should remove it (or at least the documentation). First-class options need changes in half a dozen places, among them the lexer and parser (rcfile*.l and .y), the command-line parser, help output, manual page, fetchmailconf -- whereas the environment option just needs changes to the code in one place and to the man page. Thanks for the test and reports. Time to get back to the drawing board for me. Best regards, -- Matthias Andree |
From: Thomas J. <tho...@in...> - 2011-05-26 17:17:59
|
On Tuesday, 24. May 2011 22:30:45 Matthias Andree wrote: > just tried downloading the .tar.bz2 from a different computer, no > troubles here. What were the symptoms of the problem? The download just stalled. The issue appears using konqueror only, firefox downloads the file just fine. So let's forget about this. > > I think I've seen a kind of unrelated bug: If you let the TLS > > negotation time out on the "server side", fetchmail will proceed to > > send the user name before shutting down: "USER xyz". Bug in the state > > machine? > > Probably. I suspect now that we get socket-level errors in a few > places, we're ferreting up such protocol bugs that used to be concealed > behind a freeze or possibly an alarm timer followed by siglongjmp. Makes sense. > >> * fetchmail now supports an environment variable to suppress marking > > > > Out of curiosity, why is this an environment variable > > instead of a configuration option? > > It was sort-of experimental and is much less effort that way -- and > turned out to be non-functional. Perhaps I should remove it (or at > least the documentation). > > First-class options need changes in half a dozen places, among them the > lexer and parser (rcfile*.l and .y), the command-line parser, help > output, manual page, fetchmailconf -- whereas the environment option > just needs changes to the code in one place and to the man page. Those easy "hacks" have a tendency to stay around forever :o) If it's non-functional, it's worth removing I'd say. Thomas |