fetchmail-devel — Development related discussion

[fetchmail-devel] [Bug #18794] BUG in libdispatch client - OSX ML 10.8.2

[<ad...@be...>]

Bug #18794, was updated on 2012-Nov-14 11:12
Here is a current snapshot of the bug.

Project: fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: madman1234
Assigned to : m-a
Summary: BUG in libdispatch client - OSX ML 10.8.2

Details: I installed fetchmail using mac ports. But everytime I start fetchmail (SSL included), I receive the following message in /var/log/mail.log

[...]
BUG in libdispatch client: Do not close random Unix descriptors
[...]

I have compiled the latest fetchmail version by myself, but I receive the same message. 

Fetchmail itself works, but what does it means? 

If I should ran any debugs, just let me know :)

Thanks
Soeren

Follow-Ups:

Date: 2012-Nov-15 08:08
By: m-a

Comment:
Some questions and notes:

1. What is libdispatch?

2. Why is fetchmail on macports linking to it?

3. Why does libdispatch require an application to keep random file descriptors open?

4. If fetchmail spams the logs due to macports specific, why do the macports involved people believe this bug should be reported upstream?

Note #1: Fetchmail deliberately closes unneeded file descriptors in a tradition of daemonizing (going to background and detaching from the controlling TTY/terminal) with a clean environment, so as to leave as few attack vectors as possible - closing foreign file descriptors fetchmail may have inherited from the invoking process avoids collateral damage in the case that a bug were to be discovered that let fetchmail write to a random file descriptor (no such bug is known in the latest release version at this time), and it also avoids an incomplete detachment.

Note #2: fetchmail is single-threaded, so if libdispatch is about multithreading, macports should not be using it.

Note #3, referring to the macports bug: getmail does not (can not, due to missing features in the Python library that getmail is using) detect man in the middle attacks on SSL/TLS connections. fetchmail does, if used with --sslcertck.

5. As a workaround, please try and see if running fetchmail with "-d 0" avoids the problem.

Please report back with answers to these four questions, and possibly with macports statements to the notes.
-------------------------------------------------------

Date: 2012-Nov-14 11:27
By: madman1234

Comment:
Here is the answer from macports:

[...]
http://trac.macports.org/ticket/36984#comment:3

This message is printed by libdispatch (and is thus Apple-specific). Apparently fetchmail closes file descriptors that are used by libdispatch, which terminates the process using this message when this happens. I guess one would have to attach a debugger, see where the file descriptor is closed and see whether this actually is a "random" file descriptor and what can be done about it. The maintainer might also consider forwarding this to upstream fetchmail. 
[...]

Do you have such a debug script? 

Thanks

-------------------------------------------------------

For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=18794&group_id=1824

[fetchmail-devel] [Bug #18794] BUG in libdispatch client - OSX ML 10.8.2

[<ad...@be...>]

Bug #18794, was updated on 2012-Nov-14 01:12
Here is a current snapshot of the bug.

Project: fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: madman1234
Assigned to : none
Summary: BUG in libdispatch client - OSX ML 10.8.2

Details: I installed fetchmail using mac ports. But everytime I start fetchmail (SSL included), I receive the following message in /var/log/mail.log

[...]
BUG in libdispatch client: Do not close random Unix descriptors
[...]

I have compiled the latest fetchmail version by myself, but I receive the same message. 

Fetchmail itself works, but what does it means? 

If I should ran any debugs, just let me know :)

Thanks
Soeren

Follow-Ups:

Date: 2012-Nov-14 01:27
By: madman1234

Comment:
Here is the answer from macports:

[...]
http://trac.macports.org/ticket/36984#comment:3

This message is printed by libdispatch (and is thus Apple-specific). Apparently fetchmail closes file descriptors that are used by libdispatch, which terminates the process using this message when this happens. I guess one would have to attach a debugger, see where the file descriptor is closed and see whether this actually is a "random" file descriptor and what can be done about it. The maintainer might also consider forwarding this to upstream fetchmail. 
[...]

Do you have such a debug script? 

Thanks

-------------------------------------------------------

For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=18794&group_id=1824

[fetchmail-devel] [Bug #18794] BUG in libdispatch client - OSX ML 10.8.2

[<ad...@be...>]

Bug #18794, was updated on 2012-Nov-14 01:12
Here is a current snapshot of the bug.

Project: fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: madman1234
Assigned to : none
Summary: BUG in libdispatch client - OSX ML 10.8.2

Details: I installed fetchmail using mac ports. But everytime I start fetchmail (SSL included), I receive the following message in /var/log/mail.log

[...]
BUG in libdispatch client: Do not close random Unix descriptors
[...]

I have compiled the latest fetchmail version by myself, but I receive the same message. 

Fetchmail itself works, but what does it means? 

If I should ran any debugs, just let me know :)

Thanks
Soeren

For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=18794&group_id=1824

[fetchmail-devel] [Patch #3351] Support Multiple envelope headers

[<ad...@be...>]

Patch #3351 has been updated. 

Project: fetchmail
Category: None
Status: Open
Submitted by: cr212
Assigned to : none
Summary: Support Multiple envelope headers

Follow-Ups:

Date: 2012-Oct-17 04:27
By: cr212

Comment:
When an e-mail is addressed to two or more users at the same domain, my ISP stores a single message, with multiple X-Original-To headers, one per recipient.  This patch builds a list of all such headers and adds all recipients to the delivery target.

-------------------------------------------------------

-------------------------------------------------------
For more info, visit:

http://developer.berlios.de/patch/?func=detailpatch&patch_id=3351&group_id=1824

[fetchmail-devel] [Patch #3351] Support Multiple envelope headers

[<ad...@be...>]

Patch #3351 has been updated. 

Project: fetchmail
Category: None
Status: Open
Submitted by: cr212
Assigned to : none
Summary: Support Multiple envelope headers

-------------------------------------------------------
For more info, visit:

http://developer.berlios.de/patch/?func=detailpatch&patch_id=3351&group_id=1824

Re: [fetchmail-devel] [patch][RFC] proper kill management of the MDA process

[Matthias A. <mat...@gm...>]

Am 06.09.2012 14:12, schrieb Fabio Rossi:
>>> Am 09.03.2012 23:02, schrieb Fabio Rossi:
>>>
>>> Has anyone tried the patch? Is the implementation correct? As it solves a 
>>> problem with the current fetchmail release I think it's a starting point to 
> fix 
>>> the above described problem.
>>
>> Fabio,
>>
>> grazie mille.
>>
>> I've been meaning to look at the patch, but didn't get around to do much
>> fetchmail work in the past months; I have three issues pending
>> work/review for 6.3.22: (a) a change in OpenSSL compatibility options
>> (mostly done, in Git), (b) your MDA kill management (pending review),
>> (c) a fix for --plugin and "-f -" from a Debian user (in the Debian BTS,
>> pending review).
>>
>> Also, some translation updates have accumulated that will be included in
>> the next release.
> 
> Matthias, 
> I have just seen the latest fetchmail 6.3.22/7.0.0 releases but unfortunately 
> the patch hasn't been included. Is there any plan for the next release?

Fabio,

I have been looking at it today. Your approach is the right one, the
devil, however, hides in the details, as your original changelog hinted.

For instance, we lose file descriptors (from mda_pipe) and do not kill
the child if fdopen() or anything else fails during writing.  This would
hurt in daemon mode because we'd then run out of file descriptors sooner
or later if the machine is under load.

Another problem, the exit status is not properly fetched from the child.
It would seem that that the SIGCHLD handler runs asynchronously *before*
we wait_for_child(mda_pid), and with its "while (waitpid(-1, &status,
WONOHANG) > 0) continue;" steals the exit code. Try:

fetchmail --all --fetchlimit 3 --keep --mda "cat >/dev/null && exit 75"

- this "exit 75" goes unnoticed in my testing (it should postpone
messages -- check with the debugger; I have added --keep to protect the
innocent, which will force leaving the messages on the server).

I tend to believe that if fetchmail needs to use a SIGCHLD handler to
reap random (grand-)children, that there are bugs (possibly with
plugin/plugout) where the waitpid() was missing.  The global mda_pid may
be insufficient here, and we may need to pay more attention when we call
waitpid(-1, ...).

It would seem fine to call this at the end of a poll, in daemon-mode.

Also, I find using 0 as "unset" for mda_pid unfortunate, because
waitpid(0, ...) is valid and means "wait for children from our process
group".  Either we need a separate BOOL (flag) to state that mda_pid is
valid, or (what I've currently been experimenting with) use mda_pid =
MDA_PID_UNSET; with enum { MDA_PID_UNSET = 1 }; on the assumption that
we will never wait for the init(8) process (which is special in that it
always has PID #1).

This needs more attention, and I am not sure if I want to integrate this
into 6.3.

Best regards,
Matthias

Re: [fetchmail-devel] [patch][RFC] proper kill management of the MDA process

[Fabio R. <ro...@in...>]

>> Am 09.03.2012 23:02, schrieb Fabio Rossi:
>>
>> Has anyone tried the patch? Is the implementation correct? As it solves a 
>> problem with the current fetchmail release I think it's a starting point to 
fix 
>> the above described problem.
>
>Fabio,
>
>grazie mille.
>
>I've been meaning to look at the patch, but didn't get around to do much
>fetchmail work in the past months; I have three issues pending
>work/review for 6.3.22: (a) a change in OpenSSL compatibility options
>(mostly done, in Git), (b) your MDA kill management (pending review),
>(c) a fix for --plugin and "-f -" from a Debian user (in the Debian BTS,
>pending review).
>
>Also, some translation updates have accumulated that will be included in
>the next release.

Matthias, 
I have just seen the latest fetchmail 6.3.22/7.0.0 releases but unfortunately 
the patch hasn't been included. Is there any plan for the next release?

Best,
Fabio

[fetchmail-devel] fetchmail 7.0.0-alpha3 alpha preview release

[Matthias A. <mat...@gm...>]

Greetings,

in an effort to get sufficient testing, I have released the next alpha
version of fetchmail 7.0.0. This merges post-6.3.20 changes in, to
fix security (CVE-2012-3482) and otherwise important bugs; I am
including the changelog below.

7.0.0-alpha2+MAPI was not vulnerable to CVE-2011-3389.

The snapshot is without MAPI this time, because I cannot test the MAPI feature.

If you want to help out, please arrange an Exchange account for me that
I can send test messages to that I can retrieve via MAPI and IMAP.

The alpha version isn't available through BerliOS, but only from this
DOWNLOAD: <http://home.pages.de/~mandree/fetchmail/>

The corresponding git tag is SNAPSHOT_7-0-0-alpha3, the branch is
"master". The repository is at <http://gitorious.org/fetchmail>.

Please send feedback to fet...@li....

Happy fetches!
Matthias

--------------------------------------------------------------------------------
fetchmail-7.0.0 (not yet released):

NOTE THIS IS AN ALPHA RELEASE THAT HAS NOT BEEN THOROUGHLY TESTED!

# MAJOR CHANGES
* The UIDL handler code is now much faster, especially noticable with lots of
  mail kept on a POP3 server. Where the 6.3.X code was of O(n^2) complexity,
  we're down to O(n log n).
  Contributed by Rainer Weikusat, MAD Partners Ltd./MSS GmbH.
* The POP3 code now always uses UIDL, except if "fetchall" is in effect.
  Fixes BerliOS Bug #16172. Fixes Debian Bug#345788.
* Fetchmail now enables SSL support by default. If this is undesired,
  ./configure --without-ssl should help.
* The OpenSSL code now excludes the SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS option.
  This can cause interoperability problems with certain buggy servers, but is
  required to defang chosen-plaintext attacks against AES.  While probably hard
  to mount against fetchmail, let's play it safe rather than be sorry later.

# FEATURES ADDED
* Fetchmail can now retrieve credentials from PWMD. This needs to be enabled at
  compile-time and requires run-time configuration. See README.PWMD for details.
  Contributed by Ben Kibbey, author of libpwmd and pwmd.
* Fetchmail now supports a retrieve-error command line or rcfile option that
  takes exactly one argument, abort (default), continue or markseen.  This
  specifies the policy used by fetchmail to handle messages whose bodies
  fail to be retrieved due to server errors.  Both the continue and markseen
  options will skip the message with errors and allow the session to
  continue so that subsequent messages can be retrieved.  The markseen
  option will also mark the message with errors as seen.
  The default policy is to abort the session whenever a server error occurs.
  Contributed by Craig Brown.
* Fetchmailconf offers cram-md5 and apop authentication.

# REMOVED FEATURES
* IMAP2 protocol support was removed.
* POP2 protocol support was removed.
* RPOP (not actually a protocol, but a variant of POP3) was removed
* POP3: the uidl option has been removed. It is always on.
* POP3: LAST is no longer used. It was removed from POP3 in 1994, and it could
  cause mail loss when the connection was interrupted or if clients besides
  fetchmail polled the mailbox.
* Trio was removed, fetchmail expects reasonable stdio.h quality levels.
* Support for systems that do not conform to C89 and POSIX 2001 was removed,
  this means that BeOS, EMX, NeXTSTEP quirks are no longer worked around.
* The MX and host alias DNS lookups that fetchmail performs in multidrop mode
  have been removed. They were based on the mistaken assumption that the
  IMAP/POP3 server was also the MX server, which is rarely the case.  They have
  never supported IPv6 (including IPv6-mapped IPv4) either.
  Non-DNS based alias keywords such as "aka" remain.
* Kerberos IV support was removed.
* fetchmail no longer supports SSL v2, nor the corresponding SSL2 option to
  --sslproto. SSLv2 is insecure and had been deprecated 15 years ago. fetchmail
  will actively forbid SSLv2 negotiation by means of SSL_OP_NO_SSLv2.
  To fix Debian Bug#622054.
* A lot of outdated and/or unsafe-to-use material got dropped from contrib/.

# REGRESSION FIXES
* The mimedecode feature now properly detects multipart/mixed-type matches, so
  that quoted-printable-encoded multipart messages can get decoded.
  (Regression in 5.0.0 on 1999-03-27, as a side effect of a PGP-mimedecode fix
  attributed to Henrik Storner.)

# BUG FIXES
* The mimedecode feature failed to ship the last line of the body if it was
  encoded as quoted-printable and had a MIME soft line break in the very last
  line.  Reported by Lars Hecking in June 2011.
  Bug introduced on 1998-03-20 when the mimedecode support was added by ESR
  before release 4.4.1 through code contributed by Henrik Storner.
  Workaround for older releases: do not use mimedecode feature.
* Fetchmail now detects singly-quoted % expansions in the mda option and refuses
  to deliver for safety reasons. Fixes Debian Bug#347909.
* The Server certificate: message in verbose mode now appears on stdout like the
  remainder of the output. Reported by Henry Jensen, to fix Debian Bug #639807.

# CHANGES
* A foreground fetchmail can now accept a few more options while another copy is
  running in the background.
* APOP is no longer a protocol, but an authentication method. In order to use
  it, use protocol POP3 auth APOP, or on the commandline, -p pop3 --auth apop.
  If no authentication method is specified, APOP is automatically tried if
  offered by the server before we resort to sending the password as clear text.

# KNOWN BUGS AND WORKAROUNDS
  (This section floats upwards through the NEWS file so it stays with the
  current release information)
* Fetchmail does not handle messages without Message-ID header well
  (See sourceforge.net bug #780933)
* BSMTP is mostly untested and errors can cause corrupt output.
* Sun Workshop 6 (SPARC) is known to miscompile the configuration file lexer in
  64-bit mode.  Either compile 32-bit code or use GCC to compile 64-bit
  fetchmail.  Note that fetchmail doesn't take advantage of 64-bit code,
  so compiling 32-bit SPARC code should not cause any difficulties.
* Fetchmail does not track pending deletes across crashes.
* The command line interface is sometimes a bit stubborn, for instance,
  fetchmail -s doesn't work with a daemon running.
* Linux systems may return duplicates of an IP address in some circumstances if
  no or no global IPv6 addresses are configured.
  (No workaround. Ubuntu Bug#582585, Novell Bug#606980.)
* Kerberos 5 may be broken, particularly on Heimdal, and provide bogus error
  messages. This will not be fixed, because the maintainer has no Kerberos 5
  server to test against. Use GSSAPI.

--------------------------------------------------------------------------------

fetchmail-6.3.23 (not yet released)

# NOTE THAT THE RELEASE OF FUTURE FETCHMAIL 6.3.X VERSIONS IS UNCLEAR.
Should a 7.0 release be made earlier, chances are that the 6.3.X branch
is abandoned and its changes be folded into the 7.0 release, with changes
after 6.3.22 not available on their own in a newer 6.3.X release.

# REGRESSION FIXES
* Fix compilation with OpenSSL implementations before 0.9.8m that lack
  SSL_CTX_clear_options. Patch by Earl Chew.
  Note that the use of older OpenSSL versions with fetchmail is unsupported and
  *not* recommended.

# BUG FIXES
* Fix combination of --plugin and -f -. Patch by Alexander Zangerl,
  to fix Debian Bug#671294.


fetchmail-6.3.22 (released 2012-08-29, 26077 LoC):

# SECURITY FIXES
* for CVE-2012-3482:
  NTLM: fetchmail mistook an error message that the server sent in response to
  an NTLM request for protocol exchange, tried to decode it, and crashed while
  reading from a bad memory location.
  Also, with a carefully crafted NTLM challenge packet sent from the server, it
  would be possible that fetchmail conveyed confidential data not meant for the
  server through the NTLM response packet.
  Fix: Detect base64 decoding errors, validate the NTLM challenge, and abort
  NTLM authentication in case of error.
  See fetchmail-SA-2012-02.txt for further details.
  Reported by J. Porter Clark.

* for CVE-2011-3389:
  SSL/TLS (wrapped and STARTTLS): fetchmail used to disable a countermeasure
  against a certain kind of attack against cipher block chaining initialization
  vectors (SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS).
  Whether this creates an exploitable situation, depends on the server and the
  negotiated ciphers.
  As a precaution, fetchmail 6.3.22 enables the countermeasure, by clearing
  SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS.

  NOTE that this can cause connections to certain non-conforming servers to
  fail, in which case you can set the environment variable
  FETCHMAIL_DISABLE_CBC_IV_COUNTERMEASURE to any non-empty value when starting
  fetchmail to re-instate the compatibility option at the expense of security.

  Reported by Apple Product Security.

  For technical details, refer to <http://www.openssl.org/~bodo/tls-cbc.txt>.
  See fetchmail-SA-2012-01.txt for further details.

# BUG FIX
* The Server certificate: message in verbose mode now appears on stdout like the
  remainder of the output. Reported by Henry Jensen, to fix Debian Bug #639807.

* The GSSAPI-related autoconf code now matches gssapi.c better, and uses
  a different check to look for GSS_C_NT_HOSTBASED_SERVICE.
  This fixes the GSSAPI-enabled build on NetBSD 6 Beta.

# CHANGES
* On systems where SSLv2_client_method isn't defined in OpenSSL (such as
  newer Debian, and Ubuntu starting with 11.10 oneiric ocelot), don't
  reference it (to fix the build) and if configured, print a run-time error
  that the OS does not support SSLv2. Fixes Debian Bug #622054,
  but note that that bug report has a more thorough patch that does away with
  SSLv2 altogether.

* The security and errata notices fetchmail-{EN,SA}-20??-??.txt are now
  under the more relaxed CC BY-ND 3.0 license (the noncommercial clause
  was dropped). The Creative Commons address was updated.

* The Python-related Makefile.am parts were simplified to avoid an automake
  1.11.X bug around noinst_PYTHON, Automake Bug #10995.

* Configuring fetchmail without SSL now triggers a configure warning,
  and asks the user to consider running configure --with-ssl.

# WORKAROUND
* Some servers, notably Zimbra, return A1234 987 FETCH () in response to
  a header request, in the face of message corruption.  fetchmail now treats
  these as temporary errors. Report and Patch by Mikulas Patocka, Red Hat.

* Some servers, notably Microsoft Exchange, return "A0009 OK FETCH completed."
  without any header in response to a header request for meeting reminder
  messages (with a "meeting.ics" attachment). fetchmail now treats these as
  transient errors.  Report by John Connett, Patch by Sunil Shetye.

# TRANSLATION UPDATES
* [cs]    Czech, by Petr Pisar
* [de]    German
* [fr]    French, by Frédéric Marchal
* [ja]    Japanese, by Takeshi Hamasaki
* [pl]    Polish, by Jakub Bogusz
* [sv]    Swedish, by Göran Uddeborg --- NEW TRANSLATION - Thank you!
* [vi]    Vietnamese, by Trần Ngọc Quân


fetchmail-6.3.21 (released 2011-08-21, 26011 LoC):

# CRITICAL BUG FIX
* The IMAP client no longer inserts NUL bytes into the last line of a message
  when it is not closed with a LF or CRLF sequence. Reported by Antoine Levitt.
  As a side effect of the fix, and in order to avoid a full rewrite, fetchmail
  will now CRLF-terminate the last line fetched through IMAP, even if it is
  originally not terminated by LF or CRLF. This bears no relevance if your
  messages end up in mbox, but adds line termination for storages (like Maildir)
  that do not require that the last line be LF- or CRLF-terminated.

# CONTRIB/ addition
* There is a patch against fetchnews's source, contrib/rawlog.patch, that can
  log (and hexdump non-printing characters) raw socket data to a file. It proved
  useful to debug Antoine's bug described above.
--------------------------------------------------------------------------------

Re: [fetchmail-devel] PATCH: Clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS without SSL_CTX_clear_options()

[Earl C. <ear...@ya...>]

Matthias,

> I will queue your patch in Git now that it is there and that it fixes a regression, but whether there will
> be a fetchmail 6.3.23 release is open.
Yes, thanks. I understand.

Earl

Re: [fetchmail-devel] PATCH: Clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS without SSL_CTX_clear_options()

[Matthias A. <mat...@gm...>]

Am 03.09.2012 21:43, schrieb Earl Chew:
> Matthias,
> 
>> The question I am asking myself is: What use is there in supporting
>> "older implementations" (meaning before 0.9.8m)?  How many systems are
>> affected (and I would tend to ignore enterprise distributions - they
>> ought to patch OpenSSL instead).  I suppose those older implementations
>> would likely also be vulnerable to several other attacks, which might
>> subvert the effort of closing this (minor) hole.
> 
> Yes, a reasonable question to ask.
> 
> I think that there are two separate considerations:
> 
> a. The absence/presence of SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
> b. The absence/presence of SSL_CTX_clear_options()
> 
> 
> My reading is that (a) is the key to closing the vulnerability. The availability of SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
> indicates that the vulnerability is closed and it is important that SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
> is _not_ set in this scenario (except when overridden by FETCHMAIL_DISABLE_CBC_IV_COUNTERMEASURE).

Earl,

we agree on the assessment of how to close the vulnerability, and on
ways to leave the flag unset, assuming that OpenSSL defaults to "off"
(which it does for official 0.9.8 and 1.0.0 releases, and probably
future releases).

The SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS feature dates back to 2002, so we
can check that list item off.

Perhaps I can reword my question on how wide-spread the issue is a
little bit: are there open-source distributions that are still supported
by their "vendor" where this is a real-world problem, rather than just
continued use of an unsupported version?

> The patch I submitted keeps the vulnerability closed, but does not require SSL_CTX_clear_options().

That we also agree on - providing that nobody hacked the SSL defaults.

> As you have probably figured out, one of the OpenSSL installations that I have (1.0.0-0.10.beta3.fc12) supports
> SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS but does not support SSL_CTX_clear_options().

The "0.10.beta3" and "fc12" labels reveal that your package is from a
distribution (Fedora 12 "Constantine") that went unsupported almost two
years ago (*), and as such is not supported by fetchmail either.

Fedora 12 was using a pre-1.0.0 beta release of OpenSSL, and went out of
support before OpenSSL 1.0.0 got released.

I will queue your patch in Git now that it is there and that it fixes a
regression, but whether there will be a fetchmail 6.3.23 release is open.

(*)
<https://lists.fedoraproject.org/pipermail/announce/2010-December/002895.html>

[fetchmail-devel] RFC PATCH: Allow envelope to use regular expression to identify recipient

[Earl C. <ear...@ya...>]

Per my post in fetchmail-users (https://lists.berlios.de/pipermail/fetchmail-users/2012-September/003231.html)
here is a candidate patch to enable additional envelope functionality to better identify the intended
recipient in multidrop scenarios.


This patch approaches the enhancement in a straightforward way and is relatively unintrusive.


Apologies for the tab mangling in this RFC.  I can offer up a unmangled version if required to
commit the patch.

Earl



--- fetchmail.h.orig    2012-08-13 13:02:41.000000000 -0700
+++ fetchmail.h    2012-09-03 09:17:37.000000000 -0700
@@ -33,6 +33,7 @@
 #endif
 #include <netdb.h>
 #include <stdio.h>
+#include <regex.h>
 
 /* Import Trio if needed */
 #if !defined(HAVE_SNPRINTF) || !defined(HAVE_VSNPRINTF)
@@ -270,6 +271,7 @@
     int interval;            /* # cycles to skip between polls */
     int authenticate;            /* authentication mode to try */
     int timeout;            /* inactivity timout in seconds */
+    regex_t *envre;            /* envelope address list header regex */
     char *envelope;            /* envelope address list header */
     int envskip;            /* skip to numbered envelope header */
     char *qvirtual;            /* prefix removed from local user id */
--- transact.c.orig    2012-09-03 11:40:42.000000000 -0700
+++ transact.c    2012-09-03 13:04:12.000000000 -0700
@@ -104,6 +104,88 @@
     }
 }
 
+static int match_envelope(char *hdr,
+              struct query *ctl)
+/** compare the envelope with the RFC822 header and potentially rewrite */
+/** \param hdr         RFC822 header in question */
+/** \param ctl         envelope */
+{
+    static regex_t regex_disabled;
+
+    if (hdr == (char *)NULL ||
+    !ctl->server.envelope ||
+    ctl->server.envelope == STRING_DISABLED)
+    {
+       return 0;
+    }
+
+    if (ctl->server.envelope[0] != '^')
+    return strncasecmp(ctl->server.envelope,
+               hdr, strlen(ctl->server.envelope));
+
+    if (ctl->server.envre == (regex_t*)NULL)
+    {
+    int regerr;
+
+    if (outlevel >= O_DEBUG)
+        report(stdout,
+        GT_("compiling envelope regexp %s\n"), ctl->server.envelope);
+
+    ctl->server.envre = (regex_t *)xmalloc(sizeof(*ctl->server.envre));
+    regerr = regcomp(ctl->server.envre, ctl->server.envelope, REG_EXTENDED);
+
+    if (regerr)
+    {
+        report(stderr,
+        GT_("Error %d compling regex: %s\n"),
+        regerr, ctl->server.envelope);
+
+        xfree(ctl->server.envre);
+        ctl->server.envre = &regex_disabled;
+    }
+    }
+
+    if (ctl->server.envre != &regex_disabled)
+    {
+    regmatch_t regmatch[2];
+
+    int matcherr = regexec(ctl->server.envre, hdr, 2, regmatch, 0);
+
+    if (matcherr == 0)
+    {
+        if (regmatch[1].rm_so != -1)
+        {
+        int matchlen = regmatch[1].rm_eo - regmatch[1].rm_so;
+        char *hp = hdr + matchlen + 1;
+
+        memmove(hdr+1, hdr + regmatch[1].rm_so, matchlen);
+        hdr[0] = ':';
+
+        while (*hp != '\0' && *hp != '\r' && *hp != '\n')
+            *hp++ = ' ';
+
+        if (outlevel >= O_DEBUG)
+            report(stdout,
+            GT_("extracted envelope address %s\n"), hdr+1);
+        }
+
+        return 1;
+    }
+
+    if (matcherr != REG_NOMATCH)
+    {
+        char msgbuf[256];
+
+        regerror(matcherr, ctl->server.envre, msgbuf, sizeof(msgbuf));
+
+        report(stderr,
+        GT_("Error matching %s: %s\n"), ctl->server.envelope, msgbuf);
+    }
+    }
+
+    return 0;
+}
+
 static void find_server_names(const char *hdr,
                   struct query *ctl,
                   struct idlist **xmit_names)
@@ -936,9 +1018,7 @@
         if (ctl->server.envelope 
             && strcasecmp(ctl->server.envelope, "Received"))
         {
-            if (env_offs == -1 && !strncasecmp(ctl->server.envelope,
-                               line,
-                               strlen(ctl->server.envelope)))
+            if (env_offs == -1 && match_envelope(line, ctl))
             {                
             if (skipcount++ < ctl->server.envskip)
                 continue;

Re: [fetchmail-devel] PATCH: Clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS without SSL_CTX_clear_options()

[Earl C. <ear...@ya...>]

Matthias,

> The question I am asking myself is: What use is there in supporting
> "older implementations" (meaning before 0.9.8m)?  How many systems are
> affected (and I would tend to ignore enterprise distributions - they
> ought to patch OpenSSL instead).  I suppose those older implementations
> would likely also be vulnerable to several other attacks, which might
> subvert the effort of closing this (minor) hole.

Yes, a reasonable question to ask.

I think that there are two separate considerations:

a. The absence/presence of SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
b. The absence/presence of SSL_CTX_clear_options()


My reading is that (a) is the key to closing the vulnerability. The availability of SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
indicates that the vulnerability is closed and it is important that SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS
is _not_ set in this scenario (except when overridden by FETCHMAIL_DISABLE_CBC_IV_COUNTERMEASURE).

There are two obvious ways to achieve this. One way is to simply clear the bit in SSL_OP_ALL,
and for prior art, I refer you to:


http://www.opensource.apple.com/source/curl/curl-69.1/patches/curl-dont-insert-empty-fragments.patch


This brings me to (b) which use SSL_CTX_clear_options() as an alternate way to manipulate the options. But as
the prior art shows, it is entirely unnecessary in this scenario to use this new function.



> I suppose those older implementations
> would likely also be vulnerable to several other attacks, which might
> subvert the effort of closing this (minor) hole.


As I described above, the whole question of using SSL_CTX_clear_options() is a separate
one from the ability to close down the vulnerability.

The patch I submitted keeps the vulnerability closed, but does not require SSL_CTX_clear_options().

As you have probably figured out, one of the OpenSSL installations that I have (1.0.0-0.10.beta3.fc12) supports
SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS but does not support SSL_CTX_clear_options().


Earl

Re: [fetchmail-devel] PATCH: Clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS without SSL_CTX_clear_options()

[Matthias A. <mat...@gm...>]

Am 03.09.2012 17:24, schrieb Earl Chew:
> 
> 
> A patch to clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS was added recently:
> 
>   http://gitorious.org/fetchmail/fetchmail/commit/48809c5b9f6c9081f4031fa938dd63b060c18a4b?format=patch
> 
> Older implementations of OpenSSL do not support SSL_CTX_clear_options().

Earl,

Thanks a lot for the problem report and the patch.

The question I am asking myself is: What use is there in supporting
"older implementations" (meaning before 0.9.8m)?  How many systems are
affected (and I would tend to ignore enterprise distributions - they
ought to patch OpenSSL instead).  I suppose those older implementations
would likely also be vulnerable to several other attacks, which might
subvert the effort of closing this (minor) hole.

Any insights?

Best regards,
Matthias

[fetchmail-devel] PATCH: Clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS without SSL_CTX_clear_options()

[Earl C. <ear...@ya...>]

A patch to clear SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS was added recently:

  http://gitorious.org/fetchmail/fetchmail/commit/48809c5b9f6c9081f4031fa938dd63b060c18a4b?format=patch

Older implementations of OpenSSL do not support SSL_CTX_clear_options().



This patch reworks the previous change to avoid the use of SL_CTX_clear_options()
and instead clears the corresponding bit in SSL_OP_ALL before calling
SSL_CTX_set_options().



--- socket.c.orig       2012-08-13 13:02:41.000000000 -0700
+++ socket.c    2012-09-03 08:11:49.000000000 -0700
@@ -844,6 +844,7 @@
 {
         struct stat randstat;
         int i;
+        long sslopts = SSL_OP_ALL;

        SSL_load_error_strings();
        SSL_library_init();
@@ -899,14 +900,14 @@
                return(-1);
        }

-       SSL_CTX_set_options(_ctx[sock], SSL_OP_ALL);
-
        {
            char *tmp = getenv("FETCHMAIL_DISABLE_CBC_IV_COUNTERMEASURE");
            if (tmp == NULL || *tmp == '\0' || strspn(tmp, " \t") == strlen(tmp))
-               SSL_CTX_clear_options(_ctx[sock], SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS);
+               sslopts &= ~ SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS;
        }

+       SSL_CTX_set_options(_ctx[sock], sslopts);
+
        if (certck) {
                SSL_CTX_set_verify(_ctx[sock], SSL_VERIFY_PEER, SSL_ck_verify_callback);
        } else {

[fetchmail-devel] New Polish PO file for 'fetchmail' (version 6.3.21.1)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Polish team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/pl.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New Japanese PO file for 'fetchmail' (version 6.3.21.1)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Japanese team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/ja.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New Swedish PO file for 'fetchmail' (version 6.3.21.1)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Swedish team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/sv.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New French PO file for 'fetchmail' (version 6.3.21.1)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the French team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/fr.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New Czech PO file for 'fetchmail' (version 6.3.21.1)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Czech team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/cs.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New Vietnamese PO file for 'fetchmail' (version 6.3.21.1)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Vietnamese team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/vi.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

[fetchmail-devel] New template for 'fetchmail' made available

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.  (If you have
any questions, send them to <coo...@tr...>.)

A new POT file for textual domain 'fetchmail' has been made available
to the language teams for translation.  It is archived as:

    http://translationproject.org/POT-files/fetchmail-6.3.21.1.pot

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

Below is the URL which has been provided to the translators of your
package.  Please inform the translation coordinator, at the address
at the bottom, if this information is not current:

    http://home.pages.de/~mandree/fetchmail/fetchmail-6.3.21.1.tar.bz2

We can arrange things so that translated PO files are automatically e-mailed
to you when they arrive.  Ask at the address below if you want this.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

Re: [fetchmail-devel] [patch][RFC] proper kill management of the MDA process

[Fabio R. <ro...@in...>]

>mat...@gm... wrote on 03/05/2012 8.18
>
>I've been meaning to look at the patch, but didn't get around to do much
>fetchmail work in the past months; I have three issues pending
>work/review for 6.3.22: (a) a change in OpenSSL compatibility options
>(mostly done, in Git), (b) your MDA kill management (pending review),
>(c) a fix for --plugin and "-f -" from a Debian user (in the Debian BTS,
>pending review).

Thanks for giving a status update! I guess the patch needs some care but I 
look forward in having it integrated in the next release :-)

Best,
Fabio

Re: [fetchmail-devel] [patch][RFC] proper kill management of the MDA process

[Matthias A. <mat...@gm...>]

Am 09.03.2012 23:02, schrieb Fabio Rossi:

> Has anyone tried the patch? Is the implementation correct? As it solves a 
> problem with the current fetchmail release I think it's a starting point to fix 
> the above described problem.

Fabio,

grazie mille.

I've been meaning to look at the patch, but didn't get around to do much
fetchmail work in the past months; I have three issues pending
work/review for 6.3.22: (a) a change in OpenSSL compatibility options
(mostly done, in Git), (b) your MDA kill management (pending review),
(c) a fix for --plugin and "-f -" from a Debian user (in the Debian BTS,
pending review).

Also, some translation updates have accumulated that will be included in
the next release.

Best regards,
Matthias

[fetchmail-devel] New Vietnamese PO file for 'fetchmail' (version 6.3.20-rc3)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'fetchmail' has been submitted
by the Vietnamese team of translators.  The file is available at:

    http://translationproject.org/latest/fetchmail/vi.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    http://translationproject.org/latest/fetchmail/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    http://translationproject.org/domain/fetchmail.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

Re: [fetchmail-devel] [patch][RFC] proper kill management of the MDA process

[Fabio R. <ro...@in...>]

>In the past there was a discussion with subject "the message fetch should be 
>completed before quitting" on the fetchmail-user mailing list. It was clear 
the 
>possibility of delivering incomplete mail messages when fetchmail is 
>interrupted during a mail dispatching process. The issue doesn't lead to a 
loss 
>of mail but produces some garbage in the maildir/mailbox (corrupted 
duplicated 
>messages).
>
>The problem is related to the use of the popen() function which doesn't 
>provide the PID number of the MDA process. In this way is not possible to 
track 
>properly the MDA process, i.e. to kill it in response to a quit command 
issued 
>to fetchmail. Using the killpg() in response to the SIGINT (or SIGTERM) 
signal 
>doesn't help because it kills the parent process before killing the child 
>process.
>
>Here is attached a first proposal to solve the issue, it seems to work on my 
>system. As I don't know the fetchmail code I'm pretty sure there are some 
>mistakes. First of all I don't understand the reason in the original 
>release_sink() function there is no check of the return from popen(). For 
this 
>reason I kept the same behavior with the new implementation. I also modified 
>the SIGCHLD handler to reuse the waiting procedure for the forked MDA 
process.

Has anyone tried the patch? Is the implementation correct? As it solves a 
problem with the current fetchmail release I think it's a starting point to fix 
the above described problem.

Fabio