fetchmail-users — User questions and discussion

[Fetchmail-users] SSL error

[Joe Acquisto-j. <jo...@j4...>]

Getting this:

------------------------------------------------
Thu Feb 11 19:15:08 EST 2021
fetchmail: OpenSSL reported: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
fetchmail: mail.myisphost.com: SSL connection failed.
fetchmail: socket error while fetching from in...@j4...@mail.myisphost.com
fetchmail: Query status=2 (SOCKET)
----------------------------------------------

This is fetchmail release 6.4.2-rc3+SSL-SSLv2-SSLv3+NLS.

I was getting the error earlier today.  I think I somehow munged the .fetchmailrc file at some point and created an issue. 
At this point, though, I am dazed and confused.  (we will not speak of my arrogance in not making a backup of
the file before hammering away at it.  No definitely we will not mention that. . . .)

I will be periodically checking mail in a round about way,as my "normal" way is, umm, broken. . . .

joe a.

Re: [Fetchmail-users] ANNOUNCE: The 6.4.16 release of fetchmail is available (bug fixes, --version prints trust store paths)

[Matthias A. <mat...@gm...>]

Am 10.02.21 um 01:23 schrieb Joe Acquisto-j4:
> Sourceforge reports "Whoops, we can't find that page."

Hi Joe,

you are right, the URLs for the download .tar.xz/.tar.lz were broken.
I've re-sent the announcement with corrected URLs.

Thanks for the heads-up!

Cheers,
Matthias

[Fetchmail-users] ANNOUNCE: The 6.4.16 release of fetchmail is available (correction #1, URL in announcement)

[Matthias A. <mat...@gm...>]

[re-send with corrected download URLs]

Greetings,

The 6.4.16 release of fetchmail is now available at the usual locations,
including <https://sourceforge.net/projects/fetchmail/branch_6.4/>.

It contains a few bug fixes and is more willing to give information on the
trust store paths it is using (with -V or --version).

The source archive is available at:
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.tar.xz/download>
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.tar.lz/download>

Detached GnuPG signatures for the respective tarballs are at:
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.tar.xz.asc/download>
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.tar.lz.asc/download>

SHA256 hash values for the tarballs:
SHA256(fetchmail-6.4.16.tar.lz)= 0cc50212d62a7c9912e0c7fdf795ba205db554195d05a45e36c94671ec54c089
SHA256(fetchmail-6.4.16.tar.xz)= 044b9a0ac03afbae7744979defe3e2e32e39141bca68fd0c8deda2ed40884fb9

Here are the release notes:

---------------------------------------------------------------------------------
fetchmail-6.4.16 (released 2021-02-08, 27707 LoC):

# BUG FIXES
* fetchmail's --configdump, and fetchmailconf, lacked support for the
  sslcertfile option. --configdump support added by Earl Chew,
  Gitlab issue #25, merge request !28.
* fetchmail's manual page was never updated to reflect 6.2.5's change about the
  duplicate-killer code for multidrop mode, which read
  "* Dup-killer code now keys on an MD5 hash of the raw headers."
  ...instead of just the Message-ID. [commit 9dd8400, 2003-10-10 by esr]
  The manual page was now updated accordingly and documents
  historic behaviour:
  start to 5.0.7 no duplicate suppression;
  5.0.8 to 6.2.4 duplicate suppression only by Message-ID;
  6.2.5 to 6.4.X duplicate suppression by entire raw header.
  Manpage bug found by Julian Bane debugging "duplicate message" behaviour.
* ./configure no longer runs AC_LIB_LINKFLAGS (how to link) checks
  when called --without-ssl

# FEATURES
* fetchmail --version [fetchmail -V] now queries and prints the SSL/TLS
  library's "SSL default trusted certificate" file or directory (mind the word
  "default"), where the OpenSSL-compatible TLS implementation will look for
  trusted root, meaning certification authority (CA), certificates.
  NOTE 1: watch the output carefully if the line prints the defaults
  or the configured path (without "default").
  NOTE 2: SSL_CERT_DIR and SSL_CERT_FILE are documented environment variables
  for OpenSSL 1.1.1 to override the *default* locations (those compiled into
  OpenSSL or possibly in its configuration file).
  This was added when Gene Heskett was debugging his setup and the
  information "where does OpenSSL look" was missing.
* fetchmail --version now prints version of the OpenSSL library that
  it was compiled against, and that it is using at runtime, and also
  the OPENSSL_DIR and OPENSSL_ENGINES_DIR (if available).

# TRANSLATION UPDATES
  These fine people have contributed updated translations for fetchmail,
  in no particular order:
* sq:    Besnik Bleta [Albanian]
* eo:    Keith Bowes [Esperanto]
* cs:    Petr Pisar [Czech]
* pl:    Jakub Bogusz [Polish]
* sv:    Göran Uddeborg [Swedish]
* fr:    Frédéric Marchal [French]

# KNOWN BUGS AND WORKAROUNDS
  (This section floats upwards through the NEWS file so it stays with the
  current release information)
* Fetchmail does not handle messages without Message-ID header well
  (See sourceforge.net bug #780933)
* Fetchmail currently uses 31-bit signed integers in several places
  where unsigned and/or wider types should have been used, for instance,
  for mailbox sizes, and misreports sizes of 2 GibiB and beyond.
  Fixing this requires C89 compatibility to be relinquished.
* BSMTP is mostly untested and errors can cause corrupt output.
* Fetchmail does not track pending deletes across crashes.
* The command line interface is sometimes a bit stubborn, for instance,
  fetchmail -s doesn't work with a daemon running.
* Linux systems may return duplicates of an IP address in some circumstances if
  no or no global IPv6 addresses are configured.
  (No workaround. Ubuntu Bug#582585, Novell Bug#606980.)
* Kerberos 5 may be broken, particularly on Heimdal, and provide bogus error
  messages. This will not be fixed, because the maintainer has no Kerberos 5
  server to test against. Use GSSAPI.

---------------------------------------------------------------------------------

Happy fetches,
Matthias

Re: [Fetchmail-users] ANNOUNCE: The 6.4.16 release of fetchmail is available (bug fixes, --version prints trust store paths)

[Joe Acquisto-j. <jo...@j4...>]

Sourceforge reports "Whoops, we can't find that page."

> Greetings,
> 
> The 6.4.16 release of fetchmail is now available at the usual
locations,
> including <https://sourceforge.net/projects/fetchmail/branch_6.4/>.
> 
> It contains a few bug fixes and is more willing to give information
on the
> trust store paths it is using (with -V or --version).
> 
> The source archive is available at:
>
<https://sourceforge.net/projects/fetchmail/branch_6.4/fetchmail-6.4.16.tar.xz

> /download>
>
<https://sourceforge.net/projects/fetchmail/branch_6.4/fetchmail-6.4.16.tar.lz

> /download>
> 
> Detached GnuPG signatures for the respective tarballs are at:
>
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.
> tar.xz.asc/download>
>
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.
> tar.lz.asc/download>
> 
> SHA256 hash values for the tarballs:
> SHA256(fetchmail-6.4.16.tar.lz)= 
> 0cc50212d62a7c9912e0c7fdf795ba205db554195d05a45e36c94671ec54c089
> SHA256(fetchmail-6.4.16.tar.xz)= 
> 044b9a0ac03afbae7744979defe3e2e32e39141bca68fd0c8deda2ed40884fb9
> 
> Here are the release notes:
> 
>
---------------------------------------------------------------------------------
> fetchmail-6.4.16 (released 2021-02-08, 27707 LoC):
> 
> # BUG FIXES
> * fetchmail's --configdump, and fetchmailconf, lacked support for
the
>   sslcertfile option. --configdump support added by Earl Chew,
>   Gitlab issue #25, merge request !28.
> * fetchmail's manual page was never updated to reflect 6.2.5's change
about 
> the
>   duplicate-killer code for multidrop mode, which read
>   "* Dup-killer code now keys on an MD5 hash of the raw headers."
>   ...instead of just the Message-ID. [commit 9dd8400, 2003-10-10 by
esr]
>   The manual page was now updated accordingly and documents
>   historic behaviour:
>   start to 5.0.7 no duplicate suppression;
>   5.0.8 to 6.2.4 duplicate suppression only by Message-ID;
>   6.2.5 to 6.4.X duplicate suppression by entire raw header.
>   Manpage bug found by Julian Bane debugging "duplicate message"
behaviour.
> * ./configure no longer runs AC_LIB_LINKFLAGS (how to link) checks
>   when called --without-ssl
> 
> # FEATURES
> * fetchmail --version [fetchmail -V] now queries and prints the
SSL/TLS
>   library's "SSL default trusted certificate" file or directory (mind
the 
> word
>   "default"), where the OpenSSL-compatible TLS implementation will
look for
>   trusted root, meaning certification authority (CA), certificates.
>   NOTE 1: watch the output carefully if the line prints the defaults
>   or the configured path (without "default").
>   NOTE 2: SSL_CERT_DIR and SSL_CERT_FILE are documented environment 
> variables
>   for OpenSSL 1.1.1 to override the *default* locations (those
compiled into
>   OpenSSL or possibly in its configuration file).
>   This was added when Gene Heskett was debugging his setup and the
>   information "where does OpenSSL look" was missing.
> * fetchmail --version now prints version of the OpenSSL library that
>   it was compiled against, and that it is using at runtime, and also
>   the OPENSSL_DIR and OPENSSL_ENGINES_DIR (if available).
> 
> # TRANSLATION UPDATES
>   These fine people have contributed updated translations for
fetchmail,
>   in no particular order:
> * sq:    Besnik Bleta [Albanian]
> * eo:    Keith Bowes [Esperanto]
> * cs:    Petr Pisar [Czech]
> * pl:    Jakub Bogusz [Polish]
> * sv:    Göran Uddeborg [Swedish]
> * fr:    Frédéric Marchal [French]
> 
> # KNOWN BUGS AND WORKAROUNDS
>   (This section floats upwards through the NEWS file so it stays with
the
>   current release information)
> * Fetchmail does not handle messages without Message-ID header well
>   (See sourceforge.net bug #780933)
> * Fetchmail currently uses 31-bit signed integers in several places
>   where unsigned and/or wider types should have been used, for
instance,
>   for mailbox sizes, and misreports sizes of 2 GibiB and beyond.
>   Fixing this requires C89 compatibility to be relinquished.
> * BSMTP is mostly untested and errors can cause corrupt output.
> * Fetchmail does not track pending deletes across crashes.
> * The command line interface is sometimes a bit stubborn, for
instance,
>   fetchmail -s doesn't work with a daemon running.
> * Linux systems may return duplicates of an IP address in some
circumstances 
> if
>   no or no global IPv6 addresses are configured.
>   (No workaround. Ubuntu Bug#582585, Novell Bug#606980.)
> * Kerberos 5 may be broken, particularly on Heimdal, and provide
bogus error
>   messages. This will not be fixed, because the maintainer has no
Kerberos 5
>   server to test against. Use GSSAPI.
> 
>
---------------------------------------------------------------------------------
> 
> Happy fetches,
> Matthias

[Fetchmail-users] ANNOUNCE: The 6.4.16 release of fetchmail is available (bug fixes, --version prints trust store paths)

[Matthias A. <mat...@gm...>]

Greetings,

The 6.4.16 release of fetchmail is now available at the usual locations,
including <https://sourceforge.net/projects/fetchmail/branch_6.4/>.

It contains a few bug fixes and is more willing to give information on the
trust store paths it is using (with -V or --version).

The source archive is available at:
<https://sourceforge.net/projects/fetchmail/branch_6.4/fetchmail-6.4.16.tar.xz/download>
<https://sourceforge.net/projects/fetchmail/branch_6.4/fetchmail-6.4.16.tar.lz/download>

Detached GnuPG signatures for the respective tarballs are at:
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.tar.xz.asc/download>
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16.tar.lz.asc/download>

SHA256 hash values for the tarballs:
SHA256(fetchmail-6.4.16.tar.lz)= 0cc50212d62a7c9912e0c7fdf795ba205db554195d05a45e36c94671ec54c089
SHA256(fetchmail-6.4.16.tar.xz)= 044b9a0ac03afbae7744979defe3e2e32e39141bca68fd0c8deda2ed40884fb9

Here are the release notes:

---------------------------------------------------------------------------------
fetchmail-6.4.16 (released 2021-02-08, 27707 LoC):

# BUG FIXES
* fetchmail's --configdump, and fetchmailconf, lacked support for the
  sslcertfile option. --configdump support added by Earl Chew,
  Gitlab issue #25, merge request !28.
* fetchmail's manual page was never updated to reflect 6.2.5's change about the
  duplicate-killer code for multidrop mode, which read
  "* Dup-killer code now keys on an MD5 hash of the raw headers."
  ...instead of just the Message-ID. [commit 9dd8400, 2003-10-10 by esr]
  The manual page was now updated accordingly and documents
  historic behaviour:
  start to 5.0.7 no duplicate suppression;
  5.0.8 to 6.2.4 duplicate suppression only by Message-ID;
  6.2.5 to 6.4.X duplicate suppression by entire raw header.
  Manpage bug found by Julian Bane debugging "duplicate message" behaviour.
* ./configure no longer runs AC_LIB_LINKFLAGS (how to link) checks
  when called --without-ssl

# FEATURES
* fetchmail --version [fetchmail -V] now queries and prints the SSL/TLS
  library's "SSL default trusted certificate" file or directory (mind the word
  "default"), where the OpenSSL-compatible TLS implementation will look for
  trusted root, meaning certification authority (CA), certificates.
  NOTE 1: watch the output carefully if the line prints the defaults
  or the configured path (without "default").
  NOTE 2: SSL_CERT_DIR and SSL_CERT_FILE are documented environment variables
  for OpenSSL 1.1.1 to override the *default* locations (those compiled into
  OpenSSL or possibly in its configuration file).
  This was added when Gene Heskett was debugging his setup and the
  information "where does OpenSSL look" was missing.
* fetchmail --version now prints version of the OpenSSL library that
  it was compiled against, and that it is using at runtime, and also
  the OPENSSL_DIR and OPENSSL_ENGINES_DIR (if available).

# TRANSLATION UPDATES
  These fine people have contributed updated translations for fetchmail,
  in no particular order:
* sq:    Besnik Bleta [Albanian]
* eo:    Keith Bowes [Esperanto]
* cs:    Petr Pisar [Czech]
* pl:    Jakub Bogusz [Polish]
* sv:    Göran Uddeborg [Swedish]
* fr:    Frédéric Marchal [French]

# KNOWN BUGS AND WORKAROUNDS
  (This section floats upwards through the NEWS file so it stays with the
  current release information)
* Fetchmail does not handle messages without Message-ID header well
  (See sourceforge.net bug #780933)
* Fetchmail currently uses 31-bit signed integers in several places
  where unsigned and/or wider types should have been used, for instance,
  for mailbox sizes, and misreports sizes of 2 GibiB and beyond.
  Fixing this requires C89 compatibility to be relinquished.
* BSMTP is mostly untested and errors can cause corrupt output.
* Fetchmail does not track pending deletes across crashes.
* The command line interface is sometimes a bit stubborn, for instance,
  fetchmail -s doesn't work with a daemon running.
* Linux systems may return duplicates of an IP address in some circumstances if
  no or no global IPv6 addresses are configured.
  (No workaround. Ubuntu Bug#582585, Novell Bug#606980.)
* Kerberos 5 may be broken, particularly on Heimdal, and provide bogus error
  messages. This will not be fixed, because the maintainer has no Kerberos 5
  server to test against. Use GSSAPI.

---------------------------------------------------------------------------------

Happy fetches,
Matthias

[Fetchmail-users] ANNOUNCE: The 6.4.16-rc1 release candidate #1 of fetchmail is available (bug fixes, documentation update, SSL info in fetchmail --version)

[Matthias A. <mat...@gm...>]

Greetings,

The 6.4.16-rc1 release candidate of fetchmail is now available at the usual locations,
including <https://sourceforge.net/projects/fetchmail/branch_6.4/>.

It contains a few bug fixes that were contributed via Gitlab.

The source archive is available at:
<https://sourceforge.net/projects/fetchmail/branch_6.4/fetchmail-6.4.16-rc1.tar.xz/download>
<https://sourceforge.net/projects/fetchmail/branch_6.4/fetchmail-6.4.16-rc1.tar.lz/download>

Detached GnuPG signatures for the respective tarballs are at:
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16-rc1.tar.xz.asc/download>
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/fetchmail-6.4.16-rc1.tar.lz.asc/download>

SHA256 hash values for the tarballs:
SHA256(fetchmail-6.4.16-rc1.tar.lz)= 48ca15ba5d87b564050b7642732ffc0b2faf82b83a9eb630032e4ae7c760c866
SHA256(fetchmail-6.4.16-rc1.tar.xz)= 2ac20bb2858b9cad575b1d4b95855d02561a57f65dbd5abf20868234977a73aa

Here are the release notes:

---------------------------------------------------------------------------------
fetchmail-6.4.16 (not yet released):

# BUG FIXES
* fetchmail's --configdump, and fetchmailconf, lacked support for the
  sslcertfile option. --configdump support added by Earl Chew,
  Gitlab issue #25, merge request !28.
* fetchmail's manual page was never updated to reflect 6.2.5's change about the
  duplicate-killer code for multidrop mode, which read
  "* Dup-killer code now keys on an MD5 hash of the raw headers."
  ...instead of just the Message-ID. [commit 9dd8400, 2003-10-10 by esr]
  The manual page was now updated accordingly and documents
  historic behaviour:
  start to 5.0.7 no duplicate suppression;
  5.0.8 to 6.2.4 duplicate suppression only by Message-ID;
  6.2.5 to 6.4.X duplicate suppression by entire raw header.
  Manpage bug found by Julian Bane debugging "duplicate message" behaviour.
* ./configure no longer runs AC_LIB_LINKFLAGS (how to link) checks
  when called --without-ssl

# FEATURE
* fetchmail --version [fetchmail -V] now queries and prints the SSL/TLS
  library's "SSL default trusted certificate" file or directory (mind the word
  "default"), where the OpenSSL-compatible TLS implementation will look for
  trusted root, meaning certification authority (CA), certificates.
  NOTE 1: watch the output carefully if the line prints the defaults
  or the configured path (without "default").
  NOTE 2: SSL_CERT_DIR and SSL_CERT_FILE are documented environment variables
  for OpenSSL 1.1.1 to override the *default* locations (those compiled into
  OpenSSL or possibly in its configuration file).
  This was added when Gene Heskett was debugging his setup and the
  information "where does OpenSSL look" was missing.
* fetchmail --version now prints version of the OpenSSL library that
  it was compiled against, and that it is using at runtime, and also
  the OPENSSL_DIR and OPENSSL_ENGINES_DIR (if available).

# KNOWN BUGS AND WORKAROUNDS
  (This section floats upwards through the NEWS file so it stays with the
  current release information)
* Fetchmail does not handle messages without Message-ID header well
  (See sourceforge.net bug #780933)
* Fetchmail currently uses 31-bit signed integers in several places
  where unsigned and/or wider types should have been used, for instance,
  for mailbox sizes, and misreports sizes of 2 GibiB and beyond.
  Fixing this requires C89 compatibility to be relinquished.
* BSMTP is mostly untested and errors can cause corrupt output.
* Fetchmail does not track pending deletes across crashes.
* The command line interface is sometimes a bit stubborn, for instance,
  fetchmail -s doesn't work with a daemon running.
* Linux systems may return duplicates of an IP address in some circumstances if
  no or no global IPv6 addresses are configured.
  (No workaround. Ubuntu Bug#582585, Novell Bug#606980.)
* Kerberos 5 may be broken, particularly on Heimdal, and provide bogus error
  messages. This will not be fixed, because the maintainer has no Kerberos 5
  server to test against. Use GSSAPI.
---------------------------------------------------------------------------------

Happy fetches,
Matthias

Re: [Fetchmail-users] Duplicate Emails

[Matthias A. <mat...@gm...>]

Am 29.01.21 um 19:37 schrieb Matthias Andree:
> Definitely yes because what you quote is plainly does not refer to what
> the code does. I will see to fixing the manual page. I've filed an issue
> so I don't forget, because I can't get to it right away. This is 6.4 stuff.
>
> https://gitlab.com/fetchmail/fetchmail/-/issues/29

Julian,

you have indeed found a bug that was standing since 2003-ish or so, fix
pushed to Git, and to appear in 6.4.16 soonish.

Since I've added a few translatable strings, we'll do a 6.4.16-rc1 first
(also with documentation and code enhancements suggested in Gene
Heskett's certs threads.)

Manpage change here:
https://gitlab.com/fetchmail/fetchmail/-/commit/1df193714c62e6b12f1b8f1dab10fd23b6d06e51

Thanks for your insisting after my initial harsh reaction to your using
6.3.X. The behaviour was there all the way since 6.2.5, but I ask for
everyone's understanding that I would really like to only focus on 6.4
and future versions, but I shall try to look a bit deeper before
refusing to look outright.

Regards,
Matthias

Re: [Fetchmail-users] Certs again

[Matthias A. <mat...@gm...>]

Am 30.01.21 um 03:36 schrieb Gene Heskett:
> Greetings  Matthias Andree;
>> Gene,
>
>> I agree, but I am not (currently) aware of a function that lets me query
>> this information from OpenSSL in an easy manner. I'll go look again to
>> see if I find it now, or if such information was added...
>>
>> And that is why I was proposing ... to override it.
>>
>> Five minutes later, a hail to Joel for stackexchange.com, I figured this:
>>
>> https://stackoverflow.com/questions/4138139/how-to-find-out-the-path-for-openssl-trusted-certificate
>>
>> https://stackoverflow.com/questions/37035300/how-to-determine-the-default-location-for-openssl-cnf/54282217
>>
>> One is useful for me as a hint for having fetchmail print the
>> certificate paths used in verbose mode, and one is useful for you now to
>> check where your openSSL libraries go looking:
>>
>> /usr/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"
>>
>> /usr/local/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"
>
> gene@coyote:~$ /usr/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"
> /usr/local/ssl
> gene@coyote:~$ /usr/local/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"
> /usr/local/ssl
>
> So both of those are pointing to the newly built and
> installed openssl-1.1.1i version in /usr/local.

So I expect that /usr/local/ssl/certs (mind the trailing /certs) would
usually be the store for trusted certificates.

Either concatenated as cert.pem, or as individual files to be rehashed
into additional symlinks of the 98765432.0-style with the relevant
c_rehash program.

It appears Debian might had that typical package of Mozilla-curated
certificates, and I think it's called ca-certificates, but - not being a
Debian user - I think is still rather unlikely to install those certs
into /usr/local/ssl/certs.
Also I don't know how exactly the hash used in c_rehash changed over
OpenSSL versions.
It *has* changed across the 0.9.x -> 1.0.0 transition at the time, but I
haven't checked when else.

> Perhaps a missing make clean before the reconfigure, make,
> sudo make install to put it in /usr?

Overwriting /usr,is, um, daring.

I think Debian is normally rather good at integrating all its packages
well, and if you stomp an incompatible OpenSSL version over packaged
versions, this is outside how many other packages is this going to break?

[Fetchmail-users] Certs again

[Gene H. <ghe...@sh...>]

Greetings  Matthias Andree;
Gene,   

I agree, but I am not (currently) aware of a function that lets me query
this information from OpenSSL in an easy manner. I'll go look again to
see if I find it now, or if such information was added...

And that is why I was proposing ... to override it.

Five minutes later, a hail to Joel for stackexchange.com, I figured this:

https://stackoverflow.com/questions/4138139/how-to-find-out-the-path-for-openssl-trusted-certificate

https://stackoverflow.com/questions/37035300/how-to-determine-the-default-location-for-openssl-cnf/54282217

One is useful for me as a hint for having fetchmail print the
certificate paths used in verbose mode, and one is useful for you now to
check where your openSSL libraries go looking:

/usr/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"

/usr/local/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"

gene@coyote:~$ /usr/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"
/usr/local/ssl
gene@coyote:~$ /usr/local/bin/openssl version -a | grep OPENSSLDIR | cut -f2 -d\"
/usr/local/ssl

So both of those are pointing to the newly built and 
installed openssl-1.1.1i version in /usr/local.

Perhaps a missing make clean before the reconfigure, make,
sudo make install to put it in /usr?

IDK.

Thanks for more guidance.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] Duplicate Emails

[Julian B. <jul...@bt...>]

Hi Matthias,

Thanks for the feedback and appreciate that the original functionality was more applicable to a bygone era (older time). I will pass on the X-UID question; it’s all Greek to me ;-)

For those that may search and find this post. I think I have a solution to the de-duplication as I was always going to use dovecot as my LDA and find I can use a sieve filter to achieve exactly what I want along with setting Thunderbird labels. This is not a reflection on fetchmail but rather an observation on a pragmatic workflow in today’s era.

Thank you very much for your time.

Best wishes,

Julian

On 29 Jan 2021, at 18:38, Matthias Andree <mat...@gm...> wrote:
> 
> Am 29.01.21 um 08:59 schrieb Julian Bane:
>> Good morning Mattias and all,
>> 
>> I have managed to suppress the X-Original-To header by
>> setting*enable_original_recipient = no*" in the postfix configuration
>> file. But still don't get duplicate emails suppressed, though the
>> X-Original-To header is. (Matthias - I can send you the mailbox file
>> if you want).
>> 
>> I have cut out two emails and verified they still have differences in
>> the headers (some de-personalization done):
>> 
>>    1,3c1,3
>>    < From XXX...@co...  Fri Jan 29 07:38:31 2021
>>    < Return-Path: <XXX...@co...>
>>    < Delivered-To: up...@my...
>>    ---
>>> From jul...@co...  Fri Jan 29 07:38:29 2021
>>> Return-Path: <XXX...@co...>
>>> Delivered-To: do...@my...
>>    38c38
>>    < X-UID: 24
>>    ---
>>> X-UID: 23
>> 
>> It's hard to see how you could ever get two identical sets of headers
>> for de-duplication to work.
> 
> Julian,
> 
> As stated earlier, the deduplication code is from another era, when the
> envelope-recording headers such as X-Original-To, Envelope-To,
> Delivered-To were not in widespread use, and it was meant as a hack to
> avoid local spool explosions when fetchmail resorted to guessing
> recipients from the To:/Cc: headers (which breaks with Bcc: quite
> obviously) because then it did not have a 1:1 relationship between
> server-side message and message-to-ship-onwards.
> 
> About X-UID, is it actually transmitted over the wire, or will your
> server software suppress it? I guess it goes hand in hand with the
> folder-internal status message that is also not sent over the wire.
> 
>> 
>> Not wishing to either make trouble or make work for others but could
>> the documentation be reworded to reflect how the de-duplication works.
>> Currently it says "/A piece of mail is considered duplicate if it has
>> the same message-ID as the message immediately preceding and more than
>> one addressee/".
> 
> Definitely yes because what you quote is plainly does not refer to what
> the code does. I will see to fixing the manual page. I've filed an issue
> so I don't forget, because I can't get to it right away. This is 6.4 stuff.
> 
> https://gitlab.com/fetchmail/fetchmail/-/issues/29
> 
> Best regards,
> MNatthias
> 
> 
> _______________________________________________
> Fetchmail-users mailing list
> Fet...@li...
> https://lists.sourceforge.net/lists/listinfo/fetchmail-users

Re: [Fetchmail-users] cert probelm

[Matthias A. <mat...@gm...>]

Am 29.01.21 um 13:48 schrieb Gene Heskett:
> Greetings all;
>
> locally built 6.4.14 client here on debian stretch.
>
> Been running fine for months.
>
> last night around 22:00 local mail stopped.
>
> openssl-1.1.1l is now installed locally built, installed once with --prefix=/usr, and once as the default /usr/local.
>
> ~/.fetchmailrc active section:
> poll	imap.shentel.net with proto imap
> 	user $USR with password $PW is gene
> 	fetchall
> 	ssl
> 	pass8bits
> 	nokeep
>
> Restarting fetchmail gets:
>
> fetchmail: Server certificate verification error: self signed certificate in certificate chain
> fetchmail: Missing trust anchor certificate: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
> fetchmail: This could mean that the root CA's signing certificate is not in the trusted CA certificate location, or that
> c_rehash needs to be run on the certificate directory. For details, please see the documentation of --sslcertpath
> and --sslcertfile in the manual page. See README.SSL for details.
> fetchmail: OpenSSL reported: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
> fetchmail: imap.shentel.net: SSL connection failed.
> fetchmail: socket error while fetching from $US...@im...
>
> I can go in with a browser using that same $USR and $PW and see it there ok
> Please define the certificate directory. I do have that cert if the
> spaces are replaced by underscores. And c_rehash has been run, several
> times with the -v option, looks legit but me not an expert.

c_rehash has worked if you have lots of 12345678.0 symlinks pointing to
the MyFineRootCA.pem and AnotherRootCA.pem files in that directory.

So either you can point fetchmail to the directory you have hashed, with
the "sslcertpath /path/to/my/certs" option (you'd give it the directory
1. that contains the root certificates and 2. that you should have run
c_rehash on. This should be the same ;-)), or globally, make sure that
your OpenSSL configuration points to the right directory. I'm not sure
if I would feel comfortable with OpenSSL installed in two places, that's
two places to maintain.

> Where is the bad or missing cert, here, or at my ISP's dovecot server?

On your end, the default OpenSSL library that fetchmail links to at
run-time cannot find the root certificate (digicert) of the chain
presented by the server.

Gene, any proposals on where I should update README.SSL, or how could I
reword the error message you quoted above?

Hope that helps.

Regards,
Matthias

Re: [Fetchmail-users] Duplicate Emails

[Matthias A. <mat...@gm...>]

Am 29.01.21 um 08:59 schrieb Julian Bane:
> Good morning Mattias and all,
>
> I have managed to suppress the X-Original-To header by
> setting*enable_original_recipient = no*" in the postfix configuration
> file. But still don't get duplicate emails suppressed, though the
> X-Original-To header is. (Matthias - I can send you the mailbox file
> if you want).
>
> I have cut out two emails and verified they still have differences in
> the headers (some de-personalization done):
>
>     1,3c1,3
>     < From XXX...@co...  Fri Jan 29 07:38:31 2021
>     < Return-Path: <XXX...@co...>
>     < Delivered-To: up...@my...
>     ---
>     > From jul...@co...  Fri Jan 29 07:38:29 2021
>     > Return-Path: <XXX...@co...>
>     > Delivered-To: do...@my...
>     38c38
>     < X-UID: 24
>     ---
>     > X-UID: 23
>
> It's hard to see how you could ever get two identical sets of headers
> for de-duplication to work.

Julian,

As stated earlier, the deduplication code is from another era, when the
envelope-recording headers such as X-Original-To, Envelope-To,
Delivered-To were not in widespread use, and it was meant as a hack to
avoid local spool explosions when fetchmail resorted to guessing
recipients from the To:/Cc: headers (which breaks with Bcc: quite
obviously) because then it did not have a 1:1 relationship between
server-side message and message-to-ship-onwards.

About X-UID, is it actually transmitted over the wire, or will your
server software suppress it? I guess it goes hand in hand with the
folder-internal status message that is also not sent over the wire.

>
> Not wishing to either make trouble or make work for others but could
> the documentation be reworded to reflect how the de-duplication works.
> Currently it says "/A piece of mail is considered duplicate if it has
> the same message-ID as the message immediately preceding and more than
> one addressee/".

Definitely yes because what you quote is plainly does not refer to what
the code does. I will see to fixing the manual page. I've filed an issue
so I don't forget, because I can't get to it right away. This is 6.4 stuff.

https://gitlab.com/fetchmail/fetchmail/-/issues/29

Best regards,
MNatthias

Re: [Fetchmail-users] cert probelm

[grarpamp <gra...@gm...>]

> fetchmail: Missing trust anchor certificate: /C=US/O=DigiCert
> Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
> fetchmail: This could mean that the root CA's signing certificate is not in
> the trusted CA certificate location, or that
> c_rehash needs to be run on the certificate directory. For details, please
> see the documentation of --sslcertpath
> and --sslcertfile in the manual page. See README.SSL for details.

> Please define the certificate directory. I do have that cert if the
> spaces are replaced by underscores. And c_rehash has been run, several
> times with the -v option, looks legit but me not an expert.

> Where is the bad or missing cert, here, or at my ISP's dovecot server?

openssl s_client -connect imap.shentel.net:imaps -showcerts < /dev/null
openssl x509 -text -fingerprint -sha1 < each_cert_block

Check that your system's cacert / ca_bundle / nss file is up to date,
/etc/ssl or elsewhere.

Server is supplying two certs from other parties,
ca intermediate and ca root,
when they should supply only local intermediates,
which those two aren't.

Check validity chain of fingerprints, expirations,
and add to cacert file, c_rehash dir,
or whatever other local method is in use, as needed.

Authority certs are in urls output in -text above.

[Fetchmail-users] cert probelm

[Gene H. <ghe...@sh...>]

Greetings all;

locally built 6.4.14 client here on debian stretch.

Been running fine for months.

last night around 22:00 local mail stopped.

openssl-1.1.1l is now installed locally built, installed once with --prefix=/usr, and once as the default /usr/local.

~/.fetchmailrc active section:
poll	imap.shentel.net with proto imap
	user $USR with password $PW is gene
	fetchall
	ssl
	pass8bits
	nokeep

Restarting fetchmail gets:

fetchmail: Server certificate verification error: self signed certificate in certificate chain
fetchmail: Missing trust anchor certificate: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
fetchmail: This could mean that the root CA's signing certificate is not in the trusted CA certificate location, or that 
c_rehash needs to be run on the certificate directory. For details, please see the documentation of --sslcertpath 
and --sslcertfile in the manual page. See README.SSL for details.
fetchmail: OpenSSL reported: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
fetchmail: imap.shentel.net: SSL connection failed.
fetchmail: socket error while fetching from $US...@im...

I can go in with a browser using that same $USR and $PW and see it there ok
Please define the certificate directory. I do have that cert if the 
spaces are replaced by underscores. And c_rehash has been run, several 
times with the -v option, looks legit but me not an expert.
 
Where is the bad or missing cert, here, or at my ISP's dovecot server?

Thank you.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

Re: [Fetchmail-users] Duplicate Emails

[Julian B. <jul...@bt...>]

Good morning Mattias and all,

I have managed to suppress the X-Original-To header by setting 
"*enable_original_recipient = no*" in the postfix configuration file. 
But still don't get duplicate emails suppressed, though the 
X-Original-To header is. (Matthias - I can send you the mailbox file if 
you want).

I have cut out two emails and verified they still have differences in 
the headers (some de-personalization done):

    1,3c1,3
    < From XXX...@co... Fri Jan 29 07:38:31 2021
    < Return-Path: <XXX...@co...>
    < Delivered-To: up...@my...
    ---
     > From jul...@co... Fri Jan 29 07:38:29 2021
     > Return-Path: <XXX...@co...>
     > Delivered-To: do...@my...
    38c38
    < X-UID: 24
    ---
     > X-UID: 23

It's hard to see how you could ever get two identical sets of headers 
for de-duplication to work.

Not wishing to either make trouble or make work for others but could the 
documentation be reworded to reflect how the de-duplication works. 
Currently it says "/A piece of mail is considered duplicate if it has 
the same message-ID as the message immediately preceding and more than 
one addressee/".

Many thanks and have a good weekend,

Julian

On 29/01/2021 00:04, Matthias Andree wrote:
> Am 28.01.21 um 09:00 schrieb Julian Bane via Fetchmail-users:
>> I have exactly the same results on fetchmail 6.4.15 - openSUSE
>> Tumbleweed current release which reports:
>>
>>      "This is fetchmail release
>> 6.4.15+POP2+GSS+RPA+NTLM+SDPS+SSL-SSLv2-SSLv3+OPIE+NLS+KRB5."
>>
>>
>> On 27/01/2021 13:41, Julian Bane via Fetchmail-users wrote:
>>> I have been running a test using fetchmail (6.3.26) on an openSUSE
>>> Leap 15.2 server to retrieve email from a local email account. That
>>> account has been sent 5 seperate emails. Three of the emails are to
>>> single addresses, one is to two addresses and another is to three
>>> addresses. As expected there are 8 separate emails in the local mail
>>> spool file from the 5 I sent. As a test I am using the fetchmailrc file:
>>>
>>>      set postmaster root
>>>      poll cumulus protocol POP3 local mydomain.net envelope
>>> "X-Original-To:"
>>>              username "user" password "pass" to * here fetchall keep
>>>              mda "echo 'Hello World - %T - %F'"
>>>
>>> The mda is just a dummy to see what is going on when I run it from
>>> the command line.
>>>
>>> It retrieves all 8 emails correctly identifying the origonal
>>> recipient and %T is substituted correctly for the different email
>>> adresses.
>>> Doing "fetchmail -V -f fetchmailrc" confirms that it is running in
>>> multidrop mode.
>>>
>>> If I substitute 'myname' for the '*' (asterisk) in fetchmailrc,
>>> fetchmail goes into singledrop mode as you would expect (checked with
>>> the -V option) but also retrieves all 8 emails.
>>>
>>> I am not sure which way round it should be, multidrop or singledrop,
>>> but I do want to remove duplicates and so want to retrieve just 5
>>> emails from the spool file (have been reading "THE USE AND ABUSE OF
>>> MULTIDROP MAILBOXES" on the man page). I have checked that the
>>> message-IDs are the same and there are multiple recipients on the
>>> 'To:' header of the relevant emails. I also have a copy of the spool
>>> file if it would help.
>>>
>>> Any advice or insight would be most welcome.
> Greetings,
>
> So after reviewing 6.4.15 logs that Julian sent to me off-list, some
> remarks:
>
> * The server in the test scenario contains messages with these
> Message-ID shown below. Meaning that the duplication already happened on
> the server-side mailbox:
>
> Message-ID: <16116718
> Message-ID: <6ec00e74
> Message-ID: <bbe93702
> Message-ID: <f1711007
> Message-ID: <f1711007
> Message-ID: <f1711007
> Message-ID: <f30f6378
> Message-ID: <7c6c3256
> Message-ID: <7c6c3256
>
> * fetchmail works as I believe it should: the server's mailbox in
> Julian's test scenario contains eight messages and fetchmail obtains and
> delivers eight. So fetchmail is not duplicating messages. Neither is it
> deduplicating, see next:
>
> * fetchmail has some deduplication code, but it is for a very specific
> and historic purpose: if the most-dangerous "To:/Cc:" guessing is being
> applied for lack of a header that carries a copy of the envelope's (SMTP
> dialogue's) destination address (Julian is using X-Original-To), then
> deduplication is supposed to kick in. The dedup'ing code compares the
> ENTIRE header (by way of calculating its MD5 hash value) in a run of
> message and suppresses all copies but the first.
>
> This code however is not supposed to work and also technically can not
> work in Julian's scenario: the headers of the server-side messages are
> *not exactly* the same, and differ in the X-Original-To: headers.
> Enhancing this deduplication code would mean changing behaviour and
> hence should not happen before fetchmail 7.0.0. An idea is to filter out
> a certain list of headers (among which the envelope's destination
> address recording headers) and only then hashing what is left of the
> header. But given the proposal below I'd say we don't actually need to
> change fetchmail:
>
> * Possible solution: install the standalone "maildrop" package from the
> Courier mailserver suite and use it, or a wrapper script for it, as mda.
> Maildrop ships with a "reformail" tool that can track Message-IDs of
> seen messages in a database and its exit status can be used to suppress
> these duplicates by message ID (it's option -D). The maildropex(7)
> manual page contains an example, just look for the first occurrence of:
> reformail -D
>
> https://www.courier-mta.org/maildrop/maildrop.html
>
>
> Note I strongly discourage using procmail. It has been unmaintained for
> like two decades and is inherently unsafe: All error handling and
> locking needs to be put in your recipes explicitly to achieve
> deterministic behavior in adverse conditions (such as under load).
> maildrop is safer to use and its .mailfilter files are easier to
> understand and write, and it is well-behaved by default, unlike procmail.
>
> HTH
>
> Happy fetching & deduplicating,
>
> Matthias
>
>
>
> _______________________________________________
> Fetchmail-users mailing list
> Fet...@li...
> https://lists.sourceforge.net/lists/listinfo/fetchmail-users

Re: [Fetchmail-users] Duplicate Emails

[Matthias A. <mat...@gm...>]

(note I truncated the Message-IDs to hide personal data, and yes I know
there are 9 IDs, the extra Message-ID is that of a folder-internal-data
status "message" that the mail server does not expose to fetchmail)

Re: [Fetchmail-users] Duplicate Emails

[Matthias A. <mat...@gm...>]

Am 28.01.21 um 09:00 schrieb Julian Bane via Fetchmail-users:
> I have exactly the same results on fetchmail 6.4.15 - openSUSE
> Tumbleweed current release which reports:
>
>     "This is fetchmail release
> 6.4.15+POP2+GSS+RPA+NTLM+SDPS+SSL-SSLv2-SSLv3+OPIE+NLS+KRB5."
>
>
> On 27/01/2021 13:41, Julian Bane via Fetchmail-users wrote:
>> I have been running a test using fetchmail (6.3.26) on an openSUSE
>> Leap 15.2 server to retrieve email from a local email account. That
>> account has been sent 5 seperate emails. Three of the emails are to
>> single addresses, one is to two addresses and another is to three
>> addresses. As expected there are 8 separate emails in the local mail
>> spool file from the 5 I sent. As a test I am using the fetchmailrc file:
>>
>>     set postmaster root
>>     poll cumulus protocol POP3 local mydomain.net envelope
>> "X-Original-To:"
>>             username "user" password "pass" to * here fetchall keep
>>             mda "echo 'Hello World - %T - %F'"
>>
>> The mda is just a dummy to see what is going on when I run it from
>> the command line.
>>
>> It retrieves all 8 emails correctly identifying the origonal
>> recipient and %T is substituted correctly for the different email
>> adresses.
>> Doing "fetchmail -V -f fetchmailrc" confirms that it is running in
>> multidrop mode.
>>
>> If I substitute 'myname' for the '*' (asterisk) in fetchmailrc,
>> fetchmail goes into singledrop mode as you would expect (checked with
>> the -V option) but also retrieves all 8 emails.
>>
>> I am not sure which way round it should be, multidrop or singledrop,
>> but I do want to remove duplicates and so want to retrieve just 5
>> emails from the spool file (have been reading "THE USE AND ABUSE OF
>> MULTIDROP MAILBOXES" on the man page). I have checked that the
>> message-IDs are the same and there are multiple recipients on the
>> 'To:' header of the relevant emails. I also have a copy of the spool
>> file if it would help.
>>
>> Any advice or insight would be most welcome.

Greetings,

So after reviewing 6.4.15 logs that Julian sent to me off-list, some
remarks:

* The server in the test scenario contains messages with these
Message-ID shown below. Meaning that the duplication already happened on
the server-side mailbox:

Message-ID: <16116718
Message-ID: <6ec00e74
Message-ID: <bbe93702
Message-ID: <f1711007
Message-ID: <f1711007
Message-ID: <f1711007
Message-ID: <f30f6378
Message-ID: <7c6c3256
Message-ID: <7c6c3256

* fetchmail works as I believe it should: the server's mailbox in
Julian's test scenario contains eight messages and fetchmail obtains and
delivers eight. So fetchmail is not duplicating messages. Neither is it
deduplicating, see next:

* fetchmail has some deduplication code, but it is for a very specific
and historic purpose: if the most-dangerous "To:/Cc:" guessing is being
applied for lack of a header that carries a copy of the envelope's (SMTP
dialogue's) destination address (Julian is using X-Original-To), then
deduplication is supposed to kick in. The dedup'ing code compares the
ENTIRE header (by way of calculating its MD5 hash value) in a run of
message and suppresses all copies but the first.

This code however is not supposed to work and also technically can not
work in Julian's scenario: the headers of the server-side messages are
*not exactly* the same, and differ in the X-Original-To: headers.
Enhancing this deduplication code would mean changing behaviour and
hence should not happen before fetchmail 7.0.0. An idea is to filter out
a certain list of headers (among which the envelope's destination
address recording headers) and only then hashing what is left of the
header. But given the proposal below I'd say we don't actually need to
change fetchmail:

* Possible solution: install the standalone "maildrop" package from the
Courier mailserver suite and use it, or a wrapper script for it, as mda.
Maildrop ships with a "reformail" tool that can track Message-IDs of
seen messages in a database and its exit status can be used to suppress
these duplicates by message ID (it's option -D). The maildropex(7)
manual page contains an example, just look for the first occurrence of:
reformail -D

https://www.courier-mta.org/maildrop/maildrop.html


Note I strongly discourage using procmail. It has been unmaintained for
like two decades and is inherently unsafe: All error handling and
locking needs to be put in your recipes explicitly to achieve
deterministic behavior in adverse conditions (such as under load).
maildrop is safer to use and its .mailfilter files are easier to
understand and write, and it is well-behaved by default, unlike procmail.

HTH

Happy fetching & deduplicating,

Matthias

Re: [Fetchmail-users] Duplicate Emails

[Matthias A. <mat...@gm...>]

Julian,

Thanks for confirming that, but please still provide logs, see previous
email for instructions. Thank you.

Re: [Fetchmail-users] Duplicate Emails

[Julian B. <jul...@bt...>]

I have exactly the same results on fetchmail 6.4.15 - openSUSE 
Tumbleweed current release which reports:

     "This is fetchmail release 
6.4.15+POP2+GSS+RPA+NTLM+SDPS+SSL-SSLv2-SSLv3+OPIE+NLS+KRB5."


On 27/01/2021 13:41, Julian Bane via Fetchmail-users wrote:
> I have been running a test using fetchmail (6.3.26) on an openSUSE 
> Leap 15.2 server to retrieve email from a local email account. That 
> account has been sent 5 seperate emails. Three of the emails are to 
> single addresses, one is to two addresses and another is to three 
> addresses. As expected there are 8 separate emails in the local mail 
> spool file from the 5 I sent. As a test I am using the fetchmailrc file:
>
>     set postmaster root
>     poll cumulus protocol POP3 local mydomain.net envelope 
> "X-Original-To:"
>             username "user" password "pass" to * here fetchall keep
>             mda "echo 'Hello World - %T - %F'"
>
> The mda is just a dummy to see what is going on when I run it from the 
> command line.
>
> It retrieves all 8 emails correctly identifying the origonal recipient 
> and %T is substituted correctly for the different email adresses.
> Doing "fetchmail -V -f fetchmailrc" confirms that it is running in 
> multidrop mode.
>
> If I substitute 'myname' for the '*' (asterisk) in fetchmailrc, 
> fetchmail goes into singledrop mode as you would expect (checked with 
> the -V option) but also retrieves all 8 emails.
>
> I am not sure which way round it should be, multidrop or singledrop, 
> but I do want to remove duplicates and so want to retrieve just 5 
> emails from the spool file (have been reading "THE USE AND ABUSE OF 
> MULTIDROP MAILBOXES" on the man page). I have checked that the 
> message-IDs are the same and there are multiple recipients on the 
> 'To:' header of the relevant emails. I also have a copy of the spool 
> file if it would help.
>
> Any advice or insight would be most welcome.
>
> Thanks
>
> _______________________________________________
> Fetchmail-users mailing list
> Fet...@li...
> https://lists.sourceforge.net/lists/listinfo/fetchmail-users

Re: [Fetchmail-users] Duplicate Emails

[Matthias A. <mat...@gm...>]

Am 27.01.21 um 14:41 schrieb Julian Bane via Fetchmail-users:
> Hi,
>
> I have been running a test using fetchmail (6.3.26)

I've stopped reading here. This list is not meant to support software
museums.

Update, then see https://www.fetchmail.info/fetchmail-FAQ.html#G3

[Fetchmail-users] Duplicate Emails

[Julian B. <jul...@bt...>]

Hi,

I have been running a test using fetchmail (6.3.26) on an openSUSE Leap 
15.2 server to retrieve email from a local email account. That account 
has been sent 5 seperate emails. Three of the emails are to single 
addresses, one is to two addresses and another is to three addresses. As 
expected there are 8 separate emails in the local mail spool file from 
the 5 I sent. As a test I am using the fetchmailrc file:

     set postmaster root
     poll cumulus protocol POP3 local mydomain.net envelope "X-Original-To:"
             username "user" password "pass" to * here fetchall keep
             mda "echo 'Hello World - %T - %F'"

The mda is just a dummy to see what is going on when I run it from the 
command line.

It retrieves all 8 emails correctly identifying the origonal recipient 
and %T is substituted correctly for the different email adresses.
Doing "fetchmail -V -f fetchmailrc" confirms that it is running in 
multidrop mode.

If I substitute 'myname' for the '*' (asterisk) in fetchmailrc, 
fetchmail goes into singledrop mode as you would expect (checked with 
the -V option) but also retrieves all 8 emails.

I am not sure which way round it should be, multidrop or singledrop, but 
I do want to remove duplicates and so want to retrieve just 5 emails 
from the spool file (have been reading "THE USE AND ABUSE OF MULTIDROP 
MAILBOXES" on the man page). I have checked that the message-IDs are the 
same and there are multiple recipients on the 'To:' header of the 
relevant emails. I also have a copy of the spool file if it would help.

Any advice or insight would be most welcome.

Thanks

Re: [Fetchmail-users] Problem with GMail changing sslfingerprint

[Matthias A. <mat...@gm...>]

Am 17.01.21 um 18:09 schrieb Héctor Abreu:
> Hi,
>
> I work via ssh using Mutt + Fetchmail and GMail POP3 service. I frequently stop receiving emails because the sslfingerprint is changed, and when I enter the new value in .fetchmailrc and go to GMail settings to re-enable POP3, GMail settings offers me the option of downloading all the messages again or only messages starting from that very moment; I choose the latter, so I lose in my email client all the messages from the moment the sslfingerprint was changed until the moment I adjust its value to the new one and re-enable GMail POP3 settings. This is very annoying.

Héctor, why do you need to re-enable POP3 settings in GMail once they
replace certificates? I don't see what one thing has to do with the
other and for my test account I don't seem to ever having had to
re-enable anything.

And for Google, why do you add sslfingerprint if it's such a hassle in
the first place? Google have properly signed certificate chains in place.

Re: [Fetchmail-users] Problem with GMail changing sslfingerprint

[Matthias A. <mat...@gm...>]

Am 18.01.21 um 00:32 schrieb grarpamp:
> "gmail pop3 settings" / account lockouts / download gaps,
> are unrelated to TLS fingerprint changes, TLS checking comes
> before providers ever see an account name to abuse like that.
>
>
>
> Many services, including some from google, and every service
> using LetsEncrypt puts new sigs over their certs every n months.
> This forces security conscious and auto-MITM resistant
> users to waste time validating and updating fingerprints when
> no part of the underlying security context actually changed...
>
> In many services doing this sig rotation model,
> the underlying TLS pubkey does not change since
> the key itself was never actually compromised yet,
> nor was the cert swapped out for a new privkey, but
> was just sig "expiring", which is bogus and dating back to
> days of revenue $cam forced on services by CA's model.
>
> Fetchmail needs to support the
> --pinnedpubkey sha256//...
> option of curl(1), it was posted here earlier.

I am not so sure that applications on the whole that use a TLS library
should reinvent all the checks and implement new mechanisms. There is a
specified and established approach, and that is working through the
certificate chain until you find a trust anchor. Traditionally, this
would then be a root CA. In practice, there is a list of 100+ CA
certificates that many applications trust, and this list is curated by
Mozilla as part of the NSS library based on certain policies.

Lots of code is present from the OpenSSL 0.9.Xy releases from aeons ago
that would not perform the typical checks on certificates beyond pure
signature verification, so applications resorted to doing their own
checks on top.
Also, some ten years ago many service providers did not really care
about even providing proper certficate chains, so that applications
hardly ever could really establish trust. This changed when browsers and
GUI-based mailers started verifying SSL chains by default.

I would propose most of this mechanism and default policy actually
belongs into the crypto libraries such as OpenSSL.

LibreSSL has libtls which does some of such heavy lifting, but LibreSSL
itself has issues on its own, especially around TLS v1.3 and
OpenSSL-compatible API, so it's currently not viable.

> For interim use,
> users could post a simple few line patch that would convert the
> existing DER pin check in the code into a pubkey pin check.
Can you draft a specification on how this should look? fetchmail 7.x
might make that change because effectively most of the code is already
there and we'd rewrite it to look into other places for the sig, and to
make that change clear, rename the options.
> For endpoints that are actually swapping out the entire cert/privkey...
>
> Fetchmail would want to allow specifying fingerprint hashes,
> either of pubkey, or of entire DER cert with sig, interchangably
> as users desire, for each cert in the entire path from root,
> down to whichever last intermediate cert is deemed static
> enough by the user. This way you pin down to there, and
> allow the service endpoint certificate to swap, provided it
> still matches the "cert chain" checks (if configured to check
> that, which you always should). Part of this can be done
> with cert files, yet manual fingerprint can be useful to assure
> override of untrustable system ca cert bundle/update interactions,
> or to just avoid files entirely and update just one/few line hash config.
Pretty much wishful thinking to implement this in an all-too
programmable way. If that makes it harder to explain to users, I don't
think I will want that. And I certainly don't want a fetchmail
configuration to look like Exim concepts or make indirections. fetchmail
needs to be usable by normal users.
> For example, users/systems/companies with multiple accounts
> at an upstream service would only have to update one modular
> TLS fingerprint pinning line, not per each account/employee/etc.

Text editors have had "search/replace" functions for about as long as I
can remember. Maybe longer. I therefore don't see a need to write
gazillions of lines of a new domain-specific language specification
along with documentation, test suite and whatnot. Would look a bit
overengineered to me.

It might be easier to refactor fetchmailconf into a configuration
parsing/writing module under a new name and the GUI module unter
fetchmailconf name that uses the former so you can programmatically
read/modify configurations.

Re: [Fetchmail-users] Problem with GMail changing sslfingerprint

[grarpamp <gra...@gm...>]

"gmail pop3 settings" / account lockouts / download gaps,
are unrelated to TLS fingerprint changes, TLS checking comes
before providers ever see an account name to abuse like that.



Many services, including some from google, and every service
using LetsEncrypt puts new sigs over their certs every n months.
This forces security conscious and auto-MITM resistant
users to waste time validating and updating fingerprints when
no part of the underlying security context actually changed...

In many services doing this sig rotation model,
the underlying TLS pubkey does not change since
the key itself was never actually compromised yet,
nor was the cert swapped out for a new privkey, but
was just sig "expiring", which is bogus and dating back to
days of revenue $cam forced on services by CA's model.

Fetchmail needs to support the
--pinnedpubkey sha256//...
option of curl(1), it was posted here earlier.

It will solve many users fingerprint pinning editing pain
they experience with some services.

See the curl docs for how to print the pubkey fingerprint hash
that would eventually put in a fetchmail conf.

For interim use,
users could post a simple few line patch that would convert the
existing DER pin check in the code into a pubkey pin check.


That is for service endpoint certs that keep pubkey but update sigs.


For endpoints that are actually swapping out the entire cert/privkey...

Fetchmail would want to allow specifying fingerprint hashes,
either of pubkey, or of entire DER cert with sig, interchangably
as users desire, for each cert in the entire path from root,
down to whichever last intermediate cert is deemed static
enough by the user. This way you pin down to there, and
allow the service endpoint certificate to swap, provided it
still matches the "cert chain" checks (if configured to check
that, which you always should). Part of this can be done
with cert files, yet manual fingerprint can be useful to assure
override of untrustable system ca cert bundle/update interactions,
or to just avoid files entirely and update just one/few line hash config.

If fetchmail gets a modular labeled config for 7.x, that will make
abstracting and then constructing together the component
selections from among users, servers, protocols, cert chains, polls,
mda's, etc... into easier more flexible and even hierarchical usage contexts.

For example, users/systems/companies with multiple accounts
at an upstream service would only have to update one modular
TLS fingerprint pinning line, not per each account/employee/etc.

[Fetchmail-users] Problem with GMail changing sslfingerprint

[<rus...@gm...>]

	Addressing Sr. Héctor Abreu:

	I fetchmail from GMail.  About twice monthly GMail changes the
sslfingerprint.  I have no problem other than having to generate a new
fingerprint then trying again.  Why do you have to re-enable POP3 in
GMail?  I never disable it.

russell bell