fetchmail-users — User questions and discussion

[Fetchmail-users] Fetchmail only writes the most recent mail

[<dan...@pr...>]

Hello !

I'm trying fetchmail on the command line, here's what I came up with

fetchmail -p IMAP \ # use the IMAP protocol
-r "INBOX.Brigands" \ # to fetch mail from the Brigands folder
-k \ # keep mail on the server
-a \ # retrieve seen and unseen mail
-u user@domain mail.server.com \ # my username
--mda "/bin/sh -c 'cat > /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new/$(date +%''s_%N)'" # my MDA



Here's what output on the terminal

ychaouche#ychaouche-PC 11:52:46 ~ $ fetchmail -r "INBOX.Brigands" [...]
reading message user@do...@ma...:1 of 6 (2149 header octets) (1994 body octets) not flushed
reading message user@do...@ma...:2 of 6 (2103 header octets) (72981 body octets) not flushed
reading message user@do...@ma...:3 of 6 (2204 header octets) (71475 body octets) not flushed
reading message user@do...@ma...:4 of 6 (1966 header octets) (1348 body octets) not flushed
reading message user@do...@ma...:5 of 6 (1874 header octets) (1267 body octets) not flushed
reading message user@do...@ma...:6 of 6 (2351 header octets) (71410 body octets) not flushed
ychaouche#ychaouche-PC 11:53:26 ~ $


Mail should be put in /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new/, but here's the only thing I see :

ychaouche#ychaouche-PC 12:19:47 /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new $ ls
total 72K
-rwxrwxrwx 1 root root 72K Aug  9 12:19 1533813591_348482458
ychaouche#ychaouche-PC 12:19:59 /mnt/partage_local/DATA/MAIL/principale-fetchmail/DRAFT/new $


And it's not an mbox file with multilpe emails, it's really just one e-mail, the most recent one to be more precise.

Anybody want to help me fix this ?

Re: [Fetchmail-users] fetchmail: puts no messages, quits

[Matthias A. <mat...@gm...>]

Am 08.08.2018 um 23:07 schrieb grarpamp:
>> Options for retrieving from ...@mail-olympus.userservices.net:
> You might also have problems, later past the hash / bundle step,
> with the dns mismatch between the above and what's in the cert.
> There's probably a config option to disable just the name checks,
> I don't recall offhand what some tools call it.

No, there is no such option. They goofed up their "subject alternative
name" tags on the server's certificate, that SAN list is incomplete.

$ host -t any mail-olympus.userservices.net
mail-olympus.userservices.net is an alias for
mail.olympus.net.cust.b.hostedemail.com.

So the workaround might be to use the latter name as the server's name
instead. I am not sure if getaddrinfo() with AI_CANONNAME were good
enough to canonicalize the name. I would not want to reintroduce
res_search().

[Fetchmail-users] 7.x: Public Key Pinning, Intermediate Key Trust, Certificate Transparency Observatories, Fingerprints, TLS

[grarpamp <gra...@gm...>]

Certificate fingerprint pinning is fairly common and
some here have posted new applicability schemes
for it in 7.x.

Some more features that may not have been mentioned yet...


Public Key Pinning
There should likely also be an option to pin down
and trust just the public key of the certs (or in combination
with other checks as warranted) since some entities
push the same key into subsequent CSR's when
refreshing their expiry, instead of say generating
a new key.
Thus pubkey pinning is nicely a bit less finegrained
than fingerprint pinning, working across renewals,
while still not having to trust CA's and the third party
cert stores.

See:
https://curl.haxx.se/libcurl/c/CURLOPT_PINNEDPUBLICKEY.html
wget [--ca-certificate= ...] --pinnedpubkey=<sha256// pubkey der>
wget [--ca-certificate= ...] --pinnedpubkey=<pubkeyfile pem|der>


Intermediate Key Trust
It should be possible to say, I trust
 (via PKP or fingerprint, or hashdir, or file)
- Letsencrypt, Google, Whoever's intermediate key[s]
that aren't in the global cert stores,
and may or may not be supplied by the server,
thus letting all server certs signed by such an intermediate
key be accepted, subject to passing any other checks
configured for that server stanza module, including possibly
ignoring the CA's and cert store checks.


TLS
Before release 7.x should rename all
references to "SSL / ssl" to the now explicitly
correct decade old "TLS", except where some
actual context of explicitly named SSL version
number grants use of "ssl" word.


Certificate Transparency Observatories
Don't know what the state of libraries for checking these is.
I think google has also created a CT project.


Surveying other command line utilities that interface
with TLS services as to these features might be useful
to guide and help 7.x continue being a fine TLS example :)

Re: [Fetchmail-users] fetchmail: puts no messages, quits

[grarpamp <gra...@gm...>]

> Options for retrieving from ...@mail-olympus.userservices.net:

You might also have problems, later past the hash / bundle step,
with the dns mismatch between the above and what's in the cert.
There's probably a config option to disable just the name checks,
I don't recall offhand what some tools call it.

There's also a config option to fingerprint pin down the cert
if that's needed.


        Subject: C=CA, ST=Ontario, L=Toronto, O=Tucows Inc,
OU=Operations, CN=*.b.hostedemail.com
            X509v3 Subject Alternative Name:
                DNS:*.b.hostedemail.com

Re: [Fetchmail-users] fetchmail: puts no messages, quits

[Matthias A. <mat...@gm...>]

Am 08.08.2018 um 20:08 schrieb lothar via Fetchmail-users:
> i have spent hours and days building and rebuilding fetchmail, studying documents and user reports, and fiddling with configuration and settings.
>
> 1. why is fetchmail not working?

To give a short answer up front, because the kind person who has been
packaging fetchmail for Cygwin used to have a different opinion than I
on when a fetchmail package update was warranted, and I have given up
trying to persuade him to package newer versions, or make the Cygwin
project find another maintainer for the fetchmail package. Personally, I
am not using Cygwin in production or for hacking, and grarpamp's
suggestion to try and build legacy_64 is probably the best one. You will
need to install a few more packages than for building a tarball, and
bootstrap, see README.git - or else fetch the 6.4.0.beta4 tarball from
sourceforge.net here:
<https://sourceforge.net/projects/fetchmail/files/branch_6.4/> - be sure
to install the xz package so you can unpack it.

Newer fetchmail versions print messages that are a bit more helpful when
certificate verification fails.

> fetchmail runs silently for maybe 30 seconds, then puts the message shown below and quits.
> also there is nothing in the logfile.

It's either console messages or logfile, not both.

> 2. why do i see no progress messages, when i have put to give them?
> there is nothing like:
>   fetchmail: 6.3.22 querying someisp.net (protocol POP3) at Tue 31 Jul 2018 02:00:00 UCT: poll started
>   Trying to connect to 178.47.11.211/995...connected.
>
> 3. i can communicate with the mail provider with openssl to the point where openssl sends
>   +OK POP3 ready

But you haven't showed OpenSSL's s_client the way to the certificate
directory (openssl s_client -CApath ..., or openssl s_client -CAfile...,
respectively), so you have been running OpenSSL in a less strict
configuration than fetchmail, and that is the reason why it works and
fetchmail does not: you are not performing equivalent tests.

> 8. aside from the fact that there are no messages but one, that one complains about certificate verification.

...and that leads to an early abort, so there is no progress to show
(answering #2 and #8 at the same time).

> i have changed sslcertpath in ~/.fetchmailrc to:
>   /etc/ssl/certs
>   /usr/ssl/certs
>   /var/lib/fetchmail/certs
> all equally in vain.

These would have to be directories with individual PEM files, and also
symlinks that the c_rehash script shipping with OpenSSL would produce,
and are named like 1234abcd.0 (or perhaps .1).
I don't know if Cygwin's openssl packages and the ca-certificates
package match and configure the right default paths, so that *omitting*
the sslcertpath option might have helped. I assume these do, these days.

> $ uname -a
> CYGWIN_NT-10.0 hooha 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin

This, too, appears to be dated, Cygwin used to release quite often. Be
sure to keep things updated.

>
> ## in /var/lib/fetchmail/certs:
> $ ls -lgGt "--time-style=$TIMESTYLE"
> -rw-r--r-- 1 6148 2018.07.31 02:00 mail.someisp.net.pem
> lrwxrwxrwx 1   49 2018.07.31 02:00 tls-ca-bundle.pem -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

As grarpamp has mentioned, you need to use sslcertfile if you use one
big certificate bundle, not sslcertpath. Mind the last four letters. If
Cygwin's OpenSSL is matched to the ca-certificates package, omitting
sslcertfile/sslcertpath should work out of the box. It does with some
popular Linux distributions, and FreeBSD.

> ## mail.userservices.net.pem was created with:
> $ openssl s_client -connect mail.someisp.net:995 -showcerts > mail.someisp.net.pem

This is not recommended nor necessary.

Hope that clarifies some of your trouble.

Best regards,
Matthias

Re: [Fetchmail-users] fetchmail: puts no messages, quits

[grarpamp <gra...@gm...>]

On Wed, Aug 8, 2018 at 2:08 PM, lothar via Fetchmail-users
<fet...@li...> wrote:
> 1. why is fetchmail not working?

>   fetchmail: 6.3.22

That is ancient and unsupported.

Upgrade to at least 6.3.26,
and since you know how to compile, clone the gitlab repo
and select the legacy_64 branch which is the default,
and helps move fetchmail forward to master 7.x.

> 8. aside from the fact that there are no messages but one, that one complains about certificate verification.

> 4294956672:error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:s3_clnt.c:1269:

>   sslcertpath /var/lib/fetchmail/certs

> ## in /var/lib/fetchmail/certs:
> $ ls -lgGt "--time-style=$TIMESTYLE"
> -rw-r--r-- 1 6148 2018.07.31 02:00 mail.someisp.net.pem
> lrwxrwxrwx 1   49 2018.07.31 02:00 tls-ca-bundle.pem -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Read the man page.
sslcertpath must be a dir of openssl hashed certs,
 your dir is not.
sslcertfile if you want a combined pem bundle.

[Fetchmail-users] fetchmail: puts no messages, quits

[lothar <lo...@pr...>]

i have spent hours and days building and rebuilding fetchmail, studying documents and user reports, and fiddling with configuration and settings.

1. why is fetchmail not working?
fetchmail runs silently for maybe 30 seconds, then puts the message shown below and quits.
also there is nothing in the logfile.

2. why do i see no progress messages, when i have put to give them?
there is nothing like:
  fetchmail: 6.3.22 querying someisp.net (protocol POP3) at Tue 31 Jul 2018 02:00:00 UCT: poll started
  Trying to connect to 178.47.11.211/995...connected.

3. i can communicate with the mail provider with openssl to the point where openssl sends
  +OK POP3 ready

4. i can receive and send mail from the mail provider with seamonkey.

5. the file named in the message is
  s3_clnt.c
the search for same at
  https://cygwin.com/packages/
shows
    ...
    openssl-debuginfo-1.0.2o-1 - openssl-debuginfo: Debug info for openssl
s3_clnt.c must be in the main body of openssl code, but it is listed oddly as part of openssl-debuginfo.

6. i built fetchmail with:
  ./configure --with-ssl --bindir=/usr/bin --mandir=/usr/share/man
some of the text from the configure output is shown below.

7. the fact that there are no messages suggests a build error, but i have built the thing more than once.

8. aside from the fact that there are no messages but one, that one complains about certificate verification.
i have changed sslcertpath in ~/.fetchmailrc to:
  /etc/ssl/certs
  /usr/ssl/certs
  /var/lib/fetchmail/certs
all equally in vain.

$ fetchmail -cvvv
4294956672:error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:s3_clnt.c:1269:

$ fetchmail -V
This is fetchmail release 6.3.22+SSL+NLS.

Copyright (C) 2002, 2003 Eric S. Raymond
Copyright (C) 2004 Matthias Andree, Eric S. Raymond,
                   Robert M. Funk, Graham Wilson
Copyright (C) 2005 - 2006, 2010 - 2011 Sunil Shetye
Copyright (C) 2005 - 2011 Matthias Andree
Fetchmail comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. For details,
please see the file COPYING in the source or documentation directory.
This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit. (http://www.openssl.org/)

Fallback MDA: (none)
CYGWIN_NT-10.0 hooha 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin

Taking options from command line and /home/lam/.fetchmailrc
Logfile is /home/lam/.fetchmaillog
Idfile is /home/lam/.fetchids
Progress messages will be logged via syslog
Fetchmail will forward misaddressed multidrop messages to mh.
Options for retrieving from mi...@us...@mail-olympus.userservices.net:
  True name of server is mail-olympus.userservices.net.
  Protocol is POP3 (using service 995).
  All available authentication methods will be tried.
  SSL encrypted sessions enabled.
  SSL server certificate checking enabled.
  SSL trusted certificate directory: /var/lib/fetchmail/certs
  SSL key fingerprint (checked against the server key): 11:64:3A:38:DC:EB:CF:17:3A:83:E3:33:03:42:EB:94
  Server nonresponse timeout is 300 seconds (default).
  Default mailbox selected.
  Only new messages will be retrieved (--all off).

... ## text omitted

$ cat ~/.fetchmail
poll mail.someisp.net
  proto POP3
  port 995
  user 'la...@so...'
  there with password 'vxxv'
  is 'la...@so...' here
  mda "/usr/bin/procmail -d %T"
options
  no keep
  ssl
  sslcertck
  sslproto tls1
  sslcertpath /var/lib/fetchmail/certs
... ## text omitted

$ openssl s_client -connect mail.someisp.net:995 -showcerts
CONNECTED(00000003)
---
Certificate chain
0 s:/C=CA/ST=Ontario/L=Toronto/O=Tucows Inc/OU=Operations/CN=*.b.hostedemail.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=GeoTrust RSA CA 2018
-----BEGIN CERTIFICATE-----
MIIGDjCCBPagAwIBAgIQB0QktDd1at6/85CVOXSaUjANBgkqhkiG9w0BAQsFADBe
... ## text omitted

$ uname -a
CYGWIN_NT-10.0 hooha 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin

## in /var/lib/fetchmail/certs:
$ ls -lgGt "--time-style=$TIMESTYLE"
-rw-r--r-- 1 6148 2018.07.31 02:00 mail.someisp.net.pem
lrwxrwxrwx 1   49 2018.07.31 02:00 tls-ca-bundle.pem -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

## mail.userservices.net.pem was created with:
$ openssl s_client -connect mail.someisp.net:995 -showcerts > mail.someisp.net.pem
## tls-ca-bundle.pem is the standard bundled certificates list provided by the ca-certificates package.

## in /wxe/fetchmail/fetchmail-6.3.22
$ grep -ni -e ssl config.h
53:/* Define to 1 if you have the declaration of `SSLv2_client_method', and to 0
55:#define HAVE_DECL_SSLV2_CLIENT_METHOD 1
373:/* Define if you want SSL support compiled in */
374:#define SSL_ENABLE 1

$ grep -i -e ssl config.log
$ ./configure --with-ssl --bindir=/usr/bin --mandir=/usr/share/man
configure:9985: Enabling OpenSSL support in /usr.
configure:9997: checking for additional library dependencies of SSL
configure:10014: gcc -o conftest.exe -g -O2 -I/usr/kerberos/include   -L/usr/lib conftest.c  -L/usr/lib   -lssl -lcrypto  >&5
configure:10039: checking whether SSLv2_client_method is declared
configure:10221: gcc -o conftest.exe -g -O2 -I/usr/kerberos/include   -L/usr/lib conftest.c   -lssl -lcrypto  >&5
#define SSL_ENABLE 1
#define SSL_ENABLE 1
#define HAVE_DECL_SSLV2_CLIENT_METHOD 1
configure:10246: gcc -o conftest.exe -g -O2 -I/usr/kerberos/include   -L/usr/lib conftest.c   -lssl -lcrypto  >&5
| #define SSL_ENABLE 1
| #define SSL_ENABLE 1
| #define HAVE_DECL_SSLV2_CLIENT_METHOD 1
configure:10271: gcc -o conftest.exe -g -O2 -I/usr/kerberos/include   -L/usr/lib conftest.c   -lssl -lcrypto  >&5
| #define SSL_ENABLE 1
| #define SSL_ENABLE 1
| #define HAVE_DECL_SSLV2_CLIENT_METHOD 1
ac_cv_have_decl_SSLv2_client_method=yes
LIBS='  -lssl -lcrypto '
#define SSL_ENABLE 1
#define SSL_ENABLE 1
#define HAVE_DECL_SSLV2_CLIENT_METHOD 1

$ grep -ni -e ssl config.status
442:ac_cs_config="'--with-ssl' '--bindir=/usr/bin' '--mandir=/usr/share/man'"
534:  set X '/bin/sh' './configure'  '--with-ssl' '--bindir=/usr/bin' '--mandir=/usr/share/man' $ac_configure_extra_args --no-create --no-recursion
757:S["LIBS"]="  -lssl -lcrypto "
926:D["SSL_ENABLE"]=" 1"
927:D["SSL_ENABLE"]=" 1"
928:D["HAVE_DECL_SSLV2_CLIENT_METHOD"]=" 1"

Sent with [ProtonMail](https://protonmail.com) Secure Email.

Re: [Fetchmail-users] fetchmail, SSL is not enabled at ssl

[Matthias A. <mat...@gm...>]

Am 27.06.2018 um 22:58 schrieb lothar via Fetchmail-users:
> why does fetchmail give this message and then quit?
> the syntax of ~/.fetchmailrc looks correct.
> i have the right openssl libraries.
> i can show that ssl authentication works.

> fetchmail:/home/lh/.fetchmailrc:10: SSL is not enabled at ssl
>
> $ fetchmail -V
> This is fetchmail release 6.3.22+NLS.
> ... ## text omitted
>
> $ uname -a
> CYGWIN_NT-10.0 hoho 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin

Lothar,

It appears that you are using the outdated fetchmail package that was
built for Cygwin x86_64, and apparently SSL wasn't compiled in, as Ralph
already mentioned.

I suggest you build fetchmail 6.3.26 or 6.4.0-beta4 yourself from source
with all the required features and you configure it with SSL/TLS support
(--with-ssl for 6.3, it's automatic for 6.4.0-beta4).

I haven't used Cygwin on a serious basis for the past few years though,
so I cannot give much input on what the required development packages
(the compiler, for instance) are. It would also seem that a polite
notice on the relevant Cygwin mailing lists and/or its package
maintainer to ask for a newer version of the package that also enables SSL.

Best regards,
Matthias

Re: [Fetchmail-users] fetchmail, SSL is not enabled at ssl

[Ralph C. <ra...@in...>]

Hi lothar,

> $ fetchmail --mda 'procmail -d lh'
> fetchmail:/home/lh/.fetchmailrc:10: SSL is not enabled at ssl
...
> $ fetchmail -V
> This is fetchmail release 6.3.22+NLS.
> ... ## text omitted

Looks like it wasn't built to use SSL?

    $ fetchmail -V
    This is fetchmail release 6.3.26+SSL+NLS.
    ...
    This product includes software developed by the OpenSSL Project
    for use in the OpenSSL Toolkit. (http://www.openssl.org/)
    ...

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Fetchmail-users] fetchmail, SSL is not enabled at ssl

[lothar <lo...@pr...>]

why does fetchmail give this message and then quit?
the syntax of ~/.fetchmailrc looks correct.
i have the right openssl libraries.
i can show that ssl authentication works.

$ fetchmail --mda 'procmail -d lh'
fetchmail:/home/lh/.fetchmailrc:10: SSL is not enabled at ssl

$ cat ~/.fetchmail
poll mail.someisp.net
  proto POP3
  port 995
  user '[lh...@so...](mailto:la...@so...)'
  there with password 'vxxv'
  is '[lh...@so...](mailto:la...@so...)' here
  mda "/usr/bin/procmail -d %T"
options
  no keep
  ssl
  sslcertck
  sslcertpath /etc/ssl/certs/

$ openssl s_client -connect mail.someisp.net:995 -showcerts
CONNECTED(00000003)
depth=2 C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", CN = Go Daddy Root Certificate Authority - G2
verify return:1
depth=1 C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2
verify return:1
depth=0 OU = Domain Control Validated, CN = *.userservices.net
verify return:1
---
Certificate chain
... ## text omitted

$ fetchmail -V
This is fetchmail release 6.3.22+NLS.
... ## text omitted

$ uname -a
CYGWIN_NT-10.0 hoho 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin

$ cygcheck -csv

Cygwin Package Information
Last downloaded files to: h:\hx
Last downloaded files from:

Package                                 Version                    Status
... ## text omitted
fetchmail                               6.3.22-1                   OK
... ## text omitted
libopenssl100                           1.0.2o-1                   OK
... ## text omitted
openssl                                 1.0.2o-1                   OK
openssl-devel                           1.0.2o-1                   OK
... ## text omitted

Sent with [ProtonMail](https://protonmail.com) Secure Email.

Re: [Fetchmail-users] Flush `Trying to connect...' Message.

[Matthias A. <mat...@gm...>]

Trying to send the attachment...

Re: [Fetchmail-users] Flush `Trying to connect...' Message.

[Matthias A. <mat...@gm...>]

Am 13.06.2018 um 07:51 schrieb Ralph Corderoy:
> Hi Mattias,
>
>>> If a server is slow to accept a connection then I find the `Trying
>>> to connect to 1.2.3.4/993' message only appears on SIGINT whereas it
>>> would be useful to know that's what's being attempted during the
>>> delay.  Please flush this part of the line rather than having it
>>> appear only when the whole line is finished.
>>>
>>> This `two part line' style of output may occur elsewhere and need
>>> flushing benefit from flushing there too.
>> Please heed <http://www.fetchmail.info/fetchmail-FAQ.html#G3>
> fetchmail 6.3.26-6 on Arch Linux.
>
> To re-create the problem, run this Perl script to listen(2) on TCP
> socket 31415, but not accept(2).
>
>     #! /usr/bin/perl -w
>
>     use Socket;
>
>     socket(Server, PF_INET, SOCK_STREAM, getprotobyname('tcp')) &&
>     bind(Server, sockaddr_in(31415, INADDR_ANY)) &&
>     listen(Server, 0) or die;
>
>     print "sleeping\n";
>     sleep(3600);
>
> Then run fetchmail whilst Perl's sleeping, hitting Enter when prompted
> for a password.
>
>     LC_ALL=C fetchmail -f /dev/null -v -p imap -P 31415 127.1
>
> I see
>
>     Enter password for ralph@127.1: 
>     fetchmail: 6.3.26 querying 127.1 (protocol IMAP) at Wed Jun 13 06:46:17 2018: poll started
>
> After waiting a while I Ctrl-C to send SIGINT.
>
>     ^CTrying to connect to 127.0.0.1/31415...fetchmail: terminated with signal 2
>     $
>
> If that doesn't happen because the connection succeeds then stop
> fetchmail and re-run it;  it's probably the listen(2)'s queue implicitly
> accepting the connection.
>
> My desired output is
>
>     Trying to connect to 127.0.0.1/31415...^Cfetchmail: terminated with signal 2
>
> where the text before `^C' appeared without the SIGINT.
>
Hi Ralph,

Confirmed. Fetchmail buffers that internally, trying to collect a full
log line, with the result printed or syslogged on the same line, that's
why it buffers the part of the line without sending it anywhere. The
thing is also that internally, this isn't error logging...
The immediate but expensive patch for this particular situation is
attached (in case the list strips it, I am Cc'ing you) against the
legacy_64 branch from Git (on gitlab.com), but a proper fix needs more
changes all over the map.

Best regards,
Matthias

Re: [Fetchmail-users] Flush `Trying to connect...' Message.

[Ralph C. <ra...@in...>]

Hi Mattias,

> > If a server is slow to accept a connection then I find the `Trying
> > to connect to 1.2.3.4/993' message only appears on SIGINT whereas it
> > would be useful to know that's what's being attempted during the
> > delay.  Please flush this part of the line rather than having it
> > appear only when the whole line is finished.
> >
> > This `two part line' style of output may occur elsewhere and need
> > flushing benefit from flushing there too.
>
> Please heed <http://www.fetchmail.info/fetchmail-FAQ.html#G3>

fetchmail 6.3.26-6 on Arch Linux.

To re-create the problem, run this Perl script to listen(2) on TCP
socket 31415, but not accept(2).

    #! /usr/bin/perl -w

    use Socket;

    socket(Server, PF_INET, SOCK_STREAM, getprotobyname('tcp')) &&
    bind(Server, sockaddr_in(31415, INADDR_ANY)) &&
    listen(Server, 0) or die;

    print "sleeping\n";
    sleep(3600);

Then run fetchmail whilst Perl's sleeping, hitting Enter when prompted
for a password.

    LC_ALL=C fetchmail -f /dev/null -v -p imap -P 31415 127.1

I see

    Enter password for ralph@127.1: 
    fetchmail: 6.3.26 querying 127.1 (protocol IMAP) at Wed Jun 13 06:46:17 2018: poll started

After waiting a while I Ctrl-C to send SIGINT.

    ^CTrying to connect to 127.0.0.1/31415...fetchmail: terminated with signal 2
    $

If that doesn't happen because the connection succeeds then stop
fetchmail and re-run it;  it's probably the listen(2)'s queue implicitly
accepting the connection.

My desired output is

    Trying to connect to 127.0.0.1/31415...^Cfetchmail: terminated with signal 2

where the text before `^C' appeared without the SIGINT.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [Fetchmail-users] Flush `Trying to connect...' Message.

[Matthias A. <mat...@gm...>]

Am 12.06.2018 um 09:46 schrieb Ralph Corderoy:
> Hi,
>
> If a server is slow to accept a connection then I find the `Trying to
> connect to 1.2.3.4/993' message only appears on SIGINT whereas it would
> be useful to know that's what's being attempted during the delay.
> Please flush this part of the line rather than having it appear only
> when the whole line is finished.
>
> This `two part line' style of output may occur elsewhere and need
> flushing benefit from flushing there too.
>
Please heed <http://www.fetchmail.info/fetchmail-FAQ.html#G3>

[Fetchmail-users] Flush `Trying to connect...' Message.

[Ralph C. <ra...@in...>]

Hi,

If a server is slow to accept a connection then I find the `Trying to
connect to 1.2.3.4/993' message only appears on SIGINT whereas it would
be useful to know that's what's being attempted during the delay.
Please flush this part of the line rather than having it appear only
when the whole line is finished.

This `two part line' style of output may occur elsewhere and need
flushing benefit from flushing there too.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [Fetchmail-users] (Better use --sslcertck!) ?

[Matthias A. <mat...@gm...>]

Am May 8, 2018 9:06:51 AM UTC schrieb mario chiari <ml...@ma...>:
>Hi, 
>
>I get the following warning:
>
>fetchmail: Warning: the connection is insecure, continuing anyways.
>(Better use
>--sslcertck!)
>
>how do I fix it?
>thanks cheers
>mario
>
>------------------------------------------------------------------------------
>Check out the vibrant tech community on one of the world's most
>engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>_______________________________________________
>Fetchmail-users mailing list
>Fet...@li...
>https://lists.sourceforge.net/lists/listinfo/fetchmail-users

By adding that option to the command line, or by adding sslcertck to your rcfile. 

[Fetchmail-users] (Better use --sslcertck!) ?

[mario c. <ml...@ma...>]

Hi, 

I get the following warning:

fetchmail: Warning: the connection is insecure, continuing anyways. (Better use
--sslcertck!)

how do I fix it?
thanks cheers
mario

Re: [Fetchmail-users] GSSAPI SASL auth code adds trailing null byte to authorization name

[Greg H. <gh...@mi...>]

On 04/14/2018 02:51 PM, Matthias Andree wrote:
> I can confirm the two bugs that you are reporting (including the second
> you have reported in private), and have fixed them in Git.
> Please pull the "legacy_64" branch from either Gitlab or Sourceforge and
> see if they fix the issues for you - in which case I should
> forward-merge them onto the master branch as well.

I have built the legacy_64 branch from gitlab and verified that it works
against Exchange 2013 for me.

Re: [Fetchmail-users] GSSAPI SASL auth code adds trailing null byte to authorization name

[Matthias A. <mat...@gm...>]

Am 14.04.2018 um 03:45 schrieb Greg Hudson:
> [I was directed to send bug reports here by
> http://www.fetchmail.info/fetchmail-FAQ.html#G3 ; apologies if that is
> no longer correct.]
>
> In the last step of GSSAPI SASL authentication, the client sends a wrap
> token containing the security level (one byte), the buffer size (three
> bytes), and the authorization name in the remaining bytes.  In
> fetchmail, the construction of the token is at gssapi.c lines 267-280.
> Lines 271-272 are:
>
>     strlcpy(buf1+4, username, sizeof(buf1) - 4); /* server decides if
> princ is user */
>     request_buf.length = 4 + strlen(username) + 1;
>
> The "+ 1" at the end of the length computation causes a trailing null
> byte to be included in the authorization name.  Although a C server
> implementation might tolerate the null byte if it adds its own
> terminator and then treats the result as a C string, Exchange 2013
> rejects the authorization name with the extra null byte and is within
> its rights to do so.  Cyrus SASL does not add an extra null byte, and
> interoperates with Exchange 2013.


Hi Greg,

the reporting address is more or less correct, and the list has relayed
your message and added it to its archives mbox, but it's indeed not
shown in its archives.
I have filed a (private, because it shows mail headers) ticket with the
sourceforge.net site support, and this has been repaired by sf.net ops,
the message has appeared:
https://sourceforge.net/p/fetchmail/mailman/message/36290701/

I can confirm the two bugs that you are reporting (including the second
you have reported in private), and have fixed them in Git.
Please pull the "legacy_64" branch from either Gitlab or Sourceforge and
see if they fix the issues for you - in which case I should
forward-merge them onto the master branch as well.

Repository viewers and clone URLs at:

  * https://gitlab.com/fetchmail/fetchmail
  * https://sourceforge.net/p/fetchmail/git/ci/legacy_64/tree/

Thanks for taking the time to analyse and report this!

Best regards,
Matthias

[Fetchmail-users] GSSAPI SASL auth code adds trailing null byte to authorization name

[Greg H. <gh...@mi...>]

[I was directed to send bug reports here by
http://www.fetchmail.info/fetchmail-FAQ.html#G3 ; apologies if that is
no longer correct.]

In the last step of GSSAPI SASL authentication, the client sends a wrap
token containing the security level (one byte), the buffer size (three
bytes), and the authorization name in the remaining bytes.  In
fetchmail, the construction of the token is at gssapi.c lines 267-280.
Lines 271-272 are:

    strlcpy(buf1+4, username, sizeof(buf1) - 4); /* server decides if
princ is user */
    request_buf.length = 4 + strlen(username) + 1;

The "+ 1" at the end of the length computation causes a trailing null
byte to be included in the authorization name.  Although a C server
implementation might tolerate the null byte if it adds its own
terminator and then treats the result as a C string, Exchange 2013
rejects the authorization name with the extra null byte and is within
its rights to do so.  Cyrus SASL does not add an extra null byte, and
interoperates with Exchange 2013.

Re: [Fetchmail-users] Can fetchmail print or log how many emails it fetched in a run?

[Carlos E. R. <rob...@te...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On Wednesday, 2017-11-15 at 00:20 +0100, I wrote:

> Hi,
>
> I wonder if there is a way to know how many emails a fetchmail run
> fetched? Better if per account. Perhaps printing it to the log, then
> finding that entry. Or printing to the terminal where it runs, whatever.

I found some way:

fetchmail -v ... &
PID=$!
wait
...
grep fetchmail /var/log/mail | grep $PID | grep "message.*for .* at .*" \
   | cut -d" " -f6-


I gets the data for the accounts polled in that run, but before the 
actual poll. Ie, it says how may mails are in the folder, not how many 
were actually fetched.


- -- 
Cheers,
        Carlos E. R.
        (from openSUSE 42.3 x86_64 "Malachite" at Telcontar)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlp/gCUACgkQtTMYHG2NR9W+pACgjjE9p3DG7ap8Vtstnl7cL50T
91UAnRY/yJdJlegA1V8HLUcbGWOJF1ky
=jQaE
-----END PGP SIGNATURE-----

Re: [Fetchmail-users] fetchmail, and multiple Kerberos accounts

[Matthias A. <mat...@gm...>]

Matěj Cepl wrote:
> When Fedora enabled Kerberos on id.fedoraproject.org, I have
> immediately started to use it, and I am suspicious I am now
> member of rather exclusive group of people with multiple Kerberos
> tickets on their system (my completely unevidenced suspicion is
> that 99% of Kerberos users have only one ticket from their
> employer/school).
>
> On the advice of somebody (or some Mojo document, not sure) I
> have added configuration files like (also similar ones for
> REDHAT.COM realm)
>
>     mitmanek:~# cat /etc/krb5.conf.d/fedoraproject_org 
>     [realms]
>      FEDORAPROJECT.ORG = {
>             kdc =     https://id.fedoraproject.org/KdcProxy
>  }
> [domain_realm]
>  .fedoraproject.org = FEDORAPROJECT.ORG
>  fedoraproject.org = FEDORAPROJECT.ORG
> mitmanek:~# 

Hi Matěj,

I think that is quite a typical setup, I had similar things in my
krb5.conf while still working at a University that used Kerberos, and
managed several domains, too (albeit with a hierarchical LDAP behind).

> Some programs (Gnome apps after some small amount of torture of
> the glib-networking maintainer, Firefox) seems to work perfectly
> well, using the right ticket for the right domain, but some other
> apps are hopelessly lost facing multiple Kerberos tickets.
[...]
> When I presented one of my colleagues working with Kerberos (I
> work for Red Hat), he answered me with these notes:
>
[...]
> 2. If fetchmail is using exclusively GSSAPI calls, then you
> should open ugs against our krb5 packages (as gssapi is provided
> by those) at first. There are some ways to use GSSAPI that work
> better than others, but in general clients should just work.

Fetchmail has code in place for three ways of using Kerberos:

* Kerberos IV (4), which is deemed obsolete and I won't make any
promises that it works

* Kerberos V (5), of which I am somewhat suspicious function-wise, which
would be calling krb5_*() functions, and is broken on some of the KRB5
implementations, for instance, I marked it b0rked on Heimdal.

* GSS-API, with whatever providers it will use and which I'd say is the
thing that should work, and which I would be willing to help with. The
code is self-contained in gssapi.c, with a few calls into it from pop3.c
or imap.c, the guts of the code are, however, 14...17 years old.

For any debugging and fixing, I propose to look at the "legacy_64"
branch for now, since it has several I'd say worthwhile fixes, OpenSSL
overhaul, and I will forward-port potential GSSAPI fixes for ticket
selection from that branch to the master branch.

Repositories at:
https://sourceforge.net/p/fetchmail/git/ci/legacy_64/tree/
https://gitlab.com/fetchmail/fetchmail

> 3. If fetchmail is doing any direct libkrb5 calls, or running any
> kinit/klist command line tools then yeah they should stop.

Yup, configure --without-kerberos --without-kerberos5 --with-gssapi then
to fix this part.

> 4. In general IMAP/POP use SASL, and most software uses cyrus-
> sasl to deal with it. Cyrus sasl will use only GSSAPI calls in
> this case. However if upstream built their own SASL
> library/wrappers ... then I would perhaps rather consider
> dropping its use rather than  try to fix it ... they probably
> have way bigger security issues.

fetchmail does not use SASL libraries, authentication protocols are
implemented inside fetchmail, leaving aside most of the crypto stuff
(fetchmail uses the KRB, GSSAPI, and OpenSSL libraries).

HTH
Matthias

[Fetchmail-users] fetchmail, and multiple Kerberos accounts

[Matěj C. <mc...@ce...>]

Hi,

When Fedora enabled Kerberos on id.fedoraproject.org, I have
immediately started to use it, and I am suspicious I am now
member of rather exclusive group of people with multiple Kerberos
tickets on their system (my completely unevidenced suspicion is
that 99% of Kerberos users have only one ticket from their
employer/school).

On the advice of somebody (or some Mojo document, not sure) I
have added configuration files like (also similar ones for
REDHAT.COM realm)

    mitmanek:~# cat /etc/krb5.conf.d/fedoraproject_org 
    [realms]
     FEDORAPROJECT.ORG = {
            kdc =     https://id.fedoraproject.org/KdcProxy
 }
[domain_realm]
 .fedoraproject.org = FEDORAPROJECT.ORG
 fedoraproject.org = FEDORAPROJECT.ORG
mitmanek:~# 

Some programs (Gnome apps after some small amount of torture of
the glib-networking maintainer, Firefox) seems to work perfectly
well, using the right ticket for the right domain, but some other
apps are hopelessly lost facing multiple Kerberos tickets.

When I start Gnome session, GOA collects both Kerberos tickets,
but plain klist command shows nothing. I have to run klist -A and
that shows me both tickets. When I try to run some programs,
fetchmail among them, they fail because they are apparently not
able to find the appropriate ticket. I have to run kswitch -p
PRINCIPAL (and plain klist showing it) to get these other
programs working.

When I presented one of my colleagues working with Kerberos (I
work for Red Hat), he answered me with these notes:

1. Any application directly using kerberos calls will probably
fail this way, they need to have the right creds in the "default"
cache, which is why you need to use kswitch.

2. If fetchmail is using exclusively GSSAPI calls, then you
should open ugs against our krb5 packages (as gssapi is provided
by those) at first. There are some ways to use GSSAPI that work
better than others, but in general clients should just work.

3. If fetchmail is doing any direct libkrb5 calls, or running any
kinit/klist command line tools then yeah they should stop.

4. In general IMAP/POP use SASL, and most software uses cyrus-
sasl to deal with it. Cyrus sasl will use only GSSAPI calls in
this case. However if upstream built their own SASL
library/wrappers ... then I would perhaps rather consider
dropping its use rather than  try to fix it ... they probably
have way bigger security issues.

Does anybody have any understanding of how Fetchmail works with
Kerberos? Do you use some libraries, or does fetchmail do all the
work itself?

Best,

Matěj

-- 
http://matej.ceplovi.cz/blog/, Jabber: mcepl<at>ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
 
Oh, to be young, and to feel love’s keen sting.
  -- Albus Dumbledore

Re: [Fetchmail-users] Using Fetchmail with IMAP

[Matthias A. <mat...@gm...>]

Am 13.12.2017 um 04:04 schrieb Kamil Jońca:
>
> (I cannot push to gitlab - I am not sure if I make everything properly)

Hi Kamil,

You cannot push into my repository - with most Git repositories, the
model is:
you fork/branch/clone your own repository from mine, push into your own
and then send me a so-called "pull request" telling me to merge.

> At 
> https://drive.google.com/file/d/1KanPVHBXLhTbkCFlaXSySEys4pdlmlF-/view?usp=sharing
>
> are patch made against master branch, and also there is some
> corrections:
> 1. uid file is not hardcoded now, but its name is derived from uidfile
> with "-imap" suffix
> 2. functions which  read/write imap uids have better debug
> 3. sanity checks when remove "\n" from data lines

Dziękuję! I hope to evaluate this over the week-end.

Re: [Fetchmail-users] Using Fetchmail with IMAP

[<kj...@o2...>]

Matthias Andree <mat...@gm...> writes:


> Hello Kamil,
>
> thanks for taking the initiative. I've been meaning to respond earlier,
> but real live has interfered, and unfortunately I wasn't able to tell
> you in time that this should either be against the "master" branch (if
> intrusive) or the "legacy_64" branch (if just an extension isolated well
> from the rest) because 6.3.26 is basically a dead end, and I have merged

Yes, but this version is currently in Debian, and I wanted to use
something I can revert quickly. :)

>
> So, if you want to play with things and are happy to use Git, I propose
> that you clone out from https://gitlab.com/fetchmail/fetchmail and
> create a new kj-imap-uids branch there branching off RELEASE_6-3-26, and

(I cannot push to gitlab - I am not sure if I make everything properly)

At 
https://drive.google.com/file/d/1KanPVHBXLhTbkCFlaXSySEys4pdlmlF-/view?usp=sharing

are patch made against master branch, and also there is some
corrections:
1. uid file is not hardcoded now, but its name is derived from uidfile
with "-imap" suffix
2. functions which  read/write imap uids have better debug
3. sanity checks when remove "\n" from data lines



KJ


-- 
http://stopstopnop.pl/stop_stopnop.pl_o_nas.html
Password: