fetchmail-users

[Fetchmail-users] 2 feature requests for 7.x

[Martin K. <mk...@gm...>]

Hi,

in my configuration I use fetchmail to fetch mails for several users 
from several servers in one rcfile. So lets say one rcfile has e.g. 50 
users and 5 servers, so 10 users per server. (I have several of those 
configurations.)

As I just read the message:

   Working around "Bad sender address syntax" error?

I would like to add 2 wishlist items for fetchmail 7.x:


(1) When it will be possible to repack broken messages into a new mail 
as an attachment (I like this idea, and I also encountered broken 
headers/addresses several times), I wish to be able to inform users in 
that same way, when a message has not been fetched because of size 
limit.


(2) Looking at the new features of 7.x shows that ssl options were 
changed. So maybe another change could be done, if it already changed 
anyway:

With fetchmail 6.3 I see a limitation because --ssl is not 
a server option but a user option. When e.g. 2 of the 5 servers in my 
example above need a different ssl configuration, as they may not 
support ssl at all or have a self signed certificate, I currently must 
add this to _every_ user definition:


defaults:
 	proto pop3 timeout 300 sslproto 'TLS1' ssl
 	sslcertfile /usr/ssl/certs/ca-bundle.trust.crt
 	# sslcertck
 	# sslfingerprint "82:5F:*"


poll fully-ssl-compatible-server:
 	user "s1a" there with pass "*" is "s1...@do..." here
 	sslcertck
 	nokeep fetchall

 	user "s1b" there with pass "*" is "s1...@do..." here
 	sslcertck
 	nokeep fetchall


poll self-signed-certificate-server:
 	user "s2a" there with pass "*" is "s2...@do..." here
 	sslfingerprint "82:5F:*"
 	nokeep fetchall

 	user "s2b" there with pass "*" is "s2...@do..." here
 	sslfingerprint "82:5F:*"
 	nokeep fetchall



So "sslcertck" and "sslfingerprint" must be repeated over and over. 
Also if one server doesn't support ssl at all, it's even worse. I 
can't specify "ssl" in defaults section, because there is no (or at 
least not to my knowledge) no-ssl keyword, which I could apply to all 
user accounts on that server. So instead I added "ssl" to every user 
account on every other server, just because one server doesn't support 
it. "sslcertck" is the same.


IMO all these SSL related options should be server options, not user 
options, as they apply before any user name is transmitted to the 
server, so there should be no need have it configured differently for 
every user. Ok, the server could behave diferently after user logon 
depending on the previously negotiated connection. But then I could 
have 2 different poll sections for the same server.


Martin

Re: [Fetchmail-users] 2 feature requests for 7.x

[grarpamp <gra...@gm...>]

On Wed, Sep 10, 2014 at 4:47 AM, Martin Koeppe <mk...@gm...> wrote:
> (2) Looking at the new features of 7.x shows that ssl options were
> changed. So maybe another change could be done, if it already changed
> anyway:
>
> With fetchmail 6.3 I see a limitation because --ssl is not
> a server option but a user option. When e.g. 2 of the 5 servers in my
> example above need a different ssl configuration, as they may not
> support ssl at all or have a self signed certificate, I currently must
> add this to _every_ user definition:
>
> defaults:
>         proto pop3 timeout 300 sslproto 'TLS1' ssl
>         sslcertfile /usr/ssl/certs/ca-bundle.trust.crt
>         # sslcertck
>         # sslfingerprint "82:5F:*"
> poll fully-ssl-compatible-server:
>         user "s1a" there with pass "*" is "s1...@do..." here
>         sslcertck
>         nokeep fetchall
> poll self-signed-certificate-server:
>         user "s2a" there with pass "*" is "s2...@do..." here
>         sslfingerprint "82:5F:*"
>         nokeep fetchall
>
> So "sslcertck" and "sslfingerprint" must be repeated over and over.
> Also if one server doesn't support ssl at all, it's even worse. I
> can't specify "ssl" in defaults section, because there is no (or at
> least not to my knowledge) no-ssl keyword, which I could apply to all
> user accounts on that server. So instead I added "ssl" to every user
> account on every other server, just because one server doesn't support
> it. "sslcertck" is the same.
>
> IMO all these SSL related options should be server options, not user
> options, as they apply before any user name is transmitted to the
> server, so there should be no need have it configured differently for
> every user. Ok, the server could behave diferently after user logon
> depending on the previously negotiated connection. But then I could
> have 2 different poll sections for the same server.

I already proposed, with syntax, separating 'user definitions' at
'server definitions'
abstractions years ago for 7.x. I think I split out three classes of
things, brought
them to the config file and command line, redfined what 'poll' meant, etc.
It (and a bunch of fingerprint and TLS handling stuff) is in some tickets that I
think got destroyed when the ticket system got destroyed/moved, or maybe
on this list, or something like that, I don't remember. The idea is worthwhile
and needed.

Re: [Fetchmail-users] 2 feature requests for 7.x

[Matthias A. <mat...@gm...>]

Am 15.09.2014 um 04:17 schrieb grarpamp:

> I already proposed, with syntax, separating 'user definitions' at
> 'server definitions'
> abstractions years ago for 7.x. I think I split out three classes of
> things, brought
> them to the config file and command line, redfined what 'poll' meant, etc.
> It (and a bunch of fingerprint and TLS handling stuff) is in some tickets that I
> think got destroyed when the ticket system got destroyed/moved, or maybe
> on this list, or something like that, I don't remember. The idea is worthwhile
> and needed.

Dear grarpamp,

thank you for the reminder.  I downloaded the ticket data from BerliOS
but have not yet found a way to recreate Sourceforge tickets from them
authentically, the instructions that were given on sf.net in May were a
bit vague and not at all turnkey solutions, quite on the contrary.

Best,
Matthias

Re: [Fetchmail-users] 2 feature requests for 7.x

[Matthias A. <mat...@gm...>]

Am 10.09.2014 um 10:47 schrieb Martin Koeppe:

> So "sslcertck" and "sslfingerprint" must be repeated over and over. 
> Also if one server doesn't support ssl at all, it's even worse. I 
> can't specify "ssl" in defaults section, because there is no (or at 
> least not to my knowledge) no-ssl keyword, which I could apply to all 
> user accounts on that server. So instead I added "ssl" to every user 
> account on every other server, just because one server doesn't support 
> it. "sslcertck" is the same.

I thought I had checked for missing no-* options but apparently I missed
those.  These might even be 6.3.X stuff depending on how complex the fix
will be in code, I will check that.

> IMO all these SSL related options should be server options, not user 
> options, as they apply before any user name is transmitted to the 
> server, so there should be no need have it configured differently for 
> every user. Ok, the server could behave diferently after user logon 
> depending on the previously negotiated connection. But then I could 
> have 2 different poll sections for the same server.

I have always wondered if we should be adding some notion of scope to
settings, but this would then have to go along with parenthesizing
(perhaps using curly braces) of some form, perhaps apply defaults ... to
... end defaults statements.

Sooner or later I will have to rewrite the bison parser because it
invariably must leak memory and has some minor quirks, but have not
decided on a solution.

Lemon (from sqlite3's author) is one option, going for C++ with
Boost::Spirit that follows a strict full resource tracking through
objects is another, and there are surely more solutions.

This will entail a parser rewrite and might be a good point in time to
specify such extensions.

(Bison recommends to consider garbage collection, but I'm not sure that
it's the right approach, it can lead to apparently nondeterministic
timing behaviour.)

Thanks for your suggestions!

Re: [Fetchmail-users] 2 feature requests for 7.x

[Martin K. <mk...@gm...>]

On Mon, 15 Sep 2014, Matthias Andree wrote:
> Am 10.09.2014 um 10:47 schrieb Martin Koeppe:
>
>> IMO all these SSL related options should be server options, not user
>> options, as they apply before any user name is transmitted to the
>> server, so there should be no need have it configured differently for
>> every user. Ok, the server could behave diferently after user logon
>> depending on the previously negotiated connection. But then I could
>> have 2 different poll sections for the same server.
>
> I have always wondered if we should be adding some notion of scope to
> settings, but this would then have to go along with parenthesizing
> (perhaps using curly braces) of some form, perhaps apply defaults ... to
> ... end defaults statements.
>
> Sooner or later I will have to rewrite the bison parser because it
> invariably must leak memory and has some minor quirks, but have not
> decided on a solution.


If you need to / want to do some incompatible changes anyway, why not 
switch to a totally different config file format? The one that 
bind uses (basically C like) would be my favorite. With the C-style 
/* multi-line comments */ the skip keyword isn't needed anymore IMO.
Also powershell comments would be ok: "#" for single line, "<#" start 
comment, and "#>" end comment.

Then a config could look like this:

{
 	server = "pop.example.com"
 	ssl = "none"

 	poll {
 		user = "pop-server-user"
 		pass = "SECRET"
 		sendto = "us...@do..."
 	}

 	/* nested defaults */
 	{
 		ssl = starttls
 		sendto = "us...@do..."
 		poll {
 			user = "user2"
 			pass = "123456"
 			verbose = yes // to debug this particular poll
 		}

 		poll {
 			user = "user3"
 			pass = "7890"
 			keep = yes // overwrite default
 		}
 		// continue nested defaults block
 		keep = no
 		fetchall = yes
 	} // end of nested defaults block
}
{
 	server = "another.server.com"
 	poll {
 		/* ... */
 	}
}

There are 2 types of blocks, default blocks (unnamed above) which can 
be thought as inner tree nodes, and poll blocks (leaf nodes). The 
whole file can be thought of being in a big default block (root node), 
where the compiled-in defaults are listed. No limit for the nesting 
level. Poll blocks must not contain any other blocks. A poll is done 
for each poll block.

No difference between server and user options, as it may be desirable 
to move e.g. the account name used on several servers to a default 
block. The poll block must just have enough options in it and above 
in the tree to do the poll.


A minimal config for only one account could just look like:
poll {
 	server = "servername"
 	user = "user"
 	pass = "*"
 	sendto = "loc@l"
}

Or even like this:

server = "servername"
user = "user"
pass = "*"
/* sometimes the same password is used for several accounts */
sendto = "loc@l"
poll {}


I only use fetchmail for POP3 polls with single-drop. So maybe not all 
of fetchmail's functions can be covered with this. Block names and 
option names can be changed of course. A ";" like in C might be 
appended to each line. Or, as in powershell, is optional, unless you 
want to specify 2 options on one line, where you could delimit both of 
these with a ";". OTOH if there are multi-line values 
(certificates?), then always a terminating ";" might be good, unless 
the block ends anyway.

Just as an idea.

Martin