Re: [Sqlgrey-users] Comments and features requests

[Michel B. <mi...@bo...>]

Le Lundi 10 Janvier 2005 15:08, Lionel Bouton a =E9crit :
>
> >2/ The latest version of Postgrey allows specifying the result (both code
> > and text) that Postgrey should give when a connection is greylisted. It
> > would be nice if SQLgrey could do the same.
> >Currently SQLgrey returns "defer_if_permit" and this cannot be modified
> >(except by fiddling in the code).
> >However, some may prefer to reject immediately with a 450 temporary
> > failure, rather than carry on with further potentially expensive Postfix
> > tests (such as checking external DNSBLs or verifying sender existence),
> > where the connection will in the end be rejected anyway...
>
> I suppose you use a 550 rejection at least with DNSBLs. In this case
> you'll lose time by forcing a temporary failure instead of rejecting on
> the first connection.

Let me explain my view : The huge majority of trash that my mailserver=20
currently receives comes from zombie virus-sending Windows machines, or=20
spambots that don't have a normal retry pattern. The whole point in=20
greylisting is precisely to eliminate them, because they will *NOT* retry. =
So=20
rejecting their mail attempt with any code, either 450 or 550 results in th=
e=20
same effect : They don't come back (at least using the same from/to/IP).

I personally prefer to put first in my Postfix series of tests all the test=
s=20
that can be "local and quick", and only for messages that passes these,=20
perform tests that may be "longer or network or resources consuming", such =
as=20
sender address verification.

That's why I'd like that a "Greylisted" event could immediately reject the=
=20
mail on first attempt (I wouldn't mind a "defer_if_permit" on an "early=20
reconnection" BTW, because it would allow me to check sender now that it is=
=20
probable that the same mail will come back again).

=46urthermore, if Greylisting caused an immediate reject, it would permit m=
e not=20
to uselessly bother remote servers with address verification for addresses=
=20
that are dummy, and messages that won't come back.

=46urthermore, I prefer not to "pollute" my address verification DB where i=
t can=20
be avoided, to prevent it from growing too much. SQLgrey uses a nice=20
PostgreSQL database on my system, it's size is no problem and it can be=20
easily maintained (vacuum analyze etc...), which is not the case of the=20
Postfix address verification daemon DB, which uses a simpler BerkeleyDB.

Last but not least, I don't mind rejecting first with a 450 a spam that wil=
l=20
actually come back and that I will reject with a 554 on 2nd attempt (origin=
=20
blacklisted, non-existent sender...). By rejecting the message twice, I the=
n=20
put a double load on the spammer's server, and I don't hate the idea of=20
giving them more work to get rejected twice ;-)

> With sender verification it can be more complex, I'm not sure of the
> cases where 450 and 550 are used.

Basically the Postfix verification daemon gives the same family of error th=
at=20
the remote server gave when performing verification. If the verification=20
failed with a 5xx, Postfix will issue a 5xx. If the verification failed wit=
h=20
a 4xx (or could not be done), Postfix will issue a 4xx.

> If the sender verification use 550,=20
> you end up in the same case than the one described above : you lose time
> and performance if a 450 is used then it will save one 450 but not the
> subsequent ones.
>
> So in the end, I'm not sure you will gain much from it or if in fact you
> won't lose something...
> But it's a simple thing to add, so if you find a case where it clearly
> is beneficial, I'll code it.

Thanks :-)


> >3/ It would be nice to have a whitelist of senders (using regexps) for
> > whom greylisting should never be done. This list should by default
> > include the addresses that some MTA use to perform "sender address
> > verification", as we should avoid greylisting those verifications as mu=
ch
> > as possible, the combination of greylisting and sender address
> > verification introducing problems and rather long delays. These
> > verification origin addresses are, among the most common:
> ><> (null sender)
> >MAILER-DAEMON@*
> >postmaster@*
>
> I don't think greylisting and sender address verification clash with
> each other.

=46rom my experience they actually do.

> The common problem is the following :=20
> Server A uses greylisting, Server B uses sender address verification.
> - Server A formard a mail to Server B,
> - Server B wants to check that the sender exists and Server A is the MX
> for the corresponding domain, Server A is not in the auto-whitelists of
> Server B
> - Server A don't trust Server B yet, so Server B verification fails and
> (temporarily) refuses the mail,
> - Server B will retry (around 20 minutes later with default Postfix
> configuration under light load), then Server A will be able to verify
> (and will be for 60 days and more with default SQLgrey config), the mail
> gets through.

No, because server B will probably *NOT* verify again so quickly. Server B=
=20
will probably have a "negative cache TTL" telling it not to retry a failed=
=20
verification before a given delay (I personally use 3 hours delay) to avoid=
=20
repeteadly bothering remote servers with possible repeated requests for the=
=20
same non-existent address.

If this scheme, what happens is as follows:
=2D Server A wants to send mail to server B
=2D Server B tries to verify sender address on server A, but gets greyliste=
d,=20
and its verification receives a "450".
=2D Server B keeps this 450 in cache for its "negative TTL time" (let's say=
 3=20
hours).
=2D Server A will retry several times to send its mail to server B, and wil=
l be=20
refused with 450 immediately as long as the sender address verification=20
negative TTL isn't expired.
=2D After at least 3 hours (but some sending servers may have retry delays =
that=20
grows exponentially, such as qmail, so it could well be 8 or 10 hours),=20
server A will retry, server B will retry its verification, and, provided=20
server A satisfied the verification fast enough, the mail will in the end b=
e=20
accepted. If server A is a bit long to answer, the verification will timeou=
t=20
and be finished asychronously, the mail will be refused once more, and serv=
er=20
A will have to retry once more to send it.

With the result that on average, the mail from A to B will not have been=20
delayed by "5 minutes or so" by the greylisting made on A, but rather by 4=
=20
hours or more (after a dozen of attempts), by the combination of greylistin=
g=20
on A, verification on B, and negative TTL on B's verification.

Well, 4 hours or so, that's becoming a delay long enough to cause trouble t=
o=20
many sysadmins ;-))

> The problem with adding hard whitelists for origin addressses is that
> you will have many SPAMs coming through (postmaster, MAILER-DAEMON and
> <> are used rather often).

Not that often on my experience. "postmaster" has actually been used by rec=
ent=20
worm-viruses (as well as hostmaster and webmaster), but I have seen very fe=
w=20
spams or viruses "From: <>" and even less "From: MAILER-DAEMON@" (talking=20
about SMTP enveloppe Froms).

Thanks again for SQLgrey, I like it ;-)

=2D-=20
Michel Bouissou <mi...@bo...> OpenPGP ID 0xDDE8AC6E

Les taxis de la Marne ont fait la guerre.
Alors moi je rigole quand je vois un taxi qui a peur d'aller en banlieue.