Re: FULL versus SMART (was Re: [Sqlgrey-users] Spammer alex@hotmail.com benefits from from_awl)

[Lionel B. <lio...@bo...>]

Michael Storz wrote the following on 29.04.2005 11:28 :

>- Higher delay FULL can have compared to SMART:
>  The right medizine against that are good whitelists!
>

whitelists are good for delays I agree. But until you see a new IP, you
don't put it in your whitelists...

> Good means the
>  number of whitelists and the content of whitelists. In addition to the
>  standard sqlgrey algorithms for filling the tables via traffic analysis,
>  we have implemented our own algorithms :-)
>
>  * fast propagation (fills from_awl): This algorithm is based on the
>    trust we have about a sending MTA. If we trust it, we accept the
>    email, even if there is no entry about this triple in the whitelists.
>  * MX-check (fills domain_awl): if outgoing and incoming MTAs are the
>    same, put an entry in domain_awl.
>  * A-check (fills domain_awl): if sending MTA sends emails for its
>    hostname only, put it in domain_awl.
>
>  These additional algorithms give us a lot of entries in our from_awl and
>  domain_awl and therefore reduce the delay significantly. And the last 2
>  algorithms only work with FULL, not with SMART, with the current design
>  of sqlgrey. Sorry for you, guys :-)
>
>  
>

How do you implement them ? One way to do it would be to modify
SQLgrey's log to log the whole IP address of each new connection (in
fact I was sorry to not have them in log, so I'll have the log format
changed to do this) and have a little daemon tailing the log file
looking for
"^<syslog_header> sqlgrey: grey: new: (<ip>) .*" lines
doing DNS requests and adding entries in the database.

>  About additional whitelists, forward_awl/rcpt_awl is one of them. At the
>  moment fast propagation replaces this table, because most of the time we
>  accept immidiately all the spam mails from forwarding where the remote
>  MTA does not use greylisting, but at the cost of many unnecessary
>  entries in from_awl.
>
>  Another one would be prevalidation as implented in other greylisting
>  software. here you put the tuple originator/recipient without an ip
>  address in a table for every email you send out.
>  
>

Interesting. How is it done? The policy service is called before
permit_mynetworks and the policy service is taught the value of
"mynetworks" ?
This can be problematic :
Here is part of my smtpd_recipient_restrictions :
(...)
        permit_mynetworks
        reject_unknown_recipient_domain
        reject_unauth_destination
        check_helo_access hash:/etc/postfix/smtp_helo_blacklist
        reject_unlisted_recipient
        check_policy_service inet:127.0.0.1:2501

If I move check_policy_service before permit_my_networks, I'll have lots
of crap passing through the policy service which will be rejected later.
One solution would be to have two separate policy services, one called
before permit_mynetworks which would only process "mynetworks" sources
in order to whitelist some tuples and the classic greylister later.
That's for the distant future of SQLgrey :-)

>- Trust in the sending MTA:
>  SMART reduces trust in the sending MTA about the handling of temporary
>  errors in a well behaved way based on the relevant RFCs.
>
>  Well behaved means
>
>  * trying to retransmit a message several times until a timeout of 3 - 5
>    days occur.
>  * retransmitting emails in a timely manner (minutes) and not only once
>    in 24 hours
>
>  For me this is the reason to use FULL. I want to get trust in the
>  sending MTA, the more the better. And if I have trust in a MTA, I will
>  accept emails as fast as I can.
>
>  
>

I understand your goals.

>  Actually, what I want is to strengthen the trust in the sending MTA,
>  e.g. to use the domain from HELO/ELHO
>

how do use this domain (isn't it an hostname btw)? I'm thinking about
MXs/relays for several domains.

Lionel.