Re: [Sqlgrey-users] warn_if_reject to populate the db

[Lionel B. <lio...@bo...>]

Michel Bouissou wrote the following on 26.06.2005 15:37 :

>>1. localhost:2501 vs. Unix socket
>>Wouldn't a socket be slightly faster than TCP?
>>    
>>
>
>Probably, but works only if on the same machine. TCP is more general, so to 
>use sockets, Lionel would need to implement both methods. Given the little 
>amount of data transferred between Postfix and sqlgrey for a given 
>connection, using sockets would probably make a difference only on very high 
>volume system.
>
>  
>

Even there the difference would probably be unnoticable. There are far
more complex computations involved than a TCP/IP connection. Putting
back Unix sockets would be easy, but it will be one more parameter to
set up...

>>2. Running under control of master(8)
>>That would be convenient, start and stop with Postfix; are there other
>>benefits? Why the standalone choice?
>>    
>>
>
>Running under the control of master would be interesting to me as well.
>
>  
>

Being standalone allows SQLgrey to be put anywhere the admin see fit.
Although running under the control of master would be a definitive plus.
I'll add it to my TODO.

>>3. Database population commands
>>I'm totally lost with SQL (hence the poor choice of mysql), can someone
>>help with the manual commands I'd use to add to the database?
>>    
>>
>
>Reading the fine MySQL doc would probably help with the basic SQL commands, 
>but you'd better let the DB populate by itself.
>
>BTW, why state that MySQL is a poor choice ?
>  
>

It's often buggy and doesn't follow the standard ?

>  
>
>>4. Database population scripts
>>Is there something I could run against user's maildirs which would add
>>entries to the AWL? If not should I commission such a project from my
>>private farm of Perl coders[2]; I mean, would there be interest?
>>    
>>
>
>I don't think there's any interest in trying to artificially populate the DB. 
>Just let it run ;-)
>
>  
>

One could imagine various ways of (dynamically) modifying the DB to fine
tune the greylisting processus. Everyone is welcomed to experiment with
this, but I've no guidelines on this subject, the keyword is 'experiment'.

>>6. dyn_fqdn.regexp
>>That's quite an expression.
>>    
>>
>
>;-)
>  
>

I know Michel is proud of this one :-)

>  
>
>>I didn't figure the whole thing out, but I 
>>did look for a string "\.res\." which is commonly used for dynamic
>>space, e.g., *.res.rr.com. (residential customers.) Perhaps the second
>>dot should be any non-alpha character (-, _, ., digit), and to be safe
>>there should be at least 2 domain segments following and at least one
>>segment preceding (implied by the leading dot.)
>>    
>>
>
>I don't pretend that the regexp is perfect, it's only heuristic, but it would 
>be interesting adding your modification only if you find samples of existing 
>hostnames that don't get properly classified. (i.e. your example may already 
>get classified corresctly if the last byte of the IP address is part of the 
>name) The more things you add in the regexp, the longer it will take to 
>process, and the higher chances it could collide with other non-dynamic names 
>in an undesired manner.
>
>But feel free to experiment with your own copy and suggest improvements that 
>show useful...
>  
>

Yep, we can even redistribute the modifications you make to clients
running sqlgrey_update_config.

>>8. Beyond grey
>>This is a biggie which probably warrants its own thread. This is all
>>about spam abatement. What about integrating other antispam strategies
>>under the roof of the same policy service? Yes, this belongs in its own
>>thread. I'll write more of my thoughts about that later.
>>    
>>
>
>I think about it the Unix way. I prefer to use several distinct tools of my 
>choice, each one doing _one_ thing and doing it well. I wouldn't like any 
>bigger system that would integrate different methods, I personally prefer 
>doing my own cooking.
>
>Postfix can call several "policy services" without any problem...
>
>  
>

Combining results of various checks at the Postfix level is rather
cumbersome. This is why I added a reference to SPF in my TODO: my idea
was that it wouldn't bring much benefit to greylist already known good
MTAs. We could combine a domain whitelist with SPF checks: if the source
domain is in the whitelist and the SPF checks are OK, don't greylist.
This is far in the future, though, I'll need to break SQLgrey into
modules first.

>>Thanks, Lionel, this looks good so far. I went live with a small but
>>heavily-spammed domain yesterday evening, and no spam has been seen
>>there since. (The sqlgrey is last in a long list of restrictions with
>>numerous RBL checks.)
>>    
>>
>
>My 2 cents: Put SQLgrey _before_ RBLs. You'll save external network calls so 
>your system will run faster, and you'll save unnecessary load onto the RBL 
>servers as well...
>  
>

That would be my recommendation too. Only put RBLs first if you want to
have statistics about how much more greylisting brings to you if you
have RBLs already defined.

Lionel