sqlgrey-users

Re: [Sqlgrey-users] sqlstats need explanation

[Len C. <lc...@Go...>]

>Len Conrad a écrit, le 06/01/2009 05:49 PM :
>> /usr/local/bin/sqlgrey-stats.sh
>>
>> GREY NEW:          70491
>> GREY EARLY RECON:  790
>> GREY RECON OK:     954
>> GREY DOMAWL:       7805
>> GREY FROM AWL:     795
>> WHITELIST:         191
>> SPAM:              33515
>> SMTPD GREYLISTED:  4015
>>
>> ... for about 11 hours Monday morning.
>>   
>
>I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
>report. That said the categories seem to match SQLgrey's own log
>categories so I assume it makes sums of log lines matching these.
>
>> I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.  
>>
>> For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?
>>   
>
>Why do you think there aren't ? Nothing above supports this claim: for
>each "grey new" and "grey early recon" line the server should return a
>temporary reject. If it doesn't, then it's a SMTP server configuration
>issue.

NEW=`egrep -ic "sqlgrey: grey: new:"             /var/log/mx1.hctc.net/maillog`
EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/mx1.hctc.net/maillog`
RECON=`egrep -ic "sqlgrey: grey: reconnect ok:"    /var/log/mx1.hctc.net/maillog`
DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/mx1.hctc.net/maillog`
FRMAWL=`egrep -ic "sqlgrey: grey: from awl:"        /var/log/mx1.hctc.net/maillog`
WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/mx1.hctc.net/maillog`
SPAM=`egrep -i "sqlgrey: spam:" /var/log/mx1.hctc.net/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/mx1.hctc.net/maillog`


Len

Re: [Sqlgrey-users] sqlstats need explanation

[Len C. <lc...@Go...>]

---------- Original Message ----------------------------------
From: Lionel Bouton <lio...@bo...>
Date:  Mon, 01 Jun 2009 19:38:38 +0200

>Len Conrad a écrit, le 06/01/2009 05:49 PM :
>> /usr/local/bin/sqlgrey-stats.sh
>>
>> GREY NEW:          70491
>> GREY EARLY RECON:  790
>> GREY RECON OK:     954
>> GREY DOMAWL:       7805
>> GREY FROM AWL:     795
>> WHITELIST:         191
>> SPAM:              33515
>> SMTPD GREYLISTED:  4015
>>
>> ... for about 11 hours Monday morning.
>>   
>
>I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
>report. That said the categories seem to match SQLgrey's own log
>categories so I assume it makes sums of log lines matching these.
>
>> I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.  
>>
>> For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?
>>   
>
>Why do you think there aren't ? Nothing above supports this claim: for
>each "grey new" and "grey early recon" line the server should return a
>temporary reject. If it doesn't, then it's a SMTP server configuration
>issue.
>

ok, found my misconfig:

changed these from "delay" to:

reject_first_attempt = immed
reject_early_reconnect = immed 


"greylisted" smtpd log lines are streaming up the screen now.

thanks
Len

Re: [Sqlgrey-users] sqlstats need explanation

[Lionel B. <lio...@bo...>]

Len Conrad a écrit, le 06/01/2009 08:05 PM :
>> Len Conrad a écrit, le 06/01/2009 05:49 PM :
>>     
>>> /usr/local/bin/sqlgrey-stats.sh
>>>
>>> GREY NEW:          70491
>>> GREY EARLY RECON:  790
>>> GREY RECON OK:     954
>>> GREY DOMAWL:       7805
>>> GREY FROM AWL:     795
>>> WHITELIST:         191
>>> SPAM:              33515
>>> SMTPD GREYLISTED:  4015
>>>
>>> ... for about 11 hours Monday morning.
>>>   
>>>       
>> I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
>> report. That said the categories seem to match SQLgrey's own log
>> categories so I assume it makes sums of log lines matching these.
>>
>>     
>>> I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.  
>>>
>>> For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?
>>>   
>>>       
>> Why do you think there aren't ? Nothing above supports this claim: for
>> each "grey new" and "grey early recon" line the server should return a
>> temporary reject. If it doesn't, then it's a SMTP server configuration
>> issue.
>>     
>
> NEW=`egrep -ic "sqlgrey: grey: new:"             /var/log/mx1.hctc.net/maillog`
> EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/mx1.hctc.net/maillog`
> RECON=`egrep -ic "sqlgrey: grey: reconnect ok:"    /var/log/mx1.hctc.net/maillog`
> DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/mx1.hctc.net/maillog`
> FRMAWL=`egrep -ic "sqlgrey: grey: from awl:"        /var/log/mx1.hctc.net/maillog`
> WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/mx1.hctc.net/maillog`
> SPAM=`egrep -i "sqlgrey: spam:" /var/log/mx1.hctc.net/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
> GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/mx1.hctc.net/maillog`
>   

GLIST counts the rejects where the SMTP server based it's decision on
SQLgrey's result.
It doesn't count mails rejected because they have been rejected by both
SQLgrey and another rule in relevant smtpd_*_restriction configuration
entries that takes precedence (probably because it does a permanent
reject instead of the temporary one SQLgrey tells Postfix to return).
This is expected behavior if you use RBLs (especially if they cover
ranges of residential ip addresses).

If you look into your logs you should see that the messages triggering
the "grey new" logs are permanently refused a short time after in
Postfix logs.

Lionel

Re: [Sqlgrey-users] sqlstats need explanation

[Roddie H. <ro...@kr...>]

Here's the rest of the script for anyone else who wants to try it:

>> NEW=`egrep -ic "sqlgrey: grey: new:"             /var/log/mx1.hctc.net/maillog`
>> EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/mx1.hctc.net/maillog`
>> RECON=`egrep -ic "sqlgrey: grey: reconnect ok:"    /var/log/mx1.hctc.net/maillog`
>> DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/mx1.hctc.net/maillog`
>> FRMAWL=`egrep -ic "sqlgrey: grey: from awl:"        /var/log/mx1.hctc.net/maillog`
>> WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/mx1.hctc.net/maillog`
>> SPAM=`egrep -i "sqlgrey: spam:" /var/log/mx1.hctc.net/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
>> GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/mx1.hctc.net/maillog`


krweb:/root# cat sqlgrey-stats.sh

NEW=`egrep -ic "sqlgrey: grey: new:" /var/log/maillog`
EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/maillog`
RECON=`egrep -ic "sqlgrey: grey: reconnect ok:" /var/log/maillog`
DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/maillog`
FRMAWL=`egrep -ic "sqlgrey: grey: from awl:" /var/log/maillog`
WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/maillog`
SPAM=`egrep -i "sqlgrey: spam:" /var/log/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/maillog`
echo "GREY NEW:" $NEW
echo "GREY EARLY RECON:" $EARLY
echo "GREY RECON OK:" $RECON
echo "GREY DOMAWL:" $DOMAWL
echo "GREY FROM AWL:" $FRMAWL
echo "WHITELIST:" $WHITE
echo "SPAM:" $SPAM
echo "SMTPD GREYLISTED:" $GLIST

Roddie