sqlgrey-users

[Sqlgrey-users] auto-whitelisting google/gmail/yahoo

[Alex R. <mys...@gm...>]

Hi,

It appears the default installation includes an auto-whitelist entry for 
google.com and yahoo.com.

Is there a particular reason for this that would prevent me from 
removing it? I'm seeing quite a bit of spam that I believe could 
otherwise be blocked if the whole domain wasn't whitelisted.

Ideas greatly appreciated.
Thanks,
Alex

Re: [Sqlgrey-users] auto-whitelisting google/gmail/yahoo

[Karl O. P. <ko...@me...>]

On Wed, 01 Jul 2015 20:39:57 -0400
Alex Regan <mys...@gm...> wrote:

> It appears the default installation includes an auto-whitelist entry
> for google.com and yahoo.com.
> 
> Is there a particular reason for this that would prevent me from 
> removing it? I'm seeing quite a bit of spam that I believe could 
> otherwise be blocked if the whole domain wasn't whitelisted.

The google and yahoo servers are going to obey the rules
and retry after an appropriate interval and deliver
the spam anyway.  You may as well reduce the load and
accept it immediately.

They may or may not also resend from a different box,
doing a round-robin on outbound mail gateways.
Mail (probably) eventually gets delivered when all the boxes (IPs)
are whitelisted.  But it means that legitimate mail
can get overly delayed.  (Dunno who's doing this any more.)

Regards,

Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: [Sqlgrey-users] auto-whitelisting google/gmail/yahoo

[Alex <mys...@gm...>]

Hi,

On Thu, Jul 2, 2015 at 1:08 AM, Karl O. Pinc <ko...@me...> wrote:
> On Wed, 01 Jul 2015 20:39:57 -0400
> Alex Regan <mys...@gm...> wrote:
>
>> It appears the default installation includes an auto-whitelist entry
>> for google.com and yahoo.com.
>>
>> Is there a particular reason for this that would prevent me from
>> removing it? I'm seeing quite a bit of spam that I believe could
>> otherwise be blocked if the whole domain wasn't whitelisted.
>
> The google and yahoo servers are going to obey the rules
> and retry after an appropriate interval and deliver
> the spam anyway.  You may as well reduce the load and
> accept it immediately.

Ah, right, makes sense.

> They may or may not also resend from a different box,
> doing a round-robin on outbound mail gateways.
> Mail (probably) eventually gets delivered when all the boxes (IPs)
> are whitelisted.  But it means that legitimate mail
> can get overly delayed.  (Dunno who's doing this any more.)

I'm using db_cluster to avoid this. round-robin issue. This was one of
the main reasons I chose sqlgrey over the others like postgrey. Is
this not what you're talking about?

Thanks,
Alex

Re: [Sqlgrey-users] auto-whitelisting google/gmail/yahoo

[<da...@ha...>]

Hi Alex..

On 2015-07-03 03:45, Alex wrote:
> They may or may not also resend from a different box,
> doing a round-robin on outbound mail gateways.
> Mail (probably) eventually gets delivered when all the boxes (IPs)
> are whitelisted.  But it means that legitimate mail
> can get overly delayed.  (Dunno who's doing this any more.)
> I'm using db_cluster to avoid this. round-robin issue. This was one of
> the main reasons I chose sqlgrey over the others like postgrey. Is
> this not what you're talking about?
No db_cluster (or a central db for that matter) solves the problem of 1
foreign IP hitting multiple servers at YOUR end. Not multiple foreign
servers, with different IP's, hitting your mail server(s).

sqlgrey stores the ip of the sender, either full or as partial, in the
db. But if that ip changes, sqlgrey sees it as an entirely new
"connection" and has to add that one as well. So now sqlgrey has 2 new
connections from the same mail-cluster and both new ip's must resubmit
to be whitelisted and be able to send mail through sqlgrey. Hence; they
are treated as 2 different mail setups.


One always has to remember the purpose of SQLGrey & Greylisting. The
purpose is not filtering spam. The purpose is filtering out mail-senders
that aren't real mailservers. if it IS a real mailserver but it still
sends spam, that is NOT something sqlgrey can do anything about. Thats a
job for other spam-filtering mechanisms.

So, if we KNOW that eg. google.com are actually real mailservers and
that they will alway behave as such, then there is no danger of
whitelisting, as you are only whitelisting the fact that they ARE real
mailservers; not that they arent sending spam.
And as Karl points out, you save time for the clients by whitelisting,
as they dont have to do the whole resubmit-dance to determine that they
are in fact a real mailserver, when we already know it is.

> Thanks,
> Alex
>

Regards
- Dan

[Sqlgrey-users] Auto-whitelisting recipient email addresses

[Bruce B. <bb...@bo...>]

Good day,

I'm wondering how many of you have implemented the sqlgrey mod described 
here...  http://www.hyllander.org/sqlgrey_whitelisting It's not very 
often that I receive a call from a customer informing me that the email 
that he attempted to send to us resulted in an error message  (from his 
perspective).  And the last time was when he was replying to a message 
that I sent to him!  Of course, the "auto-whitelisting of recipient 
addresses" wouldn't help a bit until someone in my domain writes to a 
customer but it might add an additional layer of automation. I realize 
that the problem relates to broken mail servers but it's our customers 
who are inconvenienced and our business that might suffer.

Does anyone have experience with this auto-whitelisting of recipients mod?

Thanks,
~Bruce

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Lionel B. <lio...@bo...>]

Hi Bruce,

On 07/03/15 18:13, Bruce Bodger wrote:
> Good day,
>
> I'm wondering how many of you have implemented the sqlgrey mod described 
> here...  http://www.hyllander.org/sqlgrey_whitelisting It's not very 
> often that I receive a call from a customer informing me that the email 
> that he attempted to send to us resulted in an error message  (from his 
> perspective).  And the last time was when he was replying to a message 
> that I sent to him!  Of course, the "auto-whitelisting of recipient 
> addresses" wouldn't help a bit until someone in my domain writes to a 
> customer but it might add an additional layer of automation. I realize 
> that the problem relates to broken mail servers but it's our customers 
> who are inconvenienced and our business that might suffer.

It's unlikely your business will suffer: if your customer reports this
kind of errors, he probably have them with multiple other destinations
(unless his/her admin was temporarily playing with parameters of the
mail server) as greylisting and temporary errors are commonplace.

If you aren't prepared to explain that everything is fine there is
support for whitelisting your customer mail servers (instead of his
domain or addresses as implemented in the patch above which would make
easy to circumvent greylisting by simply using fake addresses derived
from your known customers).
Just add entries to /etc/sqlgrey/clients_fqdn_whitelist.local or
/etc/sqlgrey/clients_ip_whitelist.local depending on how you want to
whitelist (by fqdn or ip). You don't have to restart SQLgrey, these
files are automatically reloaded when modified.

Best regards,

Lionel

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[<da...@ha...>]

Just a side note.
   

On 2015-07-03 18:13, Bruce Bodger wrote:
> often that I receive a call from a customer informing me that the email 
> that he attempted to send to us resulted in an error message  (from his 
>

If you by "error message" mean that the sender receives a mail that
looks like a bounce (but isn't), I eliminated most of those at my old
job, by using
reject_code = 451

which is actually why i added the reject_code option in the first place :)..

- Dan

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Karl O. P. <ko...@me...>]

On Fri, 3 Jul 2015 11:13:03 -0500
Bruce Bodger <bb...@bo...> wrote:

> I'm wondering how many of you have implemented the sqlgrey mod
> described here...  http://www.hyllander.org/sqlgrey_whitelisting

Some thoughts, take them for what you will:

If it were me I'd avoid modifying sqlgrey and write a little
postfix filter that frobs the database whitelist directly.
I'm no advocate of re-inventing the wheel, and honestly
didn't read the article in detail, but simpler seems better
than more complicated. 

On a related note, to actually solve the problem you must
know exactly what the problem is.  It's not clear from your
description just why there was an issue in the first place.

Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Bruce B. <bb...@bo...>]

Thanks for your feedback Karl, Dan, and Lionel..

On 7/3/15 12:33 PM, Karl O. Pinc wrote:
> On Fri, 3 Jul 2015 11:13:03 -0500
> Bruce Bodger <bb...@bo...> wrote:
>
>> I'm wondering how many of you have implemented the sqlgrey mod
>> described here...  http://www.hyllander.org/sqlgrey_whitelisting
> Some thoughts, take them for what you will:
>
> If it were me I'd avoid modifying sqlgrey and write a little
> postfix filter that frobs the database whitelist directly.
> I'm no advocate of re-inventing the wheel, and honestly
> didn't read the article in detail, but simpler seems better
> than more complicated.
>
I understand your perspectives.  I've been running sqlgrey since 2009.  
I was just wondering if you saw this issue as a big deal and, obviously, 
you don't. As soon as I hear from a recipient having any problem sending 
us email I check the mail log and then add their domain to 
clients_fqdn_whitelist.local if I see that they've truly connected.  
Again, thanks for the replies.  BTW, I've kept the reject_code = 451 
default since original installation.

P.S.  Even though I've been running sqlgrey for 6 years I've never been 
sure what the whitelist requires.  Is   troubled.com equivalent to  
*.troubled.com?   My intention is to add all servers and users in the 
troubled.com domain.  I've been adding both.

Thanks again,
Bruce

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Bruce B. <bb...@bo...>]

On Jul 3, 2015, at 5:26 PM, Lionel Bouton <lio...@bo...> wrote:

> On 07/04/15 00:21, Bruce Bodger wrote:
>> [...]
>> 
>> P.S.  Even though I've been running sqlgrey for 6 years I've never been 
>> sure what the whitelist requires.  Is   troubled.com equivalent to  
>> *.troubled.com?   My intention is to add all servers and users in the 
>> troubled.com domain.  I've been adding both.
> 
> The whitelisting is about the ips or fqdns of the mail servers
> contacting yours and not the addresses of the sender (so they might not
> share the same domain). If you are not sure about what they are you
> should be able to find them in your postfix logs.

Lionel,

Does this comment from the distributed clients_fqdn_whitelist still apply?

# Note you need the following two lines to allow both
# <lots of mtas>.example.com and example.com
# *.example.com
# example.com

Thank you,
Bruce

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Lionel B. <lio...@bo...>]

Hi,


Le 23/07/2015 15:40, Bruce Bodger a écrit :
> [...]
> Lionel,
>
> Does this comment from the distributed clients_fqdn_whitelist still apply?
>
> # Note you need the following two lines to allow both
> # <lots of mtas>.example.com and example.com
> # *.example.com
> # example.com

Yes it does.
To be honest I wasn't happy about this little quirk when I coded the
whitelists support and with experience I would not do it again the same
way if I were starting from scratch. IIRC (I didn't look at SQLgrey code
for quite some time) this is needlessly linked to internal SQLgrey
structures involved in the whitelisting: regexp, globs and identity
matching are done separately for performance reasons. I think I should
have used a cleaner syntax and made the effort to translate it to the
internal structures instead of looking for a direct mapping.
As it didn't seem too bad and cleaner solutions would have broken
backward compatibility this was left like this (and the comment was
added to make sure people reading documentation like you were aware of it).

Best regards,

Lionel

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Lionel B. <lio...@bo...>]

On 07/04/15 00:21, Bruce Bodger wrote:
> [...]
>
> P.S.  Even though I've been running sqlgrey for 6 years I've never been 
> sure what the whitelist requires.  Is   troubled.com equivalent to  
> *.troubled.com?   My intention is to add all servers and users in the 
> troubled.com domain.  I've been adding both.

The whitelisting is about the ips or fqdns of the mail servers
contacting yours and not the addresses of the sender (so they might not
share the same domain). If you are not sure about what they are you
should be able to find them in your postfix logs.

Lionel

Re: [Sqlgrey-users] Auto-whitelisting recipient email addresses

[Bruce B. <bb...@bo...>]

On 7/3/15 5:26 PM, Lionel Bouton wrote:
> The whitelisting is about the ips or fqdns of the mail servers
> contacting yours and not the addresses of the sender (so they might not
> share the same domain). If you are not sure about what they are you
> should be able to find them in your postfix logs.
>
Thank you, Lionel.  Understood.

Regards,
~Bruce