Menu
▾
▴
fail2ban-users
|
From: Alex <mys...@gm...> - 2024-05-31 00:38:24
|
Hi, I'd like to add the following pattern to the postfix filter. May 30 19:01:42 xavier postfix-116/postscreen[245785]: NOQUEUE: reject: RCPT from [173.248.144.200]:57963: 550 5.5.1 Protocol error; from=< web...@my...>, to=<jo...@ex...>, proto=SMTP, helo=< rdnsb18.mytrueguide.com> I have a basic understanding of how the postfix.conf filter works. If I'm correct, I need to modify the mdpr-normal pattern. However, I'm not sure I understand the difference between mdpr-normal and mdre-normal? mdpr-normal = (?:\w+: (?:milter-)?reject:|(?:improper command pipelining|too many errors) after \S+) mdre-normal=^%(_pref)s from [^[]*\[<HOST>\]%(_port)s: [45][50][04] [45]\.\d\.\d+ (?:(?:<[^>]*>)?: )?(?:(?:Helo command|(?:Sender|Recipient) address) rejected: )?(?:Service unavailable|(?:Client host|Command|Data command) rejected|Relay access denied|(?:Host|Domain) not found|need fully-qualified hostname|match%(exre-user)s)\b ^from [^[]*\[<HOST>\]%(_port)s:? Ideally, I'd like to not have to modify that regexp and be able to add my own, much like what appears to be happening with mdre-errors. Steps on where to go from here would be greatly appreciated. Thanks, Alex |
|
From: Tim B. <ti...@bo...> - 2024-05-31 13:13:34
|
Am Donnerstag, dem 30.05.2024 um 20:37 -0400 schrieb Alex:
>
> Ideally, I'd like to not have to modify that regexp and be able to
> add my own, much like what appears to be happening with mdre-errors.
You don't have to. Append your own rules in a new line and test your
changed rule file with
fail2ban-regex /log/file postfix
and it should reply with text output like
| 13) [3] warning: non-SMTP command from .*\[<HOST>\]
| 14) [16] PREGREET [0-9]+ after [0-9]\.[0-9]{1,2} from \[<HOST>\]
| 15) [10] BARE NEWLINE from \[<HOST>\]
| 18) [5] lost connection after STARTTLS from .*\[<HOST>\]
| 20) [19] NON-SMTP COMMAND from \[<HOST>\]
| 21) [431] warning: hostname .* does not resolve to address <HOST>
| 24) [71] RCPT from unknown\[<HOST>\]: 450 4\.7\.25 Client host
rejected: cannot find your hostname
meaning that my rule no. 24 was found 71 times in my log file.
If it throws errors in your regex, it will let you know in some or
other way ;-)
(One thing i never fixed was this: After editing my filter file,
previously working regexes started failing, e. g. they didn't match
any more - despite being unmodified.)
Cheers,
tim
|
|
From: Alex <mys...@gm...> - 2024-05-31 23:59:48
|
Hi, > > Ideally, I'd like to not have to modify that regexp and be able to > > add my own, much like what appears to be happening with mdre-errors. > > You don't have to. Append your own rules in a new line and test your > changed rule file with > > fail2ban-regex /log/file postfix > > and it should reply with text output like > Yes, I understand that - I suppose it's the actual details of doing that which I don't understand. What's the difference between the pr and re rules? For example: mdpr-errors = too many errors after \S+ mdre-errors = ^from [^[]*\[<HOST>\]%(_port)s$ I'm assuming the re version is the regexp necessary just to capture the IP? So to add a new rule, I would simply copy this format with a new name, like: mdpr-proto = Protocol error; mdre-proto = ^from [^[]*\[<HOST>\]%(_port)s$ (One thing i never fixed was this: After editing my filter file, > previously working regexes started failing, e. g. they didn't match > any more - despite being unmodified.) Did you change the mode to no longer include those other regexes? mode = errors Or specific in the jail.conf? [postfix] filter = postfix[mode=aggressive] maxretry = 1 bantime = 48h enabled = true Thanks, Alex |
|
From: Nick H. <ni...@ho...> - 2024-06-01 08:29:52
|
On 01/06/2024 00:59, Alex wrote: > > Hi, > > > Ideally, I'd like to not have to modify that regexp and be able to > > add my own, much like what appears to be happening with mdre-errors. > > You don't have to. Append your own rules in a new line and test your > changed rule file with > > fail2ban-regex /log/file postfix > > and it should reply with text output like > > > Yes, I understand that - I suppose it's the actual details of doing > that which I don't understand. > > What's the difference between the pr and re rules? For example: > > mdpr-errors = too many errors after \S+ > mdre-errors = ^from [^[]*\[<HOST>\]%(_port)s$ > > I'm assuming the re version is the regexp necessary just to capture > the IP? > > So to add a new rule, I would simply copy this format with a new name, > like: > > mdpr-proto = Protocol error; > mdre-proto = ^from [^[]*\[<HOST>\]%(_port)s$ > > (One thing i never fixed was this: After editing my filter file, > previously working regexes started failing, e. g. they didn't match > any more - despite being unmodified.) > > > Did you change the mode to no longer include those other regexes? > mode = errors > > Or specific in the jail.conf? > > [postfix] > filter = postfix[mode=aggressive] > maxretry = 1 > bantime = 48h > enabled = true > > Thanks, > Alex > I find the postfix filters really hard to follow, but as far as I can see, if you go down your route, you then need to activate your protocol filters by building them into something like mdpr-extra/mdre-extra or have another jail just calling "mode=proto". Now, mdre-proto is already part of mdre-normal which seems to be called by every filter so could be unnecessary. You could add a new line to mdpr-normal if you wanted and your filter would work with "mode = more", or you could adjust the mdpr-normal directly. Note that to do an override, you generally leave the filter.d/postfix.conf alone and create a filter.d/postfix.local. In it you could put: [Definition] mdpr-normal = (?:\w+: reject:|(?:improper command pipelining|too many errors) after \S+) Protocol error; Nick |
|
From: Alex <mys...@gm...> - 2024-06-16 02:06:35
|
Hi, Finally able to get back to this.... On Sat, Jun 1, 2024 at 4:30 AM Nick Howitt via Fail2ban-users < fai...@li...> wrote: > > On 01/06/2024 00:59, Alex wrote: > > > Hi, > >> > Ideally, I'd like to not have to modify that regexp and be able to >> > add my own, much like what appears to be happening with mdre-errors. >> >> You don't have to. Append your own rules in a new line and test your >> changed rule file with >> >> fail2ban-regex /log/file postfix >> >> and it should reply with text output like >> > > Yes, I understand that - I suppose it's the actual details of doing that > which I don't understand. > > What's the difference between the pr and re rules? For example: > > mdpr-errors = too many errors after \S+ > mdre-errors = ^from [^[]*\[<HOST>\]%(_port)s$ > > I'm assuming the re version is the regexp necessary just to capture the IP? > > So to add a new rule, I would simply copy this format with a new name, > like: > > mdpr-proto = Protocol error; > mdre-proto = ^from [^[]*\[<HOST>\]%(_port)s$ > > (One thing i never fixed was this: After editing my filter file, >> previously working regexes started failing, e. g. they didn't match >> any more - despite being unmodified.) > > > Did you change the mode to no longer include those other regexes? > mode = errors > > Or specific in the jail.conf? > > [postfix] > filter = postfix[mode=aggressive] > maxretry = 1 > bantime = 48h > enabled = true > > Thanks, > Alex > > I find the postfix filters really hard to follow, but as far as I can see, > if you go down your route, you then need to activate your protocol filters > by building them into something like mdpr-extra/mdre-extra or have another > jail just calling "mode=proto". > > Now, mdre-proto is already part of mdre-normal which seems to be called by > every filter so could be unnecessary. You could add a new line to > mdpr-normal if you wanted and your filter would work with "mode = more", or > you could adjust the mdpr-normal directly. Note that to do an override, you > generally leave the filter.d/postfix.conf alone and create a > filter.d/postfix.local. In it you could put: > > [Definition] > mdpr-normal = (?:\w+: reject:|(?:improper command pipelining|too many > errors) after \S+) > Protocol error; > Adding the above did not work. Before I create a new filter that only processes these events, do you have any other ideas on what I should do? Thanks, Alex |
|
From: Alex <mys...@gm...> - 2024-06-16 02:09:42
|
> > >> I find the postfix filters really hard to follow, but as far as I can >> see, if you go down your route, you then need to activate your protocol >> filters by building them into something like mdpr-extra/mdre-extra or have >> another jail just calling "mode=proto". >> >> Now, mdre-proto is already part of mdre-normal which seems to be called >> by every filter so could be unnecessary. You could add a new line to >> mdpr-normal if you wanted and your filter would work with "mode = more", or >> you could adjust the mdpr-normal directly. Note that to do an override, you >> generally leave the filter.d/postfix.conf alone and create a >> filter.d/postfix.local. In it you could put: >> >> [Definition] >> mdpr-normal = (?:\w+: reject:|(?:improper command pipelining|too many >> errors) after \S+) >> Protocol error; >> > > Adding the above did not work. Before I create a new filter that only > processes these events, do you have any other ideas on what I should do? > I should have repeated that I'm trying to modify the postfix filter to also identify these postscreen lines: Jun 15 22:00:00 xavier postfix-116/postscreen[1600704]: NOQUEUE: reject: RCPT from [72.18.139.104]:42495: 550 5.5.1 Protocol error; from=< web...@my...>, to=<sal...@ex...>, proto=SMTP, helo=<rdnsq98.mytrueguide.com> > > Thanks, > Alex > > |
|
From: Nick H. <ni...@ho...> - 2024-06-16 07:23:41
|
On 01/06/2024 09:29, Nick Howitt wrote: > > On 01/06/2024 00:59, Alex wrote: >> >> Hi, >> >> > Ideally, I'd like to not have to modify that regexp and be able to >> > add my own, much like what appears to be happening >> with mdre-errors. >> >> You don't have to. Append your own rules in a new line and test your >> changed rule file with >> >> fail2ban-regex /log/file postfix >> >> and it should reply with text output like >> >> >> Yes, I understand that - I suppose it's the actual details of doing >> that which I don't understand. >> >> What's the difference between the pr and re rules? For example: >> >> mdpr-errors = too many errors after \S+ >> mdre-errors = ^from [^[]*\[<HOST>\]%(_port)s$ >> >> I'm assuming the re version is the regexp necessary just to capture >> the IP? >> >> So to add a new rule, I would simply copy this format with a new >> name, like: >> >> mdpr-proto = Protocol error; >> mdre-proto = ^from [^[]*\[<HOST>\]%(_port)s$ >> >> (One thing i never fixed was this: After editing my filter file, >> previously working regexes started failing, e. g. they didn't match >> any more - despite being unmodified.) >> >> >> Did you change the mode to no longer include those other regexes? >> mode = errors >> >> Or specific in the jail.conf? >> >> [postfix] >> filter = postfix[mode=aggressive] >> maxretry = 1 >> bantime = 48h >> enabled = true >> >> Thanks, >> Alex >> > I find the postfix filters really hard to follow, but as far as I can > see, if you go down your route, you then need to activate your > protocol filters by building them into something like > mdpr-extra/mdre-extra or have another jail just calling "mode=proto". > > Now, mdre-proto is already part of mdre-normal which seems to be > called by every filter so could be unnecessary. You could add a new > line to mdpr-normal if you wanted and your filter would work with > "mode = more", or you could adjust the mdpr-normal directly. Note that > to do an override, you generally leave the filter.d/postfix.conf alone > and create a filter.d/postfix.local. In it you could put: > > [Definition] > mdpr-normal = (?:\w+: reject:|(?:improper command pipelining|too many > errors) after \S+) > Protocol error; > > Nick What are the log lines you are trying to match? |
|
From: Nick H. <ni...@ho...> - 2024-06-16 08:33:24
|
On 16/06/2024 08:23, Nick Howitt via Fail2ban-users wrote: > > > On 01/06/2024 09:29, Nick Howitt wrote: >> >> On 01/06/2024 00:59, Alex wrote: >>> >>> Hi, >>> >>> > Ideally, I'd like to not have to modify that regexp and be able to >>> > add my own, much like what appears to be happening >>> with mdre-errors. >>> >>> You don't have to. Append your own rules in a new line and test your >>> changed rule file with >>> >>> fail2ban-regex /log/file postfix >>> >>> and it should reply with text output like >>> >>> >>> Yes, I understand that - I suppose it's the actual details of doing >>> that which I don't understand. >>> >>> What's the difference between the pr and re rules? For example: >>> >>> mdpr-errors = too many errors after \S+ >>> mdre-errors = ^from [^[]*\[<HOST>\]%(_port)s$ >>> >>> I'm assuming the re version is the regexp necessary just to capture >>> the IP? >>> >>> So to add a new rule, I would simply copy this format with a new >>> name, like: >>> >>> mdpr-proto = Protocol error; >>> mdre-proto = ^from [^[]*\[<HOST>\]%(_port)s$ >>> >>> (One thing i never fixed was this: After editing my filter file, >>> previously working regexes started failing, e. g. they didn't match >>> any more - despite being unmodified.) >>> >>> >>> Did you change the mode to no longer include those other regexes? >>> mode = errors >>> >>> Or specific in the jail.conf? >>> >>> [postfix] >>> filter = postfix[mode=aggressive] >>> maxretry = 1 >>> bantime = 48h >>> enabled = true >>> >>> Thanks, >>> Alex >>> >> I find the postfix filters really hard to follow, but as far as I can >> see, if you go down your route, you then need to activate your >> protocol filters by building them into something like >> mdpr-extra/mdre-extra or have another jail just calling "mode=proto". >> >> Now, mdre-proto is already part of mdre-normal which seems to be >> called by every filter so could be unnecessary. You could add a new >> line to mdpr-normal if you wanted and your filter would work with >> "mode = more", or you could adjust the mdpr-normal directly. Note >> that to do an override, you generally leave the filter.d/postfix.conf >> alone and create a filter.d/postfix.local. In it you could put: >> >> [Definition] >> mdpr-normal = (?:\w+: reject:|(?:improper command pipelining|too many >> errors) after \S+) >> Protocol error; >> >> Nick > What are the log lines you are trying to match? > > > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users Never mind. I've seen your followup. BTW, I can't crack it for the moment. |
|
From: Nick H. <ni...@ho...> - 2024-06-16 11:58:54
|
On 16/06/2024 09:33, Nick Howitt via Fail2ban-users wrote:
>
> On 16/06/2024 08:23, Nick Howitt via Fail2ban-users wrote:
>>
>>
>> On 01/06/2024 09:29, Nick Howitt wrote:
>>>
>>> On 01/06/2024 00:59, Alex wrote:
>>>>
>>>> Hi,
>>>>
>>>> > Ideally, I'd like to not have to modify that regexp and be
>>>> able to
>>>> > add my own, much like what appears to be happening
>>>> with mdre-errors.
>>>>
>>>> You don't have to. Append your own rules in a new line and test
>>>> your
>>>> changed rule file with
>>>>
>>>> fail2ban-regex /log/file postfix
>>>>
>>>> and it should reply with text output like
>>>>
>>>>
>>>> Yes, I understand that - I suppose it's the actual details of doing
>>>> that which I don't understand.
>>>>
>>>> What's the difference between the pr and re rules? For example:
>>>>
>>>> mdpr-errors = too many errors after \S+
>>>> mdre-errors = ^from [^[]*\[<HOST>\]%(_port)s$
>>>>
>>>> I'm assuming the re version is the regexp necessary just to capture
>>>> the IP?
>>>>
>>>> So to add a new rule, I would simply copy this format with a new
>>>> name, like:
>>>>
>>>> mdpr-proto = Protocol error;
>>>> mdre-proto = ^from [^[]*\[<HOST>\]%(_port)s$
>>>>
>>>> (One thing i never fixed was this: After editing my filter file,
>>>> previously working regexes started failing, e. g. they didn't match
>>>> any more - despite being unmodified.)
>>>>
>>>>
>>>> Did you change the mode to no longer include those other regexes?
>>>> mode = errors
>>>>
>>>> Or specific in the jail.conf?
>>>>
>>>> [postfix]
>>>> filter = postfix[mode=aggressive]
>>>> maxretry = 1
>>>> bantime = 48h
>>>> enabled = true
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>> I find the postfix filters really hard to follow, but as far as I
>>> can see, if you go down your route, you then need to activate your
>>> protocol filters by building them into something like
>>> mdpr-extra/mdre-extra or have another jail just calling "mode=proto".
>>>
>>> Now, mdre-proto is already part of mdre-normal which seems to be
>>> called by every filter so could be unnecessary. You could add a new
>>> line to mdpr-normal if you wanted and your filter would work with
>>> "mode = more", or you could adjust the mdpr-normal directly. Note
>>> that to do an override, you generally leave the
>>> filter.d/postfix.conf alone and create a filter.d/postfix.local. In
>>> it you could put:
>>>
>>> [Definition]
>>> mdpr-normal = (?:\w+: reject:|(?:improper command pipelining|too
>>> many errors) after \S+)
>>> Protocol error;
>>>
>>> Nick
>> What are the log lines you are trying to match?Never mind. I've seen
>> your followup.
>
> BTW, I can't crack it for the moment.
OK so this isn't going to be quite so neat. You need to add a line:
^RCPT from [^[]*\[<HOST>\]%(_port)s:? 550 5\.5\.1 Protocol error;
to the mdre-normal section. Generally the recommended way is to create a
postfix.local file, but this would need to contain:
[Definition]
mdre-normal=^RCPT from [^[]*\[<HOST>\]%(_port)s: 55[04] 5\.7\.1\s
^RCPT from [^[]*\[<HOST>\]%(_port)s: 45[04] 4\.7\.\d+
(?:Service unavailable\b|Client host rejected: cannot find your
(reverse )?hostname\b)
^RCPT from [^[]*\[<HOST>\]%(_port)s: 450 4\.7\.\d+
(<[^>]*>)?: Helo command rejected: Host not found\b
^EHLO from [^[]*\[<HOST>\]%(_port)s: 504 5\.5\.\d+
(<[^>]*>)?: Helo command rejected: need fully-qualified hostname\b
^(RCPT|VRFY) from [^[]*\[<HOST>\]%(_port)s: 550 5\.1\.1\s
^RCPT from [^[]*\[<HOST>\]%(_port)s: 450 4\.1\.\d+
(<[^>]*>)?: Sender address rejected: Domain not found\b
^from [^[]*\[<HOST>\]%(_port)s:?
^RCPT from [^[]*\[<HOST>\]%(_port)s:? 550 5\.5\.1
Protocol error;
So you need to duplicate everything there then add your extra line.
Nick
|
|
From: Alex <mys...@gm...> - 2024-06-17 00:46:33
|
Hi,
> BTW, I can't crack it for the moment.
> OK so this isn't going to be quite so neat. You need to add a line:
>
> ^RCPT from [^[]*\[<HOST>\]%(_port)s:? 550 5\.5\.1 Protocol error;
>
> to the mdre-normal section. Generally the recommended way is to create a
> postfix.local file, but this would need to contain:
>
This got mangled by gmail, but I was able to copy the postfix.conf to
postfix.local and make it somewhat resemble what you pasted, and it appears
to work.
It's what I had also done originally, but had the formal wrong - I thought
it more replaced the postfix.conf rather than supplement it.
mdre-normal=^RCPT from [^[]*\[<HOST>\]%(_port)s: 55[04] 5\.7\.1\s
> ^RCPT from [^[]*\[<HOST>\]%(_port)s: 45[04] 4\.7\.\d+
>
I still don't understand the difference between mdre-* and mdpr-* :-(
Also, how does it match 'postscreen' when the prefix doesn't contain that
phrase?
_daemon = postfix(-\w+)?/\w+(?:/smtp[ds])?
prefregex = ^%(__prefix_line)s<mdpr-<mode>> <F-CONTENT>.+</F-CONTENT>$
Thanks,
Alex
|
|
From: Nick H. <ni...@ho...> - 2024-06-17 09:38:32
|
On 17/06/2024 01:46, Alex wrote: > Hi, > > > BTW, I can't crack it for the moment. > OK so this isn't going to be quite so neat. You need to add a line: > > ^RCPT from [^[]*\[<HOST>\]%(_port)s:? 550 5\.5\.1 Protocol error; > > to the mdre-normal section. Generally the recommended way is to > create a > postfix.local file, but this would need to contain: > > > This got mangled by gmail, but I was able to copy the postfix.conf to > postfix.local and make it somewhat resemble what you pasted, and it > appears to work. > > It's what I had also done originally, but had the formal wrong - I > thought it more replaced the postfix.conf rather than supplement it. > > mdre-normal=^RCPT from [^[]*\[<HOST>\]%(_port)s: 55[04] 5\.7\.1\s > ^RCPT from [^[]*\[<HOST>\]%(_port)s: 45[04] 4\.7\.\d+ > > > I still don't understand the difference between mdre-* and mdpr-* :-( I struggled with that yesterday because I don't understand <F-CONTENT> > > Also, how does it match 'postscreen' when the prefix doesn't contain > that phrase? > _daemon = postfix(-\w+)?/\w+(?:/smtp[ds])? The \w+ catches one or more letters/numbers/"a few other bits like _" so catches postscreen. I don't really see the point of (?:/smtp[ds])? as it is optional (the trailing ?). > prefregex = ^%(__prefix_line)s<mdpr-<mode>> <F-CONTENT>.+</F-CONTENT>$ I have not managed to decode this line, but it looks like the prefregex matches ^%(__prefix_line)s directly followed by the mdpr-* string. The ^%(__prefix_line)s matches the date, server and daemon strings. The failregex is mdre-<mode> which has to appear after the mdre-* string > > Thanks, > Alex > Someone with very detailed knowledge of regex has been playing a lot with this filter and it makes it hard to read. Why are there so many ^[]* and what do they mean? I am sure there is a much easier way of expressing it. Similarly there are many ?: and ?! which I think are OTT and make it harder to read. |
|
From: Alex <mys...@gm...> - 2024-06-17 02:04:17
Attachments:
postfix-filter.txt
|
> > BTW, I can't crack it for the moment. >> OK so this isn't going to be quite so neat. You need to add a line: >> >> ^RCPT from [^[]*\[<HOST>\]%(_port)s:? 550 5\.5\.1 Protocol error; >> >> to the mdre-normal section. Generally the recommended way is to create a >> postfix.local file, but this would need to contain: >> > > This got mangled by gmail, but I was able to copy the postfix.conf to > postfix.local and make it somewhat resemble what you pasted, and it appears > to work. > Actually, it works with fail2ban-regex but isn't catching them from the live logs. Here's what I have in my jail.conf: [postfix] filter = postfix maxretry = 1 bantime = 48h enabled = true mode = normal I've also attached my whole postfix.conf here, just in case. |
|
From: Alex <mys...@gm...> - 2024-06-18 15:37:30
|
Hi, > BTW, I can't crack it for the moment. >>> OK so this isn't going to be quite so neat. You need to add a line: >>> >>> ^RCPT from [^[]*\[<HOST>\]%(_port)s:? 550 5\.5\.1 Protocol error; >>> >>> to the mdre-normal section. Generally the recommended way is to create a >>> postfix.local file, but this would need to contain: >>> >> >> This got mangled by gmail, but I was able to copy the postfix.conf to >> postfix.local and make it somewhat resemble what you pasted, and it appears >> to work. >> > > Actually, it works with fail2ban-regex but isn't catching them from the > live logs. > Fixed it. It turned out that even though syslog_mail in paths-fedora.conf was pointing to the proper maillog, it apparently wasn't being considered by the postfix.conf filter. I had to add logpath to my jail.conf: [postfix] filter = postfix maxretry = 1 bantime = 48h enabled = true mode = normal logpath = %(syslog_mail)s |
Thanks for helping keep SourceForge clean.
X