fail2ban-users

[Fail2ban-users] How to implement ban using ipset

[Phillip C. <dom...@en...>]

I have been using csf/lfd as my firewall for several years on several 
versions of CentOS, currently CentOS7. I am using several ipset-managed 
blocklists supported directly by csf. Some of these are fairly huge 
(such as whole country blocks), and it changes them in fractions of a 
second. Very happy with everything it does.

However, csf syntax for custom regex applied to logs is relatively 
clumsy and error-prone so I have installed fail2ban in hopes of using 
that for custom log-based bans.

For my initial testing I have set up one jail and a corresponding 
filters. (I found that all very simple.)

My intent:
On filter matches, immediately ban the host IP for one full day. Use 
ipset to implement the bans.

The test case basically watches my exim reject.log (using inotify) and 
unerringly finds the naughty hosts I want to ban.

My setup:
jail.local has:

> [exim-reject]
> mode      = normal
> port      = smtp,ssmtp
> logpath   = /var/log/exim/reject.log
> filter    = exim-reject
> maxmatches = 1
> maxretry   = 1
> backend   = auto
> bantime   = 1d
> banaction = iptables-ipset-proto6
> enabled   = true

And exim-reject.conf contains:

> [INCLUDES]
> before = exim-common.conf
> [Definition]
> failregex = <HOST> is listed at zen.spamhaus.org
>             \[<HOST>\]:25 dropped: too many syntax or protocol errors

The contents of fail2ban.log indicates everything is working. It says it 
found the lines I expected it to find, and has issued bans (and unbans a 
day later).

However, when I list the ipset sets on the console, the only sets listed 
are those managed by csf. Clearly I have implemented something 
incorrectly. I am hoping somebody on the list can set me straight. Is it 
possibly a permissions problem?

Re: [Fail2ban-users] How to implement ban using ipset

[Richard S. <hob...@gm...>]

On Tue, Sep 1, 2020 at 7:45 PM Phillip Carroll <
dom...@en...> wrote:

> I have been using csf/lfd as my firewall for several years on several
> versions of CentOS, currently CentOS7. I am using several ipset-managed
> blocklists supported directly by csf. Some of these are fairly huge
> (such as whole country blocks), and it changes them in fractions of a
> second. Very happy with everything it does.
>
> However, csf syntax for custom regex applied to logs is relatively
> clumsy and error-prone so I have installed fail2ban in hopes of using
> that for custom log-based bans.
>
> For my initial testing I have set up one jail and a corresponding
> filters. (I found that all very simple.)
>
> My intent:
> On filter matches, immediately ban the host IP for one full day. Use
> ipset to implement the bans.
>
> The test case basically watches my exim reject.log (using inotify) and
> unerringly finds the naughty hosts I want to ban.
>
> My setup:
> jail.local has:
>
> > [exim-reject]
> > mode      = normal
> > port      = smtp,ssmtp
> > logpath   = /var/log/exim/reject.log
> > filter    = exim-reject
> > maxmatches = 1
> > maxretry   = 1
> > backend   = auto
> > bantime   = 1d
> > banaction = iptables-ipset-proto6
> > enabled   = true
>
> And exim-reject.conf contains:
>
> > [INCLUDES]
> > before = exim-common.conf
> > [Definition]
> > failregex = <HOST> is listed at zen.spamhaus.org
> >             \[<HOST>\]:25 dropped: too many syntax or protocol errors
>
> The contents of fail2ban.log indicates everything is working. It says it
> found the lines I expected it to find, and has issued bans (and unbans a
> day later).
>
> However, when I list the ipset sets on the console, the only sets listed
> are those managed by csf. Clearly I have implemented something
> incorrectly. I am hoping somebody on the list can set me straight. Is it
> possibly a permissions problem?
>

That quite a bit more complex installation than I use so can't help you
there, but fail2ban version and source (EPEL, self install, etc) would be
helpful.

Thanks,
Richard

Re: [Fail2ban-users] How to implement ban using ipset

[Phillip C. <dom...@en...>]

On 9/1/2020 6:04 PM, Richard Shaw wrote:
> On Tue, Sep 1, 2020 at 7:45 PM Phillip Carroll 
> <dom...@en... 
> <mailto:dom...@en...>> wrote:
> 
>     I have been using csf/lfd as my firewall for several years on several
>     versions of CentOS, currently CentOS7. I am using several ipset-managed
>     blocklists supported directly by csf. Some of these are fairly huge
>     (such as whole country blocks), and it changes them in fractions of a
>     second. Very happy with everything it does.
> 
>     However, csf syntax for custom regex applied to logs is relatively
>     clumsy and error-prone so I have installed fail2ban in hopes of using
>     that for custom log-based bans.
> 
>     For my initial testing I have set up one jail and a corresponding
>     filters. (I found that all very simple.)
> 
>     My intent:
>     On filter matches, immediately ban the host IP for one full day. Use
>     ipset to implement the bans.
> 
>     The test case basically watches my exim reject.log (using inotify) and
>     unerringly finds the naughty hosts I want to ban.
> 
>     My setup:
>     jail.local has:
> 
>      > [exim-reject]
>      > mode      = normal
>      > port      = smtp,ssmtp
>      > logpath   = /var/log/exim/reject.log
>      > filter    = exim-reject
>      > maxmatches = 1
>      > maxretry   = 1
>      > backend   = auto
>      > bantime   = 1d
>      > banaction = iptables-ipset-proto6
>      > enabled   = true
> 
>     And exim-reject.conf contains:
> 
>      > [INCLUDES]
>      > before = exim-common.conf
>      > [Definition]
>      > failregex = <HOST> is listed at zen.spamhaus.org
>     <http://zen.spamhaus.org>
>      >             \[<HOST>\]:25 dropped: too many syntax or protocol errors
> 
>     The contents of fail2ban.log indicates everything is working. It
>     says it
>     found the lines I expected it to find, and has issued bans (and
>     unbans a
>     day later).
> 
>     However, when I list the ipset sets on the console, the only sets
>     listed
>     are those managed by csf. Clearly I have implemented something
>     incorrectly. I am hoping somebody on the list can set me straight.
>     Is it
>     possibly a permissions problem?
> 
> 
> That quite a bit more complex installation than I use so can't help you 
> there, but fail2ban version and source (EPEL, self install, etc) would 
> be helpful.
> 
> Thanks,
> Richard

@Richard,

This server has only prebuilt packages from the standard repos, managed 
using yum. It is a pretty typical headless server.

I don't use selinux because of conflicts with the ISP provided kernel. 
(Linode)

 From yum list installed:
> fail2ban.noarch                       0.11.1-9.el7.2             @epel          
> fail2ban-server.noarch                0.11.1-9.el7.2             @epel          
> ipset.x86_64                          7.1-1.el7                  @base          
> ipset-libs.x86_64                     7.1-1.el7                  @base          
> iptables.x86_64                       1.4.21-34.el7              @base          
> iptables-services.x86_64              1.4.21-34.el7              @base          

Phil

Re: [Fail2ban-users] How to implement ban using ipset

[Phillip C. <dom...@en...>]

On 9/2/2020 5:43 AM, Phillip Carroll wrote:
> On 9/1/2020 6:04 PM, Richard Shaw wrote:
>> On Tue, Sep 1, 2020 at 7:45 PM Phillip Carroll 
>> <dom...@en... 
>> <mailto:dom...@en...>> wrote:
>>
>>     I have been using csf/lfd as my firewall for several years on several
>>     versions of CentOS, currently CentOS7. I am using several 
>> ipset-managed
>>     blocklists supported directly by csf. Some of these are fairly huge
>>     (such as whole country blocks), and it changes them in fractions of a
>>     second. Very happy with everything it does.
>>
>>     However, csf syntax for custom regex applied to logs is relatively
>>     clumsy and error-prone so I have installed fail2ban in hopes of using
>>     that for custom log-based bans.
>>
>>     For my initial testing I have set up one jail and a corresponding
>>     filters. (I found that all very simple.)
>>
>>     My intent:
>>     On filter matches, immediately ban the host IP for one full day. Use
>>     ipset to implement the bans.
>>
>>     The test case basically watches my exim reject.log (using inotify) 
>> and
>>     unerringly finds the naughty hosts I want to ban.
>>
>>     My setup:
>>     jail.local has:
>>
>>      > [exim-reject]
>>      > mode      = normal
>>      > port      = smtp,ssmtp
>>      > logpath   = /var/log/exim/reject.log
>>      > filter    = exim-reject
>>      > maxmatches = 1
>>      > maxretry   = 1
>>      > backend   = auto
>>      > bantime   = 1d
>>      > banaction = iptables-ipset-proto6
>>      > enabled   = true
>>
>>     And exim-reject.conf contains:
>>
>>      > [INCLUDES]
>>      > before = exim-common.conf
>>      > [Definition]
>>      > failregex = <HOST> is listed at zen.spamhaus.org
>>     <http://zen.spamhaus.org>
>>      >             \[<HOST>\]:25 dropped: too many syntax or protocol 
>> errors
>>
>>     The contents of fail2ban.log indicates everything is working. It
>>     says it
>>     found the lines I expected it to find, and has issued bans (and
>>     unbans a
>>     day later).
>>
>>     However, when I list the ipset sets on the console, the only sets
>>     listed
>>     are those managed by csf. Clearly I have implemented something
>>     incorrectly. I am hoping somebody on the list can set me straight.
>>     Is it
>>     possibly a permissions problem?
>>
>>
>> That quite a bit more complex installation than I use so can't help 
>> you there, but fail2ban version and source (EPEL, self install, etc) 
>> would be helpful.
>>
>> Thanks,
>> Richard
> 
> @Richard,
> 
> This server has only prebuilt packages from the standard repos, managed 
> using yum. It is a pretty typical headless server.
> 
> I don't use selinux because of conflicts with the ISP provided kernel. 
> (Linode)
> 
>  From yum list installed:
>> fail2ban.noarch                       0.11.1-9.el7.2             @epel 
>> fail2ban-server.noarch                0.11.1-9.el7.2             @epel 
>> ipset.x86_64                          7.1-1.el7                  @base 
>> ipset-libs.x86_64                     7.1-1.el7                  @base 
>> iptables.x86_64                       1.4.21-34.el7              @base 
>> iptables-services.x86_64              1.4.21-34.el7              @base 
> 
> Phil
> 
> 
> _______________________________________________
> Fail2ban-users mailing list
> Fai...@li...
> https://lists.sourceforge.net/lists/listinfo/fail2ban-users

I turned on DEBUG and restarted fail2ban.  Following is the result in 
fail2ban.log.

The difficulty I have is understanding the content of the scripting 
macros used in the various commands at the time the commands are issued. 
  For instance, what does "<ipmset>" contain at the time of the ipset 
create command?  The log would appear to indicate the scripting is such 
that values are used before they are defined. After the create, I see 
the value set as 'f2b-<name>' which is dependent on the value of <name> 
which is set on a subsequent set command as 'exim-reject'.

If this order is the actual sequence it would explain the fact that the 
set f2b-exim-reject' does not exist after the restart of fail2ban.
However, perhaps I just don't understand the inner workings.

The log:

> 2020-09-02 08:34:17,939 fail2ban.server         [4077]: INFO    Reload all jails
> 2020-09-02 08:34:17,939 fail2ban.server         [4077]: INFO    Reload jail 'exim-reject'
> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: DEBUG   Setting usedns = warn for FilterPyinotify(Jail('exim-reject'))
> 2020-09-02 08:34:17,940 fail2ban.server         [4077]: DEBUG     failregex: '<HOST> is listed at zen.spamhaus.org'
> 2020-09-02 08:34:17,940 fail2ban.server         [4077]: DEBUG     failregex: '\\[<HOST>\\]:25 dropped: too many syntax or protocol errors'
> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: INFO      maxRetry: 1
> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: INFO      encoding: UTF-8
> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: INFO      findtime: 600
> 2020-09-02 08:34:17,940 fail2ban.actions        [4077]: INFO      banTime: 86400
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG   Created <class 'fail2ban.server.action.CommandAction'>
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionunban = 'ipset del <ipmset> <ip> -exist'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionflush = 'ipset flush <ipmset>'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionstop = '<iptables> -D INPUT -p tcp -m multiport --dports smtp,ssmtp -m set --match-set <ipmset> src -j <blocktype>\nipset flush <ipmset>\nipset destroy <ipmset>'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionstart = 'ipset create <ipmset> hash:ip timeout 600<familyopt>\n<iptables> -I INPUT -p tcp -m multiport --dports smtp,ssmtp -m set --match-set <ipmset> src -j <blocktype>'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionprolong = 'ipset add <ipmset> <ip> timeout <bantime> -exist'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionban = 'ipset add <ipmset> <ip> timeout <bantime> -exist'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set protocol = 'tcp'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set chain = '<known/chain>'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set lockingopt = '-w'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set ipmset = 'f2b-<name>'
> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set blocktype = 'REJECT --reject-with icmp-port-unreachable'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set default-timeout = '600'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set blocktype?family=inet6 = 'REJECT --reject-with icmp6-port-unreachable'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set port = 'smtp,ssmtp'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set actname = 'iptables-ipset-proto6'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set iptables = 'iptables <lockingopt>'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set familyopt?family=inet6 = '<sp>family inet6'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set familyopt = ''
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set returntype = 'RETURN'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set ipmset?family=inet6 = 'f2b-<name>6'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set iptables?family=inet6 = 'ip6tables <lockingopt>'
> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set name = 'exim-reject'
> 2020-09-02 08:34:17,942 fail2ban.server         [4077]: INFO    Jail 'exim-reject' reloaded
> 2020-09-02 08:34:17,943 fail2ban.actions        [4077]: NOTICE  [exim-reject] Flush ticket(s) with iptables-ipset-proto6
> 2020-09-02 08:34:17,943 fail2ban.actions        [4077]: DEBUG     Unbanned 7, 7 ticket(s) in 'exim-reject'
> 2020-09-02 08:34:17,943 fail2ban.actions        [4077]: DEBUG   exim-reject: action iptables-ipset-proto6 terminated
> 2020-09-02 08:34:17,943 fail2ban.server         [4077]: INFO    Reload finished.

Re: [Fail2ban-users] How to implement ban using ipset

[Bill S. <bsh...@op...>]

At the bottom of /etc/fail2ban/action.d/iptables-ipset-proto6.conf:
[Init]

# Option: default-timeout
# Notes:  specifies default timeout in seconds (handled default ipset timeout only)
# Values:  [ NUM ]  Default: 600

default-timeout = 600

ipmset = f2b-<name>
familyopt =


[Init?family=inet6]

ipmset = f2b-<name>6
familyopt = <sp>family inet6

<name> comes from your jail:
banaction = iptables-ipset-proto6
I'm surprised it doesn't have the name param: banaction = iptables-ipset-proto6[name=something]

The two come together to create the ipset name.  Have a look at your ipsets names:
ipset -L | grep -e Name
Any fail2ban ipsets will be prefixed with 'f2b-'. Make sure csf or iptables is utilizing the ipset.

Bill

On 9/2/2020 1:17 PM, Phillip Carroll wrote:
> On 9/2/2020 5:43 AM, Phillip Carroll wrote:
>> On 9/1/2020 6:04 PM, Richard Shaw wrote:
>>> On Tue, Sep 1, 2020 at 7:45 PM Phillip Carroll <dom...@en... 
>>> <mailto:dom...@en...>> wrote:
>>>
>>>     I have been using csf/lfd as my firewall for several years on several
>>>     versions of CentOS, currently CentOS7. I am using several ipset-managed
>>>     blocklists supported directly by csf. Some of these are fairly huge
>>>     (such as whole country blocks), and it changes them in fractions of a
>>>     second. Very happy with everything it does.
>>>
>>>     However, csf syntax for custom regex applied to logs is relatively
>>>     clumsy and error-prone so I have installed fail2ban in hopes of using
>>>     that for custom log-based bans.
>>>
>>>     For my initial testing I have set up one jail and a corresponding
>>>     filters. (I found that all very simple.)
>>>
>>>     My intent:
>>>     On filter matches, immediately ban the host IP for one full day. Use
>>>     ipset to implement the bans.
>>>
>>>     The test case basically watches my exim reject.log (using inotify) and
>>>     unerringly finds the naughty hosts I want to ban.
>>>
>>>     My setup:
>>>     jail.local has:
>>>
>>>      > [exim-reject]
>>>      > mode      = normal
>>>      > port      = smtp,ssmtp
>>>      > logpath   = /var/log/exim/reject.log
>>>      > filter    = exim-reject
>>>      > maxmatches = 1
>>>      > maxretry   = 1
>>>      > backend   = auto
>>>      > bantime   = 1d
>>>      > banaction = iptables-ipset-proto6
>>>      > enabled   = true
>>>
>>>     And exim-reject.conf contains:
>>>
>>>      > [INCLUDES]
>>>      > before = exim-common.conf
>>>      > [Definition]
>>>      > failregex = <HOST> is listed at zen.spamhaus.org
>>>     <http://zen.spamhaus.org>
>>>      >             \[<HOST>\]:25 dropped: too many syntax or protocol errors
>>>
>>>     The contents of fail2ban.log indicates everything is working. It
>>>     says it
>>>     found the lines I expected it to find, and has issued bans (and
>>>     unbans a
>>>     day later).
>>>
>>>     However, when I list the ipset sets on the console, the only sets
>>>     listed
>>>     are those managed by csf. Clearly I have implemented something
>>>     incorrectly. I am hoping somebody on the list can set me straight.
>>>     Is it
>>>     possibly a permissions problem?
>>>
>>>
>>> That quite a bit more complex installation than I use so can't help you there, but fail2ban version and source (EPEL, self 
>>> install, etc) would be helpful.
>>>
>>> Thanks,
>>> Richard
>>
>> @Richard,
>>
>> This server has only prebuilt packages from the standard repos, managed using yum. It is a pretty typical headless server.
>>
>> I don't use selinux because of conflicts with the ISP provided kernel. (Linode)
>>
>>  From yum list installed:
>>> fail2ban.noarch 0.11.1-9.el7.2             @epel fail2ban-server.noarch 0.11.1-9.el7.2             @epel ipset.x86_64 
>>> 7.1-1.el7                  @base ipset-libs.x86_64 7.1-1.el7                  @base iptables.x86_64 
>>> 1.4.21-34.el7              @base iptables-services.x86_64 1.4.21-34.el7              @base 
>>
>> Phil
>>
>>
>> _______________________________________________
>> Fail2ban-users mailing list
>> Fai...@li...
>> https://lists.sourceforge.net/lists/listinfo/fail2ban-users
>
> I turned on DEBUG and restarted fail2ban.  Following is the result in fail2ban.log.
>
> The difficulty I have is understanding the content of the scripting macros used in the various commands at the time the 
> commands are issued.  For instance, what does "<ipmset>" contain at the time of the ipset create command?  The log would 
> appear to indicate the scripting is such that values are used before they are defined. After the create, I see the value set 
> as 'f2b-<name>' which is dependent on the value of <name> which is set on a subsequent set command as 'exim-reject'.
>
> If this order is the actual sequence it would explain the fact that the set f2b-exim-reject' does not exist after the restart 
> of fail2ban.
> However, perhaps I just don't understand the inner workings.
>
> The log:
>
>> 2020-09-02 08:34:17,939 fail2ban.server         [4077]: INFO    Reload all jails
>> 2020-09-02 08:34:17,939 fail2ban.server         [4077]: INFO Reload jail 'exim-reject'
>> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: DEBUG Setting usedns = warn for FilterPyinotify(Jail('exim-reject'))
>> 2020-09-02 08:34:17,940 fail2ban.server         [4077]: DEBUG     failregex: '<HOST> is listed at zen.spamhaus.org'
>> 2020-09-02 08:34:17,940 fail2ban.server         [4077]: DEBUG     failregex: '\\[<HOST>\\]:25 dropped: too many syntax or 
>> protocol errors'
>> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: INFO      maxRetry: 1
>> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: INFO      encoding: UTF-8
>> 2020-09-02 08:34:17,940 fail2ban.filter         [4077]: INFO      findtime: 600
>> 2020-09-02 08:34:17,940 fail2ban.actions        [4077]: INFO      banTime: 86400
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG Created <class 'fail2ban.server.action.CommandAction'>
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionunban = 'ipset del <ipmset> <ip> -exist'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionflush = 'ipset flush <ipmset>'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionstop = '<iptables> -D INPUT -p tcp -m multiport 
>> --dports smtp,ssmtp -m set --match-set <ipmset> src -j <blocktype>\nipset flush <ipmset>\nipset destroy <ipmset>'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionstart = 'ipset create <ipmset> hash:ip timeout 
>> 600<familyopt>\n<iptables> -I INPUT -p tcp -m multiport --dports smtp,ssmtp -m set --match-set <ipmset> src -j <blocktype>'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionprolong = 'ipset add <ipmset> <ip> timeout 
>> <bantime> -exist'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set actionban = 'ipset add <ipmset> <ip> timeout <bantime> 
>> -exist'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set protocol = 'tcp'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set chain = '<known/chain>'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set lockingopt = '-w'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set ipmset = 'f2b-<name>'
>> 2020-09-02 08:34:17,941 fail2ban.CommandAction  [4077]: DEBUG     Set blocktype = 'REJECT --reject-with icmp-port-unreachable'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set default-timeout = '600'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set blocktype?family=inet6 = 'REJECT --reject-with 
>> icmp6-port-unreachable'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set port = 'smtp,ssmtp'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set actname = 'iptables-ipset-proto6'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set iptables = 'iptables <lockingopt>'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set familyopt?family=inet6 = '<sp>family inet6'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set familyopt = ''
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set returntype = 'RETURN'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set ipmset?family=inet6 = 'f2b-<name>6'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set iptables?family=inet6 = 'ip6tables <lockingopt>'
>> 2020-09-02 08:34:17,942 fail2ban.CommandAction  [4077]: DEBUG     Set name = 'exim-reject'
>> 2020-09-02 08:34:17,942 fail2ban.server         [4077]: INFO Jail 'exim-reject' reloaded
>> 2020-09-02 08:34:17,943 fail2ban.actions        [4077]: NOTICE [exim-reject] Flush ticket(s) with iptables-ipset-proto6
>> 2020-09-02 08:34:17,943 fail2ban.actions        [4077]: DEBUG     Unbanned 7, 7 ticket(s) in 'exim-reject'
>> 2020-09-02 08:34:17,943 fail2ban.actions        [4077]: DEBUG exim-reject: action iptables-ipset-proto6 terminated
>> 2020-09-02 08:34:17,943 fail2ban.server         [4077]: INFO Reload finished.
>
>
>
> _______________________________________________
> Fail2ban-users mailing list
> Fai...@li...
> https://lists.sourceforge.net/lists/listinfo/fail2ban-users

Re: [Fail2ban-users] How to implement ban using ipset

[Kenneth P. <sh...@se...>]

--On Tuesday, September 01, 2020 5:48 PM -0700 Phillip Carroll 
<dom...@en...> wrote:

> The contents of fail2ban.log indicates everything is working. It says it
> found the lines I expected it to find, and has issued bans (and unbans a
> day later).
>
> However, when I list the ipset sets on the console, the only sets listed
> are those managed by csf. Clearly I have implemented something
> incorrectly. I am hoping somebody on the list can set me straight. Is it
> possibly a permissions problem?

Do the bans appear in /var/lib/fail2ban/fail2ban.sqlite3? You can use the 
sqlite3 utility from the sqlite package to inspect its contents.

Re: [Fail2ban-users] How to implement ban using ipset

[Kenneth P. <sh...@se...>]

--On Wednesday, September 02, 2020 11:17 AM -0700 Phillip Carroll 
<dom...@en...> wrote:

> I turned on DEBUG and restarted fail2ban.  Following is the result in
> fail2ban.log.

Near the end of /usr/lib/python2.7/site-packages/fail2ban/server/action.py 
I see:

        if logSys.getEffectiveLevel() < logging.DEBUG:
            logSys.log(9, realCmd)

Should that be a "<="? Otherwise I don't think it will log the final 
command to be executed.

Re: [Fail2ban-users] How to implement ban using ipset

[Kenneth P. <sh...@se...>]

--On Wednesday, September 02, 2020 11:58 AM -0700 Kenneth Porter 
<sh...@se...> wrote:

> Near the end of
> /usr/lib/python2.7/site-packages/fail2ban/server/action.py I see:
>
>         if logSys.getEffectiveLevel() < logging.DEBUG:
>             logSys.log(9, realCmd)

I changed this to the following to get the thing to log all commands. Note 
that Python is sensitive to indentation (which I hate, because tab size 
influences syntax so editor settings on spaces versus tabs are critical). 
Make sure the replacement line has the same two tabs in front of it as the 
if line.

        logSys.error("executeCmd: %s", realCmd)

(Remember to back up the old action.py file so you can restore it once you 
get this working!)

When you start the fail2ban service, you should see a new action.pyc file 
appear. (It gets recompiled when the script is loaded.) Check 
/var/log/messages if anything goes wrong in case you have a syntax error.

Re: [Fail2ban-users] How to implement ban using ipset

[Kenneth P. <sh...@se...>]

[replying back on list.]

--On Wednesday, September 02, 2020 12:58 PM -0700 Phillip Carroll 
<pos...@en...> wrote:

> I confess I am not fluent in Python. More like Python illiterate.

I can read it but not well and am pretty new at writing any.

> I notice that my server has both Python2.7 and Python3.6 libs installed.
> However, only 2.7 has ./site-packages/fail2ban.  I assume fail2ban uses
> the 2.7 version.

2.7 is the "system" version. Some packages require 3.6 and will pull that 
in and override the path to take advantage of it.

Most of the fail2ban-server is installed as a library and then invoked from 
a more conventional /usr/bin master script. This seems to be a pattern in 
scripted languages where an app has many script source files.

> I see the line you referred to is the same in my install.  Maybe someone
> else can shed light on whether that line is coded correctly.

I've also got a CentOS 7.x server and am using the distribution package 
from yum. ;)

> As to your earlier question about contents of fail2ban.sqlite3:
> I am a bit rusty on sqlite.

I'm having to look this up as I use it infrequently.

> How do I list tables in main?  I don't get any output from .tables.

I see this:

sqlite> .tables
bans        fail2banDb  jails       logs

How big is your file? Here's what I see on my public server at Linode:

-rw-------.  1 root root 32382976 Sep  2 19:28 fail2ban.sqlite3

Re: [Fail2ban-users] How to implement ban using ipset

[Phillip C. <dom...@en...>]

On 9/1/2020 4:48 PM, Phillip Carroll wrote:
> when I list the ipset sets on the console, the only sets listed 
> are those managed by csf. Clearly I have implemented something 
> incorrectly. 

Problem solved!

It turns out, I had everything configured correctly.

All I had to do to resolve the issue was reboot with the fail2ban server 
started automatically at boot time. All of my prior testing was with 
fail2ban started manually.

I just applied my computer golden rule:  "When all else fails, reboot 
the system."

Immediately after rebooting, listed ipsets and the set I had configured 
appeared in the list.

Mucho thanks to all who offered their help.

Phil

Re: [Fail2ban-users] How to implement ban using ipset

[Patrick S. <pa...@op...>]

* Phillip Carroll <dom...@en...> [09-03-20 11:27]:
> On 9/1/2020 4:48 PM, Phillip Carroll wrote:
> > when I list the ipset sets on the console, the only sets listed are
> > those managed by csf. Clearly I have implemented something incorrectly.
> 
> Problem solved!
> 
> It turns out, I had everything configured correctly.
> 
> All I had to do to resolve the issue was reboot with the fail2ban server
> started automatically at boot time. All of my prior testing was with
> fail2ban started manually.
> 
> I just applied my computer golden rule:  "When all else fails, reboot the
> system."

you must come from windows
 
> Immediately after rebooting, listed ipsets and the set I had configured
> appeared in the list.

systemctl restart fail2ban

no need to reboot linux unless dbus or kernel changes.  there may be one
or two more now, but "used to be" only necessary for changing kernel.


-- 
(paka)Patrick Shanahan       Plainfield, Indiana, USA          @ptilopteri
http://en.opensuse.org    openSUSE Community Member    facebook/ptilopteri
Photos: http://wahoo.no-ip.org/piwigo		    paka @ IRCnet freenode

Re: [Fail2ban-users] How to implement ban using ipset

[Phillip C. <dom...@en...>]

On 9/3/2020 8:23 AM, Phillip Carroll wrote:
> On 9/1/2020 4:48 PM, Phillip Carroll wrote:
>> when I list the ipset sets on the console, the only sets listed are 
>> those managed by csf. Clearly I have implemented something incorrectly. 
> 
> Problem solved!
> 

One more discovery:

Most likely I didn't need the reboot.  All I needed to do was use the 
console command:

systemctl restart fail2ban.

All of my "manual restarts" of fail2ban server were previously done 
using Webmin.  Webmin presents a nice interactive page with "Fqail2Ban 
Intrusion Detector" with buttons for restart server, and stop server.

HOWEVER:  This method of restarting is evidently seriously broken.  I 
will report that problem to the Webmin developers. I discovered that 
this morning after rebooting.  I saw a configuration error in the 
fail2ban.log, fixed it, and restarted fail2ban again using Webmin. I 
then entered on the console:
   ..]# systemctl status fail2ban

The result said it was active for 20 minutes. WHOA! I just restarted!

I then restarted using the console, checked the status, It showed one 
second up time. I then listed the ipset and FINALLY it listed entries 
for the first time since starting this adventure!

Live and learn.  I hope my reports of these interesting effects save 
somebody else from making the same mistakes.  I may write a "cheat 
sheet" howto on getting fail2ban up and running with a custom log watch.

I will post my debugging logs if anyone is interested. I am also now 
finally getting useful debug information in the fail2ban log.

Phil

Re: [Fail2ban-users] How to implement ban using ipset

[Kenneth P. <sh...@se...>]

On 9/3/2020 9:16 AM, Phillip Carroll wrote:
>
> Most likely I didn't need the reboot.  All I needed to do was use the 
> console command:
>
> systemctl restart fail2ban.
>
> All of my "manual restarts" of fail2ban server were previously done 
> using Webmin.  Webmin presents a nice interactive page with "Fqail2Ban 
> Intrusion Detector" with buttons for restart server, and stop server.
>
> HOWEVER:  This method of restarting is evidently seriously broken. 

My sysadmin motto is: GUIs lie.

I had a similar issue when setting up a Windows Server a few years ago 
as a border router and couldn't get the routing and firewall to work 
right. I ended up resetting to factory and starting over.

My love of Linux is because I can go deep, all the way to source code, 
if necessary, to see what's really going on. Windows is a beautiful 
thing when it works, but a horrible thing to debug when anything goes 
wrong. It sounds like Webmin has the same issue. It's like a Buick 
that's pleasant to drive but you have to take it to the dealer whenever 
you have an issue and pay big bucks to get "professionals" to fix it.

Your issues led me to look at my own fail2ban logs and discover a bug in 
the default port variable, which I just reported on GitHub. In the 
process of debugging, I also found it necessary to restart firewalld 
("systemctl restart firewalld"), as my fail2ban-ipset rules weren't 
getting refreshed by firewalld. That might be a bug in either firewalld 
or fail2ban but I lack the time to pursue that and it's now working, so 
I'm mentioning it in case anyone else sees this symptom.