Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin B. <kev...@gm...>]

On 2020/09/01 16:36, Christopher Engelhard wrote:
> 
> Execution speed is probably not really a focus for them, but I'm sort of
> hoping that there are some straightforward bottlenecks that simply
> nobody has bothered to identify yet. And to be fair, only being able to
> add ~5 IPs per second to a set is REALLY slow.
> 

Although I saw that the thread dried up in September last
year, I thought I'd pitch a bit more info into it.

I have just started up an SSHGuard 2.4.2 instance, on a
SLES 15-based Cray/HPE box, and have looked at the time
it appears to be taking to ingest IP addresses from the
blacklist that was placed there ahead of process starting.

I have assumed that the ingest is done a line at a time,
hence the grep -n of the blacklist for the IP address
that a systemctl status shows was being processed, so
as to give a rough timeline:


    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 1h 8min ago
            `-93206 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=119.63.84.130/32

# grep -n 119.63.84.130  /var/lib/sshguard/blacklist
7936:1616965857|100|4|119.63.84.130

    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 1h 12min ago
            `-95043 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=140.249.197.153/32

# grep -n 140.249.197.153  /var/lib/sshguard/blacklist
8405:1617116224|100|4|140.249.197.153

    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 1h 45min ago
            `-108270 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=188.166.237.18/32

# grep -n 188.166.237.18  /var/lib/sshguard/blacklist
12116:1620694731|100|4|188.166.237.18

    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 2h 0min ago
            |-61184 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=150.138.114.102/32

# grep -n 150.138.114.102  /var/lib/sshguard/blacklist
13916:1622407810|100|4|150.138.114.102


To summarise that data:


2h 00min 13916
1h 45min 12116  15mins 1800    120.00/m   2.00/s
1h 12min  8405  33mins 3711    112.45/m   1.87/s
1h  8min  7936   4mins  469    117.25/m   1.95/s


It does seem a bit slow, especially when compared to some
SLES 12-based instances here that use the IPTables backend.
(Apologies, I don't have any comparative figures to hand,
  at time of writing, but will look to get some)


Probably doesn't tell you anything you didn't already know,
Kevin