Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once (Christopher Engelhard)

[<Bu...@Bu...>]

Two thoughts... 25% is an interesting number, it implies either you are
actually getting 2Core of 2Threads each in your VM (saturating a single
processor) or something is seriously restricting your ability to beat a core
to death. VMware statistics would be interesting too.

Second thought ... rejigger your test and run it to native ipset add
commands. That will give you a best-case performance value and eliminate the
firewall-cmd and dbus overhead. That should be your yard-stick.

-----Burton



Date: Fri, 28 Aug 2020 17:18:38 +0200
From: Christopher Engelhard <ce...@lc...>
To: ssh...@li...
Subject: Re: [SSHGuard-users] performance when using firewalld:
	adding/removing many entries at once
Message-ID: <86a...@lc...>
Content-Type: text/plain; charset=utf-8

On 28.08.20 13:26, Christopher Engelhard wrote:
> & I haven't done any benchmarking yet.

OK, did some playing around. All testing with 6000 random IP block requests
at 100/s (i.e. over 1 min) on a 2 core/8GB RAM virtual machine)

Using the firewalld backend of 2.4.1, it takes 24min to add all IPs to the
blocklist, CPU load during that time is fairly consistently 25% for
firewalld & 5-10% for firewalld-cmd.

Using the "collecting" version & collecting requests for 1s doesn't
significantly change the overall load, but the process now completes in just
over 3 min.

Collecting for 5s or 10s reduces this a bit further to ~2:30 min, again with
no significant load reduction in firewalld. Firewall-cmd only causes
significant CPU load whenever it is triggered by the backend.

Given that there's no difference between 5s or 10s grouping in overall
runtime, I'd say that pretty much reflects the speed at which firewalld is
able to add ips to the ipset.

I think the bottleneck is firewalld processing the commands it receives on
DBus, not firewall-cmd sending them off, otherwise I'd expect to see much
less load on firewalld in the first test compared to the later ones.

Christopher

P.S.: The total number of IPs that ended up in the ipsets dropped to
~5200 in the last two tests, so it is possible that I'm running into some
issues with maximum string lengths/command lengths or so. Probably not a
good idea to set it this high outside of testing.