One last round of experiments.
I optimized the code that adds the the xt_recent ip's to the hash table.
I got rid of the printf that got run with every command (was called 7990 times), and shaved .1 seconds
off the time. Normally, there would be no such printf in the fail2ban ban commands.
Then, instead of running a system() call to do the echo "+ip" > /proc/net/xt_recent/recent1,
I did an fopen of that recent1 file, fprintf'd the "+<ip>" to that open file,
and then fclosed the file pointer. Got rid of the fork and shell startup... and
shaved the total run time to add 7990 ips from 5.2 seconds to 0.204 sec avg.
That is a whopping 347 times faster than adding iptables rules.... WOW!
To replicate this optimization, tho, in fail2ban, you'd have to write up some
kind of directive in the ban/unban code, that instead of running the command
lines into a forked shell, would just open the file, write the date, and close the
file directly inside fail2ban:
actionban = FILEWRITE("/proc/net/xt_recent/recent1","+<ip>")
or something similar. No system() or fork(), no shell to run. Done
from the thread directly. 347x faster!
For ipset, it might benefit in like fashion, if it could be directly executed
via fork, instead of a fork of shell... maybe something like this:
actionban = JUSTFORK("/usr/sbin/ipset FirstSet <ip>")
I'd have to program up a fork/exec chunk of code to simulate this... don't know
how much faster this would run over using system()....
All I know is, when you are under attack, and the requests are coming in at over 100/sec,
and your cpu is loaded, every millisecond counts!