One last round of experiments.

I optimized the code that adds the the xt_recent ip's to the hash table.
I got rid of the printf that got run with every command (was called 7990 times), and shaved .1 seconds
off the time. Normally, there would be no such printf in the fail2ban ban commands.
Then, instead of running a system() call to do the echo "+ip" > /proc/net/xt_recent/recent1,
I did an fopen of that recent1 file, fprintf'd the "+<ip>" to that open file,
and then fclosed the file pointer. Got rid of the fork and shell startup... and
shaved the total run time to add 7990 ips from 5.2 seconds to 0.204 sec avg.
That is a whopping 347 times faster than adding iptables rules.... WOW!

To replicate this optimization, tho, in fail2ban, you'd have to write up some
kind of directive in the ban/unban code, that instead of running the command
lines into a forked shell, would just open the file, write the date, and close the
file directly inside fail2ban:

actionban = FILEWRITE("/proc/net/xt_recent/recent1","+<ip>")

or something similar. No system() or fork(), no shell to run. Done
from the thread directly. 347x faster!

For ipset, it might benefit in like fashion, if it could be directly executed
via fork, instead of a fork of shell... maybe something like this:

actionban = JUSTFORK("/usr/sbin/ipset FirstSet <ip>")

I'd have to program up a fork/exec chunk of code to simulate this... don't know
how much faster this would run over using system()....

All I know is, when you are under attack, and the requests are coming in at over 100/sec,
and your cpu is loaded, every millisecond counts!


murf

 


On Mon, Jan 27, 2014 at 10:42 AM, Steve Murphy <murf@parsetree.com> wrote:

On Mon, Jan 27, 2014 at 1:07 AM, Steve Murphy <murf@parsetree.com> wrote:


On Sun, Jan 26, 2014 at 1:14 AM, Daniel Black <daniel.subs@internode.on.net> wrote:
On 01/26/2014 05:13 PM, Steve Murphy wrote:
> (another thing. Generating lots of new iptables rules
> did affect the mtr results while they were being inserted!)

I'd be interested in a benchmark on the iptables-ipset actions.


OK, I went to an ubuntu system and installed everything ipset, reading the docs
now. I'll happily re-run my tests in this regime and come up with some stats
on how fast they are added on the same system as the iptables/xt_recent stuff
for good comparisons. After a few minutes looking at the man pages, it looks
very interesting.

murf

​​
 
OK, done.

ipset loads 7990 new ip's in 11.99 sec.

To do it, I use the same C program, but change the system call string to be:

ipset add FirstSet 10.0.%d.%d    (last two digits count up in a for loop).


The ipset "FirstSet" is created via:

ipset create FirstSet hash:ip hashsize 65536 maxelem 32768

This is a bit larger than the 8k from xt_recent.... but I doubt that it affects the add times.

and to remove the set...

ipset destroy FirstSet

And, not yet having tested it, I'd assume that the rules to check this would look like this:

actionstart =
iptables -N fail2ban-ipset-1
iptables -A fail2ban-ipset-1 -m set --match-set FirstSet src -m limit --limit 1/minute -j LOG ...
iptables -A fail2ban-ipset-1 -m set --match-set FirstSet src -j DROP
iptables -A fail2ban-ipset-1 -j RETURN
iptables -I INPUT -j fail2ban-ipset-1

I ran thu the sequence (create, adds, destroy) several times. the adds take 11.99x seconds. So far only
the 3rd digit after the dec. pt. changes between runs. Not a lot of variability.

So, xt_recent loads ip's about twice as fast as ipset.

All I can say, is either of these methods would work well. ipset can allow you bigger sets than xt_recent...
but if you have 8k or more machines probing you, ... is your bandwidth going to support that? Wow!
Half the speed of really fast is still pretty fast... about 6x the speed of adding rules to an iptables chain.

It'd be maybe cool to add 32k to ipsets, config iptables, and document that ping times don't budge,
but I'm pretty confident that hashes being as they are, you could have > 1 million entries in the table,
and it won't be any slower than if you had 10 ip's in the table (I don't know what the hashing algorithm is,
but as long as the hashing spreads the entries evenly amongst the slots, and  hash slots are more numerous than the entries in the table, it shouldn't degrade at all. Even if the entries are more than the slots, lists 10 or 20 long will probably not even slow things down.
​ Much. I hope.​


​murf


--

Steve Murphy
ParseTree Corporation
57 Lane 17
Cody, WY 82414
✉  murf at parsetree dot com
307-899-5535





--

Steve Murphy
ParseTree Corporation
57 Lane 17
Cody, WY 82414
✉  murf at parsetree dot com
☎ 307-899-5535