[Mod-security-developers] Fwd: Performance improvement plan

[Ivan R. <iva...@gm...>]

Forwarding message, as the original did not show in the archives.

Is it getting through?


---------- Forwarded message ----------
From: Ivan Ristic <iva...@gm...>
Date: Fri, Aug 21, 2009 at 5:54 PM
Subject: Performance improvement plan
To: mod...@li...


I am planning to improve the performance of the ModSecurity rule
engine. I've reviewed the source code and come up with the following
ideas:

1. Avoid table lookup currently used to determine if a rule is
multiMatch. I don't suppose this is going to result with a significant
improvement, but it's been getting on my nerves for years now and I
want to get rid of it. I plan to add a variable to the rule structure
and set it at configuration time. Before I do anything, however, I
will investigate how SecRuleUpdateActionById interacts with action
lists.

2. Change the individual rule processing code to avoid discovering
tfns at runtime. At the moment we are re-discovering tfns for every
variable, even though the action list never changes. Resolving the
tfns at configuration time should result a significant performance
improvement.

3. Cache entire target lists. Currently a lot of time is spent
resolving complex variables, adding and removing variables from the
target list, and transforming them in the end. I want to identify the
rules that share target lists, cache a reusable target list when it is
created for the first time, only to reuse it later. The cost of each
rule in the engine should go down a lot. In the worst case scenario we
waste a hash lookup (per rule). This should be countered by the
removal of the multiMatch lookup ;)

I have a couple of other smaller things I may look into, but I suspect
the improvements from the above 3 changes will make me happy enough.

The plan is to benchmark with the available rule sets.

The only question is what to branch and when? I am hoping for the
improvements to be significant enough to warrant a middle digit bump
and I'd like to see them go into 2.6.0.

-- 
Ivan Ristic
Security assessment of your SSL servers
https://www.ssllabs.com/ssldb/