Re: [mod-security-users] ModSecurity phase timing

[Felipe Z. <fe...@zi...>]

Hi,

On Tue, Mar 27, 2018 at 8:56 AM Christian Folini <
chr...@ne...> wrote:

> Hello Felipe,
>
> Thank you for your openness to discuss this on github in detail. Still,
> I'd like to continue in this thread because it has already started and
> because it interests a wider audience due of certain implications that
> have to do with the way you handle our concerns.
>
>
The issue is open for discussion on GitHub since "Dec 9, 2015" no one seems
to care about till yesterday. On GitHub we have more audience and
everything is indexed. Please use the 2015 issue to discuss this.

In case you missed something:
https://github.com/SpiderLabs/ModSecurity/issues/1011


On Mon, Mar 26, 2018 at 05:08:50PM +0000, Felipe Zimmerle wrote:
> > I have no bug report saying that DURATION is not working and a regression
> > test that leads me to believe that it is working. Thus, I am assuming
> that
> > it is working Ok.
>
> Here is your new bug report:
>
> https://github.com/SpiderLabs/ModSecurity/issues/1725
>
>
That is a good way to work in a open source project. Report a bug that was
found. Thank you in the name of the community.

(...)
>


> I have just been commissioned to come up with a decent approach to do
> performance analysis of existing services on libModSecurity / NGINX. This
> is
> great for me, because it means business. But it is bad for ModSecurity
> because
> people should be able to do this without Christian Folini coming up with a
> smart method. It should be easier than this (and it should be documented!):
>
>
There is a difference in between run ModSecurity and code ModSecurity. In
the same way that there is a difference in create a script and run a
script. We have published the script here:
https://github.com/SpiderLabs/ModSecurity-nginx/commits/master/ngx-modsec.stp
since 20 Sep 2015. We dind't charge nobody for it. The decision to ask for
money is ours.

Maybe the script is not in the shape that you like but it is working and
100% compatible with our current user base. May advice to you in to study
the current proposed method before complain about it. There are huge
advantages in this new approach. It is in fact a good thing for our users.
Having said that, can you provide a solid argument pointing towards stap
not be good?


> The standard questions in such situations is:
> - Which requests have a performance issue?
> - Is the performance issue with ModSecurity or with the backend
> application?
>
>
Here you can find answers to all your questions:
 https://github.com/openresty/openresty-systemtap-toolkit


With ModSecurity 2.9.2 this is easy to find out. See my tutorials
> https://www.netnea.com/cms/apache-tutorial-5_extending-access-log/
> https://www.netnea.com/cms/apache-tutorial-6_embedding-modsecurity/
> that describe an access log format that brings this performance data for
> every request and the ModSec rules necessary to fill the variables.
> This settles 90% of all performance issues immediately.
>
>
I understand that the new version outdated your tutorials, etc... I am
sorry for that.


> Now NGINX does not allow to display env variables easily in the access log
> and libModSecurity does not yet support env variables anyways. So I assumed
> I would have to take a different path. Now it looks like I would need to
> write everything into the error log and _then_ calculate the timings
> externally with a new tool / script that works with floating point numbers.
>
> And additionally I will look into systemtap because you told us to do so
> and because the example you give may be useful for experienced stap users,
> but it's beyond the knowledge level of the average ModSecurity user.
>
>
It is just about execute a script. Nothing more. Still:

a) The discussion from 2015 still open. I am not telling what to do. Happy
users are using stap.
b) The version 2.x is still maintained; If you don't want to learn
something new, you can keep using v2.


> I am sorry if this message is a bit rude. I usually try to remain calm and
> diplomatic. But this time, it did not work out and sleeping it over did
> not work either.
>
>
Feel free to share your feelings.


> We are all together on this ModSecurity ship. You are the captain and you
> steer the ship based on the decisions you think are best. The rest of the
> community only notices the decision once the ship starts to turn. You
> stated libModSecurity would be feature compatible to 2.9.x. Now we learn it
> is not and we have to make fundamental changes to our setups and
> methodology.
>
>
Like a said too many times: We started a discussion back in 2015. If you
are part of this community, why I we didn't have a comment from you?


> I am totally open for a discussion on the need for the PERF_ variables.
> If they mean a performance hit, then tell us how much and tell us why you
> think it is not acceptable. How is the libModSecurity 3.0 performance
> stacking up against 2.9.x? Is it really so good, that you want to squeeze
> out the last few cpu cycles to the very limit and PERF_ is the last
> remaining target?
> Having to calculate performance via arithemtics
> and DURATION also has a performance hit, and maybe one could make the
> PERF_ variables optional (compile time flag or directive that is off by
> default). But please do not take these decisions silently and then brush
> away our concerns.
>
>
Since 2015:
 https://github.com/SpiderLabs/ModSecurity/issues/1011

Br.,
Felipe.