|
From: Christian F. <chr...@ne...> - 2018-04-04 09:22:11
|
Osama, You mean ab is like a tool from the stone age? How dare you! :) I'll investigate. Appreciated. A word of caution, though: As long as we are talking of performance boost of several hundred percents between releases, I doubt that the benchmark tool is of much concern. Wish I had known / used wrk when I wrote the performance chapter for the ModSecurity Handbook, though. I used a variety of tools there and they all had their issues. Ahoj, Christian On Wed, Apr 04, 2018 at 04:12:48AM -0500, Osama Elnaggar wrote: > Some interesting ideas here. I think using a single tool (+ a specific set > of queries) for our benchmarks would be useful + be more standardized. > > wrk is currently one of the most promising benchmarking tools and supports > Lua plugins so it is a lot more flexible than ab. I noticed that both > Robert and Andrei used it (Christian: time to ditch ab :)). I also > recently used it recently (with the below script) to benchmark an API > endpoint (ModSecurity wasn’t part of the solution so I don’t have any > ModSecurity benchmarks using wrk). It is a lot more flexible than ab. > > You might find the multiplepaths.lua script / plugin useful. > multiplepaths.lua (https://github.com/timotta/wrk-scripts) is a Lua script > that allows you to provide wrk with a file with different queries you want > to perform in your benchmark so you can cover different areas such as OS > command injection, SQL injection, XSS, etc. > > Since it is written in Lua, we can probably extend it to provide additional > customized payloads so they aren’t all in the GET request + send customized > cookies, headers, etc. > > Another option would be to write something up with Python + asyncio + > requests although that will need a little more effort > > -- > Osama Elnaggar > > On April 4, 2018 at 6:46:57 PM, Andrei Belov (de...@ng...) wrote: > > Hi folks, > > > On 04 Apr 2018, at 07:48, Christian Folini <chr...@ne...> > wrote: > > > > Hey Robert, > > > > On Tue, Apr 03, 2018 at 05:50:07PM -0700, Robert Paprocki wrote: > >> Can you share the specifics of your evaluation? Performance in modsec + > crs > >> will vary greatly depending on the request payload. Soon I would like to > do > >> some before and after trace profiling of these releases to better > illustrate > >> how libmodsec performs in various conditions. > > > > I did a minimal self-compiled NGINX with a basic ModSecurity and CRS > > as documented on https://www.netnea.com/cms/nginx-modsecurity-tutorials/ > . > > (These new tutorials are in a draft state, the quality is not yet there. > Use > > with caution.) > > [..] > > > Felipe tagged a 3.0.2 yesterday and made it available at > > https://github.com/SpiderLabs/ModSecurity/releases > > I took that one for my tests. I reckon the performance is the same as > with > > the 3.0.1 that has been announced. > > > > This perf test is obviously very superficial. A thing to note is that > even > > testrun 2 would write the error-log (to gather statistical data). > > > > But whatever the specifics, I think this big performance boost will show > in > > any setup even if the factor might not be that high. > > > > Having real perf tests done regularly would be very welcome, Robert. > > JFYI, I have created vagrant-based tools to run performance tests with > nginx and libmodsecurity some time ago: > > https://github.com/defanator/modsecurity-performance > > It creates pre-configured environment suitable for wide range of > investigations, > related both to performance and functionality. I tried to include > meaningful > configurations, e.g.: > > https://github.com/defanator/modsecurity-performance#what-is-being-tested > > I think that environment could be [relatively easily] extended to support > Apache + ModSec 2.x, in addition to nginx + ModSec 3.x, in order to > simplify > "direct" comparison and provide reproducible, statistically significant > results. > > (PRs are welcome of course.) > > > -- > Andrei Belov > Product Engineer > NGINX > ------------------------------------------------------------------------------ > > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ -- https://www.feistyduck.com/training/modsecurity-training-course https://www.feistyduck.com/books/modsecurity-handbook/ mailto:chr...@ne... twitter: @ChrFolini |
