|
From: Jai H. <jai...@mu...> - 2018-04-05 16:10:34
|
Also have results for XML body which I added to attachment below. On Thu, Apr 5, 2018 at 11:01 AM, Jai Harpalani <jai...@mu...> wrote: > Appears that the image I attached is not viewable. I've attached pdf, > instead. > > On Thu, Apr 5, 2018 at 10:55 AM, Jai Harpalani <jai...@mu... > > wrote: > >> I ran some tests with a) ModSecurity disabled, b) ModSecurity v3.0.0, and >> c) ModSecurity v3.0.1. >> I saw a speedup of about 3.5x with v3.0.1 vs v3.0.0. >> Details are below. >> >> >> >> >> >> On Thu, Apr 5, 2018 at 8:33 AM, Felipe Zimmerle <fe...@zi...> >> wrote: >> >>> Hi, >>> >>> On Wed, Apr 4, 2018 at 2:44 PM Robert Paprocki < >>> rpa...@fe...> wrote: >>> >>>> Hi, >>>> >>>> On Wed, Apr 4, 2018 at 5:03 AM, Christian Folini < >>>> chr...@ne...> wrote: >>>>> >>>>> >>>>> > However, I have limited knowledge on the following: >>>>> > - is ModSec 3.x has been ever targeted to support CRS < 3, >>>>> >>>>> See above. >>>>> >>>> >>>> It would be great to here an official stance from the development team >>>> on this. @Felipe can you comment? >>>> >>>> >>> Feature wise it should support both. ModSecurity was not written to >>> support OWASP CRS, but the SecRule Language. >>> Consequently it supports OWASP CRS. There is a milestone for that here: >>> https://github.com/SpiderLabs/ModSecurity/milestone/11 >>> >>> >>>> >>>>> > Also I'm not sure whether ModSec 2.x has its own benchmarks (not >>>>> related to any connector). >>>>> > If it does, then perhaps it would be good to compare "generic" >>>>> ModSec 2.x >>>>> > vs "generic" ModSec 3.x as well. >>>>> >>>>> Yes, that would be cool. But from what I understand, ModSec 2.9.x is >>>>> deeply >>>>> integrated into the webserver. >>>>> >>>> >>>> >>>> >>> I don't think it is possible to split v2 from Apache. >>> >>> >>>> Yeah, this is a bit of a pain to test. I modified one of the example >>>> programs that comes with the v3/master ModSecurity source code as follows: >>>> >>>> https://gist.github.com/p0pr0ck5/9b2c414641c9b03d527679d0c8cb7d86 >>>> >>>> >>> There is a benchmark utility in the repository: >>> https://github.com/SpiderLabs/ModSecurity/tree/v3/master/test/benchmark >>> >>> >>>> Note that this doesn't add any headers or body data (request or >>>> response) to the transaction, and the included "basic_rules.conf" is >>>> unchanged from what's in the example repo (and ignore the memory leak in >>>> not cleaning up the transaction). So running this very light example: >>>> >>>> $ ./test >>>> Rules: >>>> Phase: 0 (0 rules) >>>> Phase: 1 (0 rules) >>>> Phase: 2 (2 rules) >>>> Rule ID: 200000--0x1e2b6c0 >>>> Rule ID: 200001--0x1e2bc90 >>>> Phase: 3 (4 rules) >>>> Rule ID: 200002--0x1e2c720 >>>> Rule ID: 200003--0x1e2ea00 >>>> Rule ID: 200004--0x1e2f200 >>>> Rule ID: 200005--0x1e2fc10 >>>> Phase: 4 (0 rules) >>>> Phase: 5 (0 rules) >>>> Phase: 6 (0 rules) >>>> Phase: 7 (0 rules) >>>> Did 9907 >>>> Done! >>>> >>>> We see around 10k processes per second. Essentially all of our time is >>>> spent waiting on memory allocations: https://s3.amazon >>>> aws.com/p0pr0ck5-data/modsec-simple.svg >>>> >>>> Now consider the case where we include the full 3.0.0 CRS: >>>> >>>> $ ./test >>>> Rules: >>>> Phase: 0 (32 rules) >>>> Rule ID: 0--0x22ca500 >>>> Rule ID: 0--0x22d9480 >>>> Rule ID: 0--0x22f9530 >>>> Rule ID: 0--0x22f9f20 >>>> [...snip several hundred lines...] >>>> Did 319 >>>> Done! >>>> >>>> So this is about the same throughput we saw from Nginx + libmodsec >>>> integration. A flamegraph also highlights hot paths, particularly in >>>> Rule::evaluate (also largely spent on allocation wait time at this >>>> point, since each evaluation instantiates many new objects): >>>> https://s3.amazonaws.com/p0pr0ck5-data/modsec-simple-crs.svg >>>> >>>> (I also performed the same tests on 3.0.0; flamegraphs are also in this >>>> s3 bucket but I will avoid commentary here for now as this thread is gotten >>>> long in the tooth as it is). >>>> >>>> A few initial takeaways: >>>> >>>> - There is a clearly definable minimum overhead needed to execute >>>> libmodsecurity, based on its current architecture >>>> - I noted the same memory leak that defanator noted in #1729. I have >>>> yet to apply the patches noted. >>>> - HTTP integrations will induce some overhead, which may or may not be >>>> substantial. With smart memory pooling and good design, though, this should >>>> be kept to a minimum. >>>> - Leveraging the full CRS completely tanks throughput. It would take a >>>> while more to dig into specifics but I suspect that Rule::evaluate >>>> probably needs an overhaul. >>>> - I ran one more test again a mock set of dumb rules (generated via >>>> https://gist.github.com/p0pr0ck5/c99bca54734af7546d910db8d7c97ab3). >>>> Saw about 1000 transactions processed per second, indicating (as should be >>>> assumed) that linear growth in the size of the included ruleset results in >>>> similar performance reduction. Flamegraphs again point to >>>> Rule::evaluate, with Rule::getFinalVars taking up half those traces. I >>>> suspect there could be some smarter behaviors about avoiding object >>>> creation if its unneeded on these hot paths. >>>> >>>> >>> There are few things in the code that can be improved. There are even >>> "TODO:" marking. But at certain point you may want to look at the rules. >>> That is very important. >>> >>> Hopefully this is of use to some folks. I'd be interested in writing >>>> some automated scaffolding to execute this type of testing against every >>>> commit, to maintain a continuous audit trail of performance impacts; if >>>> this sounds interesting to the maintainers I'd be happy to chat further. >>>> >>> >>> Andrei does that already. Not automagically, but frequently updated. >>> Some of the results are posted here: >>> https://github.com/defanator/modsecurity-performance/wiki >>> >>> >>>> ------------------------------------------------------------ >>>> ------------------ >>>> Check out the vibrant tech community on one of the world's most >>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______ >>>> _________________________________________ >>>> mod-security-users mailing list >>>> mod...@li... >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>>> http://www.modsecurity.org/projects/commercial/rules/ >>>> http://www.modsecurity.org/projects/commercial/support/ >>>> >>> >>> ------------------------------------------------------------ >>> ------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> _______________________________________________ >>> mod-security-users mailing list >>> mod...@li... >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>> http://www.modsecurity.org/projects/commercial/rules/ >>> http://www.modsecurity.org/projects/commercial/support/ >>> >>> >> > |
