|
From: Jai H. <jai...@mu...> - 2018-04-05 16:01:48
|
Appears that the image I attached is not viewable. I've attached pdf, instead. On Thu, Apr 5, 2018 at 10:55 AM, Jai Harpalani <jai...@mu...> wrote: > I ran some tests with a) ModSecurity disabled, b) ModSecurity v3.0.0, and > c) ModSecurity v3.0.1. > I saw a speedup of about 3.5x with v3.0.1 vs v3.0.0. > Details are below. > > > > > > On Thu, Apr 5, 2018 at 8:33 AM, Felipe Zimmerle <fe...@zi...> > wrote: > >> Hi, >> >> On Wed, Apr 4, 2018 at 2:44 PM Robert Paprocki < >> rpa...@fe...> wrote: >> >>> Hi, >>> >>> On Wed, Apr 4, 2018 at 5:03 AM, Christian Folini < >>> chr...@ne...> wrote: >>>> >>>> >>>> > However, I have limited knowledge on the following: >>>> > - is ModSec 3.x has been ever targeted to support CRS < 3, >>>> >>>> See above. >>>> >>> >>> It would be great to here an official stance from the development team >>> on this. @Felipe can you comment? >>> >>> >> Feature wise it should support both. ModSecurity was not written to >> support OWASP CRS, but the SecRule Language. >> Consequently it supports OWASP CRS. There is a milestone for that here: >> https://github.com/SpiderLabs/ModSecurity/milestone/11 >> >> >>> >>>> > Also I'm not sure whether ModSec 2.x has its own benchmarks (not >>>> related to any connector). >>>> > If it does, then perhaps it would be good to compare "generic" ModSec >>>> 2.x >>>> > vs "generic" ModSec 3.x as well. >>>> >>>> Yes, that would be cool. But from what I understand, ModSec 2.9.x is >>>> deeply >>>> integrated into the webserver. >>>> >>> >>> >>> >> I don't think it is possible to split v2 from Apache. >> >> >>> Yeah, this is a bit of a pain to test. I modified one of the example >>> programs that comes with the v3/master ModSecurity source code as follows: >>> >>> https://gist.github.com/p0pr0ck5/9b2c414641c9b03d527679d0c8cb7d86 >>> >>> >> There is a benchmark utility in the repository: >> https://github.com/SpiderLabs/ModSecurity/tree/v3/master/test/benchmark >> >> >>> Note that this doesn't add any headers or body data (request or >>> response) to the transaction, and the included "basic_rules.conf" is >>> unchanged from what's in the example repo (and ignore the memory leak in >>> not cleaning up the transaction). So running this very light example: >>> >>> $ ./test >>> Rules: >>> Phase: 0 (0 rules) >>> Phase: 1 (0 rules) >>> Phase: 2 (2 rules) >>> Rule ID: 200000--0x1e2b6c0 >>> Rule ID: 200001--0x1e2bc90 >>> Phase: 3 (4 rules) >>> Rule ID: 200002--0x1e2c720 >>> Rule ID: 200003--0x1e2ea00 >>> Rule ID: 200004--0x1e2f200 >>> Rule ID: 200005--0x1e2fc10 >>> Phase: 4 (0 rules) >>> Phase: 5 (0 rules) >>> Phase: 6 (0 rules) >>> Phase: 7 (0 rules) >>> Did 9907 >>> Done! >>> >>> We see around 10k processes per second. Essentially all of our time is >>> spent waiting on memory allocations: https://s3.amazon >>> aws.com/p0pr0ck5-data/modsec-simple.svg >>> >>> Now consider the case where we include the full 3.0.0 CRS: >>> >>> $ ./test >>> Rules: >>> Phase: 0 (32 rules) >>> Rule ID: 0--0x22ca500 >>> Rule ID: 0--0x22d9480 >>> Rule ID: 0--0x22f9530 >>> Rule ID: 0--0x22f9f20 >>> [...snip several hundred lines...] >>> Did 319 >>> Done! >>> >>> So this is about the same throughput we saw from Nginx + libmodsec >>> integration. A flamegraph also highlights hot paths, particularly in >>> Rule::evaluate (also largely spent on allocation wait time at this >>> point, since each evaluation instantiates many new objects): >>> https://s3.amazonaws.com/p0pr0ck5-data/modsec-simple-crs.svg >>> >>> (I also performed the same tests on 3.0.0; flamegraphs are also in this >>> s3 bucket but I will avoid commentary here for now as this thread is gotten >>> long in the tooth as it is). >>> >>> A few initial takeaways: >>> >>> - There is a clearly definable minimum overhead needed to execute >>> libmodsecurity, based on its current architecture >>> - I noted the same memory leak that defanator noted in #1729. I have yet >>> to apply the patches noted. >>> - HTTP integrations will induce some overhead, which may or may not be >>> substantial. With smart memory pooling and good design, though, this should >>> be kept to a minimum. >>> - Leveraging the full CRS completely tanks throughput. It would take a >>> while more to dig into specifics but I suspect that Rule::evaluate >>> probably needs an overhaul. >>> - I ran one more test again a mock set of dumb rules (generated via >>> https://gist.github.com/p0pr0ck5/c99bca54734af7546d910db8d7c97ab3). Saw >>> about 1000 transactions processed per second, indicating (as should be >>> assumed) that linear growth in the size of the included ruleset results in >>> similar performance reduction. Flamegraphs again point to Rule::evaluate, >>> with Rule::getFinalVars taking up half those traces. I suspect there >>> could be some smarter behaviors about avoiding object creation if its >>> unneeded on these hot paths. >>> >>> >> There are few things in the code that can be improved. There are even >> "TODO:" marking. But at certain point you may want to look at the rules. >> That is very important. >> >> Hopefully this is of use to some folks. I'd be interested in writing some >>> automated scaffolding to execute this type of testing against every commit, >>> to maintain a continuous audit trail of performance impacts; if this sounds >>> interesting to the maintainers I'd be happy to chat further. >>> >> >> Andrei does that already. Not automagically, but frequently updated. Some >> of the results are posted here: >> https://github.com/defanator/modsecurity-performance/wiki >> >> >>> ------------------------------------------------------------ >>> ------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______ >>> _________________________________________ >>> mod-security-users mailing list >>> mod...@li... >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>> http://www.modsecurity.org/projects/commercial/rules/ >>> http://www.modsecurity.org/projects/commercial/support/ >>> >> >> ------------------------------------------------------------ >> ------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> http://www.modsecurity.org/projects/commercial/support/ >> >> > |
