|
From: Jai H. <jai...@mu...> - 2018-04-05 15:55:53
|
I ran some tests with a) ModSecurity disabled, b) ModSecurity v3.0.0, and c) ModSecurity v3.0.1. I saw a speedup of about 3.5x with v3.0.1 vs v3.0.0. Details are below. On Thu, Apr 5, 2018 at 8:33 AM, Felipe Zimmerle <fe...@zi...> wrote: > Hi, > > On Wed, Apr 4, 2018 at 2:44 PM Robert Paprocki <rpaprocki@ > fearnothingproductions.net> wrote: > >> Hi, >> >> On Wed, Apr 4, 2018 at 5:03 AM, Christian Folini < >> chr...@ne...> wrote: >>> >>> >>> > However, I have limited knowledge on the following: >>> > - is ModSec 3.x has been ever targeted to support CRS < 3, >>> >>> See above. >>> >> >> It would be great to here an official stance from the development team on >> this. @Felipe can you comment? >> >> > Feature wise it should support both. ModSecurity was not written to > support OWASP CRS, but the SecRule Language. > Consequently it supports OWASP CRS. There is a milestone for that here: > https://github.com/SpiderLabs/ModSecurity/milestone/11 > > >> >>> > Also I'm not sure whether ModSec 2.x has its own benchmarks (not >>> related to any connector). >>> > If it does, then perhaps it would be good to compare "generic" ModSec >>> 2.x >>> > vs "generic" ModSec 3.x as well. >>> >>> Yes, that would be cool. But from what I understand, ModSec 2.9.x is >>> deeply >>> integrated into the webserver. >>> >> >> >> > I don't think it is possible to split v2 from Apache. > > >> Yeah, this is a bit of a pain to test. I modified one of the example >> programs that comes with the v3/master ModSecurity source code as follows: >> >> https://gist.github.com/p0pr0ck5/9b2c414641c9b03d527679d0c8cb7d86 >> >> > There is a benchmark utility in the repository: > https://github.com/SpiderLabs/ModSecurity/tree/v3/master/test/benchmark > > >> Note that this doesn't add any headers or body data (request or response) >> to the transaction, and the included "basic_rules.conf" is unchanged from >> what's in the example repo (and ignore the memory leak in not cleaning up >> the transaction). So running this very light example: >> >> $ ./test >> Rules: >> Phase: 0 (0 rules) >> Phase: 1 (0 rules) >> Phase: 2 (2 rules) >> Rule ID: 200000--0x1e2b6c0 >> Rule ID: 200001--0x1e2bc90 >> Phase: 3 (4 rules) >> Rule ID: 200002--0x1e2c720 >> Rule ID: 200003--0x1e2ea00 >> Rule ID: 200004--0x1e2f200 >> Rule ID: 200005--0x1e2fc10 >> Phase: 4 (0 rules) >> Phase: 5 (0 rules) >> Phase: 6 (0 rules) >> Phase: 7 (0 rules) >> Did 9907 >> Done! >> >> We see around 10k processes per second. Essentially all of our time is >> spent waiting on memory allocations: https://s3. >> amazonaws.com/p0pr0ck5-data/modsec-simple.svg >> >> Now consider the case where we include the full 3.0.0 CRS: >> >> $ ./test >> Rules: >> Phase: 0 (32 rules) >> Rule ID: 0--0x22ca500 >> Rule ID: 0--0x22d9480 >> Rule ID: 0--0x22f9530 >> Rule ID: 0--0x22f9f20 >> [...snip several hundred lines...] >> Did 319 >> Done! >> >> So this is about the same throughput we saw from Nginx + libmodsec >> integration. A flamegraph also highlights hot paths, particularly in >> Rule::evaluate (also largely spent on allocation wait time at this >> point, since each evaluation instantiates many new objects): https://s3. >> amazonaws.com/p0pr0ck5-data/modsec-simple-crs.svg >> >> (I also performed the same tests on 3.0.0; flamegraphs are also in this >> s3 bucket but I will avoid commentary here for now as this thread is gotten >> long in the tooth as it is). >> >> A few initial takeaways: >> >> - There is a clearly definable minimum overhead needed to execute >> libmodsecurity, based on its current architecture >> - I noted the same memory leak that defanator noted in #1729. I have yet >> to apply the patches noted. >> - HTTP integrations will induce some overhead, which may or may not be >> substantial. With smart memory pooling and good design, though, this should >> be kept to a minimum. >> - Leveraging the full CRS completely tanks throughput. It would take a >> while more to dig into specifics but I suspect that Rule::evaluate >> probably needs an overhaul. >> - I ran one more test again a mock set of dumb rules (generated via >> https://gist.github.com/p0pr0ck5/c99bca54734af7546d910db8d7c97ab3). Saw >> about 1000 transactions processed per second, indicating (as should be >> assumed) that linear growth in the size of the included ruleset results in >> similar performance reduction. Flamegraphs again point to Rule::evaluate, >> with Rule::getFinalVars taking up half those traces. I suspect there >> could be some smarter behaviors about avoiding object creation if its >> unneeded on these hot paths. >> >> > There are few things in the code that can be improved. There are even > "TODO:" marking. But at certain point you may want to look at the rules. > That is very important. > > Hopefully this is of use to some folks. I'd be interested in writing some >> automated scaffolding to execute this type of testing against every commit, >> to maintain a continuous audit trail of performance impacts; if this sounds >> interesting to the maintainers I'd be happy to chat further. >> > > Andrei does that already. Not automagically, but frequently updated. Some > of the results are posted here: > https://github.com/defanator/modsecurity-performance/wiki > > >> ------------------------------------------------------------ >> ------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______ >> _________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> http://www.modsecurity.org/projects/commercial/support/ >> > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > |
