|
From: Felipe Z. <fe...@zi...> - 2018-04-05 13:33:41
|
Hi, On Wed, Apr 4, 2018 at 2:44 PM Robert Paprocki < rpa...@fe...> wrote: > Hi, > > On Wed, Apr 4, 2018 at 5:03 AM, Christian Folini < > chr...@ne...> wrote: >> >> >> > However, I have limited knowledge on the following: >> > - is ModSec 3.x has been ever targeted to support CRS < 3, >> >> See above. >> > > It would be great to here an official stance from the development team on > this. @Felipe can you comment? > > Feature wise it should support both. ModSecurity was not written to support OWASP CRS, but the SecRule Language. Consequently it supports OWASP CRS. There is a milestone for that here: https://github.com/SpiderLabs/ModSecurity/milestone/11 > >> > Also I'm not sure whether ModSec 2.x has its own benchmarks (not >> related to any connector). >> > If it does, then perhaps it would be good to compare "generic" ModSec >> 2.x >> > vs "generic" ModSec 3.x as well. >> >> Yes, that would be cool. But from what I understand, ModSec 2.9.x is >> deeply >> integrated into the webserver. >> > > > I don't think it is possible to split v2 from Apache. > Yeah, this is a bit of a pain to test. I modified one of the example > programs that comes with the v3/master ModSecurity source code as follows: > > https://gist.github.com/p0pr0ck5/9b2c414641c9b03d527679d0c8cb7d86 > > There is a benchmark utility in the repository: https://github.com/SpiderLabs/ModSecurity/tree/v3/master/test/benchmark > Note that this doesn't add any headers or body data (request or response) > to the transaction, and the included "basic_rules.conf" is unchanged from > what's in the example repo (and ignore the memory leak in not cleaning up > the transaction). So running this very light example: > > $ ./test > Rules: > Phase: 0 (0 rules) > Phase: 1 (0 rules) > Phase: 2 (2 rules) > Rule ID: 200000--0x1e2b6c0 > Rule ID: 200001--0x1e2bc90 > Phase: 3 (4 rules) > Rule ID: 200002--0x1e2c720 > Rule ID: 200003--0x1e2ea00 > Rule ID: 200004--0x1e2f200 > Rule ID: 200005--0x1e2fc10 > Phase: 4 (0 rules) > Phase: 5 (0 rules) > Phase: 6 (0 rules) > Phase: 7 (0 rules) > Did 9907 > Done! > > We see around 10k processes per second. Essentially all of our time is > spent waiting on memory allocations: > https://s3.amazonaws.com/p0pr0ck5-data/modsec-simple.svg > > Now consider the case where we include the full 3.0.0 CRS: > > $ ./test > Rules: > Phase: 0 (32 rules) > Rule ID: 0--0x22ca500 > Rule ID: 0--0x22d9480 > Rule ID: 0--0x22f9530 > Rule ID: 0--0x22f9f20 > [...snip several hundred lines...] > Did 319 > Done! > > So this is about the same throughput we saw from Nginx + libmodsec > integration. A flamegraph also highlights hot paths, particularly in > Rule::evaluate (also largely spent on allocation wait time at this point, > since each evaluation instantiates many new objects): > https://s3.amazonaws.com/p0pr0ck5-data/modsec-simple-crs.svg > > (I also performed the same tests on 3.0.0; flamegraphs are also in this s3 > bucket but I will avoid commentary here for now as this thread is gotten > long in the tooth as it is). > > A few initial takeaways: > > - There is a clearly definable minimum overhead needed to execute > libmodsecurity, based on its current architecture > - I noted the same memory leak that defanator noted in #1729. I have yet > to apply the patches noted. > - HTTP integrations will induce some overhead, which may or may not be > substantial. With smart memory pooling and good design, though, this should > be kept to a minimum. > - Leveraging the full CRS completely tanks throughput. It would take a > while more to dig into specifics but I suspect that Rule::evaluate > probably needs an overhaul. > - I ran one more test again a mock set of dumb rules (generated via > https://gist.github.com/p0pr0ck5/c99bca54734af7546d910db8d7c97ab3). Saw > about 1000 transactions processed per second, indicating (as should be > assumed) that linear growth in the size of the included ruleset results in > similar performance reduction. Flamegraphs again point to Rule::evaluate, > with Rule::getFinalVars taking up half those traces. I suspect there > could be some smarter behaviors about avoiding object creation if its > unneeded on these hot paths. > > There are few things in the code that can be improved. There are even "TODO:" marking. But at certain point you may want to look at the rules. That is very important. Hopefully this is of use to some folks. I'd be interested in writing some > automated scaffolding to execute this type of testing against every commit, > to maintain a continuous audit trail of performance impacts; if this sounds > interesting to the maintainers I'd be happy to chat further. > Andrei does that already. Not automagically, but frequently updated. Some of the results are posted here: https://github.com/defanator/modsecurity-performance/wiki > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > |
