|
From: Christian F. <chr...@ne...> - 2018-04-06 11:07:10
|
Dear all, dear Felipe, On Thu, Apr 05, 2018 at 06:28:06PM +0000, Felipe Zimmerle wrote: > > I cannot foresee how my priorities have to do with this thread :D :) > No, your past and your future priorities matter a big deal. Various people have presented numbers about the performance of ModSec3 and the data does not look good. I continued my measurements and arrived with the following results: Apache, no ModSec2 : 5490 requests per second Apache, ModSec2, 1 rule : 4588 - 15% (against naked webserver) Apache, ModSec2, 10 rules : 4329 - 20% Apache, ModSec2, CRS3 : 1123 - 79% NGINX, no ModSec3 : 10104 requests per second NGINX, ModSec3.0.0, 1 rule : 4619 - 54% (against naked webserver) NGINX, ModSec3.0.0, 10 rules : 3380 - 66% NGINX, ModSec3.0.0, CRS3 : 36 - 99% NGINX, ModSec3.0.2, 1 rule : 4168 - 58% NGINX, ModSec3.0.2, 10 rules : 3251 - 67% NGINX, ModSec3.0.2, CRS3 : 255 - 97% Parameters: 100000 reqs: http://localhost/index.html?test=test 3 test runs, I took the mean of the three runs. Testrule used (1 time or 10 times in succession): SecRule REQUEST_URI "@unconditionalMatch" "id:1,phase:1,pass,nolog,noauditlog" [For those not familiar with ModSec Performance. The numbers look already quite bad on Apache but that's because we are testing locally. Over the net, the numbers for ModSec2 are acceptable.] The takeaway message of the statistic above is this: ModSec3 has a huge overhead and every additional rule makes it worse. This confirms previous messages in this discussion. Namely those of Robert. And all these confirmations contradict your words when you announced 3.0 at https://www.trustwave.com/Resources/SpiderLabs-Blog/ModSecurity-Version-3-0-Announcement/ There you wrote: "This release represents a significant improvement to the ModSecurity WAF. It removes dependencies and improves performance." I knew this was only wishful thinking when ModSec3 came out, but I did not want to badmouth your work Felipe. So I waited for 3.0.1 and hoped for performance improvements. The improvements are significant (what I immediately reported). Unfortunately the performance is still miles away from what I would expect from ModSec3 on NGINX. So on April 3, I asked about your view on future performance improvements and a possible timeline. I think that is a reasonable thing to ask. You did not respond, though, and when Robert pointed out where he sees room for substantial improvements you invited him to refactor central aspects of your code by himself. We agree that it is not your job to do all the work. But please, at least come up with a plan how we can all solve this together. It's the least thing that we expect from a leader. I interpret your communication here as a denial of any substantial problem. You are the lead developer. If you do not acknowledge the problem and if you do not present a plan to the community how this problem will be fixed, then you are hurting ModSecurity. And that's why your and Trustwave's priorities matter a big deal. Best regards, Christian Folini -- Author of the ModSecurity Handbook, 2ed Author of a series of ModSecurity tutorials published at https://netnea.com Renown teacher and frequent speaker on ModSecurity topics |
