Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

[Christian F. <chr...@ne...>]

On Sun, Apr 25, 2021 at 01:32:56PM -0700, Osama Elnaggar wrote:
> Hi Henri,
> 
> I haven't tested CRS with such large JSON payloads so I can't help you in
> that regard but I just thought I'd mention that the reason that CloudFlare
> is so fast in this use case is this - "The Cloudflare WAF parses JSON
> responses to identify vulnerabilities targeted at APIs. The WAF limits JSON
> payload parsing to 128KB."  -
> https://support.cloudflare.com/hc/en-us/articles/200172016-Understanding-the-Cloudflare-Web-Application-Firewall-WAF-

Which kind of proofs my closing remark.

Ahoj,

Christian

> 
> -- 
> Osama Elnaggar
> 
> On April 26, 2021 at 6:19:35 AM, Henri Cook (henri@proteus.tech) wrote:
> 
> Hi all,
> 
> I'm in a situation where the only solution seems to be to drop modsec/CRS
> and look at something like Cloudflare's WAF (and change our security model
> out of necessity). I'm hoping the esteemed membership of this list might
> have some thoughts.
> 
> I've got about 1MB of JSON, payloads in our app might run to 20 or even
> 30MB ultimately.
> This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> to process in mod sec 3.0.4 with CRS 3.2.0
> 
> It takes 1 second to process in our API so the WAF element is a 39x slow
> down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
> seconds to scan this payload - and that's my target.
> 
> Has anyone got any idea how to improve performance? Reading blog posts
> about the development of cloudflare's waf I see that memoization of common
> function calls was one of their absolute best performance improvements over
> their modsec implementation (e.g. strlen(response_body) so it's only
> calculated once instead of once per rule OR contains('somestring',
> response_body)... you get the drift). Do we have anything like this in
> modsec today? Is that already in place and my 39 seconds is after that?
> 
> I appreciate that mod sec is fast on its own and adding complex rules can
> be said to slow it down. With CRS being by far the most common use case for
> mod sec (based on my googling) I'm surprised it's this slow, do you think
> i've missed something?
> 
> To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> but new, irregular ones that we need scanned in ideally <10 seconds that
> can range from 1MB-30MB
> 
> Best regards,
> 
> Henri Cook
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/