Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

[Osama E. <oel...@gm...>]

Hi Henri,

I haven't tested CRS with such large JSON payloads so I can't help you in
that regard but I just thought I'd mention that the reason that CloudFlare
is so fast in this use case is this - "The Cloudflare WAF parses JSON
responses to identify vulnerabilities targeted at APIs. The WAF limits JSON
payload parsing to 128KB."  -
https://support.cloudflare.com/hc/en-us/articles/200172016-Understanding-the-Cloudflare-Web-Application-Firewall-WAF-

-- 
Osama Elnaggar

On April 26, 2021 at 6:19:35 AM, Henri Cook (henri@proteus.tech) wrote:

Hi all,

I'm in a situation where the only solution seems to be to drop modsec/CRS
and look at something like Cloudflare's WAF (and change our security model
out of necessity). I'm hoping the esteemed membership of this list might
have some thoughts.

I've got about 1MB of JSON, payloads in our app might run to 20 or even
30MB ultimately.
This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
to process in mod sec 3.0.4 with CRS 3.2.0

It takes 1 second to process in our API so the WAF element is a 39x slow
down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
seconds to scan this payload - and that's my target.

Has anyone got any idea how to improve performance? Reading blog posts
about the development of cloudflare's waf I see that memoization of common
function calls was one of their absolute best performance improvements over
their modsec implementation (e.g. strlen(response_body) so it's only
calculated once instead of once per rule OR contains('somestring',
response_body)... you get the drift). Do we have anything like this in
modsec today? Is that already in place and my 39 seconds is after that?

I appreciate that mod sec is fast on its own and adding complex rules can
be said to slow it down. With CRS being by far the most common use case for
mod sec (based on my googling) I'm surprised it's this slow, do you think
i've missed something?

To note: I'm only scanning JSON payloads, typically much less than 0.5MB
but new, irregular ones that we need scanned in ideally <10 seconds that
can range from 1MB-30MB

Best regards,

Henri Cook
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/