Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi,

you can do this using exclusive rule, something like this (set the  
rule ID to something 'better'), length is in bytes:

SecRule REQUEST_BODY_LENGTH "@gt 131072" \
     "id:9999999,\
     phase:2,\
     pass,\
     t:none,\
     nolog,\
     ctl:ruleEngine=Off"

REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this  
into phase 1.

Citát Henri Cook <he...@pr...>:

> I think what I need here is a way to exempt the request body from any
> scanning when it's over 128kb (my chosen upper limit)...
>
> I'll probably try implementing this as a phase 1 rule on Content-Length
> unless anyone shouts there's a better way. `ProcessPartial` sounded like
> the holy grail, but you can't partial process JSON (because it doesn't
> parse!).
>
> It'd be cool to be able to fallback to a fulltext scan of the partial JSON
> if it's >128kb, I might have a try at looking into that also (again any
> guidance appreciated)
>
> Best Regards,
>
> Henri
>
> On Mon, 26 Apr 2021 at 15:24, Henri Cook <he...@pr...> wrote:
>
>> The follow up problem to this is:
>>
>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>> PartialProcess`
>>
>> In my mind this will process the first part of a request if it can, and
>> ignore the rest. But:
>>
>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>> Presumably because in a large request it tries to process the beginning of
>> the JSON and can't (because it won't parse, because the JSON is cut off so
>> doesn't end)
>>
>> So I think I need to find a way to skip JSON parsing entirely when the
>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is the
>> limit I want to stick with. I hadn't really considered before this point
>> that 'partial processing' of JSON was likely to be hairy but of course it
>> makes sense.
>>
>>
>>
>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>> chr...@ne...> wrote:
>>
>>> Hey Henri,
>>>
>>> From a security practice, this is obviously lacking, but in wider
>>> perspective,
>>> I see it meet "industry standard", yes.
>>>
>>> When I teach, I tell my student, that the worst WAF is the one that is
>>> switched off. So if you need to compromise and you can only apply 20% of
>>> the rules because you run the risk of business demanding it's switched
>>> off,
>>> then that 20% WAF is still better than no WAF.
>>>
>>> Cheers,
>>>
>>> Christian
>>>
>>>
>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>>> > Thanks Christian, taking this in combination with Osama's point earlier
>>> in
>>> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
>>> > limit the payload they'll scan. From my reading to 128kb
>>> (cloudflare+azure)
>>> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>>> >
>>> > I believe my modern application is already very robust in terms of
>>> defence
>>> > against sql injection as well as other OWASP top 10 attack vectors and
>>> that
>>> > a WAF primarily adds reassurance (for the business and clients who ask
>>> if I
>>> > have one) and minor frustration (for any potential attacker) layer. The
>>> > spec is to add a WAF that meets (but notably does not necessarily have
>>> to
>>> > exceed) industry standards. I believe this means that I can switch
>>> modsec
>>> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>>> > ProcessPartial' - allowing through unscanned any payloads over those
>>> sizes)
>>> > and be able to say I've got scan-size-policy-parity with an AWS or a
>>> > Cloudflare which means it is "industry standard".
>>> >
>>> > Please let me know if you think that's mad and thanks again
>>> >
>>> > Best Regards,
>>> >
>>> > Henri
>>> >
>>> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>>> chr...@ne...>
>>> > wrote:
>>> >
>>> > > Hey Henri,
>>> > >
>>> > > You are in a bad situation and as far as I can see you are right, you
>>> might
>>> > > have to drop modsec/CRS in this situation.
>>> > >
>>> > > I've had a customer with a similar problem and we did a deep dive
>>> > > investigation and I had to strike colors in the end.
>>> > >
>>> > > The point is not the JSON parser. That has shown to be really fast.
>>> The
>>> > > point
>>> > > is several hundred variables that go into CRS afterwards. If you run
>>> CRS
>>> > > on a
>>> > > standard web application you get forms with a few parameters and
>>> that's
>>> > > easy.
>>> > > But several megabytes of JSON means hundreds of arguments and CRS
>>> parses
>>> > > them
>>> > > all.
>>> > >
>>> > > So we tried to work with rule exclusions and skip the parameters we
>>> did not
>>> > > think dangerous, but here comes the bummer: ModSec 2.9 grew
>>> substantially
>>> > > slower the longer the ignore-lists of parameters became. This and a
>>> few
>>> > > very
>>> > > odd behaviors.
>>> > >
>>> > > Given the customer wanted a generic WAF without tuning of individual
>>> APIs
>>> > > we
>>> > > got to a dead end.
>>> > >
>>> > > However, if tuning was an option, then I would probably edit-CRS with
>>> > > msc_pyparser and replace the target lists with arguments I was
>>> interested
>>> > > in.
>>> > >
>>> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
>>> > >
>>> > > As a complementary practice, one could think of performing allowlist
>>> > > checks on
>>> > > some / most of the JSON. Say you have a huge JSON payload with 500
>>> > > parameters.
>>> > > You examine it and discover that 300 of them actually contain simple
>>> digits
>>> > > and asciii characters and neither special chars nor escape sequences.
>>> > > So you do a regex allowlist and apply it to these 300 parameters of
>>> said
>>> > > API. And the rest you can push into CRS. Or a subset of CRS.
>>> > >
>>> > > I have not done this and the problem is if ModSec is able to handle
>>> the
>>> > > large
>>> > > target lists in a speedy manner.
>>> > >
>>> > >
>>> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
>>> > > security
>>> > > tests of such a system. As I said, the JSON parser can be really
>>> fast. The
>>> > > difficult thing is to check several hundred parameters without losing
>>> > > performance.
>>> > >
>>> > > Good luck!
>>> > >
>>> > > Christian
>>> > >
>>> > >
>>> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>>> > > > Hi all,
>>> > > >
>>> > > > I'm in a situation where the only solution seems to be to drop
>>> modsec/CRS
>>> > > > and look at something like Cloudflare's WAF (and change our security
>>> > > model
>>> > > > out of necessity). I'm hoping the esteemed membership of this list
>>> might
>>> > > > have some thoughts.
>>> > > >
>>> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or
>>> even
>>> > > > 30MB ultimately.
>>> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>>> seconds
>>> > > > to process in mod sec 3.0.4 with CRS 3.2.0
>>> > > >
>>> > > > It takes 1 second to process in our API so the WAF element is a 39x
>>> slow
>>> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
>>> > > takes 5
>>> > > > seconds to scan this payload - and that's my target.
>>> > > >
>>> > > > Has anyone got any idea how to improve performance? Reading blog
>>> posts
>>> > > > about the development of cloudflare's waf I see that memoization of
>>> > > common
>>> > > > function calls was one of their absolute best performance
>>> improvements
>>> > > over
>>> > > > their modsec implementation (e.g. strlen(response_body) so it's only
>>> > > > calculated once instead of once per rule OR contains('somestring',
>>> > > > response_body)... you get the drift). Do we have anything like this
>>> in
>>> > > > modsec today? Is that already in place and my 39 seconds is after
>>> that?
>>> > > >
>>> > > > I appreciate that mod sec is fast on its own and adding complex
>>> rules can
>>> > > > be said to slow it down. With CRS being by far the most common use
>>> case
>>> > > for
>>> > > > mod sec (based on my googling) I'm surprised it's this slow, do you
>>> think
>>> > > > i've missed something?
>>> > > >
>>> > > > To note: I'm only scanning JSON payloads, typically much less than
>>> 0.5MB
>>> > > > but new, irregular ones that we need scanned in ideally <10 seconds
>>> that
>>> > > > can range from 1MB-30MB
>>> > > >
>>> > > > Best regards,
>>> > > >
>>> > > > Henri Cook
>>> > >
>>> > >
>>> > > > _______________________________________________
>>> > > > mod-security-users mailing list
>>> > > > mod...@li...
>>> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > > > Commercial ModSecurity Rules and Support from Trustwave's
>>> SpiderLabs:
>>> > > > http://www.modsecurity.org/projects/commercial/rules/
>>> > > > http://www.modsecurity.org/projects/commercial/support/
>>> > >
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > mod-security-users mailing list
>>> > > mod...@li...
>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > > http://www.modsecurity.org/projects/commercial/rules/
>>> > > http://www.modsecurity.org/projects/commercial/support/
>>> > >
>>>
>>>
>>> > _______________________________________________
>>> > mod-security-users mailing list
>>> > mod...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > http://www.modsecurity.org/projects/commercial/rules/
>>> > http://www.modsecurity.org/projects/commercial/support/
>>>
>>>
>>>
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>>>
>>