Thread: [mod-security-users] Performance woes - larger JSON payloads with CRS

Brought to you by: victorhora, zimmerletw

mod-security-users

[mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-04-25 20:15:55

Hi all,

I'm in a situation where the only solution seems to be to drop modsec/CRS
and look at something like Cloudflare's WAF (and change our security model
out of necessity). I'm hoping the esteemed membership of this list might
have some thoughts.

I've got about 1MB of JSON, payloads in our app might run to 20 or even
30MB ultimately.
This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
to process in mod sec 3.0.4 with CRS 3.2.0

It takes 1 second to process in our API so the WAF element is a 39x slow
down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
seconds to scan this payload - and that's my target.

Has anyone got any idea how to improve performance? Reading blog posts
about the development of cloudflare's waf I see that memoization of common
function calls was one of their absolute best performance improvements over
their modsec implementation (e.g. strlen(response_body) so it's only
calculated once instead of once per rule OR contains('somestring',
response_body)... you get the drift). Do we have anything like this in
modsec today? Is that already in place and my 39 seconds is after that?

I appreciate that mod sec is fast on its own and adding complex rules can
be said to slow it down. With CRS being by far the most common use case for
mod sec (based on my googling) I'm surprised it's this slow, do you think
i've missed something?

To note: I'm only scanning JSON payloads, typically much less than 0.5MB
but new, irregular ones that we need scanned in ideally <10 seconds that
can range from 1MB-30MB

Best regards,

Henri Cook

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Osama E. <oel...@gm...> - 2021-04-25 20:33:07

Hi Henri,

I haven't tested CRS with such large JSON payloads so I can't help you in
that regard but I just thought I'd mention that the reason that CloudFlare
is so fast in this use case is this - "The Cloudflare WAF parses JSON
responses to identify vulnerabilities targeted at APIs. The WAF limits JSON
payload parsing to 128KB." -
https://support.cloudflare.com/hc/en-us/articles/200172016-Understanding-the-Cloudflare-Web-Application-Firewall-WAF-

--
Osama Elnaggar

On April 26, 2021 at 6:19:35 AM, Henri Cook (henri@proteus.tech) wrote:

Hi all,

I'm in a situation where the only solution seems to be to drop modsec/CRS
and look at something like Cloudflare's WAF (and change our security model
out of necessity). I'm hoping the esteemed membership of this list might
have some thoughts.

I've got about 1MB of JSON, payloads in our app might run to 20 or even
30MB ultimately.
This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
to process in mod sec 3.0.4 with CRS 3.2.0

It takes 1 second to process in our API so the WAF element is a 39x slow
down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
seconds to scan this payload - and that's my target.

Has anyone got any idea how to improve performance? Reading blog posts
about the development of cloudflare's waf I see that memoization of common
function calls was one of their absolute best performance improvements over
their modsec implementation (e.g. strlen(response_body) so it's only
calculated once instead of once per rule OR contains('somestring',
response_body)... you get the drift). Do we have anything like this in
modsec today? Is that already in place and my 39 seconds is after that?

I appreciate that mod sec is fast on its own and adding complex rules can
be said to slow it down. With CRS being by far the most common use case for
mod sec (based on my googling) I'm surprised it's this slow, do you think
i've missed something?

To note: I'm only scanning JSON payloads, typically much less than 0.5MB
but new, irregular ones that we need scanned in ideally <10 seconds that
can range from 1MB-30MB

Best regards,

Henri Cook
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Christian F. <chr...@ne...> - 2021-04-25 20:38:39

On Sun, Apr 25, 2021 at 01:32:56PM -0700, Osama Elnaggar wrote:
> Hi Henri,
> 
> I haven't tested CRS with such large JSON payloads so I can't help you in
> that regard but I just thought I'd mention that the reason that CloudFlare
> is so fast in this use case is this - "The Cloudflare WAF parses JSON
> responses to identify vulnerabilities targeted at APIs. The WAF limits JSON
> payload parsing to 128KB."  -
> https://support.cloudflare.com/hc/en-us/articles/200172016-Understanding-the-Cloudflare-Web-Application-Firewall-WAF-

Which kind of proofs my closing remark.

Ahoj,

Christian

> 
> -- 
> Osama Elnaggar
> 
> On April 26, 2021 at 6:19:35 AM, Henri Cook (henri@proteus.tech) wrote:
> 
> Hi all,
> 
> I'm in a situation where the only solution seems to be to drop modsec/CRS
> and look at something like Cloudflare's WAF (and change our security model
> out of necessity). I'm hoping the esteemed membership of this list might
> have some thoughts.
> 
> I've got about 1MB of JSON, payloads in our app might run to 20 or even
> 30MB ultimately.
> This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> to process in mod sec 3.0.4 with CRS 3.2.0
> 
> It takes 1 second to process in our API so the WAF element is a 39x slow
> down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
> seconds to scan this payload - and that's my target.
> 
> Has anyone got any idea how to improve performance? Reading blog posts
> about the development of cloudflare's waf I see that memoization of common
> function calls was one of their absolute best performance improvements over
> their modsec implementation (e.g. strlen(response_body) so it's only
> calculated once instead of once per rule OR contains('somestring',
> response_body)... you get the drift). Do we have anything like this in
> modsec today? Is that already in place and my 39 seconds is after that?
> 
> I appreciate that mod sec is fast on its own and adding complex rules can
> be said to slow it down. With CRS being by far the most common use case for
> mod sec (based on my googling) I'm surprised it's this slow, do you think
> i've missed something?
> 
> To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> but new, irregular ones that we need scanned in ideally <10 seconds that
> can range from 1MB-30MB
> 
> Best regards,
> 
> Henri Cook
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-04-26 06:07:48

Thanks for this Osama, that is *very* interesting

I wonder what the rationale is behind only choosing to scan small, or the
first part, of payloads as standard. Prevents DoS with massive payloads I
understand but also... lots of wafs can be circumvented if the payload is
large enough?

Thanks,

Henri

On Sun, 25 Apr 2021 at 21:32, Osama Elnaggar <oel...@gm...> wrote:

> Hi Henri,
>
> I haven't tested CRS with such large JSON payloads so I can't help you in
> that regard but I just thought I'd mention that the reason that CloudFlare
> is so fast in this use case is this - "The Cloudflare WAF parses JSON
> responses to identify vulnerabilities targeted at APIs. The WAF limits JSON
> payload parsing to 128KB."  -
> https://support.cloudflare.com/hc/en-us/articles/200172016-Understanding-the-Cloudflare-Web-Application-Firewall-WAF-
>
> --
> Osama Elnaggar
>
> On April 26, 2021 at 6:19:35 AM, Henri Cook (henri@proteus.tech) wrote:
>
> Hi all,
>
> I'm in a situation where the only solution seems to be to drop modsec/CRS
> and look at something like Cloudflare's WAF (and change our security model
> out of necessity). I'm hoping the esteemed membership of this list might
> have some thoughts.
>
> I've got about 1MB of JSON, payloads in our app might run to 20 or even
> 30MB ultimately.
> This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> to process in mod sec 3.0.4 with CRS 3.2.0
>
> It takes 1 second to process in our API so the WAF element is a 39x slow
> down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
> seconds to scan this payload - and that's my target.
>
> Has anyone got any idea how to improve performance? Reading blog posts
> about the development of cloudflare's waf I see that memoization of common
> function calls was one of their absolute best performance improvements over
> their modsec implementation (e.g. strlen(response_body) so it's only
> calculated once instead of once per rule OR contains('somestring',
> response_body)... you get the drift). Do we have anything like this in
> modsec today? Is that already in place and my 39 seconds is after that?
>
> I appreciate that mod sec is fast on its own and adding complex rules can
> be said to slow it down. With CRS being by far the most common use case for
> mod sec (based on my googling) I'm surprised it's this slow, do you think
> i've missed something?
>
> To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> but new, irregular ones that we need scanned in ideally <10 seconds that
> can range from 1MB-30MB
>
> Best regards,
>
> Henri Cook
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Christian F. <chr...@ne...> - 2021-04-25 20:37:46

Hey Henri,

You are in a bad situation and as far as I can see you are right, you might
have to drop modsec/CRS in this situation.

I've had a customer with a similar problem and we did a deep dive
investigation and I had to strike colors in the end.

The point is not the JSON parser. That has shown to be really fast. The point
is several hundred variables that go into CRS afterwards. If you run CRS on a
standard web application you get forms with a few parameters and that's easy.
But several megabytes of JSON means hundreds of arguments and CRS parses them
all.

So we tried to work with rule exclusions and skip the parameters we did not
think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
slower the longer the ignore-lists of parameters became. This and a few very
odd behaviors.

Given the customer wanted a generic WAF without tuning of individual APIs we
got to a dead end.

However, if tuning was an option, then I would probably edit-CRS with
msc_pyparser and replace the target lists with arguments I was interested in.

https://coreruleset.org/20200901/introducing-msc_pyparser/

As a complementary practice, one could think of performing allowlist checks on
some / most of the JSON. Say you have a huge JSON payload with 500 parameters.
You examine it and discover that 300 of them actually contain simple digits
and asciii characters and neither special chars nor escape sequences.
So you do a regex allowlist and apply it to these 300 parameters of said
API. And the rest you can push into CRS. Or a subset of CRS.

I have not done this and the problem is if ModSec is able to handle the large
target lists in a speedy manner.

Now you can turn to a CDN or alternative WAF. I would do an extensive security
tests of such a system. As I said, the JSON parser can be really fast. The
difficult thing is to check several hundred parameters without losing
performance.

Good luck!

Christian

On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> Hi all,
> 
> I'm in a situation where the only solution seems to be to drop modsec/CRS
> and look at something like Cloudflare's WAF (and change our security model
> out of necessity). I'm hoping the esteemed membership of this list might
> have some thoughts.
> 
> I've got about 1MB of JSON, payloads in our app might run to 20 or even
> 30MB ultimately.
> This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> to process in mod sec 3.0.4 with CRS 3.2.0
> 
> It takes 1 second to process in our API so the WAF element is a 39x slow
> down. I appreciate there'll be some delays in WAF. Cloudflare's WAF takes 5
> seconds to scan this payload - and that's my target.
> 
> Has anyone got any idea how to improve performance? Reading blog posts
> about the development of cloudflare's waf I see that memoization of common
> function calls was one of their absolute best performance improvements over
> their modsec implementation (e.g. strlen(response_body) so it's only
> calculated once instead of once per rule OR contains('somestring',
> response_body)... you get the drift). Do we have anything like this in
> modsec today? Is that already in place and my 39 seconds is after that?
> 
> I appreciate that mod sec is fast on its own and adding complex rules can
> be said to slow it down. With CRS being by far the most common use case for
> mod sec (based on my googling) I'm surprised it's this slow, do you think
> i've missed something?
> 
> To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> but new, irregular ones that we need scanned in ideally <10 seconds that
> can range from 1MB-30MB
> 
> Best regards,
> 
> Henri Cook

> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-04-26 06:05:03

Thanks Christian, taking this in combination with Osama's point earlier in
the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
limit the payload they'll scan. From my reading to 128kb (cloudflare+azure)
or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.

I believe my modern application is already very robust in terms of defence
against sql injection as well as other OWASP top 10 attack vectors and that
a WAF primarily adds reassurance (for the business and clients who ask if I
have one) and minor frustration (for any potential attacker) layer. The
spec is to add a WAF that meets (but notably does not necessarily have to
exceed) industry standards. I believe this means that I can switch modsec
to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
ProcessPartial' - allowing through unscanned any payloads over those sizes)
and be able to say I've got scan-size-policy-parity with an AWS or a
Cloudflare which means it is "industry standard".

Please let me know if you think that's mad and thanks again

Best Regards,

Henri

On Sun, 25 Apr 2021 at 21:39, Christian Folini <chr...@ne...>
wrote:

> Hey Henri,
>
> You are in a bad situation and as far as I can see you are right, you might
> have to drop modsec/CRS in this situation.
>
> I've had a customer with a similar problem and we did a deep dive
> investigation and I had to strike colors in the end.
>
> The point is not the JSON parser. That has shown to be really fast. The
> point
> is several hundred variables that go into CRS afterwards. If you run CRS
> on a
> standard web application you get forms with a few parameters and that's
> easy.
> But several megabytes of JSON means hundreds of arguments and CRS parses
> them
> all.
>
> So we tried to work with rule exclusions and skip the parameters we did not
> think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
> slower the longer the ignore-lists of parameters became. This and a few
> very
> odd behaviors.
>
> Given the customer wanted a generic WAF without tuning of individual APIs
> we
> got to a dead end.
>
> However, if tuning was an option, then I would probably edit-CRS with
> msc_pyparser and replace the target lists with arguments I was interested
> in.
>
> https://coreruleset.org/20200901/introducing-msc_pyparser/
>
> As a complementary practice, one could think of performing allowlist
> checks on
> some / most of the JSON. Say you have a huge JSON payload with 500
> parameters.
> You examine it and discover that 300 of them actually contain simple digits
> and asciii characters and neither special chars nor escape sequences.
> So you do a regex allowlist and apply it to these 300 parameters of said
> API. And the rest you can push into CRS. Or a subset of CRS.
>
> I have not done this and the problem is if ModSec is able to handle the
> large
> target lists in a speedy manner.
>
>
> Now you can turn to a CDN or alternative WAF. I would do an extensive
> security
> tests of such a system. As I said, the JSON parser can be really fast. The
> difficult thing is to check several hundred parameters without losing
> performance.
>
> Good luck!
>
> Christian
>
>
> On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > Hi all,
> >
> > I'm in a situation where the only solution seems to be to drop modsec/CRS
> > and look at something like Cloudflare's WAF (and change our security
> model
> > out of necessity). I'm hoping the esteemed membership of this list might
> > have some thoughts.
> >
> > I've got about 1MB of JSON, payloads in our app might run to 20 or even
> > 30MB ultimately.
> > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> > to process in mod sec 3.0.4 with CRS 3.2.0
> >
> > It takes 1 second to process in our API so the WAF element is a 39x slow
> > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> takes 5
> > seconds to scan this payload - and that's my target.
> >
> > Has anyone got any idea how to improve performance? Reading blog posts
> > about the development of cloudflare's waf I see that memoization of
> common
> > function calls was one of their absolute best performance improvements
> over
> > their modsec implementation (e.g. strlen(response_body) so it's only
> > calculated once instead of once per rule OR contains('somestring',
> > response_body)... you get the drift). Do we have anything like this in
> > modsec today? Is that already in place and my 39 seconds is after that?
> >
> > I appreciate that mod sec is fast on its own and adding complex rules can
> > be said to slow it down. With CRS being by far the most common use case
> for
> > mod sec (based on my googling) I'm surprised it's this slow, do you think
> > i've missed something?
> >
> > To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> > but new, irregular ones that we need scanned in ideally <10 seconds that
> > can range from 1MB-30MB
> >
> > Best regards,
> >
> > Henri Cook
>
>
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Christian F. <chr...@ne...> - 2021-04-26 06:16:05

Hey Henri,

>From a security practice, this is obviously lacking, but in wider perspective,
I see it meet "industry standard", yes.

When I teach, I tell my student, that the worst WAF is the one that is
switched off. So if you need to compromise and you can only apply 20% of
the rules because you run the risk of business demanding it's switched off,
then that 20% WAF is still better than no WAF.

Cheers,

Christian


On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> Thanks Christian, taking this in combination with Osama's point earlier in
> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> limit the payload they'll scan. From my reading to 128kb (cloudflare+azure)
> or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
> 
> I believe my modern application is already very robust in terms of defence
> against sql injection as well as other OWASP top 10 attack vectors and that
> a WAF primarily adds reassurance (for the business and clients who ask if I
> have one) and minor frustration (for any potential attacker) layer. The
> spec is to add a WAF that meets (but notably does not necessarily have to
> exceed) industry standards. I believe this means that I can switch modsec
> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> ProcessPartial' - allowing through unscanned any payloads over those sizes)
> and be able to say I've got scan-size-policy-parity with an AWS or a
> Cloudflare which means it is "industry standard".
> 
> Please let me know if you think that's mad and thanks again
> 
> Best Regards,
> 
> Henri
> 
> On Sun, 25 Apr 2021 at 21:39, Christian Folini <chr...@ne...>
> wrote:
> 
> > Hey Henri,
> >
> > You are in a bad situation and as far as I can see you are right, you might
> > have to drop modsec/CRS in this situation.
> >
> > I've had a customer with a similar problem and we did a deep dive
> > investigation and I had to strike colors in the end.
> >
> > The point is not the JSON parser. That has shown to be really fast. The
> > point
> > is several hundred variables that go into CRS afterwards. If you run CRS
> > on a
> > standard web application you get forms with a few parameters and that's
> > easy.
> > But several megabytes of JSON means hundreds of arguments and CRS parses
> > them
> > all.
> >
> > So we tried to work with rule exclusions and skip the parameters we did not
> > think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
> > slower the longer the ignore-lists of parameters became. This and a few
> > very
> > odd behaviors.
> >
> > Given the customer wanted a generic WAF without tuning of individual APIs
> > we
> > got to a dead end.
> >
> > However, if tuning was an option, then I would probably edit-CRS with
> > msc_pyparser and replace the target lists with arguments I was interested
> > in.
> >
> > https://coreruleset.org/20200901/introducing-msc_pyparser/
> >
> > As a complementary practice, one could think of performing allowlist
> > checks on
> > some / most of the JSON. Say you have a huge JSON payload with 500
> > parameters.
> > You examine it and discover that 300 of them actually contain simple digits
> > and asciii characters and neither special chars nor escape sequences.
> > So you do a regex allowlist and apply it to these 300 parameters of said
> > API. And the rest you can push into CRS. Or a subset of CRS.
> >
> > I have not done this and the problem is if ModSec is able to handle the
> > large
> > target lists in a speedy manner.
> >
> >
> > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > security
> > tests of such a system. As I said, the JSON parser can be really fast. The
> > difficult thing is to check several hundred parameters without losing
> > performance.
> >
> > Good luck!
> >
> > Christian
> >
> >
> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > Hi all,
> > >
> > > I'm in a situation where the only solution seems to be to drop modsec/CRS
> > > and look at something like Cloudflare's WAF (and change our security
> > model
> > > out of necessity). I'm hoping the esteemed membership of this list might
> > > have some thoughts.
> > >
> > > I've got about 1MB of JSON, payloads in our app might run to 20 or even
> > > 30MB ultimately.
> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> > > to process in mod sec 3.0.4 with CRS 3.2.0
> > >
> > > It takes 1 second to process in our API so the WAF element is a 39x slow
> > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > takes 5
> > > seconds to scan this payload - and that's my target.
> > >
> > > Has anyone got any idea how to improve performance? Reading blog posts
> > > about the development of cloudflare's waf I see that memoization of
> > common
> > > function calls was one of their absolute best performance improvements
> > over
> > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > calculated once instead of once per rule OR contains('somestring',
> > > response_body)... you get the drift). Do we have anything like this in
> > > modsec today? Is that already in place and my 39 seconds is after that?
> > >
> > > I appreciate that mod sec is fast on its own and adding complex rules can
> > > be said to slow it down. With CRS being by far the most common use case
> > for
> > > mod sec (based on my googling) I'm surprised it's this slow, do you think
> > > i've missed something?
> > >
> > > To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> > > but new, irregular ones that we need scanned in ideally <10 seconds that
> > > can range from 1MB-30MB
> > >
> > > Best regards,
> > >
> > > Henri Cook
> >
> >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/
> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
> >


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Michael W. <sco...@ya...> - 2021-04-26 09:25:37

I've had a similar experience with mod_security and CRS. Some of our JSON payloads can be 4MB. Given that the regular expressions executed by mod_security are Perl based, I extracted the regular expressions from the CRS and benchmarked them individually with varying sizes of JSON documents.

All CRS rules completed within 0.03s except for one rule. Rule 941160 when executed against large payloads can take multiple seconds to complete. A 1M payload takes around 2700 seconds to complete, without PCRE JIT enabled. Running PCRE with JIT compilation does have an impact but it still takes around 385 seconds to complete.
Rule 941160 belongs to a collection of rules concerning XSS attacks.
Doing some more digging I discovered Perl based regular expression engines such as Perl and PCRE have exponential response times when the data grows in size. I also was doing some work with Google's Go language and their regular expression engine re provides a linear response time. Luckily on Perl's CPAN repository, there is a module that provides an interface with re. Running the same benchmarks again, rule 941160 took less than a second. I had to run it again and check the results because I didn't believe it.
Regards

    On Monday, 26 April 2021, 07:16:18 BST, Christian Folini <chr...@ne...> wrote:  
 
 Hey Henri,

>From a security practice, this is obviously lacking, but in wider perspective,
I see it meet "industry standard", yes.

When I teach, I tell my student, that the worst WAF is the one that is
switched off. So if you need to compromise and you can only apply 20% of
the rules because you run the risk of business demanding it's switched off,
then that 20% WAF is still better than no WAF.

Cheers,

Christian


On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> Thanks Christian, taking this in combination with Osama's point earlier in
> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> limit the payload they'll scan. From my reading to 128kb (cloudflare+azure)
> or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
> 
> I believe my modern application is already very robust in terms of defence
> against sql injection as well as other OWASP top 10 attack vectors and that
> a WAF primarily adds reassurance (for the business and clients who ask if I
> have one) and minor frustration (for any potential attacker) layer. The
> spec is to add a WAF that meets (but notably does not necessarily have to
> exceed) industry standards. I believe this means that I can switch modsec
> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> ProcessPartial' - allowing through unscanned any payloads over those sizes)
> and be able to say I've got scan-size-policy-parity with an AWS or a
> Cloudflare which means it is "industry standard".
> 
> Please let me know if you think that's mad and thanks again
> 
> Best Regards,
> 
> Henri
> 
> On Sun, 25 Apr 2021 at 21:39, Christian Folini <chr...@ne...>
> wrote:
> 
> > Hey Henri,
> >
> > You are in a bad situation and as far as I can see you are right, you might
> > have to drop modsec/CRS in this situation.
> >
> > I've had a customer with a similar problem and we did a deep dive
> > investigation and I had to strike colors in the end.
> >
> > The point is not the JSON parser. That has shown to be really fast. The
> > point
> > is several hundred variables that go into CRS afterwards. If you run CRS
> > on a
> > standard web application you get forms with a few parameters and that's
> > easy.
> > But several megabytes of JSON means hundreds of arguments and CRS parses
> > them
> > all.
> >
> > So we tried to work with rule exclusions and skip the parameters we did not
> > think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
> > slower the longer the ignore-lists of parameters became. This and a few
> > very
> > odd behaviors.
> >
> > Given the customer wanted a generic WAF without tuning of individual APIs
> > we
> > got to a dead end.
> >
> > However, if tuning was an option, then I would probably edit-CRS with
> > msc_pyparser and replace the target lists with arguments I was interested
> > in.
> >
> > https://coreruleset.org/20200901/introducing-msc_pyparser/
> >
> > As a complementary practice, one could think of performing allowlist
> > checks on
> > some / most of the JSON. Say you have a huge JSON payload with 500
> > parameters.
> > You examine it and discover that 300 of them actually contain simple digits
> > and asciii characters and neither special chars nor escape sequences.
> > So you do a regex allowlist and apply it to these 300 parameters of said
> > API. And the rest you can push into CRS. Or a subset of CRS.
> >
> > I have not done this and the problem is if ModSec is able to handle the
> > large
> > target lists in a speedy manner.
> >
> >
> > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > security
> > tests of such a system. As I said, the JSON parser can be really fast. The
> > difficult thing is to check several hundred parameters without losing
> > performance.
> >
> > Good luck!
> >
> > Christian
> >
> >
> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > Hi all,
> > >
> > > I'm in a situation where the only solution seems to be to drop modsec/CRS
> > > and look at something like Cloudflare's WAF (and change our security
> > model
> > > out of necessity). I'm hoping the esteemed membership of this list might
> > > have some thoughts.
> > >
> > > I've got about 1MB of JSON, payloads in our app might run to 20 or even
> > > 30MB ultimately.
> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> > > to process in mod sec 3.0.4 with CRS 3.2.0
> > >
> > > It takes 1 second to process in our API so the WAF element is a 39x slow
> > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > takes 5
> > > seconds to scan this payload - and that's my target.
> > >
> > > Has anyone got any idea how to improve performance? Reading blog posts
> > > about the development of cloudflare's waf I see that memoization of
> > common
> > > function calls was one of their absolute best performance improvements
> > over
> > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > calculated once instead of once per rule OR contains('somestring',
> > > response_body)... you get the drift). Do we have anything like this in
> > > modsec today? Is that already in place and my 39 seconds is after that?
> > >
> > > I appreciate that mod sec is fast on its own and adding complex rules can
> > > be said to slow it down. With CRS being by far the most common use case
> > for
> > > mod sec (based on my googling) I'm surprised it's this slow, do you think
> > > i've missed something?
> > >
> > > To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> > > but new, irregular ones that we need scanned in ideally <10 seconds that
> > > can range from 1MB-30MB
> > >
> > > Best regards,
> > >
> > > Henri Cook
> >
> >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/
> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
> >


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/



_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-04-26 09:30:51

That explains https://github.com/SpiderLabs/ModSecurity/issues/1996 !

I'll keep an eye on it

On Mon, 26 Apr 2021 at 10:28, Michael Woods via mod-security-users <
mod...@li...> wrote:

> I've had a similar experience with mod_security and CRS. Some of our JSON
> payloads can be 4MB. Given that the regular expressions executed by
> mod_security are Perl based, I extracted the regular expressions from the
> CRS and benchmarked them individually with varying sizes of JSON documents.
>
> All CRS rules completed within 0.03s except for one rule. Rule 941160 when
> executed against large payloads can take multiple seconds to complete. A 1M
> payload takes around 2700 seconds to complete, without PCRE JIT enabled.
> Running PCRE with JIT compilation does have an impact but it still takes
> around 385 seconds to complete.
>
> Rule 941160 belongs to a collection of rules concerning XSS attacks.
>
> Doing some more digging I discovered Perl based regular expression engines
> such as Perl and PCRE have exponential response times when the data grows
> in size. I also was doing some work with Google's Go language and their
> regular expression engine re provides a linear response time. Luckily on
> Perl's CPAN repository, there is a module that provides an interface with
> re. Running the same benchmarks again, rule 941160 took less than a second.
> I had to run it again and check the results because I didn't believe it.
>
> Regards
>
>
> On Monday, 26 April 2021, 07:16:18 BST, Christian Folini <
> chr...@ne...> wrote:
>
>
> Hey Henri,
>
> From a security practice, this is obviously lacking, but in wider
> perspective,
> I see it meet "industry standard", yes.
>
> When I teach, I tell my student, that the worst WAF is the one that is
> switched off. So if you need to compromise and you can only apply 20% of
> the rules because you run the risk of business demanding it's switched off,
> then that 20% WAF is still better than no WAF.
>
> Cheers,
>
> Christian
>
>
> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> > Thanks Christian, taking this in combination with Osama's point earlier
> in
> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> > limit the payload they'll scan. From my reading to 128kb
> (cloudflare+azure)
> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
> >
> > I believe my modern application is already very robust in terms of
> defence
> > against sql injection as well as other OWASP top 10 attack vectors and
> that
> > a WAF primarily adds reassurance (for the business and clients who ask
> if I
> > have one) and minor frustration (for any potential attacker) layer. The
> > spec is to add a WAF that meets (but notably does not necessarily have to
> > exceed) industry standards. I believe this means that I can switch modsec
> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> > ProcessPartial' - allowing through unscanned any payloads over those
> sizes)
> > and be able to say I've got scan-size-policy-parity with an AWS or a
> > Cloudflare which means it is "industry standard".
> >
> > Please let me know if you think that's mad and thanks again
> >
> > Best Regards,
> >
> > Henri
> >
> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <
> chr...@ne...>
> > wrote:
> >
> > > Hey Henri,
> > >
> > > You are in a bad situation and as far as I can see you are right, you
> might
> > > have to drop modsec/CRS in this situation.
> > >
> > > I've had a customer with a similar problem and we did a deep dive
> > > investigation and I had to strike colors in the end.
> > >
> > > The point is not the JSON parser. That has shown to be really fast. The
> > > point
> > > is several hundred variables that go into CRS afterwards. If you run
> CRS
> > > on a
> > > standard web application you get forms with a few parameters and that's
> > > easy.
> > > But several megabytes of JSON means hundreds of arguments and CRS
> parses
> > > them
> > > all.
> > >
> > > So we tried to work with rule exclusions and skip the parameters we
> did not
> > > think dangerous, but here comes the bummer: ModSec 2.9 grew
> substantially
> > > slower the longer the ignore-lists of parameters became. This and a few
> > > very
> > > odd behaviors.
> > >
> > > Given the customer wanted a generic WAF without tuning of individual
> APIs
> > > we
> > > got to a dead end.
> > >
> > > However, if tuning was an option, then I would probably edit-CRS with
> > > msc_pyparser and replace the target lists with arguments I was
> interested
> > > in.
> > >
> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
> > >
> > > As a complementary practice, one could think of performing allowlist
> > > checks on
> > > some / most of the JSON. Say you have a huge JSON payload with 500
> > > parameters.
> > > You examine it and discover that 300 of them actually contain simple
> digits
> > > and asciii characters and neither special chars nor escape sequences.
> > > So you do a regex allowlist and apply it to these 300 parameters of
> said
> > > API. And the rest you can push into CRS. Or a subset of CRS.
> > >
> > > I have not done this and the problem is if ModSec is able to handle the
> > > large
> > > target lists in a speedy manner.
> > >
> > >
> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > > security
> > > tests of such a system. As I said, the JSON parser can be really fast.
> The
> > > difficult thing is to check several hundred parameters without losing
> > > performance.
> > >
> > > Good luck!
> > >
> > > Christian
> > >
> > >
> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > > Hi all,
> > > >
> > > > I'm in a situation where the only solution seems to be to drop
> modsec/CRS
> > > > and look at something like Cloudflare's WAF (and change our security
> > > model
> > > > out of necessity). I'm hoping the esteemed membership of this list
> might
> > > > have some thoughts.
> > > >
> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or
> even
> > > > 30MB ultimately.
> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
> seconds
> > > > to process in mod sec 3.0.4 with CRS 3.2.0
> > > >
> > > > It takes 1 second to process in our API so the WAF element is a 39x
> slow
> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > > takes 5
> > > > seconds to scan this payload - and that's my target.
> > > >
> > > > Has anyone got any idea how to improve performance? Reading blog
> posts
> > > > about the development of cloudflare's waf I see that memoization of
> > > common
> > > > function calls was one of their absolute best performance
> improvements
> > > over
> > > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > > calculated once instead of once per rule OR contains('somestring',
> > > > response_body)... you get the drift). Do we have anything like this
> in
> > > > modsec today? Is that already in place and my 39 seconds is after
> that?
> > > >
> > > > I appreciate that mod sec is fast on its own and adding complex
> rules can
> > > > be said to slow it down. With CRS being by far the most common use
> case
> > > for
> > > > mod sec (based on my googling) I'm surprised it's this slow, do you
> think
> > > > i've missed something?
> > > >
> > > > To note: I'm only scanning JSON payloads, typically much less than
> 0.5MB
> > > > but new, irregular ones that we need scanned in ideally <10 seconds
> that
> > > > can range from 1MB-30MB
> > > >
> > > > Best regards,
> > > >
> > > > Henri Cook
> > >
> > >
> > > > _______________________________________________
> > > > mod-security-users mailing list
> > > > mod...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > > http://www.modsecurity.org/projects/commercial/rules/
> > > > http://www.modsecurity.org/projects/commercial/support/
>
> > >
> > >
> > >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/
> > >
>
>
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Christian F. <chr...@ne...> - 2021-04-26 09:33:33

Hey Michael,

You are correct. PCRE can be awfully slow. Alternative regex engines have less
functionality, but linear performance. That's why we are working hard to
drop the pcre specific functionality from CRS to make the rule compatible.

Meanwhile, the ModSec developers are also attempting to bring other engines
to ModSec, at least to ModSec3 AFAIK.

When doing this kind of research, it my be worth to check out msc_retest by
CRS dev Ervin Hegedüs / @airween:
https://coreruleset.org/20210106/introducing-msc_retest/

Best,

Christian


On Mon, Apr 26, 2021 at 09:25:19AM +0000, Michael Woods via mod-security-users wrote:
> I've had a similar experience with mod_security and CRS. Some of our JSON payloads can be 4MB. Given that the regular expressions executed by mod_security are Perl based, I extracted the regular expressions from the CRS and benchmarked them individually with varying sizes of JSON documents.
> 
> All CRS rules completed within 0.03s except for one rule. Rule 941160 when executed against large payloads can take multiple seconds to complete. A 1M payload takes around 2700 seconds to complete, without PCRE JIT enabled. Running PCRE with JIT compilation does have an impact but it still takes around 385 seconds to complete.
> Rule 941160 belongs to a collection of rules concerning XSS attacks.
> Doing some more digging I discovered Perl based regular expression engines such as Perl and PCRE have exponential response times when the data grows in size. I also was doing some work with Google's Go language and their regular expression engine re provides a linear response time. Luckily on Perl's CPAN repository, there is a module that provides an interface with re. Running the same benchmarks again, rule 941160 took less than a second. I had to run it again and check the results because I didn't believe it.
> Regards
> 
>     On Monday, 26 April 2021, 07:16:18 BST, Christian Folini <chr...@ne...> wrote:  
>  
>  Hey Henri,
> 
> >From a security practice, this is obviously lacking, but in wider perspective,
> I see it meet "industry standard", yes.
> 
> When I teach, I tell my student, that the worst WAF is the one that is
> switched off. So if you need to compromise and you can only apply 20% of
> the rules because you run the risk of business demanding it's switched off,
> then that 20% WAF is still better than no WAF.
> 
> Cheers,
> 
> Christian
> 
> 
> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> > Thanks Christian, taking this in combination with Osama's point earlier in
> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> > limit the payload they'll scan. From my reading to 128kb (cloudflare+azure)
> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
> > 
> > I believe my modern application is already very robust in terms of defence
> > against sql injection as well as other OWASP top 10 attack vectors and that
> > a WAF primarily adds reassurance (for the business and clients who ask if I
> > have one) and minor frustration (for any potential attacker) layer. The
> > spec is to add a WAF that meets (but notably does not necessarily have to
> > exceed) industry standards. I believe this means that I can switch modsec
> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> > ProcessPartial' - allowing through unscanned any payloads over those sizes)
> > and be able to say I've got scan-size-policy-parity with an AWS or a
> > Cloudflare which means it is "industry standard".
> > 
> > Please let me know if you think that's mad and thanks again
> > 
> > Best Regards,
> > 
> > Henri
> > 
> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <chr...@ne...>
> > wrote:
> > 
> > > Hey Henri,
> > >
> > > You are in a bad situation and as far as I can see you are right, you might
> > > have to drop modsec/CRS in this situation.
> > >
> > > I've had a customer with a similar problem and we did a deep dive
> > > investigation and I had to strike colors in the end.
> > >
> > > The point is not the JSON parser. That has shown to be really fast. The
> > > point
> > > is several hundred variables that go into CRS afterwards. If you run CRS
> > > on a
> > > standard web application you get forms with a few parameters and that's
> > > easy.
> > > But several megabytes of JSON means hundreds of arguments and CRS parses
> > > them
> > > all.
> > >
> > > So we tried to work with rule exclusions and skip the parameters we did not
> > > think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
> > > slower the longer the ignore-lists of parameters became. This and a few
> > > very
> > > odd behaviors.
> > >
> > > Given the customer wanted a generic WAF without tuning of individual APIs
> > > we
> > > got to a dead end.
> > >
> > > However, if tuning was an option, then I would probably edit-CRS with
> > > msc_pyparser and replace the target lists with arguments I was interested
> > > in.
> > >
> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
> > >
> > > As a complementary practice, one could think of performing allowlist
> > > checks on
> > > some / most of the JSON. Say you have a huge JSON payload with 500
> > > parameters.
> > > You examine it and discover that 300 of them actually contain simple digits
> > > and asciii characters and neither special chars nor escape sequences.
> > > So you do a regex allowlist and apply it to these 300 parameters of said
> > > API. And the rest you can push into CRS. Or a subset of CRS.
> > >
> > > I have not done this and the problem is if ModSec is able to handle the
> > > large
> > > target lists in a speedy manner.
> > >
> > >
> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > > security
> > > tests of such a system. As I said, the JSON parser can be really fast. The
> > > difficult thing is to check several hundred parameters without losing
> > > performance.
> > >
> > > Good luck!
> > >
> > > Christian
> > >
> > >
> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > > Hi all,
> > > >
> > > > I'm in a situation where the only solution seems to be to drop modsec/CRS
> > > > and look at something like Cloudflare's WAF (and change our security
> > > model
> > > > out of necessity). I'm hoping the esteemed membership of this list might
> > > > have some thoughts.
> > > >
> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or even
> > > > 30MB ultimately.
> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> > > > to process in mod sec 3.0.4 with CRS 3.2.0
> > > >
> > > > It takes 1 second to process in our API so the WAF element is a 39x slow
> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > > takes 5
> > > > seconds to scan this payload - and that's my target.
> > > >
> > > > Has anyone got any idea how to improve performance? Reading blog posts
> > > > about the development of cloudflare's waf I see that memoization of
> > > common
> > > > function calls was one of their absolute best performance improvements
> > > over
> > > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > > calculated once instead of once per rule OR contains('somestring',
> > > > response_body)... you get the drift). Do we have anything like this in
> > > > modsec today? Is that already in place and my 39 seconds is after that?
> > > >
> > > > I appreciate that mod sec is fast on its own and adding complex rules can
> > > > be said to slow it down. With CRS being by far the most common use case
> > > for
> > > > mod sec (based on my googling) I'm surprised it's this slow, do you think
> > > > i've missed something?
> > > >
> > > > To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> > > > but new, irregular ones that we need scanned in ideally <10 seconds that
> > > > can range from 1MB-30MB
> > > >
> > > > Best regards,
> > > >
> > > > Henri Cook
> > >
> > >
> > > > _______________________________________________
> > > > mod-security-users mailing list
> > > > mod...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > > http://www.modsecurity.org/projects/commercial/rules/
> > > > http://www.modsecurity.org/projects/commercial/support/
> > >
> > >
> > >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/
> > >
> 
> 
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
> 
> 
> 
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>   


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Osama E. <oel...@gm...> - 2021-04-26 09:45:03

Wow. That's bad. As you currently don't have an option of choosing your
regex engine in ModSecurity, one option you might want to investigate is
offloading this particular rule's processing to a Lua script. There is a
Lua binding for RE2 (https://github.com/cloudflare/lua-re2) which you can
use but I haven't tested it

-- 
Osama Elnaggar

On April 26, 2021 at 7:29:02 PM, Michael Woods via mod-security-users (
mod...@li...) wrote:

I've had a similar experience with mod_security and CRS. Some of our JSON
payloads can be 4MB. Given that the regular expressions executed by
mod_security are Perl based, I extracted the regular expressions from the
CRS and benchmarked them individually with varying sizes of JSON documents.

All CRS rules completed within 0.03s except for one rule. Rule 941160 when
executed against large payloads can take multiple seconds to complete. A 1M
payload takes around 2700 seconds to complete, without PCRE JIT enabled.
Running PCRE with JIT compilation does have an impact but it still takes
around 385 seconds to complete.

Rule 941160 belongs to a collection of rules concerning XSS attacks.

Doing some more digging I discovered Perl based regular expression engines
such as Perl and PCRE have exponential response times when the data grows
in size. I also was doing some work with Google's Go language and their
regular expression engine re provides a linear response time. Luckily on
Perl's CPAN repository, there is a module that provides an interface with
re. Running the same benchmarks again, rule 941160 took less than a second.
I had to run it again and check the results because I didn't believe it.

Regards


On Monday, 26 April 2021, 07:16:18 BST, Christian Folini <
chr...@ne...> wrote:


Hey Henri,

>From a security practice, this is obviously lacking, but in wider
perspective,
I see it meet "industry standard", yes.

When I teach, I tell my student, that the worst WAF is the one that is
switched off. So if you need to compromise and you can only apply 20% of
the rules because you run the risk of business demanding it's switched off,
then that 20% WAF is still better than no WAF.

Cheers,

Christian


On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> Thanks Christian, taking this in combination with Osama's point earlier in
> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> limit the payload they'll scan. From my reading to 128kb
(cloudflare+azure)
> or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>
> I believe my modern application is already very robust in terms of defence
> against sql injection as well as other OWASP top 10 attack vectors and
that
> a WAF primarily adds reassurance (for the business and clients who ask if
I
> have one) and minor frustration (for any potential attacker) layer. The
> spec is to add a WAF that meets (but notably does not necessarily have to
> exceed) industry standards. I believe this means that I can switch modsec
> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> ProcessPartial' - allowing through unscanned any payloads over those
sizes)
> and be able to say I've got scan-size-policy-parity with an AWS or a
> Cloudflare which means it is "industry standard".
>
> Please let me know if you think that's mad and thanks again
>
> Best Regards,
>
> Henri
>
> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
chr...@ne...>
> wrote:
>
> > Hey Henri,
> >
> > You are in a bad situation and as far as I can see you are right, you
might
> > have to drop modsec/CRS in this situation.
> >
> > I've had a customer with a similar problem and we did a deep dive
> > investigation and I had to strike colors in the end.
> >
> > The point is not the JSON parser. That has shown to be really fast. The
> > point
> > is several hundred variables that go into CRS afterwards. If you run CRS
> > on a
> > standard web application you get forms with a few parameters and that's
> > easy.
> > But several megabytes of JSON means hundreds of arguments and CRS parses
> > them
> > all.
> >
> > So we tried to work with rule exclusions and skip the parameters we did
not
> > think dangerous, but here comes the bummer: ModSec 2.9 grew
substantially
> > slower the longer the ignore-lists of parameters became. This and a few
> > very
> > odd behaviors.
> >
> > Given the customer wanted a generic WAF without tuning of individual
APIs
> > we
> > got to a dead end.
> >
> > However, if tuning was an option, then I would probably edit-CRS with
> > msc_pyparser and replace the target lists with arguments I was
interested
> > in.
> >
> > https://coreruleset.org/20200901/introducing-msc_pyparser/
> >
> > As a complementary practice, one could think of performing allowlist
> > checks on
> > some / most of the JSON. Say you have a huge JSON payload with 500
> > parameters.
> > You examine it and discover that 300 of them actually contain simple
digits
> > and asciii characters and neither special chars nor escape sequences.
> > So you do a regex allowlist and apply it to these 300 parameters of said
> > API. And the rest you can push into CRS. Or a subset of CRS.
> >
> > I have not done this and the problem is if ModSec is able to handle the
> > large
> > target lists in a speedy manner.
> >
> >
> > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > security
> > tests of such a system. As I said, the JSON parser can be really fast.
The
> > difficult thing is to check several hundred parameters without losing
> > performance.
> >
> > Good luck!
> >
> > Christian
> >
> >
> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > Hi all,
> > >
> > > I'm in a situation where the only solution seems to be to drop
modsec/CRS
> > > and look at something like Cloudflare's WAF (and change our security
> > model
> > > out of necessity). I'm hoping the esteemed membership of this list
might
> > > have some thoughts.
> > >
> > > I've got about 1MB of JSON, payloads in our app might run to 20 or
even
> > > 30MB ultimately.
> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
seconds
> > > to process in mod sec 3.0.4 with CRS 3.2.0
> > >
> > > It takes 1 second to process in our API so the WAF element is a 39x
slow
> > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > takes 5
> > > seconds to scan this payload - and that's my target.
> > >
> > > Has anyone got any idea how to improve performance? Reading blog posts
> > > about the development of cloudflare's waf I see that memoization of
> > common
> > > function calls was one of their absolute best performance improvements
> > over
> > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > calculated once instead of once per rule OR contains('somestring',
> > > response_body)... you get the drift). Do we have anything like this in
> > > modsec today? Is that already in place and my 39 seconds is after
that?
> > >
> > > I appreciate that mod sec is fast on its own and adding complex rules
can
> > > be said to slow it down. With CRS being by far the most common use
case
> > for
> > > mod sec (based on my googling) I'm surprised it's this slow, do you
think
> > > i've missed something?
> > >
> > > To note: I'm only scanning JSON payloads, typically much less than
0.5MB
> > > but new, irregular ones that we need scanned in ideally <10 seconds
that
> > > can range from 1MB-30MB
> > >
> > > Best regards,
> > >
> > > Henri Cook
> >
> >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/

> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
> >


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/



_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Michael W. <sco...@ya...> - 2021-04-26 10:57:13

We don't have a WAF so disabling this one rule is tolerated to gain the benefit of all the other rules provided by the CRS.
A lot of testing I expect needs to be done to introduce a new regular expression engine.
Tel:       01257 266394 Mobile: 07944032617 Email:  sco...@ya... 

    On Monday, 26 April 2021, 10:45:53 BST, Osama Elnaggar <oel...@gm...> wrote:  
 
 Wow. That's bad. As you currently don't have an option of choosing your regex engine in ModSecurity, one option you might want to investigate is offloading this particular rule's processing to a Lua script. There is a Lua binding for RE2 (https://github.com/cloudflare/lua-re2) which you can use but I haven't tested it 
 -- 
Osama Elnaggar
 

On April 26, 2021 at 7:29:02 PM, Michael Woods via mod-security-users (mod...@li...) wrote:
 
I've had a similar experience with mod_security and CRS. Some of our JSON payloads can be 4MB. Given that the regular expressions executed by mod_security are Perl based, I extracted the regular expressions from the CRS and benchmarked them individually with varying sizes of JSON documents.

All CRS rules completed within 0.03s except for one rule. Rule 941160 when executed against large payloads can take multiple seconds to complete. A 1M payload takes around 2700 seconds to complete, without PCRE JIT enabled. Running PCRE with JIT compilation does have an impact but it still takes around 385 seconds to complete.
Rule 941160 belongs to a collection of rules concerning XSS attacks.
Doing some more digging I discovered Perl based regular expression engines such as Perl and PCRE have exponential response times when the data grows in size. I also was doing some work with Google's Go language and their regular expression engine re provides a linear response time. Luckily on Perl's CPAN repository, there is a module that provides an interface with re. Running the same benchmarks again, rule 941160 took less than a second. I had to run it again and check the results because I didn't believe it.
Regards

    On Monday, 26 April 2021, 07:16:18 BST, Christian Folini <chr...@ne...> wrote:  
 
 Hey Henri,

>From a security practice, this is obviously lacking, but in wider perspective,
I see it meet "industry standard", yes.

When I teach, I tell my student, that the worst WAF is the one that is
switched off. So if you need to compromise and you can only apply 20% of
the rules because you run the risk of business demanding it's switched off,
then that 20% WAF is still better than no WAF.

Cheers,

Christian


On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> Thanks Christian, taking this in combination with Osama's point earlier in
> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> limit the payload they'll scan. From my reading to 128kb (cloudflare+azure)
> or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
> 
> I believe my modern application is already very robust in terms of defence
> against sql injection as well as other OWASP top 10 attack vectors and that
> a WAF primarily adds reassurance (for the business and clients who ask if I
> have one) and minor frustration (for any potential attacker) layer. The
> spec is to add a WAF that meets (but notably does not necessarily have to
> exceed) industry standards. I believe this means that I can switch modsec
> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> ProcessPartial' - allowing through unscanned any payloads over those sizes)
> and be able to say I've got scan-size-policy-parity with an AWS or a
> Cloudflare which means it is "industry standard".
> 
> Please let me know if you think that's mad and thanks again
> 
> Best Regards,
> 
> Henri
> 
> On Sun, 25 Apr 2021 at 21:39, Christian Folini <chr...@ne...>
> wrote:
> 
> > Hey Henri,
> >
> > You are in a bad situation and as far as I can see you are right, you might
> > have to drop modsec/CRS in this situation.
> >
> > I've had a customer with a similar problem and we did a deep dive
> > investigation and I had to strike colors in the end.
> >
> > The point is not the JSON parser. That has shown to be really fast. The
> > point
> > is several hundred variables that go into CRS afterwards. If you run CRS
> > on a
> > standard web application you get forms with a few parameters and that's
> > easy.
> > But several megabytes of JSON means hundreds of arguments and CRS parses
> > them
> > all.
> >
> > So we tried to work with rule exclusions and skip the parameters we did not
> > think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
> > slower the longer the ignore-lists of parameters became. This and a few
> > very
> > odd behaviors.
> >
> > Given the customer wanted a generic WAF without tuning of individual APIs
> > we
> > got to a dead end.
> >
> > However, if tuning was an option, then I would probably edit-CRS with
> > msc_pyparser and replace the target lists with arguments I was interested
> > in.
> >
> > https://coreruleset.org/20200901/introducing-msc_pyparser/
> >
> > As a complementary practice, one could think of performing allowlist
> > checks on
> > some / most of the JSON. Say you have a huge JSON payload with 500
> > parameters.
> > You examine it and discover that 300 of them actually contain simple digits
> > and asciii characters and neither special chars nor escape sequences.
> > So you do a regex allowlist and apply it to these 300 parameters of said
> > API. And the rest you can push into CRS. Or a subset of CRS.
> >
> > I have not done this and the problem is if ModSec is able to handle the
> > large
> > target lists in a speedy manner.
> >
> >
> > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > security
> > tests of such a system. As I said, the JSON parser can be really fast. The
> > difficult thing is to check several hundred parameters without losing
> > performance.
> >
> > Good luck!
> >
> > Christian
> >
> >
> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > Hi all,
> > >
> > > I'm in a situation where the only solution seems to be to drop modsec/CRS
> > > and look at something like Cloudflare's WAF (and change our security
> > model
> > > out of necessity). I'm hoping the esteemed membership of this list might
> > > have some thoughts.
> > >
> > > I've got about 1MB of JSON, payloads in our app might run to 20 or even
> > > 30MB ultimately.
> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> > > to process in mod sec 3.0.4 with CRS 3.2.0
> > >
> > > It takes 1 second to process in our API so the WAF element is a 39x slow
> > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > takes 5
> > > seconds to scan this payload - and that's my target.
> > >
> > > Has anyone got any idea how to improve performance? Reading blog posts
> > > about the development of cloudflare's waf I see that memoization of
> > common
> > > function calls was one of their absolute best performance improvements
> > over
> > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > calculated once instead of once per rule OR contains('somestring',
> > > response_body)... you get the drift). Do we have anything like this in
> > > modsec today? Is that already in place and my 39 seconds is after that?
> > >
> > > I appreciate that mod sec is fast on its own and adding complex rules can
> > > be said to slow it down. With CRS being by far the most common use case
> > for
> > > mod sec (based on my googling) I'm surprised it's this slow, do you think
> > > i've missed something?
> > >
> > > To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> > > but new, irregular ones that we need scanned in ideally <10 seconds that
> > > can range from 1MB-30MB
> > >
> > > Best regards,
> > >
> > > Henri Cook
> >
> >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/
> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
> >


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/



_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
  _______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Joe M. <Joe...@mo...> - 2021-04-26 12:17:58

Michael,

We've had some similar issues with large datex payloads from external systems.

We did a few things:

The external system supplier reduced the payloads to around a maximum of 2Mb (They were previously sending 20Mb) - This meant more payloads could be muti-threaded, where the 1 20Mb payload used 1 core only.
Disabled a couple of the rules which where problematic for us, but because it was an API, we figured the cons outweighed the pros.

The rules we disabled where:

932110
932115
941100

Joe.
From: Michael Woods via mod-security-users <mod...@li...>
Sent: 26 April 2021 11:57
To: mod...@li...; Osama Elnaggar <oel...@gm...>
Cc: Michael Woods <sco...@ya...>
Subject: Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

We don't have a WAF so disabling this one rule is tolerated to gain the benefit of all the other rules provided by the CRS.

A lot of testing I expect needs to be done to introduce a new regular expression engine.

Tel:       01257 266394 Mobile: 07944032617 Email:  sco...@ya...<mailto:sco...@ya...>


On Monday, 26 April 2021, 10:45:53 BST, Osama Elnaggar <oel...@gm...<mailto:oel...@gm...>> wrote:


Wow. That's bad. As you currently don't have an option of choosing your regex engine in ModSecurity, one option you might want to investigate is offloading this particular rule's processing to a Lua script. There is a Lua binding for RE2 (https://github.com/cloudflare/lua-re2<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcloudflare%2Flua-re2&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351162858%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z0MEBoD8z%2FRKtMofMVprNTCnegMOflMLgBzrx%2FN%2FjSI%3D&reserved=0>) which you can use but I haven't tested it

--
Osama Elnaggar


On April 26, 2021 at 7:29:02 PM, Michael Woods via mod-security-users (mod...@li...<mailto:mod...@li...>) wrote:
I've had a similar experience with mod_security and CRS. Some of our JSON payloads can be 4MB. Given that the regular expressions executed by mod_security are Perl based, I extracted the regular expressions from the CRS and benchmarked them individually with varying sizes of JSON documents.

All CRS rules completed within 0.03s except for one rule. Rule 941160 when executed against large payloads can take multiple seconds to complete. A 1M payload takes around 2700 seconds to complete, without PCRE JIT enabled. Running PCRE with JIT compilation does have an impact but it still takes around 385 seconds to complete.

Rule 941160 belongs to a collection of rules concerning XSS attacks.

Doing some more digging I discovered Perl based regular expression engines such as Perl and PCRE have exponential response times when the data grows in size. I also was doing some work with Google's Go language and their regular expression engine re provides a linear response time. Luckily on Perl's CPAN repository, there is a module that provides an interface with re. Running the same benchmarks again, rule 941160 took less than a second. I had to run it again and check the results because I didn't believe it.

Regards


On Monday, 26 April 2021, 07:16:18 BST, Christian Folini <chr...@ne...<mailto:chr...@ne...>> wrote:


Hey Henri,

>From a security practice, this is obviously lacking, but in wider perspective,
I see it meet "industry standard", yes.

When I teach, I tell my student, that the worst WAF is the one that is
switched off. So if you need to compromise and you can only apply 20% of
the rules because you run the risk of business demanding it's switched off,
then that 20% WAF is still better than no WAF.

Cheers,

Christian


On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> Thanks Christian, taking this in combination with Osama's point earlier in
> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> limit the payload they'll scan. From my reading to 128kb (cloudflare+azure)
> or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>
> I believe my modern application is already very robust in terms of defence
> against sql injection as well as other OWASP top 10 attack vectors and that
> a WAF primarily adds reassurance (for the business and clients who ask if I
> have one) and minor frustration (for any potential attacker) layer. The
> spec is to add a WAF that meets (but notably does not necessarily have to
> exceed) industry standards. I believe this means that I can switch modsec
> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> ProcessPartial' - allowing through unscanned any payloads over those sizes)
> and be able to say I've got scan-size-policy-parity with an AWS or a
> Cloudflare which means it is "industry standard".
>
> Please let me know if you think that's mad and thanks again
>
> Best Regards,
>
> Henri
>
> On Sun, 25 Apr 2021 at 21:39, Christian Folini <chr...@ne...<mailto:chr...@ne...>>
> wrote:
>
> > Hey Henri,
> >
> > You are in a bad situation and as far as I can see you are right, you might
> > have to drop modsec/CRS in this situation.
> >
> > I've had a customer with a similar problem and we did a deep dive
> > investigation and I had to strike colors in the end.
> >
> > The point is not the JSON parser. That has shown to be really fast. The
> > point
> > is several hundred variables that go into CRS afterwards. If you run CRS
> > on a
> > standard web application you get forms with a few parameters and that's
> > easy.
> > But several megabytes of JSON means hundreds of arguments and CRS parses
> > them
> > all.
> >
> > So we tried to work with rule exclusions and skip the parameters we did not
> > think dangerous, but here comes the bummer: ModSec 2.9 grew substantially
> > slower the longer the ignore-lists of parameters became. This and a few
> > very
> > odd behaviors.
> >
> > Given the customer wanted a generic WAF without tuning of individual APIs
> > we
> > got to a dead end.
> >
> > However, if tuning was an option, then I would probably edit-CRS with
> > msc_pyparser and replace the target lists with arguments I was interested
> > in.
> >
> > https://coreruleset.org/20200901/introducing-msc_pyparser/<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcoreruleset.org%2F20200901%2Fintroducing-msc_pyparser%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351172819%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cmztsUY72%2BZlyoO6WxiNeEYhxsN7SWD%2FbM5o%2FV0S%2BrQ%3D&reserved=0>
> >
> > As a complementary practice, one could think of performing allowlist
> > checks on
> > some / most of the JSON. Say you have a huge JSON payload with 500
> > parameters.
> > You examine it and discover that 300 of them actually contain simple digits
> > and asciii characters and neither special chars nor escape sequences.
> > So you do a regex allowlist and apply it to these 300 parameters of said
> > API. And the rest you can push into CRS. Or a subset of CRS.
> >
> > I have not done this and the problem is if ModSec is able to handle the
> > large
> > target lists in a speedy manner.
> >
> >
> > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > security
> > tests of such a system. As I said, the JSON parser can be really fast. The
> > difficult thing is to check several hundred parameters without losing
> > performance.
> >
> > Good luck!
> >
> > Christian
> >
> >
> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > Hi all,
> > >
> > > I'm in a situation where the only solution seems to be to drop modsec/CRS
> > > and look at something like Cloudflare's WAF (and change our security
> > model
> > > out of necessity). I'm hoping the esteemed membership of this list might
> > > have some thoughts.
> > >
> > > I've got about 1MB of JSON, payloads in our app might run to 20 or even
> > > 30MB ultimately.
> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40 seconds
> > > to process in mod sec 3.0.4 with CRS 3.2.0
> > >
> > > It takes 1 second to process in our API so the WAF element is a 39x slow
> > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > takes 5
> > > seconds to scan this payload - and that's my target.
> > >
> > > Has anyone got any idea how to improve performance? Reading blog posts
> > > about the development of cloudflare's waf I see that memoization of
> > common
> > > function calls was one of their absolute best performance improvements
> > over
> > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > calculated once instead of once per rule OR contains('somestring',
> > > response_body)... you get the drift). Do we have anything like this in
> > > modsec today? Is that already in place and my 39 seconds is after that?
> > >
> > > I appreciate that mod sec is fast on its own and adding complex rules can
> > > be said to slow it down. With CRS being by far the most common use case
> > for
> > > mod sec (based on my googling) I'm surprised it's this slow, do you think
> > > i've missed something?
> > >
> > > To note: I'm only scanning JSON payloads, typically much less than 0.5MB
> > > but new, irregular ones that we need scanned in ideally <10 seconds that
> > > can range from 1MB-30MB
> > >
> > > Best regards,
> > >
> > > Henri Cook
> >
> >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...<mailto:mod...@li...>
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fmod-security-users&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351172819%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TWrpAljAq8fI3cJ%2By0BnnGtYMgIc8m25gM6F7ztpgA8%3D&reserved=0>
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Frules%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351172819%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kSRxirjV4aT3PzE0gfJ5mU9Bls3eOkjMV1q6nTIo5Xg%3D&reserved=0>
> > > http://www.modsecurity.org/projects/commercial/support/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Fsupport%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351182766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EwxIxCPcanaE5ffUnlZiaD004MXZi960xuF0Nke2ye8%3D&reserved=0>

> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...<mailto:mod...@li...>
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fmod-security-users&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351182766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qv0XGnrUu9J9i8cktSRKVPUCod%2Fz8gLQXY8fSTPCL5k%3D&reserved=0>
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Frules%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351192728%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XmixwIAb6woq9bpdRqMnW9ipT021%2FaoK%2Bb9xZWrOdcA%3D&reserved=0>
> > http://www.modsecurity.org/projects/commercial/support/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Fsupport%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351192728%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ej9EJ%2FIv8WisA4MiFdYzg3tiaSH9PJPGY%2FcXIvziN24%3D&reserved=0>
> >


> _______________________________________________
> mod-security-users mailing list
> mod...@li...<mailto:mod...@li...>
> https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fmod-security-users&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351192728%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=x9M1QW%2Bh%2ForgqcTnmL%2BqYt6GIuSB8KOjl0JTjLzQei4%3D&reserved=0>
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Frules%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351202682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gtLOxYc6%2B5LnUW9EotBszKfi8DC8AAi%2BGun%2Be4%2FGGa4%3D&reserved=0>
> http://www.modsecurity.org/projects/commercial/support/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Fsupport%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351202682%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mQIDG57YbS07r6fSiM3s%2FtWMSX4Dzu%2Bb%2FnXqJHBjWYE%3D&reserved=0>



_______________________________________________
mod-security-users mailing list
mod...@li...<mailto:mod...@li...>
https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fmod-security-users&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351212638%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pdDHUxvstedahiEc%2FE6vHzUvBxHMyEGvK0jq7f3C0hU%3D&reserved=0>
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Frules%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351212638%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pRn7iEBe%2Fdi763wOjNTeA5i5r26pBTo5mFDukUU4Y7Y%3D&reserved=0>
http://www.modsecurity.org/projects/commercial/support/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Fsupport%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351212638%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZVV%2B7%2FILsF9zltvL4qrMcSmNG9Z3vwEw2k7ch845g7Y%3D&reserved=0>
_______________________________________________
mod-security-users mailing list
mod...@li...<mailto:mod...@li...>
https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fmod-security-users&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351222594%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=trCf%2BpjfcRwm7igJOgz917RIAWmdrv%2BSFhKIAwNPDt0%3D&reserved=0>
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Frules%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351222594%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MRoT6lpopmBaSXN4vYEDFHgNrBVVbVuuftYZUqHzLsU%3D&reserved=0>
http://www.modsecurity.org/projects/commercial/support/<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.modsecurity.org%2Fprojects%2Fcommercial%2Fsupport%2F&data=04%7C01%7Cjoe.madden%40mottmac.com%7Cacf9a9eb178049e0ae5d08d908a247dd%7Ca2bed0c459574f73b0c2a811407590fb%7C0%7C0%7C637550315351222594%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UiXkydb%2FYvQ7y5VjunDqvHPMnL8aU1ICoWBdGOt0hKA%3D&reserved=0>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-04-26 14:25:07

The follow up problem to this is:

Now i'm set to `SecRequestBodyLimit` 31457280 and
`SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
PartialProcess`

In my mind this will process the first part of a request if it can, and
ignore the rest. But:

- Rule 200002 is triggered, which is saying the JSON can't be parsed.
Presumably because in a large request it tries to process the beginning of
the JSON and can't (because it won't parse, because the JSON is cut off so
doesn't end)

So I think I need to find a way to skip JSON parsing entirely when the
payload is over 64kb (65536)? Does that sound right? Assuming 64kb is the
limit I want to stick with. I hadn't really considered before this point
that 'partial processing' of JSON was likely to be hairy but of course it
makes sense.



On Mon, 26 Apr 2021 at 07:17, Christian Folini <chr...@ne...>
wrote:

> Hey Henri,
>
> From a security practice, this is obviously lacking, but in wider
> perspective,
> I see it meet "industry standard", yes.
>
> When I teach, I tell my student, that the worst WAF is the one that is
> switched off. So if you need to compromise and you can only apply 20% of
> the rules because you run the risk of business demanding it's switched off,
> then that 20% WAF is still better than no WAF.
>
> Cheers,
>
> Christian
>
>
> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> > Thanks Christian, taking this in combination with Osama's point earlier
> in
> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
> > limit the payload they'll scan. From my reading to 128kb
> (cloudflare+azure)
> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
> >
> > I believe my modern application is already very robust in terms of
> defence
> > against sql injection as well as other OWASP top 10 attack vectors and
> that
> > a WAF primarily adds reassurance (for the business and clients who ask
> if I
> > have one) and minor frustration (for any potential attacker) layer. The
> > spec is to add a WAF that meets (but notably does not necessarily have to
> > exceed) industry standards. I believe this means that I can switch modsec
> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> > ProcessPartial' - allowing through unscanned any payloads over those
> sizes)
> > and be able to say I've got scan-size-policy-parity with an AWS or a
> > Cloudflare which means it is "industry standard".
> >
> > Please let me know if you think that's mad and thanks again
> >
> > Best Regards,
> >
> > Henri
> >
> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <
> chr...@ne...>
> > wrote:
> >
> > > Hey Henri,
> > >
> > > You are in a bad situation and as far as I can see you are right, you
> might
> > > have to drop modsec/CRS in this situation.
> > >
> > > I've had a customer with a similar problem and we did a deep dive
> > > investigation and I had to strike colors in the end.
> > >
> > > The point is not the JSON parser. That has shown to be really fast. The
> > > point
> > > is several hundred variables that go into CRS afterwards. If you run
> CRS
> > > on a
> > > standard web application you get forms with a few parameters and that's
> > > easy.
> > > But several megabytes of JSON means hundreds of arguments and CRS
> parses
> > > them
> > > all.
> > >
> > > So we tried to work with rule exclusions and skip the parameters we
> did not
> > > think dangerous, but here comes the bummer: ModSec 2.9 grew
> substantially
> > > slower the longer the ignore-lists of parameters became. This and a few
> > > very
> > > odd behaviors.
> > >
> > > Given the customer wanted a generic WAF without tuning of individual
> APIs
> > > we
> > > got to a dead end.
> > >
> > > However, if tuning was an option, then I would probably edit-CRS with
> > > msc_pyparser and replace the target lists with arguments I was
> interested
> > > in.
> > >
> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
> > >
> > > As a complementary practice, one could think of performing allowlist
> > > checks on
> > > some / most of the JSON. Say you have a huge JSON payload with 500
> > > parameters.
> > > You examine it and discover that 300 of them actually contain simple
> digits
> > > and asciii characters and neither special chars nor escape sequences.
> > > So you do a regex allowlist and apply it to these 300 parameters of
> said
> > > API. And the rest you can push into CRS. Or a subset of CRS.
> > >
> > > I have not done this and the problem is if ModSec is able to handle the
> > > large
> > > target lists in a speedy manner.
> > >
> > >
> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
> > > security
> > > tests of such a system. As I said, the JSON parser can be really fast.
> The
> > > difficult thing is to check several hundred parameters without losing
> > > performance.
> > >
> > > Good luck!
> > >
> > > Christian
> > >
> > >
> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> > > > Hi all,
> > > >
> > > > I'm in a situation where the only solution seems to be to drop
> modsec/CRS
> > > > and look at something like Cloudflare's WAF (and change our security
> > > model
> > > > out of necessity). I'm hoping the esteemed membership of this list
> might
> > > > have some thoughts.
> > > >
> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or
> even
> > > > 30MB ultimately.
> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
> seconds
> > > > to process in mod sec 3.0.4 with CRS 3.2.0
> > > >
> > > > It takes 1 second to process in our API so the WAF element is a 39x
> slow
> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
> > > takes 5
> > > > seconds to scan this payload - and that's my target.
> > > >
> > > > Has anyone got any idea how to improve performance? Reading blog
> posts
> > > > about the development of cloudflare's waf I see that memoization of
> > > common
> > > > function calls was one of their absolute best performance
> improvements
> > > over
> > > > their modsec implementation (e.g. strlen(response_body) so it's only
> > > > calculated once instead of once per rule OR contains('somestring',
> > > > response_body)... you get the drift). Do we have anything like this
> in
> > > > modsec today? Is that already in place and my 39 seconds is after
> that?
> > > >
> > > > I appreciate that mod sec is fast on its own and adding complex
> rules can
> > > > be said to slow it down. With CRS being by far the most common use
> case
> > > for
> > > > mod sec (based on my googling) I'm surprised it's this slow, do you
> think
> > > > i've missed something?
> > > >
> > > > To note: I'm only scanning JSON payloads, typically much less than
> 0.5MB
> > > > but new, irregular ones that we need scanned in ideally <10 seconds
> that
> > > > can range from 1MB-30MB
> > > >
> > > > Best regards,
> > > >
> > > > Henri Cook
> > >
> > >
> > > > _______________________________________________
> > > > mod-security-users mailing list
> > > > mod...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > > http://www.modsecurity.org/projects/commercial/rules/
> > > > http://www.modsecurity.org/projects/commercial/support/
> > >
> > >
> > >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> > > http://www.modsecurity.org/projects/commercial/support/
> > >
>
>
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Srikanth A. <sri...@go...> - 2021-04-26 14:30:20

Hi

What if thr attack is through the end of file. I tried and the answer is it
misses the waf scanning.

Regards
Srikanth A

On Mon, 26 Apr 2021, 15:27 Henri Cook, <henri@proteus.tech> wrote:

> The follow up problem to this is:
>
> Now i'm set to `SecRequestBodyLimit` 31457280 and
> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
> PartialProcess`
>
> In my mind this will process the first part of a request if it can, and
> ignore the rest. But:
>
> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
> Presumably because in a large request it tries to process the beginning of
> the JSON and can't (because it won't parse, because the JSON is cut off so
> doesn't end)
>
> So I think I need to find a way to skip JSON parsing entirely when the
> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is the
> limit I want to stick with. I hadn't really considered before this point
> that 'partial processing' of JSON was likely to be hairy but of course it
> makes sense.
>
>
>
> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
> chr...@ne...> wrote:
>
>> Hey Henri,
>>
>> From a security practice, this is obviously lacking, but in wider
>> perspective,
>> I see it meet "industry standard", yes.
>>
>> When I teach, I tell my student, that the worst WAF is the one that is
>> switched off. So if you need to compromise and you can only apply 20% of
>> the rules because you run the risk of business demanding it's switched
>> off,
>> then that 20% WAF is still better than no WAF.
>>
>> Cheers,
>>
>> Christian
>>
>>
>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>> > Thanks Christian, taking this in combination with Osama's point earlier
>> in
>> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
>> > limit the payload they'll scan. From my reading to 128kb
>> (cloudflare+azure)
>> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>> >
>> > I believe my modern application is already very robust in terms of
>> defence
>> > against sql injection as well as other OWASP top 10 attack vectors and
>> that
>> > a WAF primarily adds reassurance (for the business and clients who ask
>> if I
>> > have one) and minor frustration (for any potential attacker) layer. The
>> > spec is to add a WAF that meets (but notably does not necessarily have
>> to
>> > exceed) industry standards. I believe this means that I can switch
>> modsec
>> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>> > ProcessPartial' - allowing through unscanned any payloads over those
>> sizes)
>> > and be able to say I've got scan-size-policy-parity with an AWS or a
>> > Cloudflare which means it is "industry standard".
>> >
>> > Please let me know if you think that's mad and thanks again
>> >
>> > Best Regards,
>> >
>> > Henri
>> >
>> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>> chr...@ne...>
>> > wrote:
>> >
>> > > Hey Henri,
>> > >
>> > > You are in a bad situation and as far as I can see you are right, you
>> might
>> > > have to drop modsec/CRS in this situation.
>> > >
>> > > I've had a customer with a similar problem and we did a deep dive
>> > > investigation and I had to strike colors in the end.
>> > >
>> > > The point is not the JSON parser. That has shown to be really fast.
>> The
>> > > point
>> > > is several hundred variables that go into CRS afterwards. If you run
>> CRS
>> > > on a
>> > > standard web application you get forms with a few parameters and
>> that's
>> > > easy.
>> > > But several megabytes of JSON means hundreds of arguments and CRS
>> parses
>> > > them
>> > > all.
>> > >
>> > > So we tried to work with rule exclusions and skip the parameters we
>> did not
>> > > think dangerous, but here comes the bummer: ModSec 2.9 grew
>> substantially
>> > > slower the longer the ignore-lists of parameters became. This and a
>> few
>> > > very
>> > > odd behaviors.
>> > >
>> > > Given the customer wanted a generic WAF without tuning of individual
>> APIs
>> > > we
>> > > got to a dead end.
>> > >
>> > > However, if tuning was an option, then I would probably edit-CRS with
>> > > msc_pyparser and replace the target lists with arguments I was
>> interested
>> > > in.
>> > >
>> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
>> > >
>> > > As a complementary practice, one could think of performing allowlist
>> > > checks on
>> > > some / most of the JSON. Say you have a huge JSON payload with 500
>> > > parameters.
>> > > You examine it and discover that 300 of them actually contain simple
>> digits
>> > > and asciii characters and neither special chars nor escape sequences.
>> > > So you do a regex allowlist and apply it to these 300 parameters of
>> said
>> > > API. And the rest you can push into CRS. Or a subset of CRS.
>> > >
>> > > I have not done this and the problem is if ModSec is able to handle
>> the
>> > > large
>> > > target lists in a speedy manner.
>> > >
>> > >
>> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
>> > > security
>> > > tests of such a system. As I said, the JSON parser can be really
>> fast. The
>> > > difficult thing is to check several hundred parameters without losing
>> > > performance.
>> > >
>> > > Good luck!
>> > >
>> > > Christian
>> > >
>> > >
>> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>> > > > Hi all,
>> > > >
>> > > > I'm in a situation where the only solution seems to be to drop
>> modsec/CRS
>> > > > and look at something like Cloudflare's WAF (and change our security
>> > > model
>> > > > out of necessity). I'm hoping the esteemed membership of this list
>> might
>> > > > have some thoughts.
>> > > >
>> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or
>> even
>> > > > 30MB ultimately.
>> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>> seconds
>> > > > to process in mod sec 3.0.4 with CRS 3.2.0
>> > > >
>> > > > It takes 1 second to process in our API so the WAF element is a 39x
>> slow
>> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
>> > > takes 5
>> > > > seconds to scan this payload - and that's my target.
>> > > >
>> > > > Has anyone got any idea how to improve performance? Reading blog
>> posts
>> > > > about the development of cloudflare's waf I see that memoization of
>> > > common
>> > > > function calls was one of their absolute best performance
>> improvements
>> > > over
>> > > > their modsec implementation (e.g. strlen(response_body) so it's only
>> > > > calculated once instead of once per rule OR contains('somestring',
>> > > > response_body)... you get the drift). Do we have anything like this
>> in
>> > > > modsec today? Is that already in place and my 39 seconds is after
>> that?
>> > > >
>> > > > I appreciate that mod sec is fast on its own and adding complex
>> rules can
>> > > > be said to slow it down. With CRS being by far the most common use
>> case
>> > > for
>> > > > mod sec (based on my googling) I'm surprised it's this slow, do you
>> think
>> > > > i've missed something?
>> > > >
>> > > > To note: I'm only scanning JSON payloads, typically much less than
>> 0.5MB
>> > > > but new, irregular ones that we need scanned in ideally <10 seconds
>> that
>> > > > can range from 1MB-30MB
>> > > >
>> > > > Best regards,
>> > > >
>> > > > Henri Cook
>> > >
>> > >
>> > > > _______________________________________________
>> > > > mod-security-users mailing list
>> > > > mod...@li...
>> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > > > Commercial ModSecurity Rules and Support from Trustwave's
>> SpiderLabs:
>> > > > http://www.modsecurity.org/projects/commercial/rules/
>> > > > http://www.modsecurity.org/projects/commercial/support/
>> > >
>> > >
>> > >
>> > > _______________________________________________
>> > > mod-security-users mailing list
>> > > mod...@li...
>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > > http://www.modsecurity.org/projects/commercial/rules/
>> > > http://www.modsecurity.org/projects/commercial/support/
>> > >
>>
>>
>> > _______________________________________________
>> > mod-security-users mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > http://www.modsecurity.org/projects/commercial/rules/
>> > http://www.modsecurity.org/projects/commercial/support/
>>
>>
>>
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-04-29 12:53:47

I think what I need here is a way to exempt the request body from any
scanning when it's over 128kb (my chosen upper limit)...

I'll probably try implementing this as a phase 1 rule on Content-Length
unless anyone shouts there's a better way. `ProcessPartial` sounded like
the holy grail, but you can't partial process JSON (because it doesn't
parse!).

It'd be cool to be able to fallback to a fulltext scan of the partial JSON
if it's >128kb, I might have a try at looking into that also (again any
guidance appreciated)

Best Regards,

Henri

On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:

> The follow up problem to this is:
>
> Now i'm set to `SecRequestBodyLimit` 31457280 and
> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
> PartialProcess`
>
> In my mind this will process the first part of a request if it can, and
> ignore the rest. But:
>
> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
> Presumably because in a large request it tries to process the beginning of
> the JSON and can't (because it won't parse, because the JSON is cut off so
> doesn't end)
>
> So I think I need to find a way to skip JSON parsing entirely when the
> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is the
> limit I want to stick with. I hadn't really considered before this point
> that 'partial processing' of JSON was likely to be hairy but of course it
> makes sense.
>
>
>
> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
> chr...@ne...> wrote:
>
>> Hey Henri,
>>
>> From a security practice, this is obviously lacking, but in wider
>> perspective,
>> I see it meet "industry standard", yes.
>>
>> When I teach, I tell my student, that the worst WAF is the one that is
>> switched off. So if you need to compromise and you can only apply 20% of
>> the rules because you run the risk of business demanding it's switched
>> off,
>> then that 20% WAF is still better than no WAF.
>>
>> Cheers,
>>
>> Christian
>>
>>
>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>> > Thanks Christian, taking this in combination with Osama's point earlier
>> in
>> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
>> > limit the payload they'll scan. From my reading to 128kb
>> (cloudflare+azure)
>> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>> >
>> > I believe my modern application is already very robust in terms of
>> defence
>> > against sql injection as well as other OWASP top 10 attack vectors and
>> that
>> > a WAF primarily adds reassurance (for the business and clients who ask
>> if I
>> > have one) and minor frustration (for any potential attacker) layer. The
>> > spec is to add a WAF that meets (but notably does not necessarily have
>> to
>> > exceed) industry standards. I believe this means that I can switch
>> modsec
>> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>> > ProcessPartial' - allowing through unscanned any payloads over those
>> sizes)
>> > and be able to say I've got scan-size-policy-parity with an AWS or a
>> > Cloudflare which means it is "industry standard".
>> >
>> > Please let me know if you think that's mad and thanks again
>> >
>> > Best Regards,
>> >
>> > Henri
>> >
>> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>> chr...@ne...>
>> > wrote:
>> >
>> > > Hey Henri,
>> > >
>> > > You are in a bad situation and as far as I can see you are right, you
>> might
>> > > have to drop modsec/CRS in this situation.
>> > >
>> > > I've had a customer with a similar problem and we did a deep dive
>> > > investigation and I had to strike colors in the end.
>> > >
>> > > The point is not the JSON parser. That has shown to be really fast.
>> The
>> > > point
>> > > is several hundred variables that go into CRS afterwards. If you run
>> CRS
>> > > on a
>> > > standard web application you get forms with a few parameters and
>> that's
>> > > easy.
>> > > But several megabytes of JSON means hundreds of arguments and CRS
>> parses
>> > > them
>> > > all.
>> > >
>> > > So we tried to work with rule exclusions and skip the parameters we
>> did not
>> > > think dangerous, but here comes the bummer: ModSec 2.9 grew
>> substantially
>> > > slower the longer the ignore-lists of parameters became. This and a
>> few
>> > > very
>> > > odd behaviors.
>> > >
>> > > Given the customer wanted a generic WAF without tuning of individual
>> APIs
>> > > we
>> > > got to a dead end.
>> > >
>> > > However, if tuning was an option, then I would probably edit-CRS with
>> > > msc_pyparser and replace the target lists with arguments I was
>> interested
>> > > in.
>> > >
>> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
>> > >
>> > > As a complementary practice, one could think of performing allowlist
>> > > checks on
>> > > some / most of the JSON. Say you have a huge JSON payload with 500
>> > > parameters.
>> > > You examine it and discover that 300 of them actually contain simple
>> digits
>> > > and asciii characters and neither special chars nor escape sequences.
>> > > So you do a regex allowlist and apply it to these 300 parameters of
>> said
>> > > API. And the rest you can push into CRS. Or a subset of CRS.
>> > >
>> > > I have not done this and the problem is if ModSec is able to handle
>> the
>> > > large
>> > > target lists in a speedy manner.
>> > >
>> > >
>> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
>> > > security
>> > > tests of such a system. As I said, the JSON parser can be really
>> fast. The
>> > > difficult thing is to check several hundred parameters without losing
>> > > performance.
>> > >
>> > > Good luck!
>> > >
>> > > Christian
>> > >
>> > >
>> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>> > > > Hi all,
>> > > >
>> > > > I'm in a situation where the only solution seems to be to drop
>> modsec/CRS
>> > > > and look at something like Cloudflare's WAF (and change our security
>> > > model
>> > > > out of necessity). I'm hoping the esteemed membership of this list
>> might
>> > > > have some thoughts.
>> > > >
>> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or
>> even
>> > > > 30MB ultimately.
>> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>> seconds
>> > > > to process in mod sec 3.0.4 with CRS 3.2.0
>> > > >
>> > > > It takes 1 second to process in our API so the WAF element is a 39x
>> slow
>> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
>> > > takes 5
>> > > > seconds to scan this payload - and that's my target.
>> > > >
>> > > > Has anyone got any idea how to improve performance? Reading blog
>> posts
>> > > > about the development of cloudflare's waf I see that memoization of
>> > > common
>> > > > function calls was one of their absolute best performance
>> improvements
>> > > over
>> > > > their modsec implementation (e.g. strlen(response_body) so it's only
>> > > > calculated once instead of once per rule OR contains('somestring',
>> > > > response_body)... you get the drift). Do we have anything like this
>> in
>> > > > modsec today? Is that already in place and my 39 seconds is after
>> that?
>> > > >
>> > > > I appreciate that mod sec is fast on its own and adding complex
>> rules can
>> > > > be said to slow it down. With CRS being by far the most common use
>> case
>> > > for
>> > > > mod sec (based on my googling) I'm surprised it's this slow, do you
>> think
>> > > > i've missed something?
>> > > >
>> > > > To note: I'm only scanning JSON payloads, typically much less than
>> 0.5MB
>> > > > but new, irregular ones that we need scanned in ideally <10 seconds
>> that
>> > > > can range from 1MB-30MB
>> > > >
>> > > > Best regards,
>> > > >
>> > > > Henri Cook
>> > >
>> > >
>> > > > _______________________________________________
>> > > > mod-security-users mailing list
>> > > > mod...@li...
>> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > > > Commercial ModSecurity Rules and Support from Trustwave's
>> SpiderLabs:
>> > > > http://www.modsecurity.org/projects/commercial/rules/
>> > > > http://www.modsecurity.org/projects/commercial/support/
>> > >
>> > >
>> > >
>> > > _______________________________________________
>> > > mod-security-users mailing list
>> > > mod...@li...
>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > > http://www.modsecurity.org/projects/commercial/rules/
>> > > http://www.modsecurity.org/projects/commercial/support/
>> > >
>>
>>
>> > _______________________________________________
>> > mod-security-users mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > http://www.modsecurity.org/projects/commercial/rules/
>> > http://www.modsecurity.org/projects/commercial/support/
>>
>>
>>
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: <az...@po...> - 2021-04-29 17:40:28

Hi,

you can do this using exclusive rule, something like this (set the  
rule ID to something 'better'), length is in bytes:


SecRule REQUEST_BODY_LENGTH "@gt 131072" \
     "id:9999999,\
     phase:2,\
     pass,\
     t:none,\
     nolog,\
     ctl:ruleEngine=Off"



REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this  
into phase 1.



Citát Henri Cook <henri@proteus.tech>:

> I think what I need here is a way to exempt the request body from any
> scanning when it's over 128kb (my chosen upper limit)...
>
> I'll probably try implementing this as a phase 1 rule on Content-Length
> unless anyone shouts there's a better way. `ProcessPartial` sounded like
> the holy grail, but you can't partial process JSON (because it doesn't
> parse!).
>
> It'd be cool to be able to fallback to a fulltext scan of the partial JSON
> if it's >128kb, I might have a try at looking into that also (again any
> guidance appreciated)
>
> Best Regards,
>
> Henri
>
> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
>
>> The follow up problem to this is:
>>
>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>> PartialProcess`
>>
>> In my mind this will process the first part of a request if it can, and
>> ignore the rest. But:
>>
>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>> Presumably because in a large request it tries to process the beginning of
>> the JSON and can't (because it won't parse, because the JSON is cut off so
>> doesn't end)
>>
>> So I think I need to find a way to skip JSON parsing entirely when the
>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is the
>> limit I want to stick with. I hadn't really considered before this point
>> that 'partial processing' of JSON was likely to be hairy but of course it
>> makes sense.
>>
>>
>>
>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>> chr...@ne...> wrote:
>>
>>> Hey Henri,
>>>
>>> From a security practice, this is obviously lacking, but in wider
>>> perspective,
>>> I see it meet "industry standard", yes.
>>>
>>> When I teach, I tell my student, that the worst WAF is the one that is
>>> switched off. So if you need to compromise and you can only apply 20% of
>>> the rules because you run the risk of business demanding it's switched
>>> off,
>>> then that 20% WAF is still better than no WAF.
>>>
>>> Cheers,
>>>
>>> Christian
>>>
>>>
>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>>> > Thanks Christian, taking this in combination with Osama's point earlier
>>> in
>>> > the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
>>> > limit the payload they'll scan. From my reading to 128kb
>>> (cloudflare+azure)
>>> > or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>>> >
>>> > I believe my modern application is already very robust in terms of
>>> defence
>>> > against sql injection as well as other OWASP top 10 attack vectors and
>>> that
>>> > a WAF primarily adds reassurance (for the business and clients who ask
>>> if I
>>> > have one) and minor frustration (for any potential attacker) layer. The
>>> > spec is to add a WAF that meets (but notably does not necessarily have
>>> to
>>> > exceed) industry standards. I believe this means that I can switch
>>> modsec
>>> > to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>>> > ProcessPartial' - allowing through unscanned any payloads over those
>>> sizes)
>>> > and be able to say I've got scan-size-policy-parity with an AWS or a
>>> > Cloudflare which means it is "industry standard".
>>> >
>>> > Please let me know if you think that's mad and thanks again
>>> >
>>> > Best Regards,
>>> >
>>> > Henri
>>> >
>>> > On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>>> chr...@ne...>
>>> > wrote:
>>> >
>>> > > Hey Henri,
>>> > >
>>> > > You are in a bad situation and as far as I can see you are right, you
>>> might
>>> > > have to drop modsec/CRS in this situation.
>>> > >
>>> > > I've had a customer with a similar problem and we did a deep dive
>>> > > investigation and I had to strike colors in the end.
>>> > >
>>> > > The point is not the JSON parser. That has shown to be really fast.
>>> The
>>> > > point
>>> > > is several hundred variables that go into CRS afterwards. If you run
>>> CRS
>>> > > on a
>>> > > standard web application you get forms with a few parameters and
>>> that's
>>> > > easy.
>>> > > But several megabytes of JSON means hundreds of arguments and CRS
>>> parses
>>> > > them
>>> > > all.
>>> > >
>>> > > So we tried to work with rule exclusions and skip the parameters we
>>> did not
>>> > > think dangerous, but here comes the bummer: ModSec 2.9 grew
>>> substantially
>>> > > slower the longer the ignore-lists of parameters became. This and a
>>> few
>>> > > very
>>> > > odd behaviors.
>>> > >
>>> > > Given the customer wanted a generic WAF without tuning of individual
>>> APIs
>>> > > we
>>> > > got to a dead end.
>>> > >
>>> > > However, if tuning was an option, then I would probably edit-CRS with
>>> > > msc_pyparser and replace the target lists with arguments I was
>>> interested
>>> > > in.
>>> > >
>>> > > https://coreruleset.org/20200901/introducing-msc_pyparser/
>>> > >
>>> > > As a complementary practice, one could think of performing allowlist
>>> > > checks on
>>> > > some / most of the JSON. Say you have a huge JSON payload with 500
>>> > > parameters.
>>> > > You examine it and discover that 300 of them actually contain simple
>>> digits
>>> > > and asciii characters and neither special chars nor escape sequences.
>>> > > So you do a regex allowlist and apply it to these 300 parameters of
>>> said
>>> > > API. And the rest you can push into CRS. Or a subset of CRS.
>>> > >
>>> > > I have not done this and the problem is if ModSec is able to handle
>>> the
>>> > > large
>>> > > target lists in a speedy manner.
>>> > >
>>> > >
>>> > > Now you can turn to a CDN or alternative WAF. I would do an extensive
>>> > > security
>>> > > tests of such a system. As I said, the JSON parser can be really
>>> fast. The
>>> > > difficult thing is to check several hundred parameters without losing
>>> > > performance.
>>> > >
>>> > > Good luck!
>>> > >
>>> > > Christian
>>> > >
>>> > >
>>> > > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>>> > > > Hi all,
>>> > > >
>>> > > > I'm in a situation where the only solution seems to be to drop
>>> modsec/CRS
>>> > > > and look at something like Cloudflare's WAF (and change our security
>>> > > model
>>> > > > out of necessity). I'm hoping the esteemed membership of this list
>>> might
>>> > > > have some thoughts.
>>> > > >
>>> > > > I've got about 1MB of JSON, payloads in our app might run to 20 or
>>> even
>>> > > > 30MB ultimately.
>>> > > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>>> seconds
>>> > > > to process in mod sec 3.0.4 with CRS 3.2.0
>>> > > >
>>> > > > It takes 1 second to process in our API so the WAF element is a 39x
>>> slow
>>> > > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
>>> > > takes 5
>>> > > > seconds to scan this payload - and that's my target.
>>> > > >
>>> > > > Has anyone got any idea how to improve performance? Reading blog
>>> posts
>>> > > > about the development of cloudflare's waf I see that memoization of
>>> > > common
>>> > > > function calls was one of their absolute best performance
>>> improvements
>>> > > over
>>> > > > their modsec implementation (e.g. strlen(response_body) so it's only
>>> > > > calculated once instead of once per rule OR contains('somestring',
>>> > > > response_body)... you get the drift). Do we have anything like this
>>> in
>>> > > > modsec today? Is that already in place and my 39 seconds is after
>>> that?
>>> > > >
>>> > > > I appreciate that mod sec is fast on its own and adding complex
>>> rules can
>>> > > > be said to slow it down. With CRS being by far the most common use
>>> case
>>> > > for
>>> > > > mod sec (based on my googling) I'm surprised it's this slow, do you
>>> think
>>> > > > i've missed something?
>>> > > >
>>> > > > To note: I'm only scanning JSON payloads, typically much less than
>>> 0.5MB
>>> > > > but new, irregular ones that we need scanned in ideally <10 seconds
>>> that
>>> > > > can range from 1MB-30MB
>>> > > >
>>> > > > Best regards,
>>> > > >
>>> > > > Henri Cook
>>> > >
>>> > >
>>> > > > _______________________________________________
>>> > > > mod-security-users mailing list
>>> > > > mod...@li...
>>> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > > > Commercial ModSecurity Rules and Support from Trustwave's
>>> SpiderLabs:
>>> > > > http://www.modsecurity.org/projects/commercial/rules/
>>> > > > http://www.modsecurity.org/projects/commercial/support/
>>> > >
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > mod-security-users mailing list
>>> > > mod...@li...
>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > > http://www.modsecurity.org/projects/commercial/rules/
>>> > > http://www.modsecurity.org/projects/commercial/support/
>>> > >
>>>
>>>
>>> > _______________________________________________
>>> > mod-security-users mailing list
>>> > mod...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > http://www.modsecurity.org/projects/commercial/rules/
>>> > http://www.modsecurity.org/projects/commercial/support/
>>>
>>>
>>>
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>>>
>>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: <az...@po...> - 2021-04-29 18:00:15

I just noticed you are using CRS, so this one would be probably better:


SecRule REQUEST_BODY_LENGTH "@gt 131072" \
     "id:9999999,\
     phase:2,\
     pass,\
     t:none,\
     nolog,\
     ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"





Citát az...@po...:

> Hi,
>
> you can do this using exclusive rule, something like this (set the  
> rule ID to something 'better'), length is in bytes:
>
>
> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>     "id:9999999,\
>     phase:2,\
>     pass,\
>     t:none,\
>     nolog,\
>     ctl:ruleEngine=Off"
>
>
>
> REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this  
> into phase 1.
>
>
>
> Citát Henri Cook <henri@proteus.tech>:
>
>> I think what I need here is a way to exempt the request body from any
>> scanning when it's over 128kb (my chosen upper limit)...
>>
>> I'll probably try implementing this as a phase 1 rule on Content-Length
>> unless anyone shouts there's a better way. `ProcessPartial` sounded like
>> the holy grail, but you can't partial process JSON (because it doesn't
>> parse!).
>>
>> It'd be cool to be able to fallback to a fulltext scan of the partial JSON
>> if it's >128kb, I might have a try at looking into that also (again any
>> guidance appreciated)
>>
>> Best Regards,
>>
>> Henri
>>
>> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
>>
>>> The follow up problem to this is:
>>>
>>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>>> PartialProcess`
>>>
>>> In my mind this will process the first part of a request if it can, and
>>> ignore the rest. But:
>>>
>>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>>> Presumably because in a large request it tries to process the beginning of
>>> the JSON and can't (because it won't parse, because the JSON is cut off so
>>> doesn't end)
>>>
>>> So I think I need to find a way to skip JSON parsing entirely when the
>>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is the
>>> limit I want to stick with. I hadn't really considered before this point
>>> that 'partial processing' of JSON was likely to be hairy but of course it
>>> makes sense.
>>>
>>>
>>>
>>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>>> chr...@ne...> wrote:
>>>
>>>> Hey Henri,
>>>>
>>>> From a security practice, this is obviously lacking, but in wider
>>>> perspective,
>>>> I see it meet "industry standard", yes.
>>>>
>>>> When I teach, I tell my student, that the worst WAF is the one that is
>>>> switched off. So if you need to compromise and you can only apply 20% of
>>>> the rules because you run the risk of business demanding it's switched
>>>> off,
>>>> then that 20% WAF is still better than no WAF.
>>>>
>>>> Cheers,
>>>>
>>>> Christian
>>>>
>>>>
>>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>>>>> Thanks Christian, taking this in combination with Osama's point earlier
>>>> in
>>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem to
>>>>> limit the payload they'll scan. From my reading to 128kb
>>>> (cloudflare+azure)
>>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular issue.
>>>>>
>>>>> I believe my modern application is already very robust in terms of
>>>> defence
>>>>> against sql injection as well as other OWASP top 10 attack vectors and
>>>> that
>>>>> a WAF primarily adds reassurance (for the business and clients who ask
>>>> if I
>>>>> have one) and minor frustration (for any potential attacker) layer. The
>>>>> spec is to add a WAF that meets (but notably does not necessarily have
>>>> to
>>>>> exceed) industry standards. I believe this means that I can switch
>>>> modsec
>>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>>>>> ProcessPartial' - allowing through unscanned any payloads over those
>>>> sizes)
>>>>> and be able to say I've got scan-size-policy-parity with an AWS or a
>>>>> Cloudflare which means it is "industry standard".
>>>>>
>>>>> Please let me know if you think that's mad and thanks again
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> Henri
>>>>>
>>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>>>> chr...@ne...>
>>>>> wrote:
>>>>>
>>>>> > Hey Henri,
>>>>> >
>>>>> > You are in a bad situation and as far as I can see you are right, you
>>>> might
>>>>> > have to drop modsec/CRS in this situation.
>>>>> >
>>>>> > I've had a customer with a similar problem and we did a deep dive
>>>>> > investigation and I had to strike colors in the end.
>>>>> >
>>>>> > The point is not the JSON parser. That has shown to be really fast.
>>>> The
>>>>> > point
>>>>> > is several hundred variables that go into CRS afterwards. If you run
>>>> CRS
>>>>> > on a
>>>>> > standard web application you get forms with a few parameters and
>>>> that's
>>>>> > easy.
>>>>> > But several megabytes of JSON means hundreds of arguments and CRS
>>>> parses
>>>>> > them
>>>>> > all.
>>>>> >
>>>>> > So we tried to work with rule exclusions and skip the parameters we
>>>> did not
>>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
>>>> substantially
>>>>> > slower the longer the ignore-lists of parameters became. This and a
>>>> few
>>>>> > very
>>>>> > odd behaviors.
>>>>> >
>>>>> > Given the customer wanted a generic WAF without tuning of individual
>>>> APIs
>>>>> > we
>>>>> > got to a dead end.
>>>>> >
>>>>> > However, if tuning was an option, then I would probably edit-CRS with
>>>>> > msc_pyparser and replace the target lists with arguments I was
>>>> interested
>>>>> > in.
>>>>> >
>>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
>>>>> >
>>>>> > As a complementary practice, one could think of performing allowlist
>>>>> > checks on
>>>>> > some / most of the JSON. Say you have a huge JSON payload with 500
>>>>> > parameters.
>>>>> > You examine it and discover that 300 of them actually contain simple
>>>> digits
>>>>> > and asciii characters and neither special chars nor escape sequences.
>>>>> > So you do a regex allowlist and apply it to these 300 parameters of
>>>> said
>>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
>>>>> >
>>>>> > I have not done this and the problem is if ModSec is able to handle
>>>> the
>>>>> > large
>>>>> > target lists in a speedy manner.
>>>>> >
>>>>> >
>>>>> > Now you can turn to a CDN or alternative WAF. I would do an extensive
>>>>> > security
>>>>> > tests of such a system. As I said, the JSON parser can be really
>>>> fast. The
>>>>> > difficult thing is to check several hundred parameters without losing
>>>>> > performance.
>>>>> >
>>>>> > Good luck!
>>>>> >
>>>>> > Christian
>>>>> >
>>>>> >
>>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>>>>> > > Hi all,
>>>>> > >
>>>>> > > I'm in a situation where the only solution seems to be to drop
>>>> modsec/CRS
>>>>> > > and look at something like Cloudflare's WAF (and change our security
>>>>> > model
>>>>> > > out of necessity). I'm hoping the esteemed membership of this list
>>>> might
>>>>> > > have some thoughts.
>>>>> > >
>>>>> > > I've got about 1MB of JSON, payloads in our app might run to 20 or
>>>> even
>>>>> > > 30MB ultimately.
>>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>>>> seconds
>>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
>>>>> > >
>>>>> > > It takes 1 second to process in our API so the WAF element is a 39x
>>>> slow
>>>>> > > down. I appreciate there'll be some delays in WAF. Cloudflare's WAF
>>>>> > takes 5
>>>>> > > seconds to scan this payload - and that's my target.
>>>>> > >
>>>>> > > Has anyone got any idea how to improve performance? Reading blog
>>>> posts
>>>>> > > about the development of cloudflare's waf I see that memoization of
>>>>> > common
>>>>> > > function calls was one of their absolute best performance
>>>> improvements
>>>>> > over
>>>>> > > their modsec implementation (e.g. strlen(response_body) so it's only
>>>>> > > calculated once instead of once per rule OR contains('somestring',
>>>>> > > response_body)... you get the drift). Do we have anything like this
>>>> in
>>>>> > > modsec today? Is that already in place and my 39 seconds is after
>>>> that?
>>>>> > >
>>>>> > > I appreciate that mod sec is fast on its own and adding complex
>>>> rules can
>>>>> > > be said to slow it down. With CRS being by far the most common use
>>>> case
>>>>> > for
>>>>> > > mod sec (based on my googling) I'm surprised it's this slow, do you
>>>> think
>>>>> > > i've missed something?
>>>>> > >
>>>>> > > To note: I'm only scanning JSON payloads, typically much less than
>>>> 0.5MB
>>>>> > > but new, irregular ones that we need scanned in ideally <10 seconds
>>>> that
>>>>> > > can range from 1MB-30MB
>>>>> > >
>>>>> > > Best regards,
>>>>> > >
>>>>> > > Henri Cook
>>>>> >
>>>>> >
>>>>> > > _______________________________________________
>>>>> > > mod-security-users mailing list
>>>>> > > mod...@li...
>>>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
>>>> SpiderLabs:
>>>>> > > http://www.modsecurity.org/projects/commercial/rules/
>>>>> > > http://www.modsecurity.org/projects/commercial/support/
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > mod-security-users mailing list
>>>>> > mod...@li...
>>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>>> > http://www.modsecurity.org/projects/commercial/rules/
>>>>> > http://www.modsecurity.org/projects/commercial/support/
>>>>> >
>>>>
>>>>
>>>>> _______________________________________________
>>>>> mod-security-users mailing list
>>>>> mod...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>>> http://www.modsecurity.org/projects/commercial/support/
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mod-security-users mailing list
>>>> mod...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>> http://www.modsecurity.org/projects/commercial/support/
>>>>
>>>
>
>
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-05-10 13:13:52

Thanks for this azurit but neither of these worked for me. I changed the id
to 1001 and put it in my REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf - I
still see a delay processing a large json payload - do you have any idea
where I might have gone wrong?

I think my goal is to stop the JSON request body being parsed if it's over
128kb in size. An advance goal would be to have it scanned as text to
provide *some* level of protection - I think this is what cloudflare does
(from a very cursory glance)

Best Regards,

Henri

On Thu, 29 Apr 2021 at 19:02, <az...@po...> wrote:

> I just noticed you are using CRS, so this one would be probably better:
>
>
> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>      "id:9999999,\
>      phase:2,\
>      pass,\
>      t:none,\
>      nolog,\
>      ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"
>
>
>
>
>
> Citát az...@po...:
>
> > Hi,
> >
> > you can do this using exclusive rule, something like this (set the
> > rule ID to something 'better'), length is in bytes:
> >
> >
> > SecRule REQUEST_BODY_LENGTH "@gt 131072" \
> >     "id:9999999,\
> >     phase:2,\
> >     pass,\
> >     t:none,\
> >     nolog,\
> >     ctl:ruleEngine=Off"
> >
> >
> >
> > REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this
> > into phase 1.
> >
> >
> >
> > Citát Henri Cook <henri@proteus.tech>:
> >
> >> I think what I need here is a way to exempt the request body from any
> >> scanning when it's over 128kb (my chosen upper limit)...
> >>
> >> I'll probably try implementing this as a phase 1 rule on Content-Length
> >> unless anyone shouts there's a better way. `ProcessPartial` sounded like
> >> the holy grail, but you can't partial process JSON (because it doesn't
> >> parse!).
> >>
> >> It'd be cool to be able to fallback to a fulltext scan of the partial
> JSON
> >> if it's >128kb, I might have a try at looking into that also (again any
> >> guidance appreciated)
> >>
> >> Best Regards,
> >>
> >> Henri
> >>
> >> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
> >>
> >>> The follow up problem to this is:
> >>>
> >>> Now i'm set to `SecRequestBodyLimit` 31457280 and
> >>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
> >>> PartialProcess`
> >>>
> >>> In my mind this will process the first part of a request if it can, and
> >>> ignore the rest. But:
> >>>
> >>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
> >>> Presumably because in a large request it tries to process the
> beginning of
> >>> the JSON and can't (because it won't parse, because the JSON is cut
> off so
> >>> doesn't end)
> >>>
> >>> So I think I need to find a way to skip JSON parsing entirely when the
> >>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is
> the
> >>> limit I want to stick with. I hadn't really considered before this
> point
> >>> that 'partial processing' of JSON was likely to be hairy but of course
> it
> >>> makes sense.
> >>>
> >>>
> >>>
> >>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
> >>> chr...@ne...> wrote:
> >>>
> >>>> Hey Henri,
> >>>>
> >>>> From a security practice, this is obviously lacking, but in wider
> >>>> perspective,
> >>>> I see it meet "industry standard", yes.
> >>>>
> >>>> When I teach, I tell my student, that the worst WAF is the one that is
> >>>> switched off. So if you need to compromise and you can only apply 20%
> of
> >>>> the rules because you run the risk of business demanding it's switched
> >>>> off,
> >>>> then that 20% WAF is still better than no WAF.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Christian
> >>>>
> >>>>
> >>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> >>>>> Thanks Christian, taking this in combination with Osama's point
> earlier
> >>>> in
> >>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem
> to
> >>>>> limit the payload they'll scan. From my reading to 128kb
> >>>> (cloudflare+azure)
> >>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular
> issue.
> >>>>>
> >>>>> I believe my modern application is already very robust in terms of
> >>>> defence
> >>>>> against sql injection as well as other OWASP top 10 attack vectors
> and
> >>>> that
> >>>>> a WAF primarily adds reassurance (for the business and clients who
> ask
> >>>> if I
> >>>>> have one) and minor frustration (for any potential attacker) layer.
> The
> >>>>> spec is to add a WAF that meets (but notably does not necessarily
> have
> >>>> to
> >>>>> exceed) industry standards. I believe this means that I can switch
> >>>> modsec
> >>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> >>>>> ProcessPartial' - allowing through unscanned any payloads over those
> >>>> sizes)
> >>>>> and be able to say I've got scan-size-policy-parity with an AWS or a
> >>>>> Cloudflare which means it is "industry standard".
> >>>>>
> >>>>> Please let me know if you think that's mad and thanks again
> >>>>>
> >>>>> Best Regards,
> >>>>>
> >>>>> Henri
> >>>>>
> >>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
> >>>> chr...@ne...>
> >>>>> wrote:
> >>>>>
> >>>>> > Hey Henri,
> >>>>> >
> >>>>> > You are in a bad situation and as far as I can see you are right,
> you
> >>>> might
> >>>>> > have to drop modsec/CRS in this situation.
> >>>>> >
> >>>>> > I've had a customer with a similar problem and we did a deep dive
> >>>>> > investigation and I had to strike colors in the end.
> >>>>> >
> >>>>> > The point is not the JSON parser. That has shown to be really fast.
> >>>> The
> >>>>> > point
> >>>>> > is several hundred variables that go into CRS afterwards. If you
> run
> >>>> CRS
> >>>>> > on a
> >>>>> > standard web application you get forms with a few parameters and
> >>>> that's
> >>>>> > easy.
> >>>>> > But several megabytes of JSON means hundreds of arguments and CRS
> >>>> parses
> >>>>> > them
> >>>>> > all.
> >>>>> >
> >>>>> > So we tried to work with rule exclusions and skip the parameters we
> >>>> did not
> >>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
> >>>> substantially
> >>>>> > slower the longer the ignore-lists of parameters became. This and a
> >>>> few
> >>>>> > very
> >>>>> > odd behaviors.
> >>>>> >
> >>>>> > Given the customer wanted a generic WAF without tuning of
> individual
> >>>> APIs
> >>>>> > we
> >>>>> > got to a dead end.
> >>>>> >
> >>>>> > However, if tuning was an option, then I would probably edit-CRS
> with
> >>>>> > msc_pyparser and replace the target lists with arguments I was
> >>>> interested
> >>>>> > in.
> >>>>> >
> >>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
> >>>>> >
> >>>>> > As a complementary practice, one could think of performing
> allowlist
> >>>>> > checks on
> >>>>> > some / most of the JSON. Say you have a huge JSON payload with 500
> >>>>> > parameters.
> >>>>> > You examine it and discover that 300 of them actually contain
> simple
> >>>> digits
> >>>>> > and asciii characters and neither special chars nor escape
> sequences.
> >>>>> > So you do a regex allowlist and apply it to these 300 parameters of
> >>>> said
> >>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
> >>>>> >
> >>>>> > I have not done this and the problem is if ModSec is able to handle
> >>>> the
> >>>>> > large
> >>>>> > target lists in a speedy manner.
> >>>>> >
> >>>>> >
> >>>>> > Now you can turn to a CDN or alternative WAF. I would do an
> extensive
> >>>>> > security
> >>>>> > tests of such a system. As I said, the JSON parser can be really
> >>>> fast. The
> >>>>> > difficult thing is to check several hundred parameters without
> losing
> >>>>> > performance.
> >>>>> >
> >>>>> > Good luck!
> >>>>> >
> >>>>> > Christian
> >>>>> >
> >>>>> >
> >>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> >>>>> > > Hi all,
> >>>>> > >
> >>>>> > > I'm in a situation where the only solution seems to be to drop
> >>>> modsec/CRS
> >>>>> > > and look at something like Cloudflare's WAF (and change our
> security
> >>>>> > model
> >>>>> > > out of necessity). I'm hoping the esteemed membership of this
> list
> >>>> might
> >>>>> > > have some thoughts.
> >>>>> > >
> >>>>> > > I've got about 1MB of JSON, payloads in our app might run to 20
> or
> >>>> even
> >>>>> > > 30MB ultimately.
> >>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
> >>>> seconds
> >>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
> >>>>> > >
> >>>>> > > It takes 1 second to process in our API so the WAF element is a
> 39x
> >>>> slow
> >>>>> > > down. I appreciate there'll be some delays in WAF. Cloudflare's
> WAF
> >>>>> > takes 5
> >>>>> > > seconds to scan this payload - and that's my target.
> >>>>> > >
> >>>>> > > Has anyone got any idea how to improve performance? Reading blog
> >>>> posts
> >>>>> > > about the development of cloudflare's waf I see that memoization
> of
> >>>>> > common
> >>>>> > > function calls was one of their absolute best performance
> >>>> improvements
> >>>>> > over
> >>>>> > > their modsec implementation (e.g. strlen(response_body) so it's
> only
> >>>>> > > calculated once instead of once per rule OR
> contains('somestring',
> >>>>> > > response_body)... you get the drift). Do we have anything like
> this
> >>>> in
> >>>>> > > modsec today? Is that already in place and my 39 seconds is after
> >>>> that?
> >>>>> > >
> >>>>> > > I appreciate that mod sec is fast on its own and adding complex
> >>>> rules can
> >>>>> > > be said to slow it down. With CRS being by far the most common
> use
> >>>> case
> >>>>> > for
> >>>>> > > mod sec (based on my googling) I'm surprised it's this slow, do
> you
> >>>> think
> >>>>> > > i've missed something?
> >>>>> > >
> >>>>> > > To note: I'm only scanning JSON payloads, typically much less
> than
> >>>> 0.5MB
> >>>>> > > but new, irregular ones that we need scanned in ideally <10
> seconds
> >>>> that
> >>>>> > > can range from 1MB-30MB
> >>>>> > >
> >>>>> > > Best regards,
> >>>>> > >
> >>>>> > > Henri Cook
> >>>>> >
> >>>>> >
> >>>>> > > _______________________________________________
> >>>>> > > mod-security-users mailing list
> >>>>> > > mod...@li...
> >>>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
> >>>> SpiderLabs:
> >>>>> > > http://www.modsecurity.org/projects/commercial/rules/
> >>>>> > > http://www.modsecurity.org/projects/commercial/support/
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > _______________________________________________
> >>>>> > mod-security-users mailing list
> >>>>> > mod...@li...
> >>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>> > Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >>>>> > http://www.modsecurity.org/projects/commercial/rules/
> >>>>> > http://www.modsecurity.org/projects/commercial/support/
> >>>>> >
> >>>>
> >>>>
> >>>>> _______________________________________________
> >>>>> mod-security-users mailing list
> >>>>> mod...@li...
> >>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> >>>>> http://www.modsecurity.org/projects/commercial/rules/
> >>>>> http://www.modsecurity.org/projects/commercial/support/
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> mod-security-users mailing list
> >>>> mod...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> >>>> http://www.modsecurity.org/projects/commercial/rules/
> >>>> http://www.modsecurity.org/projects/commercial/support/
> >>>>
> >>>
> >
> >
> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
>
>
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Srikanth A. <sri...@go...> - 2021-05-10 13:28:33

Hi Henri

It defintly worked for me.
SecRequestBodyLimit 13107200
SecRequestBodyNoFileaLimit 131072

Restricts to 12.5 mb of large payload.

The only thing is if i had injections through the end, it passes through.
In your case i think you are fine with it.

And of course SecRequestBodyAccess On
Should be set

Thanks
Srikanth Arunachalam

On Mon, 10 May 2021, 14:17 Henri Cook, <henri@proteus.tech> wrote:

> Thanks for this azurit but neither of these worked for me. I changed the
> id to 1001 and put it in my REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf - I
> still see a delay processing a large json payload - do you have any idea
> where I might have gone wrong?
>
> I think my goal is to stop the JSON request body being parsed if it's over
> 128kb in size. An advance goal would be to have it scanned as text to
> provide *some* level of protection - I think this is what cloudflare does
> (from a very cursory glance)
>
> Best Regards,
>
> Henri
>
> On Thu, 29 Apr 2021 at 19:02, <az...@po...> wrote:
>
>> I just noticed you are using CRS, so this one would be probably better:
>>
>>
>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>>      "id:9999999,\
>>      phase:2,\
>>      pass,\
>>      t:none,\
>>      nolog,\
>>      ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"
>>
>>
>>
>>
>>
>> Citát az...@po...:
>>
>> > Hi,
>> >
>> > you can do this using exclusive rule, something like this (set the
>> > rule ID to something 'better'), length is in bytes:
>> >
>> >
>> > SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>> >     "id:9999999,\
>> >     phase:2,\
>> >     pass,\
>> >     t:none,\
>> >     nolog,\
>> >     ctl:ruleEngine=Off"
>> >
>> >
>> >
>> > REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this
>> > into phase 1.
>> >
>> >
>> >
>> > Citát Henri Cook <henri@proteus.tech>:
>> >
>> >> I think what I need here is a way to exempt the request body from any
>> >> scanning when it's over 128kb (my chosen upper limit)...
>> >>
>> >> I'll probably try implementing this as a phase 1 rule on Content-Length
>> >> unless anyone shouts there's a better way. `ProcessPartial` sounded
>> like
>> >> the holy grail, but you can't partial process JSON (because it doesn't
>> >> parse!).
>> >>
>> >> It'd be cool to be able to fallback to a fulltext scan of the partial
>> JSON
>> >> if it's >128kb, I might have a try at looking into that also (again any
>> >> guidance appreciated)
>> >>
>> >> Best Regards,
>> >>
>> >> Henri
>> >>
>> >> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
>> >>
>> >>> The follow up problem to this is:
>> >>>
>> >>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>> >>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>> >>> PartialProcess`
>> >>>
>> >>> In my mind this will process the first part of a request if it can,
>> and
>> >>> ignore the rest. But:
>> >>>
>> >>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>> >>> Presumably because in a large request it tries to process the
>> beginning of
>> >>> the JSON and can't (because it won't parse, because the JSON is cut
>> off so
>> >>> doesn't end)
>> >>>
>> >>> So I think I need to find a way to skip JSON parsing entirely when the
>> >>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is
>> the
>> >>> limit I want to stick with. I hadn't really considered before this
>> point
>> >>> that 'partial processing' of JSON was likely to be hairy but of
>> course it
>> >>> makes sense.
>> >>>
>> >>>
>> >>>
>> >>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>> >>> chr...@ne...> wrote:
>> >>>
>> >>>> Hey Henri,
>> >>>>
>> >>>> From a security practice, this is obviously lacking, but in wider
>> >>>> perspective,
>> >>>> I see it meet "industry standard", yes.
>> >>>>
>> >>>> When I teach, I tell my student, that the worst WAF is the one that
>> is
>> >>>> switched off. So if you need to compromise and you can only apply
>> 20% of
>> >>>> the rules because you run the risk of business demanding it's
>> switched
>> >>>> off,
>> >>>> then that 20% WAF is still better than no WAF.
>> >>>>
>> >>>> Cheers,
>> >>>>
>> >>>> Christian
>> >>>>
>> >>>>
>> >>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>> >>>>> Thanks Christian, taking this in combination with Osama's point
>> earlier
>> >>>> in
>> >>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs
>> seem to
>> >>>>> limit the payload they'll scan. From my reading to 128kb
>> >>>> (cloudflare+azure)
>> >>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular
>> issue.
>> >>>>>
>> >>>>> I believe my modern application is already very robust in terms of
>> >>>> defence
>> >>>>> against sql injection as well as other OWASP top 10 attack vectors
>> and
>> >>>> that
>> >>>>> a WAF primarily adds reassurance (for the business and clients who
>> ask
>> >>>> if I
>> >>>>> have one) and minor frustration (for any potential attacker) layer.
>> The
>> >>>>> spec is to add a WAF that meets (but notably does not necessarily
>> have
>> >>>> to
>> >>>>> exceed) industry standards. I believe this means that I can switch
>> >>>> modsec
>> >>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>> >>>>> ProcessPartial' - allowing through unscanned any payloads over those
>> >>>> sizes)
>> >>>>> and be able to say I've got scan-size-policy-parity with an AWS or a
>> >>>>> Cloudflare which means it is "industry standard".
>> >>>>>
>> >>>>> Please let me know if you think that's mad and thanks again
>> >>>>>
>> >>>>> Best Regards,
>> >>>>>
>> >>>>> Henri
>> >>>>>
>> >>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>> >>>> chr...@ne...>
>> >>>>> wrote:
>> >>>>>
>> >>>>> > Hey Henri,
>> >>>>> >
>> >>>>> > You are in a bad situation and as far as I can see you are right,
>> you
>> >>>> might
>> >>>>> > have to drop modsec/CRS in this situation.
>> >>>>> >
>> >>>>> > I've had a customer with a similar problem and we did a deep dive
>> >>>>> > investigation and I had to strike colors in the end.
>> >>>>> >
>> >>>>> > The point is not the JSON parser. That has shown to be really
>> fast.
>> >>>> The
>> >>>>> > point
>> >>>>> > is several hundred variables that go into CRS afterwards. If you
>> run
>> >>>> CRS
>> >>>>> > on a
>> >>>>> > standard web application you get forms with a few parameters and
>> >>>> that's
>> >>>>> > easy.
>> >>>>> > But several megabytes of JSON means hundreds of arguments and CRS
>> >>>> parses
>> >>>>> > them
>> >>>>> > all.
>> >>>>> >
>> >>>>> > So we tried to work with rule exclusions and skip the parameters
>> we
>> >>>> did not
>> >>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
>> >>>> substantially
>> >>>>> > slower the longer the ignore-lists of parameters became. This and
>> a
>> >>>> few
>> >>>>> > very
>> >>>>> > odd behaviors.
>> >>>>> >
>> >>>>> > Given the customer wanted a generic WAF without tuning of
>> individual
>> >>>> APIs
>> >>>>> > we
>> >>>>> > got to a dead end.
>> >>>>> >
>> >>>>> > However, if tuning was an option, then I would probably edit-CRS
>> with
>> >>>>> > msc_pyparser and replace the target lists with arguments I was
>> >>>> interested
>> >>>>> > in.
>> >>>>> >
>> >>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
>> >>>>> >
>> >>>>> > As a complementary practice, one could think of performing
>> allowlist
>> >>>>> > checks on
>> >>>>> > some / most of the JSON. Say you have a huge JSON payload with 500
>> >>>>> > parameters.
>> >>>>> > You examine it and discover that 300 of them actually contain
>> simple
>> >>>> digits
>> >>>>> > and asciii characters and neither special chars nor escape
>> sequences.
>> >>>>> > So you do a regex allowlist and apply it to these 300 parameters
>> of
>> >>>> said
>> >>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
>> >>>>> >
>> >>>>> > I have not done this and the problem is if ModSec is able to
>> handle
>> >>>> the
>> >>>>> > large
>> >>>>> > target lists in a speedy manner.
>> >>>>> >
>> >>>>> >
>> >>>>> > Now you can turn to a CDN or alternative WAF. I would do an
>> extensive
>> >>>>> > security
>> >>>>> > tests of such a system. As I said, the JSON parser can be really
>> >>>> fast. The
>> >>>>> > difficult thing is to check several hundred parameters without
>> losing
>> >>>>> > performance.
>> >>>>> >
>> >>>>> > Good luck!
>> >>>>> >
>> >>>>> > Christian
>> >>>>> >
>> >>>>> >
>> >>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>> >>>>> > > Hi all,
>> >>>>> > >
>> >>>>> > > I'm in a situation where the only solution seems to be to drop
>> >>>> modsec/CRS
>> >>>>> > > and look at something like Cloudflare's WAF (and change our
>> security
>> >>>>> > model
>> >>>>> > > out of necessity). I'm hoping the esteemed membership of this
>> list
>> >>>> might
>> >>>>> > > have some thoughts.
>> >>>>> > >
>> >>>>> > > I've got about 1MB of JSON, payloads in our app might run to 20
>> or
>> >>>> even
>> >>>>> > > 30MB ultimately.
>> >>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take
>> 40
>> >>>> seconds
>> >>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
>> >>>>> > >
>> >>>>> > > It takes 1 second to process in our API so the WAF element is a
>> 39x
>> >>>> slow
>> >>>>> > > down. I appreciate there'll be some delays in WAF. Cloudflare's
>> WAF
>> >>>>> > takes 5
>> >>>>> > > seconds to scan this payload - and that's my target.
>> >>>>> > >
>> >>>>> > > Has anyone got any idea how to improve performance? Reading blog
>> >>>> posts
>> >>>>> > > about the development of cloudflare's waf I see that
>> memoization of
>> >>>>> > common
>> >>>>> > > function calls was one of their absolute best performance
>> >>>> improvements
>> >>>>> > over
>> >>>>> > > their modsec implementation (e.g. strlen(response_body) so it's
>> only
>> >>>>> > > calculated once instead of once per rule OR
>> contains('somestring',
>> >>>>> > > response_body)... you get the drift). Do we have anything like
>> this
>> >>>> in
>> >>>>> > > modsec today? Is that already in place and my 39 seconds is
>> after
>> >>>> that?
>> >>>>> > >
>> >>>>> > > I appreciate that mod sec is fast on its own and adding complex
>> >>>> rules can
>> >>>>> > > be said to slow it down. With CRS being by far the most common
>> use
>> >>>> case
>> >>>>> > for
>> >>>>> > > mod sec (based on my googling) I'm surprised it's this slow, do
>> you
>> >>>> think
>> >>>>> > > i've missed something?
>> >>>>> > >
>> >>>>> > > To note: I'm only scanning JSON payloads, typically much less
>> than
>> >>>> 0.5MB
>> >>>>> > > but new, irregular ones that we need scanned in ideally <10
>> seconds
>> >>>> that
>> >>>>> > > can range from 1MB-30MB
>> >>>>> > >
>> >>>>> > > Best regards,
>> >>>>> > >
>> >>>>> > > Henri Cook
>> >>>>> >
>> >>>>> >
>> >>>>> > > _______________________________________________
>> >>>>> > > mod-security-users mailing list
>> >>>>> > > mod...@li...
>> >>>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
>> >>>> SpiderLabs:
>> >>>>> > > http://www.modsecurity.org/projects/commercial/rules/
>> >>>>> > > http://www.modsecurity.org/projects/commercial/support/
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > _______________________________________________
>> >>>>> > mod-security-users mailing list
>> >>>>> > mod...@li...
>> >>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>>> > Commercial ModSecurity Rules and Support from Trustwave's
>> SpiderLabs:
>> >>>>> > http://www.modsecurity.org/projects/commercial/rules/
>> >>>>> > http://www.modsecurity.org/projects/commercial/support/
>> >>>>> >
>> >>>>
>> >>>>
>> >>>>> _______________________________________________
>> >>>>> mod-security-users mailing list
>> >>>>> mod...@li...
>> >>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>>> Commercial ModSecurity Rules and Support from Trustwave's
>> SpiderLabs:
>> >>>>> http://www.modsecurity.org/projects/commercial/rules/
>> >>>>> http://www.modsecurity.org/projects/commercial/support/
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> mod-security-users mailing list
>> >>>> mod...@li...
>> >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> >>>> http://www.modsecurity.org/projects/commercial/rules/
>> >>>> http://www.modsecurity.org/projects/commercial/support/
>> >>>>
>> >>>
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > mod-security-users mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > http://www.modsecurity.org/projects/commercial/rules/
>> > http://www.modsecurity.org/projects/commercial/support/
>>
>>
>>
>>
>>
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-05-10 13:38:41

Thanks Srikanth. In my case I want to allow large payloads through
unscanned (or partially scanned) rather than rejecting them outright. My
issue is that scanning my large JSON payloads takes some time. Since
they're rare I'm willing to not scan them in the name of balancing security
and performance, but I'd still rather scan payloads that are <128k in size

On Mon, 10 May 2021 at 14:30, Srikanth Arunachalam via mod-security-users <
mod...@li...> wrote:

> Hi Henri
>
> It defintly worked for me.
> SecRequestBodyLimit 13107200
> SecRequestBodyNoFileaLimit 131072
>
> Restricts to 12.5 mb of large payload.
>
> The only thing is if i had injections through the end, it passes through.
> In your case i think you are fine with it.
>
> And of course SecRequestBodyAccess On
> Should be set
>
> Thanks
> Srikanth Arunachalam
>
> On Mon, 10 May 2021, 14:17 Henri Cook, <henri@proteus.tech> wrote:
>
>> Thanks for this azurit but neither of these worked for me. I changed the
>> id to 1001 and put it in my REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf - I
>> still see a delay processing a large json payload - do you have any idea
>> where I might have gone wrong?
>>
>> I think my goal is to stop the JSON request body being parsed if it's
>> over 128kb in size. An advance goal would be to have it scanned as text to
>> provide *some* level of protection - I think this is what cloudflare
>> does (from a very cursory glance)
>>
>> Best Regards,
>>
>> Henri
>>
>> On Thu, 29 Apr 2021 at 19:02, <az...@po...> wrote:
>>
>>> I just noticed you are using CRS, so this one would be probably better:
>>>
>>>
>>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>>>      "id:9999999,\
>>>      phase:2,\
>>>      pass,\
>>>      t:none,\
>>>      nolog,\
>>>      ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"
>>>
>>>
>>>
>>>
>>>
>>> Citát az...@po...:
>>>
>>> > Hi,
>>> >
>>> > you can do this using exclusive rule, something like this (set the
>>> > rule ID to something 'better'), length is in bytes:
>>> >
>>> >
>>> > SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>>> >     "id:9999999,\
>>> >     phase:2,\
>>> >     pass,\
>>> >     t:none,\
>>> >     nolog,\
>>> >     ctl:ruleEngine=Off"
>>> >
>>> >
>>> >
>>> > REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this
>>> > into phase 1.
>>> >
>>> >
>>> >
>>> > Citát Henri Cook <henri@proteus.tech>:
>>> >
>>> >> I think what I need here is a way to exempt the request body from any
>>> >> scanning when it's over 128kb (my chosen upper limit)...
>>> >>
>>> >> I'll probably try implementing this as a phase 1 rule on
>>> Content-Length
>>> >> unless anyone shouts there's a better way. `ProcessPartial` sounded
>>> like
>>> >> the holy grail, but you can't partial process JSON (because it doesn't
>>> >> parse!).
>>> >>
>>> >> It'd be cool to be able to fallback to a fulltext scan of the partial
>>> JSON
>>> >> if it's >128kb, I might have a try at looking into that also (again
>>> any
>>> >> guidance appreciated)
>>> >>
>>> >> Best Regards,
>>> >>
>>> >> Henri
>>> >>
>>> >> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
>>> >>
>>> >>> The follow up problem to this is:
>>> >>>
>>> >>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>>> >>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>>> >>> PartialProcess`
>>> >>>
>>> >>> In my mind this will process the first part of a request if it can,
>>> and
>>> >>> ignore the rest. But:
>>> >>>
>>> >>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>>> >>> Presumably because in a large request it tries to process the
>>> beginning of
>>> >>> the JSON and can't (because it won't parse, because the JSON is cut
>>> off so
>>> >>> doesn't end)
>>> >>>
>>> >>> So I think I need to find a way to skip JSON parsing entirely when
>>> the
>>> >>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb
>>> is the
>>> >>> limit I want to stick with. I hadn't really considered before this
>>> point
>>> >>> that 'partial processing' of JSON was likely to be hairy but of
>>> course it
>>> >>> makes sense.
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>>> >>> chr...@ne...> wrote:
>>> >>>
>>> >>>> Hey Henri,
>>> >>>>
>>> >>>> From a security practice, this is obviously lacking, but in wider
>>> >>>> perspective,
>>> >>>> I see it meet "industry standard", yes.
>>> >>>>
>>> >>>> When I teach, I tell my student, that the worst WAF is the one that
>>> is
>>> >>>> switched off. So if you need to compromise and you can only apply
>>> 20% of
>>> >>>> the rules because you run the risk of business demanding it's
>>> switched
>>> >>>> off,
>>> >>>> then that 20% WAF is still better than no WAF.
>>> >>>>
>>> >>>> Cheers,
>>> >>>>
>>> >>>> Christian
>>> >>>>
>>> >>>>
>>> >>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>>> >>>>> Thanks Christian, taking this in combination with Osama's point
>>> earlier
>>> >>>> in
>>> >>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs
>>> seem to
>>> >>>>> limit the payload they'll scan. From my reading to 128kb
>>> >>>> (cloudflare+azure)
>>> >>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular
>>> issue.
>>> >>>>>
>>> >>>>> I believe my modern application is already very robust in terms of
>>> >>>> defence
>>> >>>>> against sql injection as well as other OWASP top 10 attack vectors
>>> and
>>> >>>> that
>>> >>>>> a WAF primarily adds reassurance (for the business and clients who
>>> ask
>>> >>>> if I
>>> >>>>> have one) and minor frustration (for any potential attacker)
>>> layer. The
>>> >>>>> spec is to add a WAF that meets (but notably does not necessarily
>>> have
>>> >>>> to
>>> >>>>> exceed) industry standards. I believe this means that I can switch
>>> >>>> modsec
>>> >>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>>> >>>>> ProcessPartial' - allowing through unscanned any payloads over
>>> those
>>> >>>> sizes)
>>> >>>>> and be able to say I've got scan-size-policy-parity with an AWS or
>>> a
>>> >>>>> Cloudflare which means it is "industry standard".
>>> >>>>>
>>> >>>>> Please let me know if you think that's mad and thanks again
>>> >>>>>
>>> >>>>> Best Regards,
>>> >>>>>
>>> >>>>> Henri
>>> >>>>>
>>> >>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>>> >>>> chr...@ne...>
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>> > Hey Henri,
>>> >>>>> >
>>> >>>>> > You are in a bad situation and as far as I can see you are
>>> right, you
>>> >>>> might
>>> >>>>> > have to drop modsec/CRS in this situation.
>>> >>>>> >
>>> >>>>> > I've had a customer with a similar problem and we did a deep dive
>>> >>>>> > investigation and I had to strike colors in the end.
>>> >>>>> >
>>> >>>>> > The point is not the JSON parser. That has shown to be really
>>> fast.
>>> >>>> The
>>> >>>>> > point
>>> >>>>> > is several hundred variables that go into CRS afterwards. If you
>>> run
>>> >>>> CRS
>>> >>>>> > on a
>>> >>>>> > standard web application you get forms with a few parameters and
>>> >>>> that's
>>> >>>>> > easy.
>>> >>>>> > But several megabytes of JSON means hundreds of arguments and CRS
>>> >>>> parses
>>> >>>>> > them
>>> >>>>> > all.
>>> >>>>> >
>>> >>>>> > So we tried to work with rule exclusions and skip the parameters
>>> we
>>> >>>> did not
>>> >>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
>>> >>>> substantially
>>> >>>>> > slower the longer the ignore-lists of parameters became. This
>>> and a
>>> >>>> few
>>> >>>>> > very
>>> >>>>> > odd behaviors.
>>> >>>>> >
>>> >>>>> > Given the customer wanted a generic WAF without tuning of
>>> individual
>>> >>>> APIs
>>> >>>>> > we
>>> >>>>> > got to a dead end.
>>> >>>>> >
>>> >>>>> > However, if tuning was an option, then I would probably edit-CRS
>>> with
>>> >>>>> > msc_pyparser and replace the target lists with arguments I was
>>> >>>> interested
>>> >>>>> > in.
>>> >>>>> >
>>> >>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
>>> >>>>> >
>>> >>>>> > As a complementary practice, one could think of performing
>>> allowlist
>>> >>>>> > checks on
>>> >>>>> > some / most of the JSON. Say you have a huge JSON payload with
>>> 500
>>> >>>>> > parameters.
>>> >>>>> > You examine it and discover that 300 of them actually contain
>>> simple
>>> >>>> digits
>>> >>>>> > and asciii characters and neither special chars nor escape
>>> sequences.
>>> >>>>> > So you do a regex allowlist and apply it to these 300 parameters
>>> of
>>> >>>> said
>>> >>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
>>> >>>>> >
>>> >>>>> > I have not done this and the problem is if ModSec is able to
>>> handle
>>> >>>> the
>>> >>>>> > large
>>> >>>>> > target lists in a speedy manner.
>>> >>>>> >
>>> >>>>> >
>>> >>>>> > Now you can turn to a CDN or alternative WAF. I would do an
>>> extensive
>>> >>>>> > security
>>> >>>>> > tests of such a system. As I said, the JSON parser can be really
>>> >>>> fast. The
>>> >>>>> > difficult thing is to check several hundred parameters without
>>> losing
>>> >>>>> > performance.
>>> >>>>> >
>>> >>>>> > Good luck!
>>> >>>>> >
>>> >>>>> > Christian
>>> >>>>> >
>>> >>>>> >
>>> >>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>>> >>>>> > > Hi all,
>>> >>>>> > >
>>> >>>>> > > I'm in a situation where the only solution seems to be to drop
>>> >>>> modsec/CRS
>>> >>>>> > > and look at something like Cloudflare's WAF (and change our
>>> security
>>> >>>>> > model
>>> >>>>> > > out of necessity). I'm hoping the esteemed membership of this
>>> list
>>> >>>> might
>>> >>>>> > > have some thoughts.
>>> >>>>> > >
>>> >>>>> > > I've got about 1MB of JSON, payloads in our app might run to
>>> 20 or
>>> >>>> even
>>> >>>>> > > 30MB ultimately.
>>> >>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take
>>> 40
>>> >>>> seconds
>>> >>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
>>> >>>>> > >
>>> >>>>> > > It takes 1 second to process in our API so the WAF element is
>>> a 39x
>>> >>>> slow
>>> >>>>> > > down. I appreciate there'll be some delays in WAF.
>>> Cloudflare's WAF
>>> >>>>> > takes 5
>>> >>>>> > > seconds to scan this payload - and that's my target.
>>> >>>>> > >
>>> >>>>> > > Has anyone got any idea how to improve performance? Reading
>>> blog
>>> >>>> posts
>>> >>>>> > > about the development of cloudflare's waf I see that
>>> memoization of
>>> >>>>> > common
>>> >>>>> > > function calls was one of their absolute best performance
>>> >>>> improvements
>>> >>>>> > over
>>> >>>>> > > their modsec implementation (e.g. strlen(response_body) so
>>> it's only
>>> >>>>> > > calculated once instead of once per rule OR
>>> contains('somestring',
>>> >>>>> > > response_body)... you get the drift). Do we have anything like
>>> this
>>> >>>> in
>>> >>>>> > > modsec today? Is that already in place and my 39 seconds is
>>> after
>>> >>>> that?
>>> >>>>> > >
>>> >>>>> > > I appreciate that mod sec is fast on its own and adding complex
>>> >>>> rules can
>>> >>>>> > > be said to slow it down. With CRS being by far the most common
>>> use
>>> >>>> case
>>> >>>>> > for
>>> >>>>> > > mod sec (based on my googling) I'm surprised it's this slow,
>>> do you
>>> >>>> think
>>> >>>>> > > i've missed something?
>>> >>>>> > >
>>> >>>>> > > To note: I'm only scanning JSON payloads, typically much less
>>> than
>>> >>>> 0.5MB
>>> >>>>> > > but new, irregular ones that we need scanned in ideally <10
>>> seconds
>>> >>>> that
>>> >>>>> > > can range from 1MB-30MB
>>> >>>>> > >
>>> >>>>> > > Best regards,
>>> >>>>> > >
>>> >>>>> > > Henri Cook
>>> >>>>> >
>>> >>>>> >
>>> >>>>> > > _______________________________________________
>>> >>>>> > > mod-security-users mailing list
>>> >>>>> > > mod...@li...
>>> >>>>> > >
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> >>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
>>> >>>> SpiderLabs:
>>> >>>>> > > http://www.modsecurity.org/projects/commercial/rules/
>>> >>>>> > > http://www.modsecurity.org/projects/commercial/support/
>>> >>>>> >
>>> >>>>> >
>>> >>>>> >
>>> >>>>> > _______________________________________________
>>> >>>>> > mod-security-users mailing list
>>> >>>>> > mod...@li...
>>> >>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> >>>>> > Commercial ModSecurity Rules and Support from Trustwave's
>>> SpiderLabs:
>>> >>>>> > http://www.modsecurity.org/projects/commercial/rules/
>>> >>>>> > http://www.modsecurity.org/projects/commercial/support/
>>> >>>>> >
>>> >>>>
>>> >>>>
>>> >>>>> _______________________________________________
>>> >>>>> mod-security-users mailing list
>>> >>>>> mod...@li...
>>> >>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> >>>>> Commercial ModSecurity Rules and Support from Trustwave's
>>> SpiderLabs:
>>> >>>>> http://www.modsecurity.org/projects/commercial/rules/
>>> >>>>> http://www.modsecurity.org/projects/commercial/support/
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> mod-security-users mailing list
>>> >>>> mod...@li...
>>> >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> >>>> Commercial ModSecurity Rules and Support from Trustwave's
>>> SpiderLabs:
>>> >>>> http://www.modsecurity.org/projects/commercial/rules/
>>> >>>> http://www.modsecurity.org/projects/commercial/support/
>>> >>>>
>>> >>>
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > mod-security-users mailing list
>>> > mod...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > http://www.modsecurity.org/projects/commercial/rules/
>>> > http://www.modsecurity.org/projects/commercial/support/
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>>>
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: <az...@po...> - 2021-05-10 13:52:21

Sorry, now I see the problem: Rule which is doing JSON processing  
(200001) is running in phase 1 and my rule below is running in phase  
2. Problem is, as i stated below, REQUEST_BODY_LENGTH is available  
from phase 2 so my rule cannot be run in phase 1. Also, i noticed  
another problem - we cannot do 'ruleRemoveTargetByTag=OWASP_CRS' as  
rule 200001 is not part of the CRS and it's running before CRS (and  
also before rules in file REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf).

In this case, we can use Content-Length header, which won't be so  
realibe but should be sufficient. Try this:

SecRule REQUEST_HEADERS:Content-Type "application/json" \
     "id:9999999,\
     phase:1,\
     pass,\
     t:none,\
     nolog,\
     chain"
     SecRule REQUEST_HEADERS:Content-Length "@gt 131072" \
         "t:none,\
         ctl:ruleRemoveTargetById=200001;REQUEST_BODY"


This rule should be run before rule 200001 so you will, probably, need  
to put it inside modsecurity.conf before rule 200001 (but i didn't  
tested it at all).





Citát Henri Cook <henri@proteus.tech>:

> Thanks for this azurit but neither of these worked for me. I changed the id
> to 1001 and put it in my REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf - I
> still see a delay processing a large json payload - do you have any idea
> where I might have gone wrong?
>
> I think my goal is to stop the JSON request body being parsed if it's over
> 128kb in size. An advance goal would be to have it scanned as text to
> provide *some* level of protection - I think this is what cloudflare does
> (from a very cursory glance)
>
> Best Regards,
>
> Henri
>
> On Thu, 29 Apr 2021 at 19:02, <az...@po...> wrote:
>
>> I just noticed you are using CRS, so this one would be probably better:
>>
>>
>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>>      "id:9999999,\
>>      phase:2,\
>>      pass,\
>>      t:none,\
>>      nolog,\
>>      ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"
>>
>>
>>
>>
>>
>> Citát az...@po...:
>>
>> > Hi,
>> >
>> > you can do this using exclusive rule, something like this (set the
>> > rule ID to something 'better'), length is in bytes:
>> >
>> >
>> > SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>> >     "id:9999999,\
>> >     phase:2,\
>> >     pass,\
>> >     t:none,\
>> >     nolog,\
>> >     ctl:ruleEngine=Off"
>> >
>> >
>> >
>> > REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this
>> > into phase 1.
>> >
>> >
>> >
>> > Citát Henri Cook <henri@proteus.tech>:
>> >
>> >> I think what I need here is a way to exempt the request body from any
>> >> scanning when it's over 128kb (my chosen upper limit)...
>> >>
>> >> I'll probably try implementing this as a phase 1 rule on Content-Length
>> >> unless anyone shouts there's a better way. `ProcessPartial` sounded like
>> >> the holy grail, but you can't partial process JSON (because it doesn't
>> >> parse!).
>> >>
>> >> It'd be cool to be able to fallback to a fulltext scan of the partial
>> JSON
>> >> if it's >128kb, I might have a try at looking into that also (again any
>> >> guidance appreciated)
>> >>
>> >> Best Regards,
>> >>
>> >> Henri
>> >>
>> >> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
>> >>
>> >>> The follow up problem to this is:
>> >>>
>> >>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>> >>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>> >>> PartialProcess`
>> >>>
>> >>> In my mind this will process the first part of a request if it can, and
>> >>> ignore the rest. But:
>> >>>
>> >>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>> >>> Presumably because in a large request it tries to process the
>> beginning of
>> >>> the JSON and can't (because it won't parse, because the JSON is cut
>> off so
>> >>> doesn't end)
>> >>>
>> >>> So I think I need to find a way to skip JSON parsing entirely when the
>> >>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is
>> the
>> >>> limit I want to stick with. I hadn't really considered before this
>> point
>> >>> that 'partial processing' of JSON was likely to be hairy but of course
>> it
>> >>> makes sense.
>> >>>
>> >>>
>> >>>
>> >>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>> >>> chr...@ne...> wrote:
>> >>>
>> >>>> Hey Henri,
>> >>>>
>> >>>> From a security practice, this is obviously lacking, but in wider
>> >>>> perspective,
>> >>>> I see it meet "industry standard", yes.
>> >>>>
>> >>>> When I teach, I tell my student, that the worst WAF is the one that is
>> >>>> switched off. So if you need to compromise and you can only apply 20%
>> of
>> >>>> the rules because you run the risk of business demanding it's switched
>> >>>> off,
>> >>>> then that 20% WAF is still better than no WAF.
>> >>>>
>> >>>> Cheers,
>> >>>>
>> >>>> Christian
>> >>>>
>> >>>>
>> >>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>> >>>>> Thanks Christian, taking this in combination with Osama's point
>> earlier
>> >>>> in
>> >>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem
>> to
>> >>>>> limit the payload they'll scan. From my reading to 128kb
>> >>>> (cloudflare+azure)
>> >>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular
>> issue.
>> >>>>>
>> >>>>> I believe my modern application is already very robust in terms of
>> >>>> defence
>> >>>>> against sql injection as well as other OWASP top 10 attack vectors
>> and
>> >>>> that
>> >>>>> a WAF primarily adds reassurance (for the business and clients who
>> ask
>> >>>> if I
>> >>>>> have one) and minor frustration (for any potential attacker) layer.
>> The
>> >>>>> spec is to add a WAF that meets (but notably does not necessarily
>> have
>> >>>> to
>> >>>>> exceed) industry standards. I believe this means that I can switch
>> >>>> modsec
>> >>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>> >>>>> ProcessPartial' - allowing through unscanned any payloads over those
>> >>>> sizes)
>> >>>>> and be able to say I've got scan-size-policy-parity with an AWS or a
>> >>>>> Cloudflare which means it is "industry standard".
>> >>>>>
>> >>>>> Please let me know if you think that's mad and thanks again
>> >>>>>
>> >>>>> Best Regards,
>> >>>>>
>> >>>>> Henri
>> >>>>>
>> >>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>> >>>> chr...@ne...>
>> >>>>> wrote:
>> >>>>>
>> >>>>> > Hey Henri,
>> >>>>> >
>> >>>>> > You are in a bad situation and as far as I can see you are right,
>> you
>> >>>> might
>> >>>>> > have to drop modsec/CRS in this situation.
>> >>>>> >
>> >>>>> > I've had a customer with a similar problem and we did a deep dive
>> >>>>> > investigation and I had to strike colors in the end.
>> >>>>> >
>> >>>>> > The point is not the JSON parser. That has shown to be really fast.
>> >>>> The
>> >>>>> > point
>> >>>>> > is several hundred variables that go into CRS afterwards. If you
>> run
>> >>>> CRS
>> >>>>> > on a
>> >>>>> > standard web application you get forms with a few parameters and
>> >>>> that's
>> >>>>> > easy.
>> >>>>> > But several megabytes of JSON means hundreds of arguments and CRS
>> >>>> parses
>> >>>>> > them
>> >>>>> > all.
>> >>>>> >
>> >>>>> > So we tried to work with rule exclusions and skip the parameters we
>> >>>> did not
>> >>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
>> >>>> substantially
>> >>>>> > slower the longer the ignore-lists of parameters became. This and a
>> >>>> few
>> >>>>> > very
>> >>>>> > odd behaviors.
>> >>>>> >
>> >>>>> > Given the customer wanted a generic WAF without tuning of
>> individual
>> >>>> APIs
>> >>>>> > we
>> >>>>> > got to a dead end.
>> >>>>> >
>> >>>>> > However, if tuning was an option, then I would probably edit-CRS
>> with
>> >>>>> > msc_pyparser and replace the target lists with arguments I was
>> >>>> interested
>> >>>>> > in.
>> >>>>> >
>> >>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
>> >>>>> >
>> >>>>> > As a complementary practice, one could think of performing
>> allowlist
>> >>>>> > checks on
>> >>>>> > some / most of the JSON. Say you have a huge JSON payload with 500
>> >>>>> > parameters.
>> >>>>> > You examine it and discover that 300 of them actually contain
>> simple
>> >>>> digits
>> >>>>> > and asciii characters and neither special chars nor escape
>> sequences.
>> >>>>> > So you do a regex allowlist and apply it to these 300 parameters of
>> >>>> said
>> >>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
>> >>>>> >
>> >>>>> > I have not done this and the problem is if ModSec is able to handle
>> >>>> the
>> >>>>> > large
>> >>>>> > target lists in a speedy manner.
>> >>>>> >
>> >>>>> >
>> >>>>> > Now you can turn to a CDN or alternative WAF. I would do an
>> extensive
>> >>>>> > security
>> >>>>> > tests of such a system. As I said, the JSON parser can be really
>> >>>> fast. The
>> >>>>> > difficult thing is to check several hundred parameters without
>> losing
>> >>>>> > performance.
>> >>>>> >
>> >>>>> > Good luck!
>> >>>>> >
>> >>>>> > Christian
>> >>>>> >
>> >>>>> >
>> >>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>> >>>>> > > Hi all,
>> >>>>> > >
>> >>>>> > > I'm in a situation where the only solution seems to be to drop
>> >>>> modsec/CRS
>> >>>>> > > and look at something like Cloudflare's WAF (and change our
>> security
>> >>>>> > model
>> >>>>> > > out of necessity). I'm hoping the esteemed membership of this
>> list
>> >>>> might
>> >>>>> > > have some thoughts.
>> >>>>> > >
>> >>>>> > > I've got about 1MB of JSON, payloads in our app might run to 20
>> or
>> >>>> even
>> >>>>> > > 30MB ultimately.
>> >>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>> >>>> seconds
>> >>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
>> >>>>> > >
>> >>>>> > > It takes 1 second to process in our API so the WAF element is a
>> 39x
>> >>>> slow
>> >>>>> > > down. I appreciate there'll be some delays in WAF. Cloudflare's
>> WAF
>> >>>>> > takes 5
>> >>>>> > > seconds to scan this payload - and that's my target.
>> >>>>> > >
>> >>>>> > > Has anyone got any idea how to improve performance? Reading blog
>> >>>> posts
>> >>>>> > > about the development of cloudflare's waf I see that memoization
>> of
>> >>>>> > common
>> >>>>> > > function calls was one of their absolute best performance
>> >>>> improvements
>> >>>>> > over
>> >>>>> > > their modsec implementation (e.g. strlen(response_body) so it's
>> only
>> >>>>> > > calculated once instead of once per rule OR
>> contains('somestring',
>> >>>>> > > response_body)... you get the drift). Do we have anything like
>> this
>> >>>> in
>> >>>>> > > modsec today? Is that already in place and my 39 seconds is after
>> >>>> that?
>> >>>>> > >
>> >>>>> > > I appreciate that mod sec is fast on its own and adding complex
>> >>>> rules can
>> >>>>> > > be said to slow it down. With CRS being by far the most common
>> use
>> >>>> case
>> >>>>> > for
>> >>>>> > > mod sec (based on my googling) I'm surprised it's this slow, do
>> you
>> >>>> think
>> >>>>> > > i've missed something?
>> >>>>> > >
>> >>>>> > > To note: I'm only scanning JSON payloads, typically much less
>> than
>> >>>> 0.5MB
>> >>>>> > > but new, irregular ones that we need scanned in ideally <10
>> seconds
>> >>>> that
>> >>>>> > > can range from 1MB-30MB
>> >>>>> > >
>> >>>>> > > Best regards,
>> >>>>> > >
>> >>>>> > > Henri Cook
>> >>>>> >
>> >>>>> >
>> >>>>> > > _______________________________________________
>> >>>>> > > mod-security-users mailing list
>> >>>>> > > mod...@li...
>> >>>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
>> >>>> SpiderLabs:
>> >>>>> > > http://www.modsecurity.org/projects/commercial/rules/
>> >>>>> > > http://www.modsecurity.org/projects/commercial/support/
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > _______________________________________________
>> >>>>> > mod-security-users mailing list
>> >>>>> > mod...@li...
>> >>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>>> > Commercial ModSecurity Rules and Support from Trustwave's
>> SpiderLabs:
>> >>>>> > http://www.modsecurity.org/projects/commercial/rules/
>> >>>>> > http://www.modsecurity.org/projects/commercial/support/
>> >>>>> >
>> >>>>
>> >>>>
>> >>>>> _______________________________________________
>> >>>>> mod-security-users mailing list
>> >>>>> mod...@li...
>> >>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> >>>>> http://www.modsecurity.org/projects/commercial/rules/
>> >>>>> http://www.modsecurity.org/projects/commercial/support/
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> mod-security-users mailing list
>> >>>> mod...@li...
>> >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> >>>> http://www.modsecurity.org/projects/commercial/rules/
>> >>>> http://www.modsecurity.org/projects/commercial/support/
>> >>>>
>> >>>
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > mod-security-users mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > http://www.modsecurity.org/projects/commercial/rules/
>> > http://www.modsecurity.org/projects/commercial/support/
>>
>>
>>
>>
>>
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: <az...@po...> - 2021-05-10 14:32:45

Another, maybe better, solution, as you are already going to modify  
modsecurity.conf, would be to modify rule 200001 directly. Try this:


SecRule REQUEST_HEADERS:Content-Type "application/json" \
      "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,chain"
      SecRule REQUEST_HEADERS:Content-Length "@lt 131072" \
           "t:none,ctl:requestBodyProcessor=JSON"





Citát az...@po...:

> Sorry, now I see the problem: Rule which is doing JSON processing  
> (200001) is running in phase 1 and my rule below is running in phase  
> 2. Problem is, as i stated below, REQUEST_BODY_LENGTH is available  
> from phase 2 so my rule cannot be run in phase 1. Also, i noticed  
> another problem - we cannot do 'ruleRemoveTargetByTag=OWASP_CRS' as  
> rule 200001 is not part of the CRS and it's running before CRS (and  
> also before rules in file  
> REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf).
>
> In this case, we can use Content-Length header, which won't be so  
> realibe but should be sufficient. Try this:
>
> SecRule REQUEST_HEADERS:Content-Type "application/json" \
>     "id:9999999,\
>     phase:1,\
>     pass,\
>     t:none,\
>     nolog,\
>     chain"
>     SecRule REQUEST_HEADERS:Content-Length "@gt 131072" \
>         "t:none,\
>         ctl:ruleRemoveTargetById=200001;REQUEST_BODY"
>
>
> This rule should be run before rule 200001 so you will, probably,  
> need to put it inside modsecurity.conf before rule 200001 (but i  
> didn't tested it at all).
>
>
>
>
>
> Citát Henri Cook <henri@proteus.tech>:
>
>> Thanks for this azurit but neither of these worked for me. I changed the id
>> to 1001 and put it in my REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf - I
>> still see a delay processing a large json payload - do you have any idea
>> where I might have gone wrong?
>>
>> I think my goal is to stop the JSON request body being parsed if it's over
>> 128kb in size. An advance goal would be to have it scanned as text to
>> provide *some* level of protection - I think this is what cloudflare does
>> (from a very cursory glance)
>>
>> Best Regards,
>>
>> Henri
>>
>> On Thu, 29 Apr 2021 at 19:02, <az...@po...> wrote:
>>
>>> I just noticed you are using CRS, so this one would be probably better:
>>>
>>>
>>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>>>     "id:9999999,\
>>>     phase:2,\
>>>     pass,\
>>>     t:none,\
>>>     nolog,\
>>>     ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"
>>>
>>>
>>>
>>>
>>>
>>> Citát az...@po...:
>>>
>>>> Hi,
>>>>
>>>> you can do this using exclusive rule, something like this (set the
>>>> rule ID to something 'better'), length is in bytes:
>>>>
>>>>
>>>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
>>>>     "id:9999999,\
>>>>     phase:2,\
>>>>     pass,\
>>>>     t:none,\
>>>>     nolog,\
>>>>     ctl:ruleEngine=Off"
>>>>
>>>>
>>>>
>>>> REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this
>>>> into phase 1.
>>>>
>>>>
>>>>
>>>> Citát Henri Cook <henri@proteus.tech>:
>>>>
>>>>> I think what I need here is a way to exempt the request body from any
>>>>> scanning when it's over 128kb (my chosen upper limit)...
>>>>>
>>>>> I'll probably try implementing this as a phase 1 rule on Content-Length
>>>>> unless anyone shouts there's a better way. `ProcessPartial` sounded like
>>>>> the holy grail, but you can't partial process JSON (because it doesn't
>>>>> parse!).
>>>>>
>>>>> It'd be cool to be able to fallback to a fulltext scan of the partial
>>> JSON
>>>>> if it's >128kb, I might have a try at looking into that also (again any
>>>>> guidance appreciated)
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> Henri
>>>>>
>>>>> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
>>>>>
>>>>>> The follow up problem to this is:
>>>>>>
>>>>>> Now i'm set to `SecRequestBodyLimit` 31457280 and
>>>>>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
>>>>>> PartialProcess`
>>>>>>
>>>>>> In my mind this will process the first part of a request if it can, and
>>>>>> ignore the rest. But:
>>>>>>
>>>>>> - Rule 200002 is triggered, which is saying the JSON can't be parsed.
>>>>>> Presumably because in a large request it tries to process the
>>> beginning of
>>>>>> the JSON and can't (because it won't parse, because the JSON is cut
>>> off so
>>>>>> doesn't end)
>>>>>>
>>>>>> So I think I need to find a way to skip JSON parsing entirely when the
>>>>>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb is
>>> the
>>>>>> limit I want to stick with. I hadn't really considered before this
>>> point
>>>>>> that 'partial processing' of JSON was likely to be hairy but of course
>>> it
>>>>>> makes sense.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
>>>>>> chr...@ne...> wrote:
>>>>>>
>>>>>>> Hey Henri,
>>>>>>>
>>>>>>> From a security practice, this is obviously lacking, but in wider
>>>>>>> perspective,
>>>>>>> I see it meet "industry standard", yes.
>>>>>>>
>>>>>>> When I teach, I tell my student, that the worst WAF is the one that is
>>>>>>> switched off. So if you need to compromise and you can only apply 20%
>>> of
>>>>>>> the rules because you run the risk of business demanding it's switched
>>>>>>> off,
>>>>>>> then that 20% WAF is still better than no WAF.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Christian
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
>>>>>>>> Thanks Christian, taking this in combination with Osama's point
>>> earlier
>>>>>>> in
>>>>>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs seem
>>> to
>>>>>>>> limit the payload they'll scan. From my reading to 128kb
>>>>>>> (cloudflare+azure)
>>>>>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular
>>> issue.
>>>>>>>>
>>>>>>>> I believe my modern application is already very robust in terms of
>>>>>>> defence
>>>>>>>> against sql injection as well as other OWASP top 10 attack vectors
>>> and
>>>>>>> that
>>>>>>>> a WAF primarily adds reassurance (for the business and clients who
>>> ask
>>>>>>> if I
>>>>>>>> have one) and minor frustration (for any potential attacker) layer.
>>> The
>>>>>>>> spec is to add a WAF that meets (but notably does not necessarily
>>> have
>>>>>>> to
>>>>>>>> exceed) industry standards. I believe this means that I can switch
>>>>>>> modsec
>>>>>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
>>>>>>>> ProcessPartial' - allowing through unscanned any payloads over those
>>>>>>> sizes)
>>>>>>>> and be able to say I've got scan-size-policy-parity with an AWS or a
>>>>>>>> Cloudflare which means it is "industry standard".
>>>>>>>>
>>>>>>>> Please let me know if you think that's mad and thanks again
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>>
>>>>>>>> Henri
>>>>>>>>
>>>>>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
>>>>>>> chr...@ne...>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> > Hey Henri,
>>>>>>>> >
>>>>>>>> > You are in a bad situation and as far as I can see you are right,
>>> you
>>>>>>> might
>>>>>>>> > have to drop modsec/CRS in this situation.
>>>>>>>> >
>>>>>>>> > I've had a customer with a similar problem and we did a deep dive
>>>>>>>> > investigation and I had to strike colors in the end.
>>>>>>>> >
>>>>>>>> > The point is not the JSON parser. That has shown to be really fast.
>>>>>>> The
>>>>>>>> > point
>>>>>>>> > is several hundred variables that go into CRS afterwards. If you
>>> run
>>>>>>> CRS
>>>>>>>> > on a
>>>>>>>> > standard web application you get forms with a few parameters and
>>>>>>> that's
>>>>>>>> > easy.
>>>>>>>> > But several megabytes of JSON means hundreds of arguments and CRS
>>>>>>> parses
>>>>>>>> > them
>>>>>>>> > all.
>>>>>>>> >
>>>>>>>> > So we tried to work with rule exclusions and skip the parameters we
>>>>>>> did not
>>>>>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
>>>>>>> substantially
>>>>>>>> > slower the longer the ignore-lists of parameters became. This and a
>>>>>>> few
>>>>>>>> > very
>>>>>>>> > odd behaviors.
>>>>>>>> >
>>>>>>>> > Given the customer wanted a generic WAF without tuning of
>>> individual
>>>>>>> APIs
>>>>>>>> > we
>>>>>>>> > got to a dead end.
>>>>>>>> >
>>>>>>>> > However, if tuning was an option, then I would probably edit-CRS
>>> with
>>>>>>>> > msc_pyparser and replace the target lists with arguments I was
>>>>>>> interested
>>>>>>>> > in.
>>>>>>>> >
>>>>>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
>>>>>>>> >
>>>>>>>> > As a complementary practice, one could think of performing
>>> allowlist
>>>>>>>> > checks on
>>>>>>>> > some / most of the JSON. Say you have a huge JSON payload with 500
>>>>>>>> > parameters.
>>>>>>>> > You examine it and discover that 300 of them actually contain
>>> simple
>>>>>>> digits
>>>>>>>> > and asciii characters and neither special chars nor escape
>>> sequences.
>>>>>>>> > So you do a regex allowlist and apply it to these 300 parameters of
>>>>>>> said
>>>>>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
>>>>>>>> >
>>>>>>>> > I have not done this and the problem is if ModSec is able to handle
>>>>>>> the
>>>>>>>> > large
>>>>>>>> > target lists in a speedy manner.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Now you can turn to a CDN or alternative WAF. I would do an
>>> extensive
>>>>>>>> > security
>>>>>>>> > tests of such a system. As I said, the JSON parser can be really
>>>>>>> fast. The
>>>>>>>> > difficult thing is to check several hundred parameters without
>>> losing
>>>>>>>> > performance.
>>>>>>>> >
>>>>>>>> > Good luck!
>>>>>>>> >
>>>>>>>> > Christian
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
>>>>>>>> > > Hi all,
>>>>>>>> > >
>>>>>>>> > > I'm in a situation where the only solution seems to be to drop
>>>>>>> modsec/CRS
>>>>>>>> > > and look at something like Cloudflare's WAF (and change our
>>> security
>>>>>>>> > model
>>>>>>>> > > out of necessity). I'm hoping the esteemed membership of this
>>> list
>>>>>>> might
>>>>>>>> > > have some thoughts.
>>>>>>>> > >
>>>>>>>> > > I've got about 1MB of JSON, payloads in our app might run to 20
>>> or
>>>>>>> even
>>>>>>>> > > 30MB ultimately.
>>>>>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can take 40
>>>>>>> seconds
>>>>>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
>>>>>>>> > >
>>>>>>>> > > It takes 1 second to process in our API so the WAF element is a
>>> 39x
>>>>>>> slow
>>>>>>>> > > down. I appreciate there'll be some delays in WAF. Cloudflare's
>>> WAF
>>>>>>>> > takes 5
>>>>>>>> > > seconds to scan this payload - and that's my target.
>>>>>>>> > >
>>>>>>>> > > Has anyone got any idea how to improve performance? Reading blog
>>>>>>> posts
>>>>>>>> > > about the development of cloudflare's waf I see that memoization
>>> of
>>>>>>>> > common
>>>>>>>> > > function calls was one of their absolute best performance
>>>>>>> improvements
>>>>>>>> > over
>>>>>>>> > > their modsec implementation (e.g. strlen(response_body) so it's
>>> only
>>>>>>>> > > calculated once instead of once per rule OR
>>> contains('somestring',
>>>>>>>> > > response_body)... you get the drift). Do we have anything like
>>> this
>>>>>>> in
>>>>>>>> > > modsec today? Is that already in place and my 39 seconds is after
>>>>>>> that?
>>>>>>>> > >
>>>>>>>> > > I appreciate that mod sec is fast on its own and adding complex
>>>>>>> rules can
>>>>>>>> > > be said to slow it down. With CRS being by far the most common
>>> use
>>>>>>> case
>>>>>>>> > for
>>>>>>>> > > mod sec (based on my googling) I'm surprised it's this slow, do
>>> you
>>>>>>> think
>>>>>>>> > > i've missed something?
>>>>>>>> > >
>>>>>>>> > > To note: I'm only scanning JSON payloads, typically much less
>>> than
>>>>>>> 0.5MB
>>>>>>>> > > but new, irregular ones that we need scanned in ideally <10
>>> seconds
>>>>>>> that
>>>>>>>> > > can range from 1MB-30MB
>>>>>>>> > >
>>>>>>>> > > Best regards,
>>>>>>>> > >
>>>>>>>> > > Henri Cook
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > > _______________________________________________
>>>>>>>> > > mod-security-users mailing list
>>>>>>>> > > mod...@li...
>>>>>>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
>>>>>>> SpiderLabs:
>>>>>>>> > > http://www.modsecurity.org/projects/commercial/rules/
>>>>>>>> > > http://www.modsecurity.org/projects/commercial/support/
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > _______________________________________________
>>>>>>>> > mod-security-users mailing list
>>>>>>>> > mod...@li...
>>>>>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>>>>> > Commercial ModSecurity Rules and Support from Trustwave's
>>> SpiderLabs:
>>>>>>>> > http://www.modsecurity.org/projects/commercial/rules/
>>>>>>>> > http://www.modsecurity.org/projects/commercial/support/
>>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> mod-security-users mailing list
>>>>>>>> mod...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>>>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>>>>>> http://www.modsecurity.org/projects/commercial/support/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> mod-security-users mailing list
>>>>>>> mod...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>>>>> http://www.modsecurity.org/projects/commercial/support/
>>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mod-security-users mailing list
>>>> mod...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>> http://www.modsecurity.org/projects/commercial/support/
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>>>
>
>
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Performance woes - larger JSON payloads with CRS

From: Henri C. <he...@pr...> - 2021-05-10 14:39:34

This is really great, and works like a charm - thanks so much

On Mon, 10 May 2021 at 15:35, <az...@po...> wrote:

> Another, maybe better, solution, as you are already going to modify
> modsecurity.conf, would be to modify rule 200001 directly. Try this:
>
>
> SecRule REQUEST_HEADERS:Content-Type "application/json" \
>       "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,chain"
>       SecRule REQUEST_HEADERS:Content-Length "@lt 131072" \
>            "t:none,ctl:requestBodyProcessor=JSON"
>
>
>
>
>
> Citát az...@po...:
>
> > Sorry, now I see the problem: Rule which is doing JSON processing
> > (200001) is running in phase 1 and my rule below is running in phase
> > 2. Problem is, as i stated below, REQUEST_BODY_LENGTH is available
> > from phase 2 so my rule cannot be run in phase 1. Also, i noticed
> > another problem - we cannot do 'ruleRemoveTargetByTag=OWASP_CRS' as
> > rule 200001 is not part of the CRS and it's running before CRS (and
> > also before rules in file
> > REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf).
> >
> > In this case, we can use Content-Length header, which won't be so
> > realibe but should be sufficient. Try this:
> >
> > SecRule REQUEST_HEADERS:Content-Type "application/json" \
> >     "id:9999999,\
> >     phase:1,\
> >     pass,\
> >     t:none,\
> >     nolog,\
> >     chain"
> >     SecRule REQUEST_HEADERS:Content-Length "@gt 131072" \
> >         "t:none,\
> >         ctl:ruleRemoveTargetById=200001;REQUEST_BODY"
> >
> >
> > This rule should be run before rule 200001 so you will, probably,
> > need to put it inside modsecurity.conf before rule 200001 (but i
> > didn't tested it at all).
> >
> >
> >
> >
> >
> > Citát Henri Cook <henri@proteus.tech>:
> >
> >> Thanks for this azurit but neither of these worked for me. I changed
> the id
> >> to 1001 and put it in my REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf - I
> >> still see a delay processing a large json payload - do you have any idea
> >> where I might have gone wrong?
> >>
> >> I think my goal is to stop the JSON request body being parsed if it's
> over
> >> 128kb in size. An advance goal would be to have it scanned as text to
> >> provide *some* level of protection - I think this is what cloudflare
> does
> >> (from a very cursory glance)
> >>
> >> Best Regards,
> >>
> >> Henri
> >>
> >> On Thu, 29 Apr 2021 at 19:02, <az...@po...> wrote:
> >>
> >>> I just noticed you are using CRS, so this one would be probably better:
> >>>
> >>>
> >>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
> >>>     "id:9999999,\
> >>>     phase:2,\
> >>>     pass,\
> >>>     t:none,\
> >>>     nolog,\
> >>>     ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Citát az...@po...:
> >>>
> >>>> Hi,
> >>>>
> >>>> you can do this using exclusive rule, something like this (set the
> >>>> rule ID to something 'better'), length is in bytes:
> >>>>
> >>>>
> >>>> SecRule REQUEST_BODY_LENGTH "@gt 131072" \
> >>>>     "id:9999999,\
> >>>>     phase:2,\
> >>>>     pass,\
> >>>>     t:none,\
> >>>>     nolog,\
> >>>>     ctl:ruleEngine=Off"
> >>>>
> >>>>
> >>>>
> >>>> REQUEST_BODY_LENGTH is available from phase 2 so you cannot put this
> >>>> into phase 1.
> >>>>
> >>>>
> >>>>
> >>>> Citát Henri Cook <henri@proteus.tech>:
> >>>>
> >>>>> I think what I need here is a way to exempt the request body from any
> >>>>> scanning when it's over 128kb (my chosen upper limit)...
> >>>>>
> >>>>> I'll probably try implementing this as a phase 1 rule on
> Content-Length
> >>>>> unless anyone shouts there's a better way. `ProcessPartial` sounded
> like
> >>>>> the holy grail, but you can't partial process JSON (because it
> doesn't
> >>>>> parse!).
> >>>>>
> >>>>> It'd be cool to be able to fallback to a fulltext scan of the partial
> >>> JSON
> >>>>> if it's >128kb, I might have a try at looking into that also (again
> any
> >>>>> guidance appreciated)
> >>>>>
> >>>>> Best Regards,
> >>>>>
> >>>>> Henri
> >>>>>
> >>>>> On Mon, 26 Apr 2021 at 15:24, Henri Cook <henri@proteus.tech> wrote:
> >>>>>
> >>>>>> The follow up problem to this is:
> >>>>>>
> >>>>>> Now i'm set to `SecRequestBodyLimit` 31457280 and
> >>>>>> `SecRequestBodyNoFilesLimit` 65536 and `SecRequestBodyLimitAction
> >>>>>> PartialProcess`
> >>>>>>
> >>>>>> In my mind this will process the first part of a request if it can,
> and
> >>>>>> ignore the rest. But:
> >>>>>>
> >>>>>> - Rule 200002 is triggered, which is saying the JSON can't be
> parsed.
> >>>>>> Presumably because in a large request it tries to process the
> >>> beginning of
> >>>>>> the JSON and can't (because it won't parse, because the JSON is cut
> >>> off so
> >>>>>> doesn't end)
> >>>>>>
> >>>>>> So I think I need to find a way to skip JSON parsing entirely when
> the
> >>>>>> payload is over 64kb (65536)? Does that sound right? Assuming 64kb
> is
> >>> the
> >>>>>> limit I want to stick with. I hadn't really considered before this
> >>> point
> >>>>>> that 'partial processing' of JSON was likely to be hairy but of
> course
> >>> it
> >>>>>> makes sense.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, 26 Apr 2021 at 07:17, Christian Folini <
> >>>>>> chr...@ne...> wrote:
> >>>>>>
> >>>>>>> Hey Henri,
> >>>>>>>
> >>>>>>> From a security practice, this is obviously lacking, but in wider
> >>>>>>> perspective,
> >>>>>>> I see it meet "industry standard", yes.
> >>>>>>>
> >>>>>>> When I teach, I tell my student, that the worst WAF is the one
> that is
> >>>>>>> switched off. So if you need to compromise and you can only apply
> 20%
> >>> of
> >>>>>>> the rules because you run the risk of business demanding it's
> switched
> >>>>>>> off,
> >>>>>>> then that 20% WAF is still better than no WAF.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Christian
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Apr 26, 2021 at 06:57:54AM +0100, Henri Cook wrote:
> >>>>>>>> Thanks Christian, taking this in combination with Osama's point
> >>> earlier
> >>>>>>> in
> >>>>>>>> the thread that most 'big four' (AWS/GCP/Cloudflare/Azure) WAFs
> seem
> >>> to
> >>>>>>>> limit the payload they'll scan. From my reading to 128kb
> >>>>>>> (cloudflare+azure)
> >>>>>>>> or 8kb (aws+gcp) I think I'll be able to resolve our particular
> >>> issue.
> >>>>>>>>
> >>>>>>>> I believe my modern application is already very robust in terms of
> >>>>>>> defence
> >>>>>>>> against sql injection as well as other OWASP top 10 attack vectors
> >>> and
> >>>>>>> that
> >>>>>>>> a WAF primarily adds reassurance (for the business and clients who
> >>> ask
> >>>>>>> if I
> >>>>>>>> have one) and minor frustration (for any potential attacker)
> layer.
> >>> The
> >>>>>>>> spec is to add a WAF that meets (but notably does not necessarily
> >>> have
> >>>>>>> to
> >>>>>>>> exceed) industry standards. I believe this means that I can switch
> >>>>>>> modsec
> >>>>>>>> to 128kb or 8kb partial parsing ('SecResponseBodyLimitAction
> >>>>>>>> ProcessPartial' - allowing through unscanned any payloads over
> those
> >>>>>>> sizes)
> >>>>>>>> and be able to say I've got scan-size-policy-parity with an AWS
> or a
> >>>>>>>> Cloudflare which means it is "industry standard".
> >>>>>>>>
> >>>>>>>> Please let me know if you think that's mad and thanks again
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>>
> >>>>>>>> Henri
> >>>>>>>>
> >>>>>>>> On Sun, 25 Apr 2021 at 21:39, Christian Folini <
> >>>>>>> chr...@ne...>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> > Hey Henri,
> >>>>>>>> >
> >>>>>>>> > You are in a bad situation and as far as I can see you are
> right,
> >>> you
> >>>>>>> might
> >>>>>>>> > have to drop modsec/CRS in this situation.
> >>>>>>>> >
> >>>>>>>> > I've had a customer with a similar problem and we did a deep
> dive
> >>>>>>>> > investigation and I had to strike colors in the end.
> >>>>>>>> >
> >>>>>>>> > The point is not the JSON parser. That has shown to be really
> fast.
> >>>>>>> The
> >>>>>>>> > point
> >>>>>>>> > is several hundred variables that go into CRS afterwards. If you
> >>> run
> >>>>>>> CRS
> >>>>>>>> > on a
> >>>>>>>> > standard web application you get forms with a few parameters and
> >>>>>>> that's
> >>>>>>>> > easy.
> >>>>>>>> > But several megabytes of JSON means hundreds of arguments and
> CRS
> >>>>>>> parses
> >>>>>>>> > them
> >>>>>>>> > all.
> >>>>>>>> >
> >>>>>>>> > So we tried to work with rule exclusions and skip the
> parameters we
> >>>>>>> did not
> >>>>>>>> > think dangerous, but here comes the bummer: ModSec 2.9 grew
> >>>>>>> substantially
> >>>>>>>> > slower the longer the ignore-lists of parameters became. This
> and a
> >>>>>>> few
> >>>>>>>> > very
> >>>>>>>> > odd behaviors.
> >>>>>>>> >
> >>>>>>>> > Given the customer wanted a generic WAF without tuning of
> >>> individual
> >>>>>>> APIs
> >>>>>>>> > we
> >>>>>>>> > got to a dead end.
> >>>>>>>> >
> >>>>>>>> > However, if tuning was an option, then I would probably edit-CRS
> >>> with
> >>>>>>>> > msc_pyparser and replace the target lists with arguments I was
> >>>>>>> interested
> >>>>>>>> > in.
> >>>>>>>> >
> >>>>>>>> > https://coreruleset.org/20200901/introducing-msc_pyparser/
> >>>>>>>> >
> >>>>>>>> > As a complementary practice, one could think of performing
> >>> allowlist
> >>>>>>>> > checks on
> >>>>>>>> > some / most of the JSON. Say you have a huge JSON payload with
> 500
> >>>>>>>> > parameters.
> >>>>>>>> > You examine it and discover that 300 of them actually contain
> >>> simple
> >>>>>>> digits
> >>>>>>>> > and asciii characters and neither special chars nor escape
> >>> sequences.
> >>>>>>>> > So you do a regex allowlist and apply it to these 300
> parameters of
> >>>>>>> said
> >>>>>>>> > API. And the rest you can push into CRS. Or a subset of CRS.
> >>>>>>>> >
> >>>>>>>> > I have not done this and the problem is if ModSec is able to
> handle
> >>>>>>> the
> >>>>>>>> > large
> >>>>>>>> > target lists in a speedy manner.
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> > Now you can turn to a CDN or alternative WAF. I would do an
> >>> extensive
> >>>>>>>> > security
> >>>>>>>> > tests of such a system. As I said, the JSON parser can be really
> >>>>>>> fast. The
> >>>>>>>> > difficult thing is to check several hundred parameters without
> >>> losing
> >>>>>>>> > performance.
> >>>>>>>> >
> >>>>>>>> > Good luck!
> >>>>>>>> >
> >>>>>>>> > Christian
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> > On Sun, Apr 25, 2021 at 08:47:06PM +0100, Henri Cook wrote:
> >>>>>>>> > > Hi all,
> >>>>>>>> > >
> >>>>>>>> > > I'm in a situation where the only solution seems to be to drop
> >>>>>>> modsec/CRS
> >>>>>>>> > > and look at something like Cloudflare's WAF (and change our
> >>> security
> >>>>>>>> > model
> >>>>>>>> > > out of necessity). I'm hoping the esteemed membership of this
> >>> list
> >>>>>>> might
> >>>>>>>> > > have some thoughts.
> >>>>>>>> > >
> >>>>>>>> > > I've got about 1MB of JSON, payloads in our app might run to
> 20
> >>> or
> >>>>>>> even
> >>>>>>>> > > 30MB ultimately.
> >>>>>>>> > > This 1MB of somewhat nested JSON (7 or 8 levels deep) can
> take 40
> >>>>>>> seconds
> >>>>>>>> > > to process in mod sec 3.0.4 with CRS 3.2.0
> >>>>>>>> > >
> >>>>>>>> > > It takes 1 second to process in our API so the WAF element is
> a
> >>> 39x
> >>>>>>> slow
> >>>>>>>> > > down. I appreciate there'll be some delays in WAF.
> Cloudflare's
> >>> WAF
> >>>>>>>> > takes 5
> >>>>>>>> > > seconds to scan this payload - and that's my target.
> >>>>>>>> > >
> >>>>>>>> > > Has anyone got any idea how to improve performance? Reading
> blog
> >>>>>>> posts
> >>>>>>>> > > about the development of cloudflare's waf I see that
> memoization
> >>> of
> >>>>>>>> > common
> >>>>>>>> > > function calls was one of their absolute best performance
> >>>>>>> improvements
> >>>>>>>> > over
> >>>>>>>> > > their modsec implementation (e.g. strlen(response_body) so
> it's
> >>> only
> >>>>>>>> > > calculated once instead of once per rule OR
> >>> contains('somestring',
> >>>>>>>> > > response_body)... you get the drift). Do we have anything like
> >>> this
> >>>>>>> in
> >>>>>>>> > > modsec today? Is that already in place and my 39 seconds is
> after
> >>>>>>> that?
> >>>>>>>> > >
> >>>>>>>> > > I appreciate that mod sec is fast on its own and adding
> complex
> >>>>>>> rules can
> >>>>>>>> > > be said to slow it down. With CRS being by far the most common
> >>> use
> >>>>>>> case
> >>>>>>>> > for
> >>>>>>>> > > mod sec (based on my googling) I'm surprised it's this slow,
> do
> >>> you
> >>>>>>> think
> >>>>>>>> > > i've missed something?
> >>>>>>>> > >
> >>>>>>>> > > To note: I'm only scanning JSON payloads, typically much less
> >>> than
> >>>>>>> 0.5MB
> >>>>>>>> > > but new, irregular ones that we need scanned in ideally <10
> >>> seconds
> >>>>>>> that
> >>>>>>>> > > can range from 1MB-30MB
> >>>>>>>> > >
> >>>>>>>> > > Best regards,
> >>>>>>>> > >
> >>>>>>>> > > Henri Cook
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> > > _______________________________________________
> >>>>>>>> > > mod-security-users mailing list
> >>>>>>>> > > mod...@li...
> >>>>>>>> > >
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>>>>> > > Commercial ModSecurity Rules and Support from Trustwave's
> >>>>>>> SpiderLabs:
> >>>>>>>> > > http://www.modsecurity.org/projects/commercial/rules/
> >>>>>>>> > > http://www.modsecurity.org/projects/commercial/support/
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> > _______________________________________________
> >>>>>>>> > mod-security-users mailing list
> >>>>>>>> > mod...@li...
> >>>>>>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>>>>> > Commercial ModSecurity Rules and Support from Trustwave's
> >>> SpiderLabs:
> >>>>>>>> > http://www.modsecurity.org/projects/commercial/rules/
> >>>>>>>> > http://www.modsecurity.org/projects/commercial/support/
> >>>>>>>> >
> >>>>>>>
> >>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> mod-security-users mailing list
> >>>>>>>> mod...@li...
> >>>>>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>>>>> Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >>>>>>>> http://www.modsecurity.org/projects/commercial/rules/
> >>>>>>>> http://www.modsecurity.org/projects/commercial/support/
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> mod-security-users mailing list
> >>>>>>> mod...@li...
> >>>>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>>>>> Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >>>>>>> http://www.modsecurity.org/projects/commercial/rules/
> >>>>>>> http://www.modsecurity.org/projects/commercial/support/
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> mod-security-users mailing list
> >>>> mod...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> >>>> http://www.modsecurity.org/projects/commercial/rules/
> >>>> http://www.modsecurity.org/projects/commercial/support/
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> mod-security-users mailing list
> >>> mod...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> >>> http://www.modsecurity.org/projects/commercial/rules/
> >>> http://www.modsecurity.org/projects/commercial/support/
> >>>
> >
> >
> >
> >
> >
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
>
>
>
>
>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>