Thread: [mod-security-users] Retry-After header not being set?
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
mod-security-users
|
From: Jamie B. <ja...@ib...> - 2022-02-15 15:35:41
|
Hi
I’m using the following to rate limit request:
<LocationMatch "^/foo">
SecAction initcol:ip=%{REMOTE_ADDR},pass,nolog,id:100
SecAction "phase:5,deprecatevar:ip.hitcounter=1/10,pass,nolog,id:102"
SecRule IP:HITCOUNTER "@gt 60"
"phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:103"
SecAction "phase:2,pass,setvar:ip.hitcounter=+1,nolog,id:104"
Header always set Retry-After "10" env=RATELIMITED
</LocationMatch>
The limiting is working as expected, with a 429 response code when reached
but there is no Retry-After header shown in the response. Can anyone help
me to figure out why?
Thanks
Jamie
|
|
From: Andrew H. <and...@lo...> - 2022-02-16 18:05:32
|
Hi Jamie, That works for me: > GET /foo HTTP/1.1 > Host: example.com > User-Agent: curl/7.81.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 429 Too Many Requests < Date: Wed, 16 Feb 2022 16:52:32 GMT < Server: Apache < Retry-After: 10 < Content-Length: 227 < Content-Type: text/html; charset=iso-8859-1 Is your Apache config loading mod_headers? E.g.: LoadModule headers_module lib/httpd/mod_headers.so It's not a "core" Apache module, so it may not be compiled by default. For what it's worth, in my opinion, ModSecurity is really, really not a good place to do any kind of rate limiting. Especially on Apache: the underlying persistent collection mechanism is ridiculously flakey and will break your heart (and eat your RAM). The implementation of 'deprecatevar' is particularly "interesting". (Can you tell I've been burnt by all this before? :) ) I've had much more success putting HAProxy in front of Apache and using its stick tables to take care of rate limiting. I've also heard good things about using the Apache mod_qos module, although I've never tried it myself. You can also do some clever things using iptables and tc. Thanks, Andrew -- Andrew Howe Loadbalancer.org Ltd. www.loadbalancer.org +1 888 867 9504 / +44 (0)330 380 1064 |
|
From: Jamie B. <ja...@ib...> - 2022-02-16 18:45:32
|
Hi Andrew mod_headers is there; I'm using the same syntax to set a HSTS expiry header in the virtualhost block. That header is present, but not the Retry-After. Thanks Jamie -----Original Message----- From: Andrew Howe <and...@lo...> Sent: 16 February 2022 17:15 To: mod...@li... Subject: Re: [mod-security-users] Retry-After header not being set? Hi Jamie, That works for me: > GET /foo HTTP/1.1 > Host: example.com > User-Agent: curl/7.81.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 429 Too Many Requests < Date: Wed, 16 Feb 2022 16:52:32 GMT < Server: Apache < Retry-After: 10 < Content-Length: 227 < Content-Type: text/html; charset=iso-8859-1 Is your Apache config loading mod_headers? E.g.: LoadModule headers_module lib/httpd/mod_headers.so It's not a "core" Apache module, so it may not be compiled by default. For what it's worth, in my opinion, ModSecurity is really, really not a good place to do any kind of rate limiting. Especially on Apache: the underlying persistent collection mechanism is ridiculously flakey and will break your heart (and eat your RAM). The implementation of 'deprecatevar' is particularly "interesting". (Can you tell I've been burnt by all this before? :) ) I've had much more success putting HAProxy in front of Apache and using its stick tables to take care of rate limiting. I've also heard good things about using the Apache mod_qos module, although I've never tried it myself. You can also do some clever things using iptables and tc. Thanks, Andrew -- Andrew Howe Loadbalancer.org Ltd. www.loadbalancer.org +1 888 867 9504 / +44 (0)330 380 1064 _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Jamie B. <ja...@ib...> - 2022-02-16 19:13:31
|
One peculiar thing I noticed is that after the 429 response, the next request gives me a 429 even if I have waited for a long time. -----Original Message----- From: Jamie Burchell <ja...@ib...> Sent: 16 February 2022 18:45 To: mod...@li... Subject: RE: [mod-security-users] Retry-After header not being set? Hi Andrew mod_headers is there; I'm using the same syntax to set a HSTS expiry header in the virtualhost block. That header is present, but not the Retry-After. Thanks Jamie -----Original Message----- From: Andrew Howe <and...@lo...> Sent: 16 February 2022 17:15 To: mod...@li... Subject: Re: [mod-security-users] Retry-After header not being set? Hi Jamie, That works for me: > GET /foo HTTP/1.1 > Host: example.com > User-Agent: curl/7.81.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 429 Too Many Requests < Date: Wed, 16 Feb 2022 16:52:32 GMT < Server: Apache < Retry-After: 10 < Content-Length: 227 < Content-Type: text/html; charset=iso-8859-1 Is your Apache config loading mod_headers? E.g.: LoadModule headers_module lib/httpd/mod_headers.so It's not a "core" Apache module, so it may not be compiled by default. For what it's worth, in my opinion, ModSecurity is really, really not a good place to do any kind of rate limiting. Especially on Apache: the underlying persistent collection mechanism is ridiculously flakey and will break your heart (and eat your RAM). The implementation of 'deprecatevar' is particularly "interesting". (Can you tell I've been burnt by all this before? :) ) I've had much more success putting HAProxy in front of Apache and using its stick tables to take care of rate limiting. I've also heard good things about using the Apache mod_qos module, although I've never tried it myself. You can also do some clever things using iptables and tc. Thanks, Andrew -- Andrew Howe Loadbalancer.org Ltd. www.loadbalancer.org +1 888 867 9504 / +44 (0)330 380 1064 _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Michael W. <sco...@ya...> - 2022-02-17 09:30:53
|
Have you looked at mod_qos, this is a third part module which needs to be built and installed into Apache. We are having a lot of success with it in rate limiting incoming requests.
Email: sco...@ya...
On Wednesday, 16 February 2022, 19:13:52 GMT, Jamie Burchell <ja...@ib...> wrote:
One peculiar thing I noticed is that after the 429 response, the next
request gives me a 429 even if I have waited for a long time.
-----Original Message-----
From: Jamie Burchell <ja...@ib...>
Sent: 16 February 2022 18:45
To: mod...@li...
Subject: RE: [mod-security-users] Retry-After header not being set?
Hi Andrew
mod_headers is there; I'm using the same syntax to set a HSTS expiry header
in the virtualhost block. That header is present, but not the Retry-After.
Thanks
Jamie
-----Original Message-----
From: Andrew Howe <and...@lo...>
Sent: 16 February 2022 17:15
To: mod...@li...
Subject: Re: [mod-security-users] Retry-After header not being set?
Hi Jamie,
That works for me:
> GET /foo HTTP/1.1
> Host: example.com
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse < HTTP/1.1 429 Too Many Requests <
Date: Wed, 16 Feb 2022 16:52:32 GMT < Server: Apache < Retry-After: 10 <
Content-Length: 227 < Content-Type: text/html; charset=iso-8859-1
Is your Apache config loading mod_headers? E.g.:
LoadModule headers_module lib/httpd/mod_headers.so
It's not a "core" Apache module, so it may not be compiled by default.
For what it's worth, in my opinion, ModSecurity is really, really not a good
place to do any kind of rate limiting. Especially on Apache:
the underlying persistent collection mechanism is ridiculously flakey and
will break your heart (and eat your RAM). The implementation of
'deprecatevar' is particularly "interesting". (Can you tell I've been burnt
by all this before? :) )
I've had much more success putting HAProxy in front of Apache and using its
stick tables to take care of rate limiting. I've also heard good things
about using the Apache mod_qos module, although I've never tried it myself.
You can also do some clever things using iptables and tc.
Thanks,
Andrew
--
Andrew Howe
Loadbalancer.org Ltd.
www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
|
|
From: Jamie B. <ja...@ib...> - 2022-02-17 09:47:53
|
I haven’t. I’m using mod_security already for CRS stuff, so I was hoping I could just add basic limiting for a couple of URLs with it although I appreciate it might not be the best tool for the job. As I say, it works, but without the header being set. Maybe that’s because my error document is a PHP script and it’s getting removed somehow. *From:* Michael Woods via mod-security-users < mod...@li...> *Sent:* 17 February 2022 09:20 *To:* mod...@li... *Cc:* Michael Woods <sco...@ya...> *Subject:* Re: [mod-security-users] Retry-After header not being set? Have you looked at mod_qos, this is a third part module which needs to be built and installed into Apache. We are having a lot of success with it in rate limiting incoming requests. Email: sco...@ya... On Wednesday, 16 February 2022, 19:13:52 GMT, Jamie Burchell <ja...@ib...> wrote: One peculiar thing I noticed is that after the 429 response, the next request gives me a 429 even if I have waited for a long time. -----Original Message----- From: Jamie Burchell <ja...@ib...> Sent: 16 February 2022 18:45 To: mod...@li... Subject: RE: [mod-security-users] Retry-After header not being set? Hi Andrew mod_headers is there; I'm using the same syntax to set a HSTS expiry header in the virtualhost block. That header is present, but not the Retry-After. Thanks Jamie -----Original Message----- From: Andrew Howe <and...@lo...> Sent: 16 February 2022 17:15 To: mod...@li... Subject: Re: [mod-security-users] Retry-After header not being set? Hi Jamie, That works for me: > GET /foo HTTP/1.1 > Host: example.com > User-Agent: curl/7.81.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 429 Too Many Requests < Date: Wed, 16 Feb 2022 16:52:32 GMT < Server: Apache < Retry-After: 10 < Content-Length: 227 < Content-Type: text/html; charset=iso-8859-1 Is your Apache config loading mod_headers? E.g.: LoadModule headers_module lib/httpd/mod_headers.so It's not a "core" Apache module, so it may not be compiled by default. For what it's worth, in my opinion, ModSecurity is really, really not a good place to do any kind of rate limiting. Especially on Apache: the underlying persistent collection mechanism is ridiculously flakey and will break your heart (and eat your RAM). The implementation of 'deprecatevar' is particularly "interesting". (Can you tell I've been burnt by all this before? :) ) I've had much more success putting HAProxy in front of Apache and using its stick tables to take care of rate limiting. I've also heard good things about using the Apache mod_qos module, although I've never tried it myself. You can also do some clever things using iptables and tc. Thanks, Andrew -- Andrew Howe Loadbalancer.org Ltd. www.loadbalancer.org +1 888 867 9504 / +44 (0)330 380 1064 _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Andrew H. <and...@lo...> - 2022-02-17 12:30:28
|
Hi Jamie,
> One peculiar thing I noticed is that after the 429 response, the next
> request gives me a 429 even if I have waited for a long time.
I think that's because of the phases of your rules. The following
test, and its deny action if the rule matches, is executed in phase 2:
SecRule IP:HITCOUNTER "@gt 60" "phase:2,pause:300,deny,...
Even if you've waited long enough for HITCOUNTER to cool off, the
deprecatevar action to apply that 'cooling off' doesn't take place
until later on, in phase 5:
SecAction "phase:5,deprecatevar:ip.hitcounter=1/10,...
Unless HITCOUNTER has fully _expired_ it will still retain its high
value when your next request hits after a long break. That request
then gets denied, deprecatevar then 'cools off' HITCOUNTER to its
expected value, and then the *next* request after that will pass
through, as intended.
> mod_headers is there; I'm using the same syntax to set a HSTS expiry
I've just triple-checked, and your initial set of rules definitely
work for me out of the box, with no alterations. I can only think that
something in your set up is causing you problems.
Maybe try walking it back to the simplest possible setup, see if that
works, and then add each additional piece of complexity until it
breaks? For example, does this work on its own:
Header always set Retry-After "10"
If that *doesn't* work then there's something fundamentally wrong. But
if that *does* work, then does this work:
SecAction "id:103,phase:2,pass,nolog,setenv:RATELIMITED"
Header always set Retry-After "12" env=RATELIMITED
If that works, then does this work:
SecAction "id:100,pass,nolog,initcol:ip=%{REMOTE_ADDR}"
SecAction "id:110,pass,nolog,setvar:'ip.hitcounter=100'"
SecRule IP:HITCOUNTER "@gt 60"
"phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:103"
Header always set Retry-After "10" env=RATELIMITED
And so on and so forth.
Thanks,
Andrew
--
Andrew Howe
Loadbalancer.org Ltd.
www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
|
|
From: Jamie B. <ja...@ib...> - 2022-02-17 23:06:06
|
I tried setting all the phases to 2, but that didn't work. Now after the
initial 429 I get a 200 then a 429 ... forever more.
Yes, I really don't understand mod_security ☹
-----Original Message-----
From: Andrew Howe <and...@lo...>
Sent: 17 February 2022 12:30
To: mod...@li...
Subject: Re: [mod-security-users] Retry-After header not being set?
Hi Jamie,
> One peculiar thing I noticed is that after the 429 response, the next
> request gives me a 429 even if I have waited for a long time.
I think that's because of the phases of your rules. The following test, and
its deny action if the rule matches, is executed in phase 2:
SecRule IP:HITCOUNTER "@gt 60" "phase:2,pause:300,deny,...
Even if you've waited long enough for HITCOUNTER to cool off, the
deprecatevar action to apply that 'cooling off' doesn't take place until
later on, in phase 5:
SecAction "phase:5,deprecatevar:ip.hitcounter=1/10,...
Unless HITCOUNTER has fully _expired_ it will still retain its high value
when your next request hits after a long break. That request then gets
denied, deprecatevar then 'cools off' HITCOUNTER to its expected value, and
then the *next* request after that will pass through, as intended.
> mod_headers is there; I'm using the same syntax to set a HSTS expiry
I've just triple-checked, and your initial set of rules definitely work for
me out of the box, with no alterations. I can only think that something in
your set up is causing you problems.
Maybe try walking it back to the simplest possible setup, see if that works,
and then add each additional piece of complexity until it breaks? For
example, does this work on its own:
Header always set Retry-After "10"
If that *doesn't* work then there's something fundamentally wrong. But if
that *does* work, then does this work:
SecAction "id:103,phase:2,pass,nolog,setenv:RATELIMITED"
Header always set Retry-After "12" env=RATELIMITED
If that works, then does this work:
SecAction "id:100,pass,nolog,initcol:ip=%{REMOTE_ADDR}"
SecAction "id:110,pass,nolog,setvar:'ip.hitcounter=100'"
SecRule IP:HITCOUNTER "@gt 60"
"phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:103"
Header always set Retry-After "10" env=RATELIMITED
And so on and so forth.
Thanks,
Andrew
--
Andrew Howe
Loadbalancer.org Ltd.
www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
|
|
From: Jamie B. <ja...@ib...> - 2022-02-17 20:35:25
|
Hi Andrew
Thanks for taking the time to help me. I have narrowed the header issue
down. If I remove:
ErrorDocument 429 /error.php
The default Apache error document is used, and the header is in the
response. It seems that somehow it is being removed when I'm passing the
processing off to PHP-FPM.
Regarding the phasing, please can you tell me which numbers to use to make
that work? All the examples I have found use the same phase numbers. If I
set them the same, presumably the counter will never move?
Thanks
Jamie
-----Original Message-----
From: Andrew Howe <and...@lo...>
Sent: 17 February 2022 12:30
To: mod...@li...
Subject: Re: [mod-security-users] Retry-After header not being set?
Hi Jamie,
> One peculiar thing I noticed is that after the 429 response, the next
> request gives me a 429 even if I have waited for a long time.
I think that's because of the phases of your rules. The following test,
and its deny action if the rule matches, is executed in phase 2:
SecRule IP:HITCOUNTER "@gt 60" "phase:2,pause:300,deny,...
Even if you've waited long enough for HITCOUNTER to cool off, the
deprecatevar action to apply that 'cooling off' doesn't take place until
later on, in phase 5:
SecAction "phase:5,deprecatevar:ip.hitcounter=1/10,...
Unless HITCOUNTER has fully _expired_ it will still retain its high value
when your next request hits after a long break. That request then gets
denied, deprecatevar then 'cools off' HITCOUNTER to its expected value,
and then the *next* request after that will pass through, as intended.
> mod_headers is there; I'm using the same syntax to set a HSTS expiry
I've just triple-checked, and your initial set of rules definitely work
for me out of the box, with no alterations. I can only think that
something in your set up is causing you problems.
Maybe try walking it back to the simplest possible setup, see if that
works, and then add each additional piece of complexity until it breaks?
For example, does this work on its own:
Header always set Retry-After "10"
If that *doesn't* work then there's something fundamentally wrong. But if
that *does* work, then does this work:
SecAction "id:103,phase:2,pass,nolog,setenv:RATELIMITED"
Header always set Retry-After "12" env=RATELIMITED
If that works, then does this work:
SecAction "id:100,pass,nolog,initcol:ip=%{REMOTE_ADDR}"
SecAction "id:110,pass,nolog,setvar:'ip.hitcounter=100'"
SecRule IP:HITCOUNTER "@gt 60"
"phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:103"
Header always set Retry-After "10" env=RATELIMITED
And so on and so forth.
Thanks,
Andrew
--
Andrew Howe
Loadbalancer.org Ltd.
www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
|
|
From: Reindl H. <h.r...@th...> - 2022-02-17 21:42:29
|
Am 17.02.22 um 21:35 schrieb Jamie Burchell: > Hi Andrew > > Thanks for taking the time to help me. I have narrowed the header issue > down. If I remove: > > ErrorDocument 429 /error.php > > The default Apache error document is used, and the header is in the > response. It seems that somehow it is being removed when I'm passing the > processing off to PHP-FPM does your "error.php" send the correct header? |
|
From: Jamie B. <ja...@ib...> - 2022-02-17 23:03:08
|
No, I'm not sending the headers from there, but I could. I expected the headers set by Apache to work though, since the HSTS header works. -----Original Message----- From: Reindl Harald <h.r...@th...> Sent: 17 February 2022 21:26 To: mod...@li... Subject: Re: [mod-security-users] Retry-After header not being set? Am 17.02.22 um 21:35 schrieb Jamie Burchell: > Hi Andrew > > Thanks for taking the time to help me. I have narrowed the header > issue down. If I remove: > > ErrorDocument 429 /error.php > > The default Apache error document is used, and the header is in the > response. It seems that somehow it is being removed when I'm passing > the processing off to PHP-FPM does your "error.php" send the correct header? _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Reindl H. <h.r...@th...> - 2022-02-17 23:07:23
|
Am 18.02.22 um 00:02 schrieb Jamie Burchell: > No, I'm not sending the headers from there, but I could you must > I expected the > headers set by Apache to work though, since the HSTS header works. your expectation is simply wrong when you use a PHP script for error-pages the default response is 200 because you override the error rule of thumbs: don't use custom error pages (with dynamic scripts) for anything else but 403/404 and for rate-limits (which shouldn't be handeled in the application layer to begin with) it's pretty nonsense to add the burden of a dynamic script > -----Original Message----- > From: Reindl Harald <h.r...@th...> > Sent: 17 February 2022 21:26 > To: mod...@li... > Subject: Re: [mod-security-users] Retry-After header not being set? > > > > Am 17.02.22 um 21:35 schrieb Jamie Burchell: >> Hi Andrew >> >> Thanks for taking the time to help me. I have narrowed the header >> issue down. If I remove: >> >> ErrorDocument 429 /error.php >> >> The default Apache error document is used, and the header is in the >> response. It seems that somehow it is being removed when I'm passing >> the processing off to PHP-FPM > > does your "error.php" send the correct header? |
|
From: Jamie B. <ja...@ib...> - 2022-02-18 09:27:40
|
Hi Reindl > your expectation is simply wrong > when you use a PHP script for error-pages the default response is 200 because you override the error This is not true. I have the ErrorDocument for a 429 set to a very basic PHP page which outputs the unique request ID for diagnostic purposes and the response code is 429, not 200. The PHP script does not do anything else. Further, headers I am setting in Apache are in that same response: Strict-Transport-Security: max-age=63072000; includeSubDomains; preload X-UA-Compatible: IE=edge X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block ... If I add Header always set Retry-After "10" in the virtual host block, that header is in the response for all requests (including those handled by PHP) and for 200 and 429 responses. The part that seems not to work is the condition based on the environment variable in combination with processing the PHP file. Jamie -----Original Message----- From: Reindl Harald <h.r...@th...> Sent: 17 February 2022 23:07 To: mod...@li... Subject: Re: [mod-security-users] Retry-After header not being set? Am 18.02.22 um 00:02 schrieb Jamie Burchell: > No, I'm not sending the headers from there, but I could you must > I expected the > headers set by Apache to work though, since the HSTS header works. your expectation is simply wrong when you use a PHP script for error-pages the default response is 200 because you override the error rule of thumbs: don't use custom error pages (with dynamic scripts) for anything else but 403/404 and for rate-limits (which shouldn't be handeled in the application layer to begin with) it's pretty nonsense to add the burden of a dynamic script > -----Original Message----- > From: Reindl Harald <h.r...@th...> > Sent: 17 February 2022 21:26 > To: mod...@li... > Subject: Re: [mod-security-users] Retry-After header not being set? > > > > Am 17.02.22 um 21:35 schrieb Jamie Burchell: >> Hi Andrew >> >> Thanks for taking the time to help me. I have narrowed the header >> issue down. If I remove: >> >> ErrorDocument 429 /error.php >> >> The default Apache error document is used, and the header is in the >> response. It seems that somehow it is being removed when I'm passing >> the processing off to PHP-FPM > > does your "error.php" send the correct header? _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Reindl H. <h.r...@th...> - 2022-02-18 15:14:25
|
Am 18.02.22 um 10:27 schrieb Jamie Burchell:
> Hi Reindl
>
>> your expectation is simply wrong
>> when you use a PHP script for error-pages the default response is 200
> because you override the error
>
> This is not true. I have the ErrorDocument for a 429 set to a very basic
> PHP page which outputs the unique request ID for diagnostic purposes and
> the response code is 429, not 200. The PHP script does not do anything
> else.
>
> Further, headers I am setting in Apache are in that same response:
>
> Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
> X-UA-Compatible: IE=edge
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> ...
>
> If I add Header always set Retry-After "10" in the virtual host block,
> that header is in the response for all requests (including those handled
> by PHP) and for 200 and 429 responses.
>
> The part that seems not to work is the condition based on the environment
> variable in combination with processing the PHP file
anyways, ratelimits in the webserver are pretty pointless
iptables xt_recent with DROP is the solution
especially with small bursts because it don't break clients which hit
the limit by accident - the TCP layer can't decide between drop and
packet loss
so it's a short "hang" for a client behind a shared IP instead of random
errors and load is completly taken away from the webserver
have fun with apllication prcoessed ratelimts under real load where you
need them most
ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
ctstate RELATED,ESTABLISHED
DROP all -- * * 0.0.0.0/0 0.0.0.0/0
recent: UPDATE seconds: 2 reap hit_count: 150 name: all side: source
mask: 255.255.255.255
all -- * * 0.0.0.0/0 0.0.0.0/0
recent: SET name: all side: source mask: 255.255.255.255
|
|
From: Jamie B. <ja...@ib...> - 2022-02-18 16:04:21
|
You can't achieve simple URL specific rate limiting with iptables. I am
not trying to thwart DoS attacks - you do not understand my use case.
-----Original Message-----
From: Reindl Harald <h.r...@th...>
Sent: 18 February 2022 15:14
To: mod...@li...
Subject: Re: [mod-security-users] Retry-After header not being set?
Am 18.02.22 um 10:27 schrieb Jamie Burchell:
> Hi Reindl
>
>> your expectation is simply wrong
>> when you use a PHP script for error-pages the default response is 200
> because you override the error
>
> This is not true. I have the ErrorDocument for a 429 set to a very
> basic PHP page which outputs the unique request ID for diagnostic
> purposes and the response code is 429, not 200. The PHP script does
> not do anything else.
>
> Further, headers I am setting in Apache are in that same response:
>
> Strict-Transport-Security: max-age=63072000; includeSubDomains;
> preload
> X-UA-Compatible: IE=edge
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> ...
>
> If I add Header always set Retry-After "10" in the virtual host block,
> that header is in the response for all requests (including those
> handled by PHP) and for 200 and 429 responses.
>
> The part that seems not to work is the condition based on the
> environment variable in combination with processing the PHP file
anyways, ratelimits in the webserver are pretty pointless iptables
xt_recent with DROP is the solution
especially with small bursts because it don't break clients which hit the
limit by accident - the TCP layer can't decide between drop and packet
loss
so it's a short "hang" for a client behind a shared IP instead of random
errors and load is completly taken away from the webserver
have fun with apllication prcoessed ratelimts under real load where you
need them most
ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
ctstate RELATED,ESTABLISHED
DROP all -- * * 0.0.0.0/0 0.0.0.0/0
recent: UPDATE seconds: 2 reap hit_count: 150 name: all side: source
mask: 255.255.255.255
all -- * * 0.0.0.0/0 0.0.0.0/0
recent: SET name: all side: source mask: 255.255.255.255
_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
|
|
From: Andrew H. <and...@lo...> - 2022-02-21 14:03:53
|
Hi Jamie, > Regarding the phasing, please can you tell me which numbers to use to make > that work? All the examples I have found use the same phase numbers. If I > set them the same, presumably the counter will never move? You _could_ try moving your deprecatevar rule to phase 2 so that it's executed before your deny rule, but that may well introduce unintended side effects I haven't considered... (The point of having deprecatevar execute in phase 5, as I understand it, is so that it takes place unconditionally: putting it elsewhere could result in it being skipped or removed or for something unforeseen to happen, breaking the whole construct.) You might have more luck decoupling the detection and blocking logic by setting an "is_blocked" flag, checking for that, and then playing with variable expiry times. You could even zero-out your counting/detection variable on a successful block, so that it's back to 0 again once a temporary block is over. There are some examples like this in the ModSecurity Handbook, and I've seen a few tutorials online doing similar things. I'd still recommend steering clear of doing this in ModSecurity, though :) Thanks, Andrew -- Andrew Howe Loadbalancer.org Ltd. www.loadbalancer.org +1 888 867 9504 / +44 (0)330 380 1064 |
|
From: Jamie B. <ja...@ib...> - 2022-02-21 15:14:43
|
Excellent, thanks for your help. --- From: Andrew Howe <and...@lo...> Sent: 21 February 2022 14:03 To: mod...@li... Subject: Re: [mod-security-users] Retry-After header not being set? Hi Jamie, > Regarding the phasing, please can you tell me which numbers to use to > make that work? All the examples I have found use the same phase > numbers. If I set them the same, presumably the counter will never move? You _could_ try moving your deprecatevar rule to phase 2 so that it's executed before your deny rule, but that may well introduce unintended side effects I haven't considered... (The point of having deprecatevar execute in phase 5, as I understand it, is so that it takes place unconditionally: putting it elsewhere could result in it being skipped or removed or for something unforeseen to happen, breaking the whole construct.) You might have more luck decoupling the detection and blocking logic by setting an "is_blocked" flag, checking for that, and then playing with variable expiry times. You could even zero-out your counting/detection variable on a successful block, so that it's back to 0 again once a temporary block is over. There are some examples like this in the ModSecurity Handbook, and I've seen a few tutorials online doing similar things. I'd still recommend steering clear of doing this in ModSecurity, though :) Thanks, Andrew -- Andrew Howe Loadbalancer.org Ltd. www.loadbalancer.org +1 888 867 9504 / +44 (0)330 380 1064 _______________________________________________ mod-security-users mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X