Thread: [mod-security-users] Trying out "early blocking"
Brought to you by:
victorhora,
zimmerletw
From: CM <ne...@pr...> - 2025-05-08 21:46:00
|
I am testing out the latest CRS 4.14 because I was interested in the "early blocking" feature I am using Ubuntu 24.04 using its libapache2-mod-security2 package (version 2.9.7-1build3) I believe this version of modsecurity should be sufficient for running the latest CRS, however, I get various strange behaviors when turning on early blocking. As I understand it, modsecurity has a "--enable-request-early" compile flag that needs to be turned on, however, I'm not sure how to tell if my mod_security2.so was compiled with this option or not. My hypothesis is that because Ubuntu does not ship CRS 4.x yet, they might be compiling modsecurity with this functionality disabled. I had an idea to try the libapache2-mod-security2 package from Ubuntu 25.04 (which would give me 2.9.8) however I haven't done so yet. Is there a way to examine my mod_security2.so to determine if it was compiled with early blocking support or not? Here are some of the odd behaviors I'm encountering: - With early blocking enabled, requests that trigger an Apache redirect still get redirected, even though the audit log shows that they should have been blocked: --992be21d-F-- HTTP/1.1 308 Permanent Redirect ... --992be21d-H-- Message: Access denied with code 403 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 12)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"]Action: Intercepted (phase 1) - If I try to update rule 949111 to do anything other than a 403, then that action gets applied to ALL requests, not just blocks, for example with this: SecRuleUpdateActionById 949111 "status:404" then even requests with an anomaly score of ZERO get blocked: Message: Access denied with code 404 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 0)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] but if I don't modify 949111, then this doesn't happen Sent with [Proton Mail](https://proton.me/mail/home) secure email. |
From: Ervin H. <ai...@gm...> - 2025-05-09 07:31:54
|
Hi CM, On Thu, May 08, 2025 at 09:45:48PM +0000, CM via mod-security-users wrote: > I am testing out the latest CRS 4.14 because I was interested in the "early blocking" feature > > I am using Ubuntu 24.04 using its libapache2-mod-security2 package (version 2.9.7-1build3) > > I believe this version of modsecurity should be sufficient for running the latest CRS, however, I get various strange behaviors when turning on early blocking. > > As I understand it, modsecurity has a "--enable-request-early" compile flag that needs to be turned on, however, I'm not sure how to tell if my mod_security2.so was compiled with this option or not. If you take a look at the module's configure script, you can check how it works: https://github.com/owasp-modsecurity/ModSecurity/blob/v2.9.7/configure.ac#L412-L426 # Enable phase-1 in post_read_request AC_ARG_ENABLE(request-early, AS_HELP_STRING([--enable-request-early], [Place phase1 into post_read_request hook. default is hook_request_early]), [ if test "$enableval" != "no"; then request_early="-DREQUEST_EARLY" MODSEC_EXTRA_CFLAGS="$MODSEC_EXTRA_CFLAGS $request_early" else request_early= fi ], [ request_early='-DREQUEST_EARLY' ]) This means if you pass `--enable-request-early=no` (which is equivalent with `--disable-request-early`), then you explicitly DISABLE this feature. The default is on. (Btw I've nver used this option, and now I checked my generated Makefile, it contains this macro: $ grep -rI REQUEST_EARLY * apache2/Makefile:MODSEC_APXS_EXTRA_CFLAGS = -Wc,-DWITH_PCRE_STUDY ... -DREQUEST_EARLY ) > My hypothesis is that because Ubuntu does not ship CRS 4.x yet, they might be compiling modsecurity with this functionality disabled. > > I had an idea to try the libapache2-mod-security2 package from Ubuntu 25.04 (which would give me 2.9.8) however I haven't done so yet. > > Is there a way to examine my mod_security2.so to determine if it was compiled with early blocking support or not? You can check the package build flags - I can't link that directly, but here is the package's debian/ directory: http://archive.ubuntu.com/ubuntu/pool/universe/m/modsecurity-apache/modsecurity-apache_2.9.7-1build3.debian.tar.xz After you download and unpack it, you can see the debian/rules file: 16 build-stamp: 17 dh_testdir 18 dh_autoreconf 19 ./configure --prefix=/usr --with-apxs=/usr/bin/apxs2 --with-apr=/usr/bin/apr-config --with-lua=/usr/include/lua5.1 --with-pcre2 --enable-pcre-jit --enable-pcre-study CPPFLAGS='$(CPPFLAGS)' CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' As you can see there are no --enable-request-early=no nor --disable-request-early flags. > Here are some of the odd behaviors I'm encountering: > > - With early blocking enabled, requests that trigger an Apache redirect still get redirected, even though the audit log shows that they should have been blocked: > > --992be21d-F-- > HTTP/1.1 308 Permanent Redirect > ... > --992be21d-H-- > Message: Access denied with code 403 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 12)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"]Action: Intercepted (phase 1) > > - If I try to update rule 949111 to do anything other than a 403, then that action gets applied to ALL requests, not just blocks, > > for example with this: > > SecRuleUpdateActionById 949111 "status:404" > > then even requests with an anomaly score of ZERO get blocked: > > Message: Access denied with code 404 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 0)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] > > but if I don't modify 949111, then this doesn't happen I can take a look at this later. a. |
From: CM <ne...@pr...> - 2025-05-10 22:16:33
|
I tried a couple other builds of mod_security2.so (2.9.8 from Ubuntu 25.04, and a couple of Debian versions), with exactly the same results, and then I compiled my own 2.9.8, still with the same results, so I guess I was barking up the wrong tree. Next line of thought, is there a minimum Apache version to properly handle the early blocking feature? I am running Apache 2.4.58-1ubuntu8.6 so it's not too old or is there any specific Apache-side configuration that has to be added in order for early blocking to function properly, outside of the modsecurity/CRS configs? Sent with Proton Mail secure email. On Friday, May 9th, 2025 at 2:31 AM, Ervin Hegedüs <ai...@gm...> wrote: > Hi CM, > > > On Thu, May 08, 2025 at 09:45:48PM +0000, CM via mod-security-users wrote: > > > I am testing out the latest CRS 4.14 because I was interested in the "early blocking" feature > > > > I am using Ubuntu 24.04 using its libapache2-mod-security2 package (version 2.9.7-1build3) > > > > I believe this version of modsecurity should be sufficient for running the latest CRS, however, I get various strange behaviors when turning on early blocking. > > > > As I understand it, modsecurity has a "--enable-request-early" compile flag that needs to be turned on, however, I'm not sure how to tell if my mod_security2.so was compiled with this option or not. > > > If you take a look at the module's configure script, you can > check how it works: > > https://github.com/owasp-modsecurity/ModSecurity/blob/v2.9.7/configure.ac#L412-L426 > > # Enable phase-1 in post_read_request > AC_ARG_ENABLE(request-early, > AS_HELP_STRING([--enable-request-early], > [Place phase1 into post_read_request hook. default is hook_request_early]), > [ > if test "$enableval" != "no"; then > request_early="-DREQUEST_EARLY" > MODSEC_EXTRA_CFLAGS="$MODSEC_EXTRA_CFLAGS $request_early" > else > request_early= > fi > ], > [ > request_early='-DREQUEST_EARLY' > ]) > > This means if you pass `--enable-request-early=no` (which is > equivalent with `--disable-request-early`), then you explicitly > DISABLE this feature. The default is on. > > (Btw I've nver used this option, and now I checked my generated > Makefile, it contains this macro: > > $ grep -rI REQUEST_EARLY * > apache2/Makefile:MODSEC_APXS_EXTRA_CFLAGS = -Wc,-DWITH_PCRE_STUDY ... -DREQUEST_EARLY > ) > > > My hypothesis is that because Ubuntu does not ship CRS 4.x yet, they might be compiling modsecurity with this functionality disabled. > > > > I had an idea to try the libapache2-mod-security2 package from Ubuntu 25.04 (which would give me 2.9.8) however I haven't done so yet. > > > > Is there a way to examine my mod_security2.so to determine if it was compiled with early blocking support or not? > > > You can check the package build flags - I can't link that > directly, but here is the package's debian/ directory: > > http://archive.ubuntu.com/ubuntu/pool/universe/m/modsecurity-apache/modsecurity-apache_2.9.7-1build3.debian.tar.xz > > After you download and unpack it, you can see the debian/rules > file: > > 16 build-stamp: > 17 dh_testdir > 18 dh_autoreconf > 19 ./configure --prefix=/usr --with-apxs=/usr/bin/apxs2 --with-apr=/usr/bin/apr-config --with-lua=/usr/include/lua5.1 --with-pcre2 --enable-pcre-jit --enable-pcre-study CPPFLAGS='$(CPPFLAGS)' CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' > > As you can see there are no --enable-request-early=no nor > --disable-request-early flags. > > > Here are some of the odd behaviors I'm encountering: > > > > - With early blocking enabled, requests that trigger an Apache redirect still get redirected, even though the audit log shows that they should have been blocked: > > > > --992be21d-F-- > > HTTP/1.1 308 Permanent Redirect > > ... > > --992be21d-H-- > > Message: Access denied with code 403 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 12)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"]Action: Intercepted (phase 1) > > > > - If I try to update rule 949111 to do anything other than a 403, then that action gets applied to ALL requests, not just blocks, > > > > for example with this: > > > > SecRuleUpdateActionById 949111 "status:404" > > > > then even requests with an anomaly score of ZERO get blocked: > > > > Message: Access denied with code 404 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 0)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] > > > > but if I don't modify 949111, then this doesn't happen > > > I can take a look at this later. > > > > a. |
From: Ervin H. <ai...@gm...> - 2025-05-12 08:43:23
|
Hi CM, I was able to check that with the "regular" build options (that I mentioned previously - Debian and Ubuntu feliver their packages with those options. Also I checked crs-setup.conf and enabled early blocking mechanism: https://github.com/coreruleset/coreruleset/blob/main/crs-setup.conf.example#L413-L421 I set up my config to use CRS on PL4 (to be sure that engine runs as many rules as possible). Then I sent a minimal but invalid request with telnet command: telnet localhost 80 Trying ::1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1 Host: 127.0.0.1 HTTP/1.1 200 OK Date: Mon, 12 May 2025 08:32:07 GMT Server: Apache/2.4.63 (Debian) Content-Length: 0 Errors in this request: * `Host` is an IP address - rule 920350 * `User-Agent` header is missing - rule 920320 These two rules collect 5 points which is enough to trigger rule 949111, which is responsible to deny the request in phase:1 https://github.com/coreruleset/coreruleset/blob/main/rules/REQUEST-949-BLOCKING-EVALUATION.conf#L212-L222 Here is the log: ModSecurity: Warning. Operator EQ matched 1 at TX:early_blocking. [file "/home/airween/src/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 5)"] [ver "OWASP_CRS/4.15.0-dev"] (I get response 200 because the engine is in DetectionOnly mode, and this is why you see "ModSecurity: Warning", and not "Access Denied) I think early blocking works as we expected. Please check your config again - hope this summary helps you. Regards, a. |
From: CM <ne...@pr...> - 2025-05-27 03:24:54
Attachments:
auditlog1.txt
auditlog2.txt
|
But the purpose of early blocking is to block requests that Apache would have otherwise redirected, is it not? Your 200 OK implies that there's no redirection in place, so even though your log claimed that early blocking happened (my logs claim that too), it's insufficient to prove that the redirect-prevention functionality is actually functioning. What I'm seeing is that even though the logs claim early blocking is happening, Apache is still able to perform redirects as normal on those requests. Refer to attached auditlog1.txt where a score-14 request is answered with a 308 redirect to another site even though the log claims "Access denied with code 403 (phase 1)" (you won't see all the rules it matched because I put noauditlog on some of the 3-point rules, in total it violated 3 3-point rules (noauditlog'd) and 1 5-point rule, for a total score of 14) after doing some additional testing, this seems to be an incompatibility between early blocking and Apache's "ErrorDocument 403" directive. If I disable "ErrorDocument 403" then early blocking is actually able to prevent redirects but I don't really want to do that. "ErrorDocument" when pointing to an internal URL is supposed to do a transparent, internal rewrite rather than a redirect, but it sometimes has weird esoteric interactions and incompatibilities, and it appears incompatibility with early blocking is one of them so my next thought was to try to do early blocking with something other than a 403, using some code I don't have an ErrorDocument for, such as a 429 So I go to RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf and try to modify early blocking rule 949111 like this: SecRuleUpdateActionById 949111 "t:none,deny,status:429" but this breaks my entire server and causes **all** requests to be rejected with a 429, even score-0 requests see attached auditlog2.txt where a perfect score-0 request is rejected with a 429 Message: Access denied with code 429 (phase 1). [file "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 0)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] So I'm at a loss about how to proceed from here, other than turning off early blocking, and probably going back to an older CRS version, since my only reason for testing out CRS4 was because I thought I would be able to get early blocking to work. Sent with Proton Mail secure email. On Monday, May 12th, 2025 at 3:43 AM, Ervin Hegedüs <ai...@gm...> wrote: > Hi CM, > > I was able to check that with the "regular" build options (that I > mentioned previously - Debian and Ubuntu feliver their packages > with those options. > > Also I checked crs-setup.conf and enabled early blocking > mechanism: > > https://github.com/coreruleset/coreruleset/blob/main/crs-setup.conf.example#L413-L421 > > I set up my config to use CRS on PL4 (to be sure that engine runs > as many rules as possible). > > Then I sent a minimal but invalid request with telnet command: > > telnet localhost 80 > Trying ::1... > Connected to localhost. > Escape character is '^]'. > GET / HTTP/1.1 > Host: 127.0.0.1 > > HTTP/1.1 200 OK > Date: Mon, 12 May 2025 08:32:07 GMT > Server: Apache/2.4.63 (Debian) > Content-Length: 0 > > Errors in this request: > * `Host` is an IP address - rule 920350 > * `User-Agent` header is missing - rule 920320 > > These two rules collect 5 points which is enough to trigger rule > 949111, which is responsible to deny the request in phase:1 > > https://github.com/coreruleset/coreruleset/blob/main/rules/REQUEST-949-BLOCKING-EVALUATION.conf#L212-L222 > > Here is the log: > > ModSecurity: Warning. Operator EQ matched 1 at TX:early_blocking. [file "/home/airween/src/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total Score: 5)"] [ver "OWASP_CRS/4.15.0-dev"] > > (I get response 200 because the engine is in DetectionOnly mode, > and this is why you see "ModSecurity: Warning", and not "Access > Denied) > > I think early blocking works as we expected. > > > Please check your config again - hope this summary helps you. > > > Regards, > > > a. |
From: Andrew H. <and...@ow...> - 2025-06-19 13:48:17
|
Hi CM, Early blocking mode can, in some scenarios, possibly save some CPU time and memory. (Was this your use case for early blocking? Or were you trying to achieve something else?) Take an environment that is request body heavy (maybe an API service that is expecting lots of large JSON payloads or similar). If you can filter out and block a large number of requests early, in phase 1 (e.g. by inspecting the request headers), you may be able to avoid unnecessary body processing for requests that have *already* reached the inbound score threshold *before* the body processing stage. Better (maybe) to skip body parsing and processing entirely, as it is usually the heaviest part of request processing when body data is involved. The documentation for early blocking mode contains this warning: "Warning: If early blocking is not enabled, there’s a chance that the web server will interfere with the handling of a request between phases 1 and 2. Take the example where the Apache web server issues a redirect to a new location. With a request that violates CRS rules in phase 1, this may mean that the request has a higher anomaly score than the defined threshold but it gets redirected away before blocking evaluation happens." That sounds like some of the behaviour you've described. (NB: I haven't had time right now to examine your audit log data so I cannot comment on that, sorry.) Thanks, Andrew |
From: Rakesh B. <rak...@gm...> - 2025-05-27 09:40:31
|
Hey, i have apache server with modsecurity. I have one rule which drop requests and get 403 response based on ip address which mentioned in txt file. this rule is triggered first. somehow the ip addresses which mentioned in that txt file can access my website but getting 400 response code (first time it showing 403 response as my first rule triggered). i dont know why apache giving 400 response code.? it should always give 403 response. Please suggest. Thanks, Rakesh On Tue, 27 May 2025, 08:59 CM via mod-security-users, < mod...@li...> wrote: > But the purpose of early blocking is to block requests that Apache would > have otherwise redirected, is it not? > > Your 200 OK implies that there's no redirection in place, so even though > your log claimed that early blocking happened (my logs claim that too), > it's insufficient to prove that the redirect-prevention functionality is > actually functioning. > > What I'm seeing is that even though the logs claim early blocking is > happening, Apache is still able to perform redirects as normal on those > requests. > > Refer to attached auditlog1.txt where a score-14 request is answered with > a 308 redirect to another site even though the log claims "Access denied > with code 403 (phase 1)" (you won't see all the rules it matched because I > put noauditlog on some of the 3-point rules, in total it violated 3 3-point > rules (noauditlog'd) and 1 5-point rule, for a total score of 14) > > after doing some additional testing, this seems to be an incompatibility > between early blocking and Apache's "ErrorDocument 403" directive. If I > disable "ErrorDocument 403" then early blocking is actually able to prevent > redirects but I don't really want to do that. "ErrorDocument" when > pointing to an internal URL is supposed to do a transparent, internal > rewrite rather than a redirect, but it sometimes has weird esoteric > interactions and incompatibilities, and it appears incompatibility with > early blocking is one of them > > so my next thought was to try to do early blocking with something other > than a 403, using some code I don't have an ErrorDocument for, such as a 429 > > So I go to RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf and try to modify > early blocking rule 949111 like this: > > SecRuleUpdateActionById 949111 "t:none,deny,status:429" > > but this breaks my entire server and causes **all** requests to be > rejected with a 429, even score-0 requests > > see attached auditlog2.txt where a perfect score-0 request is rejected > with a 429 > > Message: Access denied with code 429 (phase 1). [file > "/usr/local/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line > "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 (Total > Score: 0)"] [ver "OWASP_CRS/4.14.0"] [tag "anomaly-evaluation"] [tag > "OWASP_CRS"] > > So I'm at a loss about how to proceed from here, other than turning off > early blocking, and probably going back to an older CRS version, since my > only reason for testing out CRS4 was because I thought I would be able to > get early blocking to work. > > > > > > > > Sent with Proton Mail secure email. > > On Monday, May 12th, 2025 at 3:43 AM, Ervin Hegedüs <ai...@gm...> > wrote: > > > Hi CM, > > > > I was able to check that with the "regular" build options (that I > > mentioned previously - Debian and Ubuntu feliver their packages > > with those options. > > > > Also I checked crs-setup.conf and enabled early blocking > > mechanism: > > > > > https://github.com/coreruleset/coreruleset/blob/main/crs-setup.conf.example#L413-L421 > > > > I set up my config to use CRS on PL4 (to be sure that engine runs > > as many rules as possible). > > > > Then I sent a minimal but invalid request with telnet command: > > > > telnet localhost 80 > > Trying ::1... > > Connected to localhost. > > Escape character is '^]'. > > GET / HTTP/1.1 > > Host: 127.0.0.1 > > > > HTTP/1.1 200 OK > > Date: Mon, 12 May 2025 08:32:07 GMT > > Server: Apache/2.4.63 (Debian) > > Content-Length: 0 > > > > Errors in this request: > > * `Host` is an IP address - rule 920350 > > * `User-Agent` header is missing - rule 920320 > > > > These two rules collect 5 points which is enough to trigger rule > > 949111, which is responsible to deny the request in phase:1 > > > > > https://github.com/coreruleset/coreruleset/blob/main/rules/REQUEST-949-BLOCKING-EVALUATION.conf#L212-L222 > > > > Here is the log: > > > > ModSecurity: Warning. Operator EQ matched 1 at TX:early_blocking. [file > "/home/airween/src/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] > [line "221"] [id "949111"] [msg "Inbound Anomaly Score Exceeded in phase 1 > (Total Score: 5)"] [ver "OWASP_CRS/4.15.0-dev"] > > > > (I get response 200 because the engine is in DetectionOnly mode, > > and this is why you see "ModSecurity: Warning", and not "Access > > Denied) > > > > I think early blocking works as we expected. > > > > > > Please check your config again - hope this summary helps you. > > > > > > Regards, > > > > > > a._______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > |
From: Andrew H. <and...@ow...> - 2025-06-19 13:27:28
|
Hi Rakesh, That is a separate issue to the early blocking discussion thread. But you could try taking a look at your Apache logs and see what is causing your '400 Bad Request' responses. Likely, the bad requests are fundamentally malformed and cause an immediate 400 response from Apache. You could move the enforcement of your IP address deny list to somewhere before the ModSecurity stage if this is a problem and you need to ensure a consistent response. As early as possible is ideal, e.g. at a perimeter firewall. Thanks, Andrew |