Thread: Re: [mod-security-users] blocking "bad spiders" with mod_security?
Brought to you by:
victorhora,
zimmerletw
From: Ryan B. <RBa...@tr...> - 2011-06-22 23:10:25
|
Correct, you can change it to whatever disruptive action you see fit. We chose the drop action for DoS responses since your network bandwidth may be at a premium and sending back http compliant/friend html response is usually pointless as the attacker is using scripts and not browsers. If you want to update the actions for any CRS rule, I highly suggest you use SecRuleUpdateActionById so that you can externally make customizations. This helps you to avoid issues when upgrading. -Ryan On 6/22/11 7:06 PM, "Tomasz Chmielewski" <tc...@wp...> wrote: >On 23.06.2011 00:54, Tomasz Chmielewski wrote: >> On 20.06.2011 00:55, Ryan Barnett wrote: >>> Yep. The OWASP ModSecurity Core Rule Set is just that - a rule set. It >>>is intended to be used together. If you only pick and choose a few rule >>>files you will probably run into a few issues. >>> >>> You found the answer to your issue. The 10 file initiates the Global >>>and IP collections that are used by other rules. >> >> Thank you. >> >> Could you give me a tip on how to set up a custom error page, instead of >> "white page of death" when the rule is triggered? >> >> To avoid false positives (at least in the testing phase), I'd rather >> show user a descriptive error page if the rule is triggered, rather than >> drop the connection. > >Ryplying to myself ;) I see something like this does the trick >(redirect:... instead of drop): > >SecRule IP:DOS_BLOCK "@eq 1" >"chain,phase:1,id:'981044',redirect:http://wpkg.org/failed.html,msg:'Denia >l of Service (DoS) Attack Identified from %{remote_addr} >(%{tx.dos_block_counter} hits since last >alert)',setvar:ip.dos_block_counter=+1" > > >-- >Tomasz Chmielewski >http://wpkg.org > This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
From: Tomasz C. <ma...@wp...> - 2011-06-22 23:15:01
|
On 23.06.2011 00:54, Tomasz Chmielewski wrote: > On 20.06.2011 00:55, Ryan Barnett wrote: >> Yep. The OWASP ModSecurity Core Rule Set is just that - a rule set. It is intended to be used together. If you only pick and choose a few rule files you will probably run into a few issues. >> >> You found the answer to your issue. The 10 file initiates the Global and IP collections that are used by other rules. > > Thank you. > > Could you give me a tip on how to set up a custom error page, instead of > "white page of death" when the rule is triggered? > > To avoid false positives (at least in the testing phase), I'd rather > show user a descriptive error page if the rule is triggered, rather than > drop the connection. Replying to myself ;) I see something like this does the trick (redirect:... instead of drop): SecRule IP:DOS_BLOCK "@eq 1" "chain,phase:1,id:'981044',redirect:http://wpkg.org/failed.html,msg:'Denial of Service (DoS) Attack Identified from %{remote_addr} (%{tx.dos_block_counter} hits since last alert)',setvar:ip.dos_block_counter=+1" -- Tomasz Chmielewski http://wpkg.org |
From: Tomasz C. <ma...@wp...> - 2011-06-25 12:58:41
|
On 23.06.2011 01:10, Ryan Barnett wrote: > Correct, you can change it to whatever disruptive action you see fit. We > chose the drop action for DoS responses since your network bandwidth may > be at a premium and sending back http compliant/friend html response is > usually pointless as the attacker is using scripts and not browsers. > > If you want to update the actions for any CRS rule, I highly suggest you > use SecRuleUpdateActionById so that you can externally make > customizations. This helps you to avoid issues when upgrading. Hi, I still have issues with blocking spiders :| I deployed it successfully on a test server - works great there. However, I can't make it work on a production server. I enabled some more logging on both servers to see where the difference is. On the test server, with these rules: # DOS Counter # Count the number of requests to non-static resoures # SecRule REQUEST_BASENAME "!\.(jpe?g|png|gif|js|css|ico)$" "phase:5,id:'981047',t:none,log,pass,setvar:ip.dos_counter=+1" # # Check DOS Counter # If the request count is greater than or equal to user settings, # we then set the burst counter # SecRule IP:DOS_COUNTER "@gt %{tx.dos_counter_threshold}" "phase:5,id:'981048',t:none,log,pass,t:none,setvar:ip.dos_burst_counter=+1,expirevar:ip.dos_burst_counter=%{tx.dos_burst_time_slice},setvar:!ip.dos_counter" I get two lines logged in the error log: [Sat Jun 25 14:49:53 2011] [error] [client 85.183.95.92] ModSecurity: Warning. Match of "rx \\\\.(jpe?g|png|gif|js|css|ico)$" against "REQUEST_BASENAME" required. [file "/etc/apache2/sites-enabled/blog.wpkg.org.conf"] [line "75"] [id "981047"] [hostname "blog.wpkg.org"] [uri "/wp-content/w3tc/pgcache//_index.html"] [unique_id "TgXZcbI-w34AAC7CAggAAAAD"] [Sat Jun 25 14:49:53 2011] [error] [client 85.183.95.92] ModSecurity: Warning. Operator GT matched 0 at IP:dos_counter. [file "/etc/apache2/sites-enabled/blog.wpkg.org.conf"] [line "82"] [id "981048"] [hostname "blog.wpkg.org"] [uri "/wp-content/w3tc/pgcache//_index.html"] [unique_id "TgXZcbI-w34AAC7CAggAAAAD"] With exactly the same config on the production server, I only get the first line logged. I.e. I don't get the "Warning. Operator GT matched 0 at IP:dos_counter". Just to check if it works at all, if I change @gt to @lt (so, we always match), all traffic is blocked. How can I debug it further? -- Tomasz Chmielewski http://wpkg.org |
From: Ryan B. <RBa...@tr...> - 2011-06-25 13:06:36
|
I suggest that you add in a custom rule to increase the debug log level to 9 when your IP accesses the site. This will allow you to review rule processing. See - http://blog.spiderlabs.com/2010/11/modsecurity-advanced-topic-of-the-week-exception-handling.html Ryan On Jun 25, 2011, at 8:58 AM, "Tomasz Chmielewski" <ma...@wp...> wrote: > On 23.06.2011 01:10, Ryan Barnett wrote: >> Correct, you can change it to whatever disruptive action you see fit. We >> chose the drop action for DoS responses since your network bandwidth may >> be at a premium and sending back http compliant/friend html response is >> usually pointless as the attacker is using scripts and not browsers. >> >> If you want to update the actions for any CRS rule, I highly suggest you >> use SecRuleUpdateActionById so that you can externally make >> customizations. This helps you to avoid issues when upgrading. > > Hi, > > I still have issues with blocking spiders :| > > > I deployed it successfully on a test server - works great there. > > However, I can't make it work on a production server. > > > I enabled some more logging on both servers to see where the difference is. > > On the test server, with these rules: > > # DOS Counter > # Count the number of requests to non-static resoures > # > SecRule REQUEST_BASENAME "!\.(jpe?g|png|gif|js|css|ico)$" "phase:5,id:'981047',t:none,log,pass,setvar:ip.dos_counter=+1" > > # > # Check DOS Counter > # If the request count is greater than or equal to user settings, > # we then set the burst counter > # > SecRule IP:DOS_COUNTER "@gt %{tx.dos_counter_threshold}" "phase:5,id:'981048',t:none,log,pass,t:none,setvar:ip.dos_burst_counter=+1,expirevar:ip.dos_burst_counter=%{tx.dos_burst_time_slice},setvar:!ip.dos_counter" > > > I get two lines logged in the error log: > > > [Sat Jun 25 14:49:53 2011] [error] [client 85.183.95.92] ModSecurity: Warning. Match of "rx \\\\.(jpe?g|png|gif|js|css|ico)$" against "REQUEST_BASENAME" required. [file "/etc/apache2/sites-enabled/blog.wpkg.org.conf"] [line "75"] [id "981047"] [hostname "blog.wpkg.org"] [uri "/wp-content/w3tc/pgcache//_index.html"] [unique_id "TgXZcbI-w34AAC7CAggAAAAD"] > > [Sat Jun 25 14:49:53 2011] [error] [client 85.183.95.92] ModSecurity: Warning. Operator GT matched 0 at IP:dos_counter. [file "/etc/apache2/sites-enabled/blog.wpkg.org.conf"] [line "82"] [id "981048"] [hostname "blog.wpkg.org"] [uri "/wp-content/w3tc/pgcache//_index.html"] [unique_id "TgXZcbI-w34AAC7CAggAAAAD"] > > > > > With exactly the same config on the production server, I only get the first line logged. I.e. I don't get the "Warning. Operator GT matched 0 at IP:dos_counter". > > Just to check if it works at all, if I change @gt to @lt (so, we always match), all traffic is blocked. > > > How can I debug it further? > > > -- > Tomasz Chmielewski > http://wpkg.org > This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
From: Tomasz C. <ma...@wp...> - 2011-06-25 15:03:06
|
On 25.06.2011 15:06, Ryan Barnett wrote: > I suggest that you add in a custom rule to increase the debug log level to 9 when your IP accesses the site. This will allow you to review rule processing. > > See - > > http://blog.spiderlabs.com/2010/11/modsecurity-advanced-topic-of-the-week-exception-handling.html Turns out it was a bug in mod_security, which caused that it worked on my "test system" purely accidentally ;) The system where I was testing it had mod_security version 2.5.11 (Ubuntu 10.04), my production system had version 2.5.12 (Debian Squeeze). After setting tx.dos_counter_threshold=1 (in modsecurity_crs_10_config.conf, down from default 100) it works as expected on both systems. Seems that dos_counter_threshold was somehow totally ignored with 2.5.11. -- Tomasz Chmielewski http://wpkg.org |