Re: [mod-security-users] How to limit access rate by header?

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Apache 2.4.x

El vie., 7 dic. 2018 09:25, Eero Volotinen <eer...@ik...> escribió:

> are you using nginx or apache?
>
> Luciano Guillermo Fantuzzi <luc...@gm...> kirjoitti pe 7.
> jouluk. 2018 klo 14.19:
>
>> Already tried that, but still the same message in debug log. Anyway, I'm
>> behind Cloudflare so I need to access that var (that contains the real IP)
>> from header. Moreover, I tried with global collection and same luck. Am I
>> missing some initialization step in modsec?
>>
>> Thanks.
>>
>> El vie., 7 dic. 2018 00:56, Scheblein, Adam <ada...@ma...>
>> escribió:
>>
>>> I had a similar problem.  You need to initialize the collection with
>>> something like this:
>>>
>>>
>>>
>>>                 SecAction
>>> id:'2000000',phase:1,nolog,pass,initcol:IP=%{REMOTE_ADDR}
>>>
>>>
>>>
>>> *From: *Luciano Guillermo Fantuzzi <luc...@gm...>
>>> *Reply-To: *"mod...@li..." <
>>> mod...@li...>
>>> *Date: *Thursday, December 6, 2018 at 8:51 PM
>>> *To: *"mod...@li..." <
>>> mod...@li...>
>>> *Subject: *Re: [mod-security-users] How to limit access rate by header?
>>>
>>>
>>>
>>> I've very frustrated... I can't make it work, even for IP control. What
>>> am I doing wrong here? It always returns:
>>>
>>> Could not set variable "IP.access_count" as the collection does not
>>> exist.
>>>
>>>
>>>
>>> 105 <LocationMatch "^/.*">
>>>
>>> 109   SecRule REQUEST_HEADERS:CF-Connecting-IP "@unconditionalMatch"
>>> "phase:2,initcol:IP=%{MATCHED_VAR},pass,nolog,id:35003"
>>>
>>> 112   SecRule IP:ACCESS_COUNT "@gt 1"
>>> "phase:2,pause:300,deny,status:503,setenv:RATELIMITED,skip:1,nolog,id:35004"
>>>
>>> 116   SecAction "phase:2,setvar:IP.access_count=+1,pass,nolog,id:35005"
>>>
>>>
>>>
>>> 119   SecAction
>>> "phase:5,deprecatevar:IP.access_count=1/10,pass,nolog,id:35006"
>>>
>>> 122   Header always set Retry-After "10" env=RATELIMITED
>>>
>>> 123 </LocationMatch>
>>>
>>> 124
>>>
>>> 125 ErrorDocument 503 "Service Unavailable"
>>>
>>>
>>>
>>>
>>>
>>> El jue., 6 de dic. de 2018 a la(s) 20:38, Luciano Guillermo Fantuzzi (
>>> luc...@gm...) escribió:
>>>
>>> Thank you for your answer, Christian. Do you think it's possible for you
>>> to just build the first part of the rule (in Modsec)? I'm trying but I'm
>>> not understanding how variables work with the global scope. I was be able
>>> to build some basic rules like:
>>>
>>>
>>>
>>> # Banned Bots and Crawlers
>>>
>>> SecRule REQUEST_HEADERS:User-Agent "@pmFromFile blacklist-bots.data" \
>>>
>>>     "id:350001,phase:1,t:none,deny,log,msg:'BANNED BOT'"
>>>
>>>
>>>
>>> # Specific IPs
>>>
>>> SecRule REMOTE_ADDR "@pmFromFile blacklist-ip.data" \
>>>
>>>     "id:350002,phase:1,t:none,deny,log,msg:'BANNED IP'"
>>>
>>>
>>>
>>> I'm trying to understand examples from stackoverflow and different
>>> places, but they are all intended to limit by IP and for specific resources
>>> (the scope of the rule). Eg.:
>>>
>>> https://gist.github.com/josnidhin/91d1ea9cd71fde386c27a9228476834e
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_josnidhin_91d1ea9cd71fde386c27a9228476834e&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=L7iKlxwUA3exA-ByaKl7gyvQkoOevQwuEjv4ZKC6hOY&e=>
>>>
>>>
>>>
>>> I'm not asking for the entire rule, just an example of how var counters
>>> work in the global scope (directly in /etc/modsecurity/modsecurity.conf)
>>> and how can I connect them to sum by header instead of IP.
>>>
>>>
>>>
>>> Thank you!
>>>
>>>
>>>
>>> El jue., 6 de dic. de 2018 a la(s) 10:30, Christian Folini (
>>> chr...@ne...) escribió:
>>>
>>> Hello Luciano,
>>>
>>> You have a peculiar use case, but I see your thinking.
>>>
>>> There are examples in the ModSecurity books that are really close to your
>>> plan. They should be easy to adopt.
>>>
>>> Other than that, you may want to look into mod_qos. It has functionality
>>> that might be useful in your case.
>>>
>>> Best,
>>>
>>> Christian
>>>
>>>
>>> On Wed, Dec 05, 2018 at 06:26:03PM -0300, Luciano Guillermo Fantuzzi
>>> wrote:
>>> > Thank you for your answer, but maybe I'm not asking it the right way or
>>> > this is not the right place to ask(?).
>>> >
>>> > I need a Modsecurity rule (I'm using it through Apache) to be able to
>>> > control hits from clients with a specific header, like
>>> > "facebookexternalhit/1.1".
>>> > Ie. to stop some agressive bots hitting too often my webservers and
>>> taking
>>> > them down eventually. I don't want to block them at all because I need
>>> some
>>> > of them (like Facebook bot to parse shared content), but I need a way
>>> to
>>> > tell them "stop, retry in some seconds".
>>> >
>>> > Thanks.
>>> >
>>> > El mié., 5 de dic. de 2018 a la(s) 16:16, Reindl Harald (
>>> > h.r...@th...) escribió:
>>> >
>>> > >
>>> > >
>>> > > Am 05.12.18 um 16:57 schrieb Luciano Guillermo Fantuzzi:
>>> > > > First of all, I'm new here so I'm not sure this is the right place
>>> for
>>> > > > asking for help (free modsec version). If it's not, I'll really
>>> > > > appreciate it if you can tell me where should I go.
>>> > > >
>>> > > > I'm trying to limit hit rate by:
>>> > > >
>>> > > > 1. Request's header (like "facebookexternalhit").
>>> > > > 2. (All hits to non static resources)
>>> > > >
>>> > > > And then return a friendly "429 Too Many Requests" and
>>> "Retry-After: 3"
>>> > > > (seconds).
>>> > > > I know I can read a file of headers like:
>>> > > >
>>> > > > SecRule REQUEST_HEADERS:User-Agent "@pmFromFile ratelimit-bots.txt"
>>> > > >
>>> > > > But I'm getting trouble building the entire rule.
>>> > > >
>>> > > > Any help would be really appreciated. Thank you!
>>> > >
>>> > > this a non-iusse
>>> > >
>>> > > normally you have rate-limits per IP in place and they should not be
>>> > > within the application layer at all and in the best case not even on
>>> the
>>> > > same machine
>>> > >
>>> > > that below is from a firewall-vm on a complete /24 network before any
>>> > > packet reaches a server at all, and for the individual servers are
>>> > > simimlar rules with lower values per 2 seconds in place
>>> > >
>>> > > when the request reachs the webserver damage is long done and if no
>>> > > damage is done you are wasting expensive ressources with the rules
>>> > >
>>> > > Chain INBOUND (2 references)
>>> > >  pkts bytes target     prot opt in     out     source
>>> > > destination
>>> > >  1914  183K IPST_ALL   all  --  *      *       0.0.0.0/0
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>>> > > 0.0.0.0/0
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>>>           recent: UPDATE seconds: 2 hit_count: 250 TTL-Match
>>> > > name: limit_all_global side: source mask: 255.255.255.255
>>> > >  149K   15M DROP_ALL   all  --  *      *       0.0.0.0/0
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>>> > > 0.0.0.0/0
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>>>           recent: UPDATE seconds: 2 reap hit_count: 150
>>> > > TTL-Match name: limit_all_global side: source mask: 255.255.255.255
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > mod-security-users mailing list
>>> > > mod...@li...
>>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
>>> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > > http://www.modsecurity.org/projects/commercial/rules/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
>>> > > http://www.modsecurity.org/projects/commercial/support/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>>> > >
>>>
>>>
>>> > _______________________________________________
>>> > mod-security-users mailing list
>>> > mod...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
>>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> > http://www.modsecurity.org/projects/commercial/rules/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
>>> > http://www.modsecurity.org/projects/commercial/support/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>>>
>>>
>>>
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
>>> http://www.modsecurity.org/projects/commercial/support/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>>>
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>>>
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>