Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for header

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Ryan,
That might not work for a single IP in x-forwarded-for header, becuase 
non-greedy match for (.*?) will return empty string.

Try:

perl -e "'1.2.3.4' =~ /^(.*?),?.*$/; print $1;"

and you'll see what I mean. I'm assuming here that modsecurity uses PCRE.

Thanks,
Oleg.

----- Original Message ----
> From: Ryan Barnett <RBa...@tr...>
> To: "ol...@gr..." <ol...@gr...>; 
>"mod...@li..." 
><mod...@li...>
> Cc: "owa...@li..." 
><owa...@li...>
> Sent: Fri, May 6, 2011 10:29:57 AM
> Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for 
>header
> 
> Good point about IPv6.  Looks like we could still do this using the same  #
> of SecRules and just not using an IP regex like this -
> 
> SecRule  REQUEST_HEADERS:User-Agent  "^(.*)$"
> "phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_v
> ar}"
> SecRule  REQUEST_HEADERS:x-forwarded-for  "^(.*?),?.*$"
> "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}"
> 
> SecRule  &TX:REAL_IP "@eq  0"
> "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}_
> %{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}"
> SecRule  &TX:REAL_IP "!@eq  0"
> "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_%
> {tx.ua_hash}"
> 
> 
> 
> --
> Ryan
> 
> 
> 
> On  5/6/11 1:16 PM, "Oleg Gryb" <ole...@ya...>  wrote:
> 
> >Ryan,
> >
> >Thanks for the update. It might have  problems with IPv6 though. Here are
> >my rules: they search for ',' and  disregard the IP structure:
> >
> >
> >SecRule  REQUEST_HEADERS:User-Agent  "^(.*)$"
> >"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_
> >var}"
> >SecRule  REQUEST_HEADERS:x-forwarded-for  "^(.*)$"
> >"phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}"
> >SecRule  TX:REAL_IP  "^(.*?),.*$"
> >"phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}"
> >
> >SecRule  &TX:REAL_IP "@eq  0"
> >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}
> >_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}"
> >SecRule  &TX:REAL_IP "!@eq  0"
> >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_
> >%{tx.ua_hash}"
> >
> >Thanks,
> >Oleg.
> >
> >---  On Thu, 5/5/11, Ryan Barnett <RBa...@tr...>  wrote:
> >
> >> From: Ryan Barnett <RBa...@tr...>
> >>  Subject: Re: [Mod-security-developers] CRS DoS protection  &
> >>x-forwarded-for header
> >> To: "Oleg Gryb" <ol...@gr...>,
> >>"mod...@li..."
> >><mod...@li...>
> >>  Cc: "owa...@li..."
> >><owa...@li...>
> >>  Date: Thursday, May 5, 2011, 10:46 AM
> >> FYI - updated the CRS 10  config file
> >> to add in this logic and uploaded it to SVN  -
> >>
> >>http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/mod
> >>security_crs_10_config.conf.example?revision=1772
> >>
> >>  #
> >> # -=[ Global and IP Collections ]=-
> >> #
> >> #  Create both Global and IP collections for rules to use
> >> # There are  some CRS rules that assume that these two
> >> collections
> >> #  have already been initiated.
> >> #
> >> SecRule  REQUEST_HEADERS:User-Agent  "^(.*)$"
> >>
> >>"phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_ha
> >>sh=%{matched_var}"
> >>  SecRule REQUEST_HEADERS:x-forwarded-for
> >>  "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b"
> >>
> >>"phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1}"
> >>  SecRule &TX:REAL_IP "!@eq  0"
> >>
> >>"phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip=%
> >>{tx.real_ip}_%{tx.ua_hash}"
> >>  SecRule &TX:REAL_IP "@eq  0"
> >>
> >>"phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip=%
> >>{remote_addr}_%{tx.ua_hash}"
> >>
> >>  The new rules will grab the first IP address listed in an
> >>  X-Forwared-For header and use that for the IP collection
> >> key.   If X-Forwarded-For is not present, then it will
> >> use  REMOTE_ADDR.
> >>
> >> Thanks for the  suggestion!
> >>
> >> -Ryan
> >>
> >>
> >> On  4/28/11 8:19 PM, "Oleg Gryb"
> >><ole...@ya...<mailto:ole...@ya...>>
> >>  wrote:
> >>
> >> I've just realized that there might be a problem  with
> >> relying on that header: if
> >> an attacker  intentionally sends different random IPs in
> >> there, DoS  protection
> >> can be efficiently by-passed. In my case it should  not
> >> happen, because an LB is
> >> the one who adds the  header, but in general we should warn
> >> engineers about  the
> >> possible exploit.
> >>
> >>
> >> Actually,  even in LB case: if a request has already had the
> >> header, LB  will
> >> create a new one with the existing value appended to  the
> >> client IP:
> >>
> >> x-forwarded-for:  real-client-ip,
> >> whatever-client-sent-to-LB
> >>
> >>  It means that we would need to rely on the the first IP in
> >> the list,  everything
> >> else should be considered as  untrusted.
> >>
> >> Thanks,
> >>  Oleg.
> >>
> >>
> >> ----- Original Message  ----
> >> From: Ryan Barnett
> >><RBa...@tr...<mailto:RBa...@tr...>>
> >>  To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>;
> >>
> >>"mod...@li...<mailto:mod-security-develo
> >>pe...@li...>"
> >>
> >><mod...@li...<mailto:mod-security-develo
> >>pe...@li...>>
> >>  Cc:
> >>"owa...@li...<mailto:owasp-modsecurity
> >>-cor...@li...>"
> >>
> >><owa...@li...<mailto:owasp-modsecurity
> >>-cor...@li...>>
> >>  Sent: Thu, April 28, 2011 4:41:14 PM
> >> Subject: Re:  [Mod-security-developers] CRS DoS protection
> >> &  x-forwarded-for
> >> header
> >> Thanks for the updates  Oleg!  This will certainly be a
> >> useful update   to
> >> not only the DoS rules buy any rules that will be based  on
> >> the client  IP.
> >> I will actually go back to check  other uses of REMOTE_ADDR
> >> and see if  we
> >> can swap  it for tx.real_ip instead.
> >> I will add this to the CRS  v2.2.0  that I am working
> >> on.
> >> For future reference - here is the  OWASP CRS
> >> mail-list -
> >> https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
> >>  --
> >> Ryan  Barnett
> >> Senior Security  Researcher
> >> Trustwave -  SpiderLabs
> >> On 4/28/11 7:32  PM, "Oleg Gryb"
> >><ole...@ya...<mailto:ole...@ya...>>
> >>  wrote:
> >> >I'm not sure if I can discuss CRS rules here. If  not,
> >> please  let me know
> >> >what
> >>  >the right place is. I want to suggest an
> >> improvement to DoS  protection in
> >> >CRS
> >> >2.1.2. The problem is  that  enterprise
> >> applications usually run behind
> >>  >load
> >> >balancers, so  relying on remote_addr doesn't  make
> >> too much sense, because
> >> >you'll
> >>  >always have an LB's IP in there.
> >> >
> >>  >
> >> >My improved  rules (attached)  check  for
> >> x-forwarded-for header and if
> >> >it's
> >>  >present, this IP will be used to initialize IP
> >> collection. If  it's  not
> >> >then the
> >> >old logic will be  used.
> >> >
> >> >It would be  great if we can include  this
> >> improvement to the next  CRS
> >>  >release.
> >> >
> >> >Thanks,
> >>  >Oleg.
> >>
> >>-------------------------------------------------------------------------
> >>-----
> >>  WhatsUp  Gold - Download Free Network Management
> >>  Software
> >> The most intuitive,  comprehensive, and  cost-effective
> >> network
> >> management toolset  available  today.  Delivers
> >> lowest initial
> >>  acquisition cost and overall TCO of any
> >> competing   solution.
> >> http://p.sf.net/sfu/whatsupgold-sd
> >>  _______________________________________________
> >>  mod-security-developers  mailing list
> >>
> >>mod...@li...<mailto:mod-security-develop
> >>er...@li...>
> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >>  ModSecurity  Services from Trustwave's SpiderLabs:
> >> https://www.trustwave.com/spiderLabs.php
> >>
> >>
> >>-------------------------------------------------------------------------
> >>-----
> >>  WhatsUp Gold - Download Free Network Management Software
> >> The most  intuitive, comprehensive, and cost-effective
> >> network
> >>  management toolset available today.  Delivers lowest
> >>  initial
> >> acquisition cost and overall TCO of any  competing
> >> solution.
> >> http://p.sf.net/sfu/whatsupgold-sd
> >>  _______________________________________________
> >>  mod-security-developers mailing list
> >>
> >>mod...@li...<mailto:mod-security-develop
> >>er...@li...>
> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >>  ModSecurity Services from Trustwave's SpiderLabs:
> >> https://www.trustwave.com/spiderLabs.php
> >>
> >>
> >>
> >>  ________________________________
> >> This transmission may contain  information that is
> >> privileged, confidential, and/or exempt from  disclosure
> >> under applicable law. If you are not the intended  recipient,
> >> you are hereby notified that any disclosure,  copying,
> >> distribution, or use of the information contained  herein
> >> (including any reliance thereon) is STRICTLY PROHIBITED.  If
> >> you received this transmission in error, please  immediately
> >> contact the sender and destroy the material in its  entirety,
> >> whether in electronic or hard copy  format.
> >>
> >>
> >
> 
> 
> This transmission may contain  information that is privileged, confidential, 
>and/or exempt from disclosure  under applicable law. If you are not the intended 
>recipient, you are hereby  notified that any disclosure, copying, distribution, 
>or use of the information  contained herein (including any reliance thereon) is 
>STRICTLY PROHIBITED. If you  received this transmission in error, please 
>immediately contact the sender and  destroy the material in its entirety, 
>whether in electronic or hard copy  format.
> 
> 