Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for header
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for header
|
From: Oleg G. <ole...@ya...> - 2011-05-07 15:13:46
|
> By the way - with regards to the pcre matching, switch your single/double
> quotes around to test -
Here is the difference between greedy and non-greedy match:
ogryb@debian:~$ perl -e '"1.2.3.4" =~ /^(.*),?.*$/; print $1;'
1.2.3.4ogryb@debian:~$ perl -e '"1.2.3.4" =~ /^(.*?),?.*$/; print $1;'
ogryb@debian:~$
But your new rule should work because you've enforced the IP characters that
exclude ',' and got rid of non-greedy match.The most importantly, you've
achieved your goal of reducing the # of SecRulles :)
One tiny suggestion though:
^([0-9a-fA-F\.\:]+),?.*$
I think, we should treat empty values in the same way as cases when no
x-forwarded-for header is present.
----- Original Message ----
> From: Ryan Barnett <RBa...@tr...>
> To: Oleg Gryb <ol...@gr...>; "mod...@li..."
><mod...@li...>
> Cc: "owa...@li..."
><owa...@li...>
> Sent: Fri, May 6, 2011 5:26:45 PM
> Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for
>header
>
> This one seems to work in ModSecurity (verified with the debug log). I
> matches a single IP, or the 1st one when there a multiple and it handles
> IPv6 addresses as well -
>
> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" \
>
>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_
> var}"
> SecRule REQUEST_HEADERS:x-forwarded-for "^([0-9a-fA-F\.\:]*),?.*$" \
> "phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1}
> SecRule &TX:REAL_IP "@eq 0" \
>
>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}
> _%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}"
> SecRule &TX:REAL_IP "!@eq 0" \
>
>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_
> %{tx.ua_hash}"
>
>
>
> By the way - with regards to the pcre matching, switch your single/double
> quotes around to test -
>
> perl -e '"1.2.3.4" =~ m/^([0-9a-fA-F\.\:]*),?.*$/; print "$1\n";'
>
> --
> Ryan Barnett
> Senior Security Researcher
> Trustwave - SpiderLabs
> Email: rba...@tr...
> Phone: (703) 794-2248
> Cell: (571) 382-0476
> www.trustwave.com
>
> This transmission may contain information that is privileged,
> confidential, and/or exempt from disclosure under applicable law. If you
> are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution, or use of the information contained
> herein (including any reliance thereon) is STRICTLY PROHIBITED. If you
> received this transmission in error, please immediately contact the sender
> and destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
>
>
>
> On 5/6/11 5:41 PM, "Oleg Gryb" <ole...@ya...> wrote:
>
> >Ryan,
> >That might not work for a single IP in x-forwarded-for header, becuase
> >non-greedy match for (.*?) will return empty string.
> >
> >Try:
> >
> >perl -e "'1.2.3.4' =~ /^(.*?),?.*$/; print $1;"
> >
> >and you'll see what I mean. I'm assuming here that modsecurity uses PCRE.
> >
> >Thanks,
> >Oleg.
> >
> >
> >
> >
> >
> >
> >----- Original Message ----
> >> From: Ryan Barnett <RBa...@tr...>
> >> To: "ol...@gr..." <ol...@gr...>;
> >>"mod...@li..."
> >><mod...@li...>
> >> Cc: "owa...@li..."
> >><owa...@li...>
> >> Sent: Fri, May 6, 2011 10:29:57 AM
> >> Subject: Re: [Mod-security-developers] CRS DoS protection &
> >>x-forwarded-for
> >>header
> >>
> >> Good point about IPv6. Looks like we could still do this using the
> >>same #
> >> of SecRules and just not using an IP regex like this -
> >>
> >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$"
> >>
> >>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched
> >>_v
> >> ar}"
> >> SecRule REQUEST_HEADERS:x-forwarded-for "^(.*?),?.*$"
> >> "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}"
> >>
> >> SecRule &TX:REAL_IP "@eq 0"
> >>
> >>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr
> >>}_
> >> %{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}"
> >> SecRule &TX:REAL_IP "!@eq 0"
> >>
> >>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}
> >>_%
> >> {tx.ua_hash}"
> >>
> >>
> >>
> >> --
> >> Ryan
> >>
> >>
> >>
> >> On 5/6/11 1:16 PM, "Oleg Gryb" <ole...@ya...> wrote:
> >>
> >> >Ryan,
> >> >
> >> >Thanks for the update. It might have problems with IPv6 though. Here
> >>are
> >> >my rules: they search for ',' and disregard the IP structure:
> >> >
> >> >
> >> >SecRule REQUEST_HEADERS:User-Agent "^(.*)$"
> >>
> >>>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matche
> >>>d_
> >> >var}"
> >> >SecRule REQUEST_HEADERS:x-forwarded-for "^(.*)$"
> >> >"phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}"
> >> >SecRule TX:REAL_IP "^(.*?),.*$"
> >> >"phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}"
> >> >
> >> >SecRule &TX:REAL_IP "@eq 0"
> >>
> >>>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_add
> >>>r}
> >> >_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}"
> >> >SecRule &TX:REAL_IP "!@eq 0"
> >>
> >>>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip
> >>>}_
> >> >%{tx.ua_hash}"
> >> >
> >> >Thanks,
> >> >Oleg.
> >> >
> >> >--- On Thu, 5/5/11, Ryan Barnett <RBa...@tr...> wrote:
> >> >
> >> >> From: Ryan Barnett <RBa...@tr...>
> >> >> Subject: Re: [Mod-security-developers] CRS DoS protection &
> >> >>x-forwarded-for header
> >> >> To: "Oleg Gryb" <ol...@gr...>,
> >> >>"mod...@li..."
> >> >><mod...@li...>
> >> >> Cc: "owa...@li..."
> >> >><owa...@li...>
> >> >> Date: Thursday, May 5, 2011, 10:46 AM
> >> >> FYI - updated the CRS 10 config file
> >> >> to add in this logic and uploaded it to SVN -
> >> >>
> >>
> >>>>http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/m
> >>>>od
> >> >>security_crs_10_config.conf.example?revision=1772
> >> >>
> >> >> #
> >> >> # -=[ Global and IP Collections ]=-
> >> >> #
> >> >> # Create both Global and IP collections for rules to use
> >> >> # There are some CRS rules that assume that these two
> >> >> collections
> >> >> # have already been initiated.
> >> >> #
> >> >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$"
> >> >>
> >>
> >>>>"phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_
> >>>>ha
> >> >>sh=%{matched_var}"
> >> >> SecRule REQUEST_HEADERS:x-forwarded-for
> >> >> "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b"
> >> >>
> >>
> >>>>"phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1
> >>>>}"
> >> >> SecRule &TX:REAL_IP "!@eq 0"
> >> >>
> >>
> >>>>"phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip
> >>>>=%
> >> >>{tx.real_ip}_%{tx.ua_hash}"
> >> >> SecRule &TX:REAL_IP "@eq 0"
> >> >>
> >>
> >>>>"phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip
> >>>>=%
> >> >>{remote_addr}_%{tx.ua_hash}"
> >> >>
> >> >> The new rules will grab the first IP address listed in an
> >> >> X-Forwared-For header and use that for the IP collection
> >> >> key. If X-Forwarded-For is not present, then it will
> >> >> use REMOTE_ADDR.
> >> >>
> >> >> Thanks for the suggestion!
> >> >>
> >> >> -Ryan
> >> >>
> >> >>
> >> >> On 4/28/11 8:19 PM, "Oleg Gryb"
> >> >><ole...@ya...<mailto:ole...@ya...>>
> >> >> wrote:
> >> >>
> >> >> I've just realized that there might be a problem with
> >> >> relying on that header: if
> >> >> an attacker intentionally sends different random IPs in
> >> >> there, DoS protection
> >> >> can be efficiently by-passed. In my case it should not
> >> >> happen, because an LB is
> >> >> the one who adds the header, but in general we should warn
> >> >> engineers about the
> >> >> possible exploit.
> >> >>
> >> >>
> >> >> Actually, even in LB case: if a request has already had the
> >> >> header, LB will
> >> >> create a new one with the existing value appended to the
> >> >> client IP:
> >> >>
> >> >> x-forwarded-for: real-client-ip,
> >> >> whatever-client-sent-to-LB
> >> >>
> >> >> It means that we would need to rely on the the first IP in
> >> >> the list, everything
> >> >> else should be considered as untrusted.
> >> >>
> >> >> Thanks,
> >> >> Oleg.
> >> >>
> >> >>
> >> >> ----- Original Message ----
> >> >> From: Ryan Barnett
> >> >><RBa...@tr...<mailto:RBa...@tr...>>
> >> >> To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>;
> >> >>
> >>
> >>>>"mod...@li...<mailto:mod-security-deve
> >>>>lo
> >> >>pe...@li...>"
> >> >>
> >>
> >>>><mod...@li...<mailto:mod-security-deve
> >>>>lo
> >> >>pe...@li...>>
> >> >> Cc:
> >>
> >>>>"owa...@li...<mailto:owasp-modsecuri
> >>>>ty
> >> >>-cor...@li...>"
> >> >>
> >>
> >>>><owa...@li...<mailto:owasp-modsecuri
> >>>>ty
> >> >>-cor...@li...>>
> >> >> Sent: Thu, April 28, 2011 4:41:14 PM
> >> >> Subject: Re: [Mod-security-developers] CRS DoS protection
> >> >> & x-forwarded-for
> >> >> header
> >> >> Thanks for the updates Oleg! This will certainly be a
> >> >> useful update to
> >> >> not only the DoS rules buy any rules that will be based on
> >> >> the client IP.
> >> >> I will actually go back to check other uses of REMOTE_ADDR
> >> >> and see if we
> >> >> can swap it for tx.real_ip instead.
> >> >> I will add this to the CRS v2.2.0 that I am working
> >> >> on.
> >> >> For future reference - here is the OWASP CRS
> >> >> mail-list -
> >> >>
> >>https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
> >> >> --
> >> >> Ryan Barnett
> >> >> Senior Security Researcher
> >> >> Trustwave - SpiderLabs
> >> >> On 4/28/11 7:32 PM, "Oleg Gryb"
> >> >><ole...@ya...<mailto:ole...@ya...>>
> >> >> wrote:
> >> >> >I'm not sure if I can discuss CRS rules here. If not,
> >> >> please let me know
> >> >> >what
> >> >> >the right place is. I want to suggest an
> >> >> improvement to DoS protection in
> >> >> >CRS
> >> >> >2.1.2. The problem is that enterprise
> >> >> applications usually run behind
> >> >> >load
> >> >> >balancers, so relying on remote_addr doesn't make
> >> >> too much sense, because
> >> >> >you'll
> >> >> >always have an LB's IP in there.
> >> >> >
> >> >> >
> >> >> >My improved rules (attached) check for
> >> >> x-forwarded-for header and if
> >> >> >it's
> >> >> >present, this IP will be used to initialize IP
> >> >> collection. If it's not
> >> >> >then the
> >> >> >old logic will be used.
> >> >> >
> >> >> >It would be great if we can include this
> >> >> improvement to the next CRS
> >> >> >release.
> >> >> >
> >> >> >Thanks,
> >> >> >Oleg.
> >> >>
> >>
> >>>>-----------------------------------------------------------------------
> >>>>--
> >> >>-----
> >> >> WhatsUp Gold - Download Free Network Management
> >> >> Software
> >> >> The most intuitive, comprehensive, and cost-effective
> >> >> network
> >> >> management toolset available today. Delivers
> >> >> lowest initial
> >> >> acquisition cost and overall TCO of any
> >> >> competing solution.
> >> >> http://p.sf.net/sfu/whatsupgold-sd
> >> >> _______________________________________________
> >> >> mod-security-developers mailing list
> >> >>
> >>
> >>>>mod...@li...<mailto:mod-security-devel
> >>>>op
> >> >>er...@li...>
> >> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >> >> ModSecurity Services from Trustwave's SpiderLabs:
> >> >> https://www.trustwave.com/spiderLabs.php
> >> >>
> >> >>
> >>
> >>>>-----------------------------------------------------------------------
> >>>>--
> >> >>-----
> >> >> WhatsUp Gold - Download Free Network Management Software
> >> >> The most intuitive, comprehensive, and cost-effective
> >> >> network
> >> >> management toolset available today. Delivers lowest
> >> >> initial
> >> >> acquisition cost and overall TCO of any competing
> >> >> solution.
> >> >> http://p.sf.net/sfu/whatsupgold-sd
> >> >> _______________________________________________
> >> >> mod-security-developers mailing list
> >> >>
> >>
> >>>>mod...@li...<mailto:mod-security-devel
> >>>>op
> >> >>er...@li...>
> >> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >> >> ModSecurity Services from Trustwave's SpiderLabs:
> >> >> https://www.trustwave.com/spiderLabs.php
> >> >>
> >> >>
> >> >>
> >> >> ________________________________
> >> >> This transmission may contain information that is
> >> >> privileged, confidential, and/or exempt from disclosure
> >> >> under applicable law. If you are not the intended recipient,
> >> >> you are hereby notified that any disclosure, copying,
> >> >> distribution, or use of the information contained herein
> >> >> (including any reliance thereon) is STRICTLY PROHIBITED. If
> >> >> you received this transmission in error, please immediately
> >> >> contact the sender and destroy the material in its entirety,
> >> >> whether in electronic or hard copy format.
> >> >>
> >> >>
> >> >
> >>
> >>
> >> This transmission may contain information that is privileged,
> >>confidential,
> >>and/or exempt from disclosure under applicable law. If you are not the
> >>intended
> >>recipient, you are hereby notified that any disclosure, copying,
> >>distribution,
> >>or use of the information contained herein (including any reliance
> >>thereon) is
> >>STRICTLY PROHIBITED. If you received this transmission in error, please
> >>immediately contact the sender and destroy the material in its entirety,
> >>whether in electronic or hard copy format.
> >>
> >>
> >
>
>
> This transmission may contain information that is privileged, confidential,
>and/or exempt from disclosure under applicable law. If you are not the intended
>recipient, you are hereby notified that any disclosure, copying, distribution,
>or use of the information contained herein (including any reliance thereon) is
>STRICTLY PROHIBITED. If you received this transmission in error, please
>immediately contact the sender and destroy the material in its entirety,
>whether in electronic or hard copy format.
>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today. Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> mod-security-developers mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> ModSecurity Services from Trustwave's SpiderLabs:
> https://www.trustwave.com/spiderLabs.php
>
|
