Thread: [Mod-security-developers] CRS DoS protection & x-forwarded-for header
Brought to you by:
victorhora,
zimmerletw
From: Oleg G. <ole...@ya...> - 2011-04-28 23:33:03
|
I'm not sure if I can discuss CRS rules here. If not, please let me know what the right place is. I want to suggest an improvement to DoS protection in CRS 2.1.2. The problem is that enterprise applications usually run behind load balancers, so relying on remote_addr doesn't make too much sense, because you'll always have an LB's IP in there. My improved rules (attached) check for x-forwarded-for header and if it's present, this IP will be used to initialize IP collection. If it's not then the old logic will be used. It would be great if we can include this improvement to the next CRS release. Thanks, Oleg. |
From: Ryan B. <RBa...@tr...> - 2011-04-28 23:41:27
|
Thanks for the updates Oleg! This will certainly be a useful update to not only the DoS rules buy any rules that will be based on the client IP. I will actually go back to check other uses of REMOTE_ADDR and see if we can swap it for tx.real_ip instead. I will add this to the CRS v2.2.0 that I am working on. For future reference - here is the OWASP CRS mail-list - https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set -- Ryan Barnett Senior Security Researcher Trustwave - SpiderLabs On 4/28/11 7:32 PM, "Oleg Gryb" <ole...@ya...> wrote: >I'm not sure if I can discuss CRS rules here. If not, please let me know >what >the right place is. I want to suggest an improvement to DoS protection in >CRS >2.1.2. The problem is that enterprise applications usually run behind >load >balancers, so relying on remote_addr doesn't make too much sense, because >you'll >always have an LB's IP in there. > > >My improved rules (attached) check for x-forwarded-for header and if >it's >present, this IP will be used to initialize IP collection. If it's not >then the >old logic will be used. > >It would be great if we can include this improvement to the next CRS >release. > >Thanks, >Oleg. |
From: Oleg G. <ole...@ya...> - 2011-04-29 00:19:46
|
I've just realized that there might be a problem with relying on that header: if an attacker intentionally sends different random IPs in there, DoS protection can be efficiently by-passed. In my case it should not happen, because an LB is the one who adds the header, but in general we should warn engineers about the possible exploit. Actually, even in LB case: if a request has already had the header, LB will create a new one with the existing value appended to the client IP: x-forwarded-for: real-client-ip, whatever-client-sent-to-LB It means that we would need to rely on the the first IP in the list, everything else should be considered as untrusted. Thanks, Oleg. ----- Original Message ---- > From: Ryan Barnett <RBa...@tr...> > To: Oleg Gryb <ol...@gr...>; "mod...@li..." ><mod...@li...> > Cc: "owa...@li..." ><owa...@li...> > Sent: Thu, April 28, 2011 4:41:14 PM > Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for >header > > Thanks for the updates Oleg! This will certainly be a useful update to > not only the DoS rules buy any rules that will be based on the client IP. > I will actually go back to check other uses of REMOTE_ADDR and see if we > can swap it for tx.real_ip instead. > > I will add this to the CRS v2.2.0 that I am working on. > > For future reference - here is the OWASP CRS mail-list - > > https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set > > -- > Ryan Barnett > Senior Security Researcher > Trustwave - SpiderLabs > > > > > On 4/28/11 7:32 PM, "Oleg Gryb" <ole...@ya...> wrote: > > >I'm not sure if I can discuss CRS rules here. If not, please let me know > >what > >the right place is. I want to suggest an improvement to DoS protection in > >CRS > >2.1.2. The problem is that enterprise applications usually run behind > >load > >balancers, so relying on remote_addr doesn't make too much sense, because > >you'll > >always have an LB's IP in there. > > > > > >My improved rules (attached) check for x-forwarded-for header and if > >it's > >present, this IP will be used to initialize IP collection. If it's not > >then the > >old logic will be used. > > > >It would be great if we can include this improvement to the next CRS > >release. > > > >Thanks, > >Oleg. > > > > ------------------------------------------------------------------------------ > WhatsUp Gold - Download Free Network Management Software > The most intuitive, comprehensive, and cost-effective network > management toolset available today. Delivers lowest initial > acquisition cost and overall TCO of any competing solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > mod-security-developers mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-developers > ModSecurity Services from Trustwave's SpiderLabs: > https://www.trustwave.com/spiderLabs.php > |
From: Ryan B. <RBa...@tr...> - 2011-05-05 14:46:29
|
FYI - updated the CRS 10 config file to add in this logic and uploaded it to SVN - http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/modsecurity_crs_10_config.conf.example?revision=1772 # # -=[ Global and IP Collections ]=- # # Create both Global and IP collections for rules to use # There are some CRS rules that assume that these two collections # have already been initiated. # SecRule REQUEST_HEADERS:User-Agent "^(.*)$" "phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_var}" SecRule REQUEST_HEADERS:x-forwarded-for "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b" "phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1}" SecRule &TX:REAL_IP "!@eq 0" "phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_%{tx.ua_hash}" SecRule &TX:REAL_IP "@eq 0" "phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}_%{tx.ua_hash}" The new rules will grab the first IP address listed in an X-Forwared-For header and use that for the IP collection key. If X-Forwarded-For is not present, then it will use REMOTE_ADDR. Thanks for the suggestion! -Ryan On 4/28/11 8:19 PM, "Oleg Gryb" <ole...@ya...<mailto:ole...@ya...>> wrote: I've just realized that there might be a problem with relying on that header: if an attacker intentionally sends different random IPs in there, DoS protection can be efficiently by-passed. In my case it should not happen, because an LB is the one who adds the header, but in general we should warn engineers about the possible exploit. Actually, even in LB case: if a request has already had the header, LB will create a new one with the existing value appended to the client IP: x-forwarded-for: real-client-ip, whatever-client-sent-to-LB It means that we would need to rely on the the first IP in the list, everything else should be considered as untrusted. Thanks, Oleg. ----- Original Message ---- From: Ryan Barnett <RBa...@tr...<mailto:RBa...@tr...>> To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>; "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>> Cc: "owa...@li...<mailto:owa...@li...>" <owa...@li...<mailto:owa...@li...>> Sent: Thu, April 28, 2011 4:41:14 PM Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for header Thanks for the updates Oleg! This will certainly be a useful update to not only the DoS rules buy any rules that will be based on the client IP. I will actually go back to check other uses of REMOTE_ADDR and see if we can swap it for tx.real_ip instead. I will add this to the CRS v2.2.0 that I am working on. For future reference - here is the OWASP CRS mail-list - https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set -- Ryan Barnett Senior Security Researcher Trustwave - SpiderLabs On 4/28/11 7:32 PM, "Oleg Gryb" <ole...@ya...<mailto:ole...@ya...>> wrote: >I'm not sure if I can discuss CRS rules here. If not, please let me know >what >the right place is. I want to suggest an improvement to DoS protection in >CRS >2.1.2. The problem is that enterprise applications usually run behind >load >balancers, so relying on remote_addr doesn't make too much sense, because >you'll >always have an LB's IP in there. > > >My improved rules (attached) check for x-forwarded-for header and if >it's >present, this IP will be used to initialize IP collection. If it's not >then the >old logic will be used. > >It would be great if we can include this improvement to the next CRS >release. > >Thanks, >Oleg. ------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ mod-security-developers mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-developers ModSecurity Services from Trustwave's SpiderLabs: https://www.trustwave.com/spiderLabs.php ------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ mod-security-developers mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-developers ModSecurity Services from Trustwave's SpiderLabs: https://www.trustwave.com/spiderLabs.php ________________________________ This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
From: Oleg G. <ole...@ya...> - 2011-05-06 17:16:15
|
Ryan, Thanks for the update. It might have problems with IPv6 though. Here are my rules: they search for ',' and disregard the IP structure: SecRule REQUEST_HEADERS:User-Agent "^(.*)$" "phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_var}" SecRule REQUEST_HEADERS:x-forwarded-for "^(.*)$" "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" SecRule TX:REAL_IP "^(.*?),.*$" "phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}" SecRule &TX:REAL_IP "@eq 0" "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" SecRule &TX:REAL_IP "!@eq 0" "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_%{tx.ua_hash}" Thanks, Oleg. --- On Thu, 5/5/11, Ryan Barnett <RBa...@tr...> wrote: > From: Ryan Barnett <RBa...@tr...> > Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for header > To: "Oleg Gryb" <ol...@gr...>, "mod...@li..." <mod...@li...> > Cc: "owa...@li..." <owa...@li...> > Date: Thursday, May 5, 2011, 10:46 AM > FYI - updated the CRS 10 config file > to add in this logic and uploaded it to SVN - > http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/modsecurity_crs_10_config.conf.example?revision=1772 > > # > # -=[ Global and IP Collections ]=- > # > # Create both Global and IP collections for rules to use > # There are some CRS rules that assume that these two > collections > # have already been initiated. > # > SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > "phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_var}" > SecRule REQUEST_HEADERS:x-forwarded-for > "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b" > "phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1}" > SecRule &TX:REAL_IP "!@eq 0" > "phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_%{tx.ua_hash}" > SecRule &TX:REAL_IP "@eq 0" > "phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}_%{tx.ua_hash}" > > The new rules will grab the first IP address listed in an > X-Forwared-For header and use that for the IP collection > key. If X-Forwarded-For is not present, then it will > use REMOTE_ADDR. > > Thanks for the suggestion! > > -Ryan > > > On 4/28/11 8:19 PM, "Oleg Gryb" <ole...@ya...<mailto:ole...@ya...>> > wrote: > > I've just realized that there might be a problem with > relying on that header: if > an attacker intentionally sends different random IPs in > there, DoS protection > can be efficiently by-passed. In my case it should not > happen, because an LB is > the one who adds the header, but in general we should warn > engineers about the > possible exploit. > > > Actually, even in LB case: if a request has already had the > header, LB will > create a new one with the existing value appended to the > client IP: > > x-forwarded-for: real-client-ip, > whatever-client-sent-to-LB > > It means that we would need to rely on the the first IP in > the list, everything > else should be considered as untrusted. > > Thanks, > Oleg. > > > ----- Original Message ---- > From: Ryan Barnett <RBa...@tr...<mailto:RBa...@tr...>> > To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>; > "mod...@li...<mailto:mod...@li...>" > <mod...@li...<mailto:mod...@li...>> > Cc: "owa...@li...<mailto:owa...@li...>" > <owa...@li...<mailto:owa...@li...>> > Sent: Thu, April 28, 2011 4:41:14 PM > Subject: Re: [Mod-security-developers] CRS DoS protection > & x-forwarded-for > header > Thanks for the updates Oleg! This will certainly be a > useful update to > not only the DoS rules buy any rules that will be based on > the client IP. > I will actually go back to check other uses of REMOTE_ADDR > and see if we > can swap it for tx.real_ip instead. > I will add this to the CRS v2.2.0 that I am working > on. > For future reference - here is the OWASP CRS > mail-list - > https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set > -- > Ryan Barnett > Senior Security Researcher > Trustwave - SpiderLabs > On 4/28/11 7:32 PM, "Oleg Gryb" <ole...@ya...<mailto:ole...@ya...>> > wrote: > >I'm not sure if I can discuss CRS rules here. If not, > please let me know > >what > >the right place is. I want to suggest an > improvement to DoS protection in > >CRS > >2.1.2. The problem is that enterprise > applications usually run behind > >load > >balancers, so relying on remote_addr doesn't make > too much sense, because > >you'll > >always have an LB's IP in there. > > > > > >My improved rules (attached) check for > x-forwarded-for header and if > >it's > >present, this IP will be used to initialize IP > collection. If it's not > >then the > >old logic will be used. > > > >It would be great if we can include this > improvement to the next CRS > >release. > > > >Thanks, > >Oleg. > ------------------------------------------------------------------------------ > WhatsUp Gold - Download Free Network Management > Software > The most intuitive, comprehensive, and cost-effective > network > management toolset available today. Delivers > lowest initial > acquisition cost and overall TCO of any > competing solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > mod-security-developers mailing list > mod...@li...<mailto:mod...@li...> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers > ModSecurity Services from Trustwave's SpiderLabs: > https://www.trustwave.com/spiderLabs.php > > ------------------------------------------------------------------------------ > WhatsUp Gold - Download Free Network Management Software > The most intuitive, comprehensive, and cost-effective > network > management toolset available today. Delivers lowest > initial > acquisition cost and overall TCO of any competing > solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > mod-security-developers mailing list > mod...@li...<mailto:mod...@li...> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers > ModSecurity Services from Trustwave's SpiderLabs: > https://www.trustwave.com/spiderLabs.php > > > > ________________________________ > This transmission may contain information that is > privileged, confidential, and/or exempt from disclosure > under applicable law. If you are not the intended recipient, > you are hereby notified that any disclosure, copying, > distribution, or use of the information contained herein > (including any reliance thereon) is STRICTLY PROHIBITED. If > you received this transmission in error, please immediately > contact the sender and destroy the material in its entirety, > whether in electronic or hard copy format. > > |
From: Ryan B. <RBa...@tr...> - 2011-05-06 17:30:06
|
Good point about IPv6. Looks like we could still do this using the same # of SecRules and just not using an IP regex like this - SecRule REQUEST_HEADERS:User-Agent "^(.*)$" "phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_v ar}" SecRule REQUEST_HEADERS:x-forwarded-for "^(.*?),?.*$" "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" SecRule &TX:REAL_IP "@eq 0" "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}_ %{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" SecRule &TX:REAL_IP "!@eq 0" "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_% {tx.ua_hash}" -- Ryan On 5/6/11 1:16 PM, "Oleg Gryb" <ole...@ya...> wrote: >Ryan, > >Thanks for the update. It might have problems with IPv6 though. Here are >my rules: they search for ',' and disregard the IP structure: > > >SecRule REQUEST_HEADERS:User-Agent "^(.*)$" >"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_ >var}" >SecRule REQUEST_HEADERS:x-forwarded-for "^(.*)$" >"phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" >SecRule TX:REAL_IP "^(.*?),.*$" >"phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}" > >SecRule &TX:REAL_IP "@eq 0" >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr} >_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" >SecRule &TX:REAL_IP "!@eq 0" >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_ >%{tx.ua_hash}" > >Thanks, >Oleg. > >--- On Thu, 5/5/11, Ryan Barnett <RBa...@tr...> wrote: > >> From: Ryan Barnett <RBa...@tr...> >> Subject: Re: [Mod-security-developers] CRS DoS protection & >>x-forwarded-for header >> To: "Oleg Gryb" <ol...@gr...>, >>"mod...@li..." >><mod...@li...> >> Cc: "owa...@li..." >><owa...@li...> >> Date: Thursday, May 5, 2011, 10:46 AM >> FYI - updated the CRS 10 config file >> to add in this logic and uploaded it to SVN - >> >>http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/mod >>security_crs_10_config.conf.example?revision=1772 >> >> # >> # -=[ Global and IP Collections ]=- >> # >> # Create both Global and IP collections for rules to use >> # There are some CRS rules that assume that these two >> collections >> # have already been initiated. >> # >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" >> >>"phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_ha >>sh=%{matched_var}" >> SecRule REQUEST_HEADERS:x-forwarded-for >> "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b" >> >>"phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1}" >> SecRule &TX:REAL_IP "!@eq 0" >> >>"phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip=% >>{tx.real_ip}_%{tx.ua_hash}" >> SecRule &TX:REAL_IP "@eq 0" >> >>"phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip=% >>{remote_addr}_%{tx.ua_hash}" >> >> The new rules will grab the first IP address listed in an >> X-Forwared-For header and use that for the IP collection >> key. If X-Forwarded-For is not present, then it will >> use REMOTE_ADDR. >> >> Thanks for the suggestion! >> >> -Ryan >> >> >> On 4/28/11 8:19 PM, "Oleg Gryb" >><ole...@ya...<mailto:ole...@ya...>> >> wrote: >> >> I've just realized that there might be a problem with >> relying on that header: if >> an attacker intentionally sends different random IPs in >> there, DoS protection >> can be efficiently by-passed. In my case it should not >> happen, because an LB is >> the one who adds the header, but in general we should warn >> engineers about the >> possible exploit. >> >> >> Actually, even in LB case: if a request has already had the >> header, LB will >> create a new one with the existing value appended to the >> client IP: >> >> x-forwarded-for: real-client-ip, >> whatever-client-sent-to-LB >> >> It means that we would need to rely on the the first IP in >> the list, everything >> else should be considered as untrusted. >> >> Thanks, >> Oleg. >> >> >> ----- Original Message ---- >> From: Ryan Barnett >><RBa...@tr...<mailto:RBa...@tr...>> >> To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>; >> >>"mod...@li...<mailto:mod-security-develo >>pe...@li...>" >> >><mod...@li...<mailto:mod-security-develo >>pe...@li...>> >> Cc: >>"owa...@li...<mailto:owasp-modsecurity >>-co...@li...>" >> >><owa...@li...<mailto:owasp-modsecurity >>-co...@li...>> >> Sent: Thu, April 28, 2011 4:41:14 PM >> Subject: Re: [Mod-security-developers] CRS DoS protection >> & x-forwarded-for >> header >> Thanks for the updates Oleg! This will certainly be a >> useful update to >> not only the DoS rules buy any rules that will be based on >> the client IP. >> I will actually go back to check other uses of REMOTE_ADDR >> and see if we >> can swap it for tx.real_ip instead. >> I will add this to the CRS v2.2.0 that I am working >> on. >> For future reference - here is the OWASP CRS >> mail-list - >> https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set >> -- >> Ryan Barnett >> Senior Security Researcher >> Trustwave - SpiderLabs >> On 4/28/11 7:32 PM, "Oleg Gryb" >><ole...@ya...<mailto:ole...@ya...>> >> wrote: >> >I'm not sure if I can discuss CRS rules here. If not, >> please let me know >> >what >> >the right place is. I want to suggest an >> improvement to DoS protection in >> >CRS >> >2.1.2. The problem is that enterprise >> applications usually run behind >> >load >> >balancers, so relying on remote_addr doesn't make >> too much sense, because >> >you'll >> >always have an LB's IP in there. >> > >> > >> >My improved rules (attached) check for >> x-forwarded-for header and if >> >it's >> >present, this IP will be used to initialize IP >> collection. If it's not >> >then the >> >old logic will be used. >> > >> >It would be great if we can include this >> improvement to the next CRS >> >release. >> > >> >Thanks, >> >Oleg. >> >>------------------------------------------------------------------------- >>----- >> WhatsUp Gold - Download Free Network Management >> Software >> The most intuitive, comprehensive, and cost-effective >> network >> management toolset available today. Delivers >> lowest initial >> acquisition cost and overall TCO of any >> competing solution. >> http://p.sf.net/sfu/whatsupgold-sd >> _______________________________________________ >> mod-security-developers mailing list >> >>mod...@li...<mailto:mod-security-develop >>er...@li...> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers >> ModSecurity Services from Trustwave's SpiderLabs: >> https://www.trustwave.com/spiderLabs.php >> >> >>------------------------------------------------------------------------- >>----- >> WhatsUp Gold - Download Free Network Management Software >> The most intuitive, comprehensive, and cost-effective >> network >> management toolset available today. Delivers lowest >> initial >> acquisition cost and overall TCO of any competing >> solution. >> http://p.sf.net/sfu/whatsupgold-sd >> _______________________________________________ >> mod-security-developers mailing list >> >>mod...@li...<mailto:mod-security-develop >>er...@li...> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers >> ModSecurity Services from Trustwave's SpiderLabs: >> https://www.trustwave.com/spiderLabs.php >> >> >> >> ________________________________ >> This transmission may contain information that is >> privileged, confidential, and/or exempt from disclosure >> under applicable law. If you are not the intended recipient, >> you are hereby notified that any disclosure, copying, >> distribution, or use of the information contained herein >> (including any reliance thereon) is STRICTLY PROHIBITED. If >> you received this transmission in error, please immediately >> contact the sender and destroy the material in its entirety, >> whether in electronic or hard copy format. >> >> > This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
From: Oleg G. <ole...@ya...> - 2011-05-06 21:41:17
|
Ryan, That might not work for a single IP in x-forwarded-for header, becuase non-greedy match for (.*?) will return empty string. Try: perl -e "'1.2.3.4' =~ /^(.*?),?.*$/; print $1;" and you'll see what I mean. I'm assuming here that modsecurity uses PCRE. Thanks, Oleg. ----- Original Message ---- > From: Ryan Barnett <RBa...@tr...> > To: "ol...@gr..." <ol...@gr...>; >"mod...@li..." ><mod...@li...> > Cc: "owa...@li..." ><owa...@li...> > Sent: Fri, May 6, 2011 10:29:57 AM > Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for >header > > Good point about IPv6. Looks like we could still do this using the same # > of SecRules and just not using an IP regex like this - > > SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > "phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_v > ar}" > SecRule REQUEST_HEADERS:x-forwarded-for "^(.*?),?.*$" > "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" > > SecRule &TX:REAL_IP "@eq 0" > "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr}_ > %{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" > SecRule &TX:REAL_IP "!@eq 0" > "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_% > {tx.ua_hash}" > > > > -- > Ryan > > > > On 5/6/11 1:16 PM, "Oleg Gryb" <ole...@ya...> wrote: > > >Ryan, > > > >Thanks for the update. It might have problems with IPv6 though. Here are > >my rules: they search for ',' and disregard the IP structure: > > > > > >SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > >"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_ > >var}" > >SecRule REQUEST_HEADERS:x-forwarded-for "^(.*)$" > >"phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" > >SecRule TX:REAL_IP "^(.*?),.*$" > >"phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}" > > > >SecRule &TX:REAL_IP "@eq 0" > >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr} > >_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" > >SecRule &TX:REAL_IP "!@eq 0" > >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_ > >%{tx.ua_hash}" > > > >Thanks, > >Oleg. > > > >--- On Thu, 5/5/11, Ryan Barnett <RBa...@tr...> wrote: > > > >> From: Ryan Barnett <RBa...@tr...> > >> Subject: Re: [Mod-security-developers] CRS DoS protection & > >>x-forwarded-for header > >> To: "Oleg Gryb" <ol...@gr...>, > >>"mod...@li..." > >><mod...@li...> > >> Cc: "owa...@li..." > >><owa...@li...> > >> Date: Thursday, May 5, 2011, 10:46 AM > >> FYI - updated the CRS 10 config file > >> to add in this logic and uploaded it to SVN - > >> > >>http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/mod > >>security_crs_10_config.conf.example?revision=1772 > >> > >> # > >> # -=[ Global and IP Collections ]=- > >> # > >> # Create both Global and IP collections for rules to use > >> # There are some CRS rules that assume that these two > >> collections > >> # have already been initiated. > >> # > >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > >> > >>"phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_ha > >>sh=%{matched_var}" > >> SecRule REQUEST_HEADERS:x-forwarded-for > >> "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b" > >> > >>"phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1}" > >> SecRule &TX:REAL_IP "!@eq 0" > >> > >>"phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip=% > >>{tx.real_ip}_%{tx.ua_hash}" > >> SecRule &TX:REAL_IP "@eq 0" > >> > >>"phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip=% > >>{remote_addr}_%{tx.ua_hash}" > >> > >> The new rules will grab the first IP address listed in an > >> X-Forwared-For header and use that for the IP collection > >> key. If X-Forwarded-For is not present, then it will > >> use REMOTE_ADDR. > >> > >> Thanks for the suggestion! > >> > >> -Ryan > >> > >> > >> On 4/28/11 8:19 PM, "Oleg Gryb" > >><ole...@ya...<mailto:ole...@ya...>> > >> wrote: > >> > >> I've just realized that there might be a problem with > >> relying on that header: if > >> an attacker intentionally sends different random IPs in > >> there, DoS protection > >> can be efficiently by-passed. In my case it should not > >> happen, because an LB is > >> the one who adds the header, but in general we should warn > >> engineers about the > >> possible exploit. > >> > >> > >> Actually, even in LB case: if a request has already had the > >> header, LB will > >> create a new one with the existing value appended to the > >> client IP: > >> > >> x-forwarded-for: real-client-ip, > >> whatever-client-sent-to-LB > >> > >> It means that we would need to rely on the the first IP in > >> the list, everything > >> else should be considered as untrusted. > >> > >> Thanks, > >> Oleg. > >> > >> > >> ----- Original Message ---- > >> From: Ryan Barnett > >><RBa...@tr...<mailto:RBa...@tr...>> > >> To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>; > >> > >>"mod...@li...<mailto:mod-security-develo > >>pe...@li...>" > >> > >><mod...@li...<mailto:mod-security-develo > >>pe...@li...>> > >> Cc: > >>"owa...@li...<mailto:owasp-modsecurity > >>-co...@li...>" > >> > >><owa...@li...<mailto:owasp-modsecurity > >>-co...@li...>> > >> Sent: Thu, April 28, 2011 4:41:14 PM > >> Subject: Re: [Mod-security-developers] CRS DoS protection > >> & x-forwarded-for > >> header > >> Thanks for the updates Oleg! This will certainly be a > >> useful update to > >> not only the DoS rules buy any rules that will be based on > >> the client IP. > >> I will actually go back to check other uses of REMOTE_ADDR > >> and see if we > >> can swap it for tx.real_ip instead. > >> I will add this to the CRS v2.2.0 that I am working > >> on. > >> For future reference - here is the OWASP CRS > >> mail-list - > >> https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set > >> -- > >> Ryan Barnett > >> Senior Security Researcher > >> Trustwave - SpiderLabs > >> On 4/28/11 7:32 PM, "Oleg Gryb" > >><ole...@ya...<mailto:ole...@ya...>> > >> wrote: > >> >I'm not sure if I can discuss CRS rules here. If not, > >> please let me know > >> >what > >> >the right place is. I want to suggest an > >> improvement to DoS protection in > >> >CRS > >> >2.1.2. The problem is that enterprise > >> applications usually run behind > >> >load > >> >balancers, so relying on remote_addr doesn't make > >> too much sense, because > >> >you'll > >> >always have an LB's IP in there. > >> > > >> > > >> >My improved rules (attached) check for > >> x-forwarded-for header and if > >> >it's > >> >present, this IP will be used to initialize IP > >> collection. If it's not > >> >then the > >> >old logic will be used. > >> > > >> >It would be great if we can include this > >> improvement to the next CRS > >> >release. > >> > > >> >Thanks, > >> >Oleg. > >> > >>------------------------------------------------------------------------- > >>----- > >> WhatsUp Gold - Download Free Network Management > >> Software > >> The most intuitive, comprehensive, and cost-effective > >> network > >> management toolset available today. Delivers > >> lowest initial > >> acquisition cost and overall TCO of any > >> competing solution. > >> http://p.sf.net/sfu/whatsupgold-sd > >> _______________________________________________ > >> mod-security-developers mailing list > >> > >>mod...@li...<mailto:mod-security-develop > >>er...@li...> > >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers > >> ModSecurity Services from Trustwave's SpiderLabs: > >> https://www.trustwave.com/spiderLabs.php > >> > >> > >>------------------------------------------------------------------------- > >>----- > >> WhatsUp Gold - Download Free Network Management Software > >> The most intuitive, comprehensive, and cost-effective > >> network > >> management toolset available today. Delivers lowest > >> initial > >> acquisition cost and overall TCO of any competing > >> solution. > >> http://p.sf.net/sfu/whatsupgold-sd > >> _______________________________________________ > >> mod-security-developers mailing list > >> > >>mod...@li...<mailto:mod-security-develop > >>er...@li...> > >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers > >> ModSecurity Services from Trustwave's SpiderLabs: > >> https://www.trustwave.com/spiderLabs.php > >> > >> > >> > >> ________________________________ > >> This transmission may contain information that is > >> privileged, confidential, and/or exempt from disclosure > >> under applicable law. If you are not the intended recipient, > >> you are hereby notified that any disclosure, copying, > >> distribution, or use of the information contained herein > >> (including any reliance thereon) is STRICTLY PROHIBITED. If > >> you received this transmission in error, please immediately > >> contact the sender and destroy the material in its entirety, > >> whether in electronic or hard copy format. > >> > >> > > > > > This transmission may contain information that is privileged, confidential, >and/or exempt from disclosure under applicable law. If you are not the intended >recipient, you are hereby notified that any disclosure, copying, distribution, >or use of the information contained herein (including any reliance thereon) is >STRICTLY PROHIBITED. If you received this transmission in error, please >immediately contact the sender and destroy the material in its entirety, >whether in electronic or hard copy format. > > |
From: Ryan B. <RBa...@tr...> - 2011-05-07 00:26:59
|
This one seems to work in ModSecurity (verified with the debug log). I matches a single IP, or the 1st one when there a multiple and it handles IPv6 addresses as well - SecRule REQUEST_HEADERS:User-Agent "^(.*)$" \ "phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_ var}" SecRule REQUEST_HEADERS:x-forwarded-for "^([0-9a-fA-F\.\:]*),?.*$" \ "phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1} SecRule &TX:REAL_IP "@eq 0" \ "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr} _%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" SecRule &TX:REAL_IP "!@eq 0" \ "phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_ %{tx.ua_hash}" By the way - with regards to the pcre matching, switch your single/double quotes around to test - perl -e '"1.2.3.4" =~ m/^([0-9a-fA-F\.\:]*),?.*$/; print "$1\n";' -- Ryan Barnett Senior Security Researcher Trustwave - SpiderLabs Email: rba...@tr... Phone: (703) 794-2248 Cell: (571) 382-0476 www.trustwave.com This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. On 5/6/11 5:41 PM, "Oleg Gryb" <ole...@ya...> wrote: >Ryan, >That might not work for a single IP in x-forwarded-for header, becuase >non-greedy match for (.*?) will return empty string. > >Try: > >perl -e "'1.2.3.4' =~ /^(.*?),?.*$/; print $1;" > >and you'll see what I mean. I'm assuming here that modsecurity uses PCRE. > >Thanks, >Oleg. > > > > > > >----- Original Message ---- >> From: Ryan Barnett <RBa...@tr...> >> To: "ol...@gr..." <ol...@gr...>; >>"mod...@li..." >><mod...@li...> >> Cc: "owa...@li..." >><owa...@li...> >> Sent: Fri, May 6, 2011 10:29:57 AM >> Subject: Re: [Mod-security-developers] CRS DoS protection & >>x-forwarded-for >>header >> >> Good point about IPv6. Looks like we could still do this using the >>same # >> of SecRules and just not using an IP regex like this - >> >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" >> >>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched >>_v >> ar}" >> SecRule REQUEST_HEADERS:x-forwarded-for "^(.*?),?.*$" >> "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" >> >> SecRule &TX:REAL_IP "@eq 0" >> >>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr >>}_ >> %{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" >> SecRule &TX:REAL_IP "!@eq 0" >> >>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip} >>_% >> {tx.ua_hash}" >> >> >> >> -- >> Ryan >> >> >> >> On 5/6/11 1:16 PM, "Oleg Gryb" <ole...@ya...> wrote: >> >> >Ryan, >> > >> >Thanks for the update. It might have problems with IPv6 though. Here >>are >> >my rules: they search for ',' and disregard the IP structure: >> > >> > >> >SecRule REQUEST_HEADERS:User-Agent "^(.*)$" >> >>>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matche >>>d_ >> >var}" >> >SecRule REQUEST_HEADERS:x-forwarded-for "^(.*)$" >> >"phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" >> >SecRule TX:REAL_IP "^(.*?),.*$" >> >"phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}" >> > >> >SecRule &TX:REAL_IP "@eq 0" >> >>>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_add >>>r} >> >_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" >> >SecRule &TX:REAL_IP "!@eq 0" >> >>>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip >>>}_ >> >%{tx.ua_hash}" >> > >> >Thanks, >> >Oleg. >> > >> >--- On Thu, 5/5/11, Ryan Barnett <RBa...@tr...> wrote: >> > >> >> From: Ryan Barnett <RBa...@tr...> >> >> Subject: Re: [Mod-security-developers] CRS DoS protection & >> >>x-forwarded-for header >> >> To: "Oleg Gryb" <ol...@gr...>, >> >>"mod...@li..." >> >><mod...@li...> >> >> Cc: "owa...@li..." >> >><owa...@li...> >> >> Date: Thursday, May 5, 2011, 10:46 AM >> >> FYI - updated the CRS 10 config file >> >> to add in this logic and uploaded it to SVN - >> >> >> >>>>http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/m >>>>od >> >>security_crs_10_config.conf.example?revision=1772 >> >> >> >> # >> >> # -=[ Global and IP Collections ]=- >> >> # >> >> # Create both Global and IP collections for rules to use >> >> # There are some CRS rules that assume that these two >> >> collections >> >> # have already been initiated. >> >> # >> >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" >> >> >> >>>>"phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_ >>>>ha >> >>sh=%{matched_var}" >> >> SecRule REQUEST_HEADERS:x-forwarded-for >> >> "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b" >> >> >> >>>>"phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1 >>>>}" >> >> SecRule &TX:REAL_IP "!@eq 0" >> >> >> >>>>"phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip >>>>=% >> >>{tx.real_ip}_%{tx.ua_hash}" >> >> SecRule &TX:REAL_IP "@eq 0" >> >> >> >>>>"phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip >>>>=% >> >>{remote_addr}_%{tx.ua_hash}" >> >> >> >> The new rules will grab the first IP address listed in an >> >> X-Forwared-For header and use that for the IP collection >> >> key. If X-Forwarded-For is not present, then it will >> >> use REMOTE_ADDR. >> >> >> >> Thanks for the suggestion! >> >> >> >> -Ryan >> >> >> >> >> >> On 4/28/11 8:19 PM, "Oleg Gryb" >> >><ole...@ya...<mailto:ole...@ya...>> >> >> wrote: >> >> >> >> I've just realized that there might be a problem with >> >> relying on that header: if >> >> an attacker intentionally sends different random IPs in >> >> there, DoS protection >> >> can be efficiently by-passed. In my case it should not >> >> happen, because an LB is >> >> the one who adds the header, but in general we should warn >> >> engineers about the >> >> possible exploit. >> >> >> >> >> >> Actually, even in LB case: if a request has already had the >> >> header, LB will >> >> create a new one with the existing value appended to the >> >> client IP: >> >> >> >> x-forwarded-for: real-client-ip, >> >> whatever-client-sent-to-LB >> >> >> >> It means that we would need to rely on the the first IP in >> >> the list, everything >> >> else should be considered as untrusted. >> >> >> >> Thanks, >> >> Oleg. >> >> >> >> >> >> ----- Original Message ---- >> >> From: Ryan Barnett >> >><RBa...@tr...<mailto:RBa...@tr...>> >> >> To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>; >> >> >> >>>>"mod...@li...<mailto:mod-security-deve >>>>lo >> >>pe...@li...>" >> >> >> >>>><mod...@li...<mailto:mod-security-deve >>>>lo >> >>pe...@li...>> >> >> Cc: >> >>>>"owa...@li...<mailto:owasp-modsecuri >>>>ty >> >>-co...@li...>" >> >> >> >>>><owa...@li...<mailto:owasp-modsecuri >>>>ty >> >>-co...@li...>> >> >> Sent: Thu, April 28, 2011 4:41:14 PM >> >> Subject: Re: [Mod-security-developers] CRS DoS protection >> >> & x-forwarded-for >> >> header >> >> Thanks for the updates Oleg! This will certainly be a >> >> useful update to >> >> not only the DoS rules buy any rules that will be based on >> >> the client IP. >> >> I will actually go back to check other uses of REMOTE_ADDR >> >> and see if we >> >> can swap it for tx.real_ip instead. >> >> I will add this to the CRS v2.2.0 that I am working >> >> on. >> >> For future reference - here is the OWASP CRS >> >> mail-list - >> >> >>https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set >> >> -- >> >> Ryan Barnett >> >> Senior Security Researcher >> >> Trustwave - SpiderLabs >> >> On 4/28/11 7:32 PM, "Oleg Gryb" >> >><ole...@ya...<mailto:ole...@ya...>> >> >> wrote: >> >> >I'm not sure if I can discuss CRS rules here. If not, >> >> please let me know >> >> >what >> >> >the right place is. I want to suggest an >> >> improvement to DoS protection in >> >> >CRS >> >> >2.1.2. The problem is that enterprise >> >> applications usually run behind >> >> >load >> >> >balancers, so relying on remote_addr doesn't make >> >> too much sense, because >> >> >you'll >> >> >always have an LB's IP in there. >> >> > >> >> > >> >> >My improved rules (attached) check for >> >> x-forwarded-for header and if >> >> >it's >> >> >present, this IP will be used to initialize IP >> >> collection. If it's not >> >> >then the >> >> >old logic will be used. >> >> > >> >> >It would be great if we can include this >> >> improvement to the next CRS >> >> >release. >> >> > >> >> >Thanks, >> >> >Oleg. >> >> >> >>>>----------------------------------------------------------------------- >>>>-- >> >>----- >> >> WhatsUp Gold - Download Free Network Management >> >> Software >> >> The most intuitive, comprehensive, and cost-effective >> >> network >> >> management toolset available today. Delivers >> >> lowest initial >> >> acquisition cost and overall TCO of any >> >> competing solution. >> >> http://p.sf.net/sfu/whatsupgold-sd >> >> _______________________________________________ >> >> mod-security-developers mailing list >> >> >> >>>>mod...@li...<mailto:mod-security-devel >>>>op >> >>er...@li...> >> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers >> >> ModSecurity Services from Trustwave's SpiderLabs: >> >> https://www.trustwave.com/spiderLabs.php >> >> >> >> >> >>>>----------------------------------------------------------------------- >>>>-- >> >>----- >> >> WhatsUp Gold - Download Free Network Management Software >> >> The most intuitive, comprehensive, and cost-effective >> >> network >> >> management toolset available today. Delivers lowest >> >> initial >> >> acquisition cost and overall TCO of any competing >> >> solution. >> >> http://p.sf.net/sfu/whatsupgold-sd >> >> _______________________________________________ >> >> mod-security-developers mailing list >> >> >> >>>>mod...@li...<mailto:mod-security-devel >>>>op >> >>er...@li...> >> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers >> >> ModSecurity Services from Trustwave's SpiderLabs: >> >> https://www.trustwave.com/spiderLabs.php >> >> >> >> >> >> >> >> ________________________________ >> >> This transmission may contain information that is >> >> privileged, confidential, and/or exempt from disclosure >> >> under applicable law. If you are not the intended recipient, >> >> you are hereby notified that any disclosure, copying, >> >> distribution, or use of the information contained herein >> >> (including any reliance thereon) is STRICTLY PROHIBITED. If >> >> you received this transmission in error, please immediately >> >> contact the sender and destroy the material in its entirety, >> >> whether in electronic or hard copy format. >> >> >> >> >> > >> >> >> This transmission may contain information that is privileged, >>confidential, >>and/or exempt from disclosure under applicable law. If you are not the >>intended >>recipient, you are hereby notified that any disclosure, copying, >>distribution, >>or use of the information contained herein (including any reliance >>thereon) is >>STRICTLY PROHIBITED. If you received this transmission in error, please >>immediately contact the sender and destroy the material in its entirety, >>whether in electronic or hard copy format. >> >> > This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
From: Oleg G. <ole...@ya...> - 2011-05-07 15:13:46
|
> By the way - with regards to the pcre matching, switch your single/double > quotes around to test - Here is the difference between greedy and non-greedy match: ogryb@debian:~$ perl -e '"1.2.3.4" =~ /^(.*),?.*$/; print $1;' 1.2.3.4ogryb@debian:~$ perl -e '"1.2.3.4" =~ /^(.*?),?.*$/; print $1;' ogryb@debian:~$ But your new rule should work because you've enforced the IP characters that exclude ',' and got rid of non-greedy match.The most importantly, you've achieved your goal of reducing the # of SecRulles :) One tiny suggestion though: ^([0-9a-fA-F\.\:]+),?.*$ I think, we should treat empty values in the same way as cases when no x-forwarded-for header is present. ----- Original Message ---- > From: Ryan Barnett <RBa...@tr...> > To: Oleg Gryb <ol...@gr...>; "mod...@li..." ><mod...@li...> > Cc: "owa...@li..." ><owa...@li...> > Sent: Fri, May 6, 2011 5:26:45 PM > Subject: Re: [Mod-security-developers] CRS DoS protection & x-forwarded-for >header > > This one seems to work in ModSecurity (verified with the debug log). I > matches a single IP, or the 1st one when there a multiple and it handles > IPv6 addresses as well - > > SecRule REQUEST_HEADERS:User-Agent "^(.*)$" \ > >"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched_ > var}" > SecRule REQUEST_HEADERS:x-forwarded-for "^([0-9a-fA-F\.\:]*),?.*$" \ > "phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1} > SecRule &TX:REAL_IP "@eq 0" \ > >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr} > _%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" > SecRule &TX:REAL_IP "!@eq 0" \ > >"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip}_ > %{tx.ua_hash}" > > > > By the way - with regards to the pcre matching, switch your single/double > quotes around to test - > > perl -e '"1.2.3.4" =~ m/^([0-9a-fA-F\.\:]*),?.*$/; print "$1\n";' > > -- > Ryan Barnett > Senior Security Researcher > Trustwave - SpiderLabs > Email: rba...@tr... > Phone: (703) 794-2248 > Cell: (571) 382-0476 > www.trustwave.com > > This transmission may contain information that is privileged, > confidential, and/or exempt from disclosure under applicable law. If you > are not the intended recipient, you are hereby notified that any > disclosure, copying, distribution, or use of the information contained > herein (including any reliance thereon) is STRICTLY PROHIBITED. If you > received this transmission in error, please immediately contact the sender > and destroy the material in its entirety, whether in electronic or hard > copy format. Thank you. > > > > On 5/6/11 5:41 PM, "Oleg Gryb" <ole...@ya...> wrote: > > >Ryan, > >That might not work for a single IP in x-forwarded-for header, becuase > >non-greedy match for (.*?) will return empty string. > > > >Try: > > > >perl -e "'1.2.3.4' =~ /^(.*?),?.*$/; print $1;" > > > >and you'll see what I mean. I'm assuming here that modsecurity uses PCRE. > > > >Thanks, > >Oleg. > > > > > > > > > > > > > >----- Original Message ---- > >> From: Ryan Barnett <RBa...@tr...> > >> To: "ol...@gr..." <ol...@gr...>; > >>"mod...@li..." > >><mod...@li...> > >> Cc: "owa...@li..." > >><owa...@li...> > >> Sent: Fri, May 6, 2011 10:29:57 AM > >> Subject: Re: [Mod-security-developers] CRS DoS protection & > >>x-forwarded-for > >>header > >> > >> Good point about IPv6. Looks like we could still do this using the > >>same # > >> of SecRules and just not using an IP regex like this - > >> > >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > >> > >>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matched > >>_v > >> ar}" > >> SecRule REQUEST_HEADERS:x-forwarded-for "^(.*?),?.*$" > >> "phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" > >> > >> SecRule &TX:REAL_IP "@eq 0" > >> > >>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_addr > >>}_ > >> %{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" > >> SecRule &TX:REAL_IP "!@eq 0" > >> > >>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip} > >>_% > >> {tx.ua_hash}" > >> > >> > >> > >> -- > >> Ryan > >> > >> > >> > >> On 5/6/11 1:16 PM, "Oleg Gryb" <ole...@ya...> wrote: > >> > >> >Ryan, > >> > > >> >Thanks for the update. It might have problems with IPv6 though. Here > >>are > >> >my rules: they search for ',' and disregard the IP structure: > >> > > >> > > >> >SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > >> > >>>"phase:1,t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_hash=%{matche > >>>d_ > >> >var}" > >> >SecRule REQUEST_HEADERS:x-forwarded-for "^(.*)$" > >> >"phase:1,t:none,pass,nolog,setvar:tx.real_ip=%{matched_var}" > >> >SecRule TX:REAL_IP "^(.*?),.*$" > >> >"phase:1,t:none,pass,nolog,capture,setvar:tx.real_ip=%{TX.1}" > >> > > >> >SecRule &TX:REAL_IP "@eq 0" > >> > >>>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{remote_add > >>>r} > >> >_%{tx.ua_hash},,setvar:tx.real_ip=%{remote_addr}" > >> >SecRule &TX:REAL_IP "!@eq 0" > >> > >>>"phase:1,t:none,pass,nolog,initcol:global=global,initcol:ip=%{tx.real_ip > >>>}_ > >> >%{tx.ua_hash}" > >> > > >> >Thanks, > >> >Oleg. > >> > > >> >--- On Thu, 5/5/11, Ryan Barnett <RBa...@tr...> wrote: > >> > > >> >> From: Ryan Barnett <RBa...@tr...> > >> >> Subject: Re: [Mod-security-developers] CRS DoS protection & > >> >>x-forwarded-for header > >> >> To: "Oleg Gryb" <ol...@gr...>, > >> >>"mod...@li..." > >> >><mod...@li...> > >> >> Cc: "owa...@li..." > >> >><owa...@li...> > >> >> Date: Thursday, May 5, 2011, 10:46 AM > >> >> FYI - updated the CRS 10 config file > >> >> to add in this logic and uploaded it to SVN - > >> >> > >> > >>>>http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/trunk/m > >>>>od > >> >>security_crs_10_config.conf.example?revision=1772 > >> >> > >> >> # > >> >> # -=[ Global and IP Collections ]=- > >> >> # > >> >> # Create both Global and IP collections for rules to use > >> >> # There are some CRS rules that assume that these two > >> >> collections > >> >> # have already been initiated. > >> >> # > >> >> SecRule REQUEST_HEADERS:User-Agent "^(.*)$" > >> >> > >> > >>>>"phase:1,id:'981217',t:none,pass,nolog,t:sha1,t:hexEncode,setvar:tx.ua_ > >>>>ha > >> >>sh=%{matched_var}" > >> >> SecRule REQUEST_HEADERS:x-forwarded-for > >> >> "^\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b" > >> >> > >> > >>>>"phase:1,id:'981225',t:none,pass,nolog,capture,setvar:tx.real_ip=%{tx.1 > >>>>}" > >> >> SecRule &TX:REAL_IP "!@eq 0" > >> >> > >> > >>>>"phase:1,id:'981226',t:none,pass,nolog,initcol:global=global,initcol:ip > >>>>=% > >> >>{tx.real_ip}_%{tx.ua_hash}" > >> >> SecRule &TX:REAL_IP "@eq 0" > >> >> > >> > >>>>"phase:1,id:'981218',t:none,pass,nolog,initcol:global=global,initcol:ip > >>>>=% > >> >>{remote_addr}_%{tx.ua_hash}" > >> >> > >> >> The new rules will grab the first IP address listed in an > >> >> X-Forwared-For header and use that for the IP collection > >> >> key. If X-Forwarded-For is not present, then it will > >> >> use REMOTE_ADDR. > >> >> > >> >> Thanks for the suggestion! > >> >> > >> >> -Ryan > >> >> > >> >> > >> >> On 4/28/11 8:19 PM, "Oleg Gryb" > >> >><ole...@ya...<mailto:ole...@ya...>> > >> >> wrote: > >> >> > >> >> I've just realized that there might be a problem with > >> >> relying on that header: if > >> >> an attacker intentionally sends different random IPs in > >> >> there, DoS protection > >> >> can be efficiently by-passed. In my case it should not > >> >> happen, because an LB is > >> >> the one who adds the header, but in general we should warn > >> >> engineers about the > >> >> possible exploit. > >> >> > >> >> > >> >> Actually, even in LB case: if a request has already had the > >> >> header, LB will > >> >> create a new one with the existing value appended to the > >> >> client IP: > >> >> > >> >> x-forwarded-for: real-client-ip, > >> >> whatever-client-sent-to-LB > >> >> > >> >> It means that we would need to rely on the the first IP in > >> >> the list, everything > >> >> else should be considered as untrusted. > >> >> > >> >> Thanks, > >> >> Oleg. > >> >> > >> >> > >> >> ----- Original Message ---- > >> >> From: Ryan Barnett > >> >><RBa...@tr...<mailto:RBa...@tr...>> > >> >> To: Oleg Gryb <ol...@gr...<mailto:ol...@gr...>>; > >> >> > >> > >>>>"mod...@li...<mailto:mod-security-deve > >>>>lo > >> >>pe...@li...>" > >> >> > >> > >>>><mod...@li...<mailto:mod-security-deve > >>>>lo > >> >>pe...@li...>> > >> >> Cc: > >> > >>>>"owa...@li...<mailto:owasp-modsecuri > >>>>ty > >> >>-co...@li...>" > >> >> > >> > >>>><owa...@li...<mailto:owasp-modsecuri > >>>>ty > >> >>-co...@li...>> > >> >> Sent: Thu, April 28, 2011 4:41:14 PM > >> >> Subject: Re: [Mod-security-developers] CRS DoS protection > >> >> & x-forwarded-for > >> >> header > >> >> Thanks for the updates Oleg! This will certainly be a > >> >> useful update to > >> >> not only the DoS rules buy any rules that will be based on > >> >> the client IP. > >> >> I will actually go back to check other uses of REMOTE_ADDR > >> >> and see if we > >> >> can swap it for tx.real_ip instead. > >> >> I will add this to the CRS v2.2.0 that I am working > >> >> on. > >> >> For future reference - here is the OWASP CRS > >> >> mail-list - > >> >> > >>https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set > >> >> -- > >> >> Ryan Barnett > >> >> Senior Security Researcher > >> >> Trustwave - SpiderLabs > >> >> On 4/28/11 7:32 PM, "Oleg Gryb" > >> >><ole...@ya...<mailto:ole...@ya...>> > >> >> wrote: > >> >> >I'm not sure if I can discuss CRS rules here. If not, > >> >> please let me know > >> >> >what > >> >> >the right place is. I want to suggest an > >> >> improvement to DoS protection in > >> >> >CRS > >> >> >2.1.2. The problem is that enterprise > >> >> applications usually run behind > >> >> >load > >> >> >balancers, so relying on remote_addr doesn't make > >> >> too much sense, because > >> >> >you'll > >> >> >always have an LB's IP in there. > >> >> > > >> >> > > >> >> >My improved rules (attached) check for > >> >> x-forwarded-for header and if > >> >> >it's > >> >> >present, this IP will be used to initialize IP > >> >> collection. If it's not > >> >> >then the > >> >> >old logic will be used. > >> >> > > >> >> >It would be great if we can include this > >> >> improvement to the next CRS > >> >> >release. > >> >> > > >> >> >Thanks, > >> >> >Oleg. > >> >> > >> > >>>>----------------------------------------------------------------------- > >>>>-- > >> >>----- > >> >> WhatsUp Gold - Download Free Network Management > >> >> Software > >> >> The most intuitive, comprehensive, and cost-effective > >> >> network > >> >> management toolset available today. Delivers > >> >> lowest initial > >> >> acquisition cost and overall TCO of any > >> >> competing solution. > >> >> http://p.sf.net/sfu/whatsupgold-sd > >> >> _______________________________________________ > >> >> mod-security-developers mailing list > >> >> > >> > >>>>mod...@li...<mailto:mod-security-devel > >>>>op > >> >>er...@li...> > >> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers > >> >> ModSecurity Services from Trustwave's SpiderLabs: > >> >> https://www.trustwave.com/spiderLabs.php > >> >> > >> >> > >> > >>>>----------------------------------------------------------------------- > >>>>-- > >> >>----- > >> >> WhatsUp Gold - Download Free Network Management Software > >> >> The most intuitive, comprehensive, and cost-effective > >> >> network > >> >> management toolset available today. Delivers lowest > >> >> initial > >> >> acquisition cost and overall TCO of any competing > >> >> solution. > >> >> http://p.sf.net/sfu/whatsupgold-sd > >> >> _______________________________________________ > >> >> mod-security-developers mailing list > >> >> > >> > >>>>mod...@li...<mailto:mod-security-devel > >>>>op > >> >>er...@li...> > >> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers > >> >> ModSecurity Services from Trustwave's SpiderLabs: > >> >> https://www.trustwave.com/spiderLabs.php > >> >> > >> >> > >> >> > >> >> ________________________________ > >> >> This transmission may contain information that is > >> >> privileged, confidential, and/or exempt from disclosure > >> >> under applicable law. If you are not the intended recipient, > >> >> you are hereby notified that any disclosure, copying, > >> >> distribution, or use of the information contained herein > >> >> (including any reliance thereon) is STRICTLY PROHIBITED. If > >> >> you received this transmission in error, please immediately > >> >> contact the sender and destroy the material in its entirety, > >> >> whether in electronic or hard copy format. > >> >> > >> >> > >> > > >> > >> > >> This transmission may contain information that is privileged, > >>confidential, > >>and/or exempt from disclosure under applicable law. If you are not the > >>intended > >>recipient, you are hereby notified that any disclosure, copying, > >>distribution, > >>or use of the information contained herein (including any reliance > >>thereon) is > >>STRICTLY PROHIBITED. If you received this transmission in error, please > >>immediately contact the sender and destroy the material in its entirety, > >>whether in electronic or hard copy format. > >> > >> > > > > > This transmission may contain information that is privileged, confidential, >and/or exempt from disclosure under applicable law. If you are not the intended >recipient, you are hereby notified that any disclosure, copying, distribution, >or use of the information contained herein (including any reliance thereon) is >STRICTLY PROHIBITED. If you received this transmission in error, please >immediately contact the sender and destroy the material in its entirety, >whether in electronic or hard copy format. > > > ------------------------------------------------------------------------------ > WhatsUp Gold - Download Free Network Management Software > The most intuitive, comprehensive, and cost-effective network > management toolset available today. Delivers lowest initial > acquisition cost and overall TCO of any competing solution. > http://p.sf.net/sfu/whatsupgold-sd > _______________________________________________ > mod-security-developers mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-developers > ModSecurity Services from Trustwave's SpiderLabs: > https://www.trustwave.com/spiderLabs.php > |