Thread: [mod-security-users] regex for matching urls
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
mod-security-users
|
From: Peter W. <prw...@gm...> - 2005-03-07 16:32:28
|
Greetings,
I'm trying to create a mod_security rule that uses regexes to match a
certain piece of text, but only if it is found within a URL. Here is
the initial attempt:
SecFilterSelective HTTP_Referer|ARGS
"[a-z]+://[\w\-_.]*poker[\w\-_.]*\.[a-z]{2,}"
This does *not* match 'poker' if it is found within a URL (e.g.
http://www.high-stakes-poker.com/). However, if you take the
'[a-z]+://' off of the beginning, it *does* work:
SecFilterSelective HTTP_Referer|ARGS "[\w\-_.]*poker[\w\-_.]*\.[a-z]{2,}"
What is wrong with '[a-z]+://' ? I tried escaping the /'s, and that
didn't seem to make any difference.
Thanks,
Peter
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
|
From: Ivan R. <iv...@we...> - 2005-03-07 16:40:32
|
Peter Wood wrote:
> Greetings,
> What is wrong with '[a-z]+://' ?
Before regular expression is applied to a piece of data
mod_security performs data normalization and reduces
redundant forward slashes. Thus "http://" becomes "http:/".
(No, I don't like it either. That's why in 1.9 normalization
will become optional and configurable per-rule.)
--
Ivan Ristic
Apache Security (O'Reilly) - http://www.apachesecurity.net
Open source web application firewall - http://www.modsecurity.org
|
|
From: Peter W. <prw...@gm...> - 2005-03-07 16:45:46
|
Ivan,
Thanks for the response. Can you suggest any way to work around this
so that we can match 'http://'? Would '/{2}' work, or would that also
be normalized?
Thanks,
Peter
On Mon, 07 Mar 2005 16:45:58 +0000, Ivan Ristic <iv...@we...> wrote:
> Peter Wood wrote:
> > Greetings,
>
> > What is wrong with '[a-z]+://' ?
>
> Before regular expression is applied to a piece of data
> mod_security performs data normalization and reduces
> redundant forward slashes. Thus "http://" becomes "http:/".
>
> (No, I don't like it either. That's why in 1.9 normalization
> will become optional and configurable per-rule.)
>
> --
> Ivan Ristic
> Apache Security (O'Reilly) - http://www.apachesecurity.net
> Open source web application firewall - http://www.modsecurity.org
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
|
From: Ivan R. <iv...@we...> - 2005-03-07 16:52:54
|
Peter Wood wrote:
> Ivan,
>
> Thanks for the response. Can you suggest any way to work around this
> so that we can match 'http://'? Would '/{2}' work, or would that also
> be normalized?
Normalisation is not performed against the regex, only against
the incoming data. In the regex just use "http:/" instead of
"http://" and you should be fine.
--
Ivan Ristic
Apache Security (O'Reilly) - http://www.apachesecurity.net
Open source web application firewall - http://www.modsecurity.org
|
|
From: Peter W. <prw...@gm...> - 2005-03-07 16:57:10
|
Ah, excellent! I just took the extra / out and it works great now. Thanks!
On Mon, 07 Mar 2005 16:58:26 +0000, Ivan Ristic <iv...@we...> wrote:
> Peter Wood wrote:
> > Ivan,
> >
> > Thanks for the response. Can you suggest any way to work around this
> > so that we can match 'http://'? Would '/{2}' work, or would that also
> > be normalized?
>
> Normalisation is not performed against the regex, only against
> the incoming data. In the regex just use "http:/" instead of
> "http://" and you should be fine.
>
> --
> Ivan Ristic
> Apache Security (O'Reilly) - http://www.apachesecurity.net
> Open source web application firewall - http://www.modsecurity.org
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
|
From: Peter W. <prw...@gm...> - 2005-03-07 17:02:56
|
P.S.
The mod_security reference manual is excellent. Had I bothered to
fully read the section on Path Normalization, I could have avoided
asking this question. :-)
Peter
On Mon, 07 Mar 2005 16:58:26 +0000, Ivan Ristic <iv...@we...> wrote:
> Peter Wood wrote:
> > Ivan,
> >
> > Thanks for the response. Can you suggest any way to work around this
> > so that we can match 'http://'? Would '/{2}' work, or would that also
> > be normalized?
>
> Normalisation is not performed against the regex, only against
> the incoming data. In the regex just use "http:/" instead of
> "http://" and you should be fine.
>
> --
> Ivan Ristic
> Apache Security (O'Reilly) - http://www.apachesecurity.net
> Open source web application firewall - http://www.modsecurity.org
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
|
From: Ivan R. <iv...@we...> - 2005-03-09 10:00:25
|
Peter Wood wrote: > P.S. > > The mod_security reference manual is excellent. Had I bothered to > fully read the section on Path Normalization, I could have avoided > asking this question. :-) Thanks. The manual does contain everything I know of ModSecurity, or at least everything I know I know. In retrospective, I wish it were more structured (e.g. a proper reference manual) but I "grew" it organically and it shows. -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: Peter W. <prw...@gm...> - 2005-03-07 20:24:13
|
I'm having someone help me test these rules. On my machine, this rule
matches just fine. On his machine, it doesn't match at all. He is
running mod_security 1.8.6, I'm running 1.8.7. Is there any difference
in between the two that might cause this not to match?
SecFilterSelective HTTP_Referer|ARGS
"[a-z]+:/[\w\-_.]*online[\w\-_.]*\.[a-z]{2,}"
Matches e.g. http://www.buy-online.com/ on my machine, but not on his.
- Peter
On Mon, 07 Mar 2005 16:58:26 +0000, Ivan Ristic <iv...@we...> wrote:
> Peter Wood wrote:
> > Ivan,
> >
> > Thanks for the response. Can you suggest any way to work around this
> > so that we can match 'http://'? Would '/{2}' work, or would that also
> > be normalized?
>
> Normalisation is not performed against the regex, only against
> the incoming data. In the regex just use "http:/" instead of
> "http://" and you should be fine.
>
> --
> Ivan Ristic
> Apache Security (O'Reilly) - http://www.apachesecurity.net
> Open source web application firewall - http://www.modsecurity.org
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
|
From: Ivan R. <iv...@we...> - 2005-03-07 20:40:21
|
Peter Wood wrote: > I'm having someone help me test these rules. On my machine, this rule > matches just fine. On his machine, it doesn't match at all. He is > running mod_security 1.8.6, I'm running 1.8.7. Is there any difference > in between the two that might cause this not to match? No, there shouldn't be any. Is he running Apache 1.x and you Apache 2.x? These two have completely different regular expression engines. (The one in Apache 2.x being much better.) In my experience (I can't test now) \w does not work in Apache 1.x. (Neither do \d and \s.) The really interesting thing is that the documentation for Apache 1.x regex engine does not exist. Try replacing \w with [:word:], but beware [:word:] works only when used in [ and ]. So [\w\-_.]* becomes [[:word:]\-_.] -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: Peter W. <prw...@gm...> - 2005-03-07 20:42:35
|
> > Is he running Apache 1.x and you Apache 2.x? > I'm running Apache 2.x... not sure what he's running, but you may be on to something here. I keep forgetting that not everybody has jumped on the Apache 2.x bandwagon. :-) > In my experience (I can't test now) \w does not work in Apache 1.x. > (Neither do \d and \s.) The really interesting thing is that the > documentation for Apache 1.x regex engine does not exist. > > Try replacing \w with [:word:], but beware [:word:] works only > when used in [ and ]. So [\w\-_.]* becomes [[:word:]\-_.] Thanks, I'll have him try this. Peter -- Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/ |
|
From: Tom A. <tan...@oa...> - 2005-03-07 20:48:00
|
----- Original Message ----- From: "Ivan Ristic" <iv...@we...> > Try replacing \w with [:word:], but beware [:word:] works only > when used in [ and ]. So [\w\-_.]* becomes [[:word:]\-_.] Or you could be a little more liberal and replace [\w\-_.]* with [^/]* Tom |
|
From: Peter W. <prw...@gm...> - 2005-03-07 16:55:44
|
Hrm, never mind, I just tried that, and it didn't work either... any
other way around it?
On Mon, 7 Mar 2005 11:45:37 -0500, Peter Wood <prw...@gm...> wrote:
> Ivan,
>
> Thanks for the response. Can you suggest any way to work around this
> so that we can match 'http://'? Would '/{2}' work, or would that also
> be normalized?
>
> Thanks,
>
> Peter
>
>
> On Mon, 07 Mar 2005 16:45:58 +0000, Ivan Ristic <iv...@we...> wrote:
> > Peter Wood wrote:
> > > Greetings,
> >
> > > What is wrong with '[a-z]+://' ?
> >
> > Before regular expression is applied to a piece of data
> > mod_security performs data normalization and reduces
> > redundant forward slashes. Thus "http://" becomes "http:/".
> >
> > (No, I don't like it either. That's why in 1.9 normalization
> > will become optional and configurable per-rule.)
> >
> > --
> > Ivan Ristic
> > Apache Security (O'Reilly) - http://www.apachesecurity.net
> > Open source web application firewall - http://www.modsecurity.org
> >
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >
>
> --
> Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
>
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
|
From: Tom A. <tan...@oa...> - 2005-03-07 17:48:41
|
SecFilterSelective HTTP_Referer|ARGS
"[a-z]+:/+[\w\-_.]*poker[\w\-_.]*\.[a-z]{2,}"
That will work whether there's one or two (or more) slashes. For just one
or two, try:
SecFilterSelective HTTP_Referer|ARGS
"[a-z]+:/{1,2}[\w\-_.]*poker[\w\-_.]*\.[a-z]{2,}"
Then it will match whether you use normalization or not. Better still:
SecFilterSelective HTTP_Referer|ARGS
"(ht|f)tps?:/{1,2}[\w\-_.]*poker[\w\-_.]*\.[a-z]{2,4}(/|$)"
Tom
----- Original Message -----
From: "Peter Wood" <prw...@gm...>
To: <mod...@li...>
Sent: Monday, March 07, 2005 11:55 AM
Subject: Re: [mod-security-users] regex for matching urls
> Hrm, never mind, I just tried that, and it didn't work either... any
> other way around it?
>
>
> On Mon, 7 Mar 2005 11:45:37 -0500, Peter Wood <prw...@gm...> wrote:
>> Ivan,
>>
>> Thanks for the response. Can you suggest any way to work around this
>> so that we can match 'http://'? Would '/{2}' work, or would that also
>> be normalized?
>>
>> Thanks,
>>
>> Peter
>>
>>
>> On Mon, 07 Mar 2005 16:45:58 +0000, Ivan Ristic <iv...@we...>
>> wrote:
>> > Peter Wood wrote:
>> > > Greetings,
>> >
>> > > What is wrong with '[a-z]+://' ?
>> >
>> > Before regular expression is applied to a piece of data
>> > mod_security performs data normalization and reduces
>> > redundant forward slashes. Thus "http://" becomes "http:/".
>> >
>> > (No, I don't like it either. That's why in 1.9 normalization
>> > will become optional and configurable per-rule.)
>> >
>> > --
>> > Ivan Ristic
>> > Apache Security (O'Reilly) - http://www.apachesecurity.net
>> > Open source web application firewall - http://www.modsecurity.org
>> >
>> > -------------------------------------------------------
>> > SF email is sponsored by - The IT Product Guide
>> > Read honest & candid reviews on hundreds of IT Products from real
>> > users.
>> > Discover which products truly live up to the hype. Start reading now.
>> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
>> > _______________________________________________
>> > mod-security-users mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >
>>
>> --
>> Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
>>
>
>
> --
> Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
|
|
From: Peter W. <prw...@gm...> - 2005-03-07 18:16:04
|
Tom,
> Then it will match whether you use normalization or not. Better still:
>
> SecFilterSelective HTTP_Referer|ARGS
> "(ht|f)tps?:/{1,2}[\w\-_.]*poker[\w\-_.]*\.[a-z]{2,4}(/|$)"
>
Thanks for this. I had considered using {2,4} for the TLD... as far as
I know there aren't any TLD's with length > 4, but it could always
happen, which is why I had left that as just {2,}... also I'd
personally prefer to be a bit liberal as to what can appear before the
:// in a URL... you never know what crazy schemes attackers will think
up...
Peter
--
Peter R. Wood | email: prw...@gm... | blog: http://prwdot.org/
|
Thanks for helping keep SourceForge clean.
X