|
From: Riemann . <rie...@gm...> - 2016-09-07 21:05:00
|
Hello,
I'm trying to write rules to track and log (not block) brute force login
attempts. I've initialized the Global and IP collection in CRS_10, and have
created a separate rule/config file for horizontal brute force attempts.
I've included my attempt at a horizontal brute force rule below ("rule
one"). The rule works exactly as I'd expect if I use Burp Intruder to
simulate a brute force attack, sending POST requests to login.jsp, with
"username" and "password" parameters in the request body, and changing only
the username on each request.
If I run the exact same 'attack,' but this time with five threads, I run
into problems. My counter may equal '1' for five consecutive requests, and
then jump to '6', but only show two usernames are stored in IP.username
(for example). I've also tried using rules similar to the horizontal brute
force rules in The Web Application Defender's Cookbook (recipe 7-2, pg
269-271), and found I had similar issues with multi-threaded attacks for
those rules too.
I know SDBM allows for concurrent access, and I've read (and re-read) about
how collections work in Chapter 8 of the ModSecurity Handbook (including
the parts about storing non-numeric values in SDBM, and race conditions).
In light of that I've attempted to use only numeric values by eliminating
all stored variables except the counter, as seen in "rule two" below (and
completely eliminating the tracking of unique usernames). Still, I had
issues where multi-threading a brute force attack resulted in the counter
value not being correct for several requests in a row.
So my questions:
1. Am I missing something, or doing something fundamentally wrong in how
I'm using persistent storage?
2. I realize the potential for race conditions with SDBM, especially
with non-numeric data, but is what I'm describing entirely expected
behavior? Numeric values in persistent storage seem to be fairly unreliable
on a per-request basis (in logs, my counter is wrong more than it is right).
3. Is there a better approach to accomplish what I'm trying to do, which
is track horizontal (by IP addr and username), and eventually vertical (by
IP addr and password hash) brute force attempts? For example, using Lua, or
some other persistent storage mechanism (memcached isn't an option).
If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23,
running on Windows.
Thanks,
Riemann
########################################
# 'Rule' One #
###########
SecRule REQUEST_FILENAME "!@pm login.jsp" \
"phase:2,t:none,pass,nolog, \
id:999321, \
skipAfter:END_BRUTE_FORCE"
SecRule REQUEST_METHOD "!@streq POST" \
"phase:2,t:none,pass,nolog, \
id:999322, \
skipAfter:END_BRUTE_FORCE"
SecRule ARGS:username|ARGS:user ".+" \
"chain,phase:2,t:none,t:lowercase,pass,nolog, \
id:999323, \
skipAfter:LOGIN_TEST"
SecRule ARGS:password|ARGS:pass ".+" \
"t:none,t:lowercase"
SecAction \
"phase:2,pass,nolog, \
skipAfter:END_BRUTE_FORCE, \
id:'1000950'"
SecMarker LOGIN_TEST
SecRule IP:username_ct "@ge 5" \
"phase:5,t:none,log,pass, \
id:'1000951', \
msg:'Multiple Username Violation - Too Many Usernames Submitted from
%{REMOTE_ADDR}.' \
logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted Usernames:
%{IP.username}.', \
setvar:IP.clear_username=1"
SecRule &IP:CLEAR_USERNAME "@eq 1" \
"phase:5,pass,nolog, \
id:'1000952', \
setvar:!IP.KEY"
SecRule &IP:username_ct "@eq 0" \
"chain,phase:2,pass,nolog, \
id:'1000953'"
SecRule ARGS:username|ARGS:user ".+" \
"t:none,t:lowercase, \
setvar:IP.username_ct=+1, \
expirevar:IP.username_ct=300, \
setvar:IP.username=%{matched_var}, \
expirevar:IP.username=300"
SecRule &IP:username_ct "@ge 1" \
"chain,phase:2,pass,nolog, \
id:'1000954'"
SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \
"t:none,t:lowercase, \
setvar:IP.username_ct=+1, \
expirevar:IP.username_ct=300, \
setvar:'IP.username=%{IP.username},%{matched_var}', \
expirevar:IP.username=300"
SecMarker END_BRUTE_FORCE
########################################
# 'Rule' Two #
###########
SecRule REQUEST_FILENAME "!@pm login.jsp" \
"phase:2,t:none,pass,nolog, \
id:999321, \
skipAfter:END_BRUTE_FORCE"
SecRule REQUEST_METHOD "!@streq POST" \
"phase:2,t:none,pass,nolog, \
id:999322, \
skipAfter:END_BRUTE_FORCE"
SecRule ARGS:username|ARGS:user ".+" \
"chain,phase:2,t:none,t:lowercase,pass,nolog, \
id:999323, \
skipAfter:LOGIN_TEST"
SecRule ARGS:password|ARGS:pass ".+" \
"t:none,t:lowercase"
SecAction \
"phase:2,pass,nolog, \
skipAfter:END_BRUTE_FORCE, \
id:'1000950'"
SecAction \
"phase:2,pass,log, \
id:'1000959', \
logdata:'Usernames Attempted Count: %{IP.username_ct}.'"
SecRule IP:username_ct "@ge 7" \
"phase:2,t:none,log,pass, \
id:'1000951', \
msg:'Multiple Username Violation - Too Many Usernames Submitted from
%{REMOTE_ADDR}.' \
logdata:'Usernames Attempted Count: %{IP.username_ct}.', \
setvar:IP.clear_username=1"
SecRule &IP:CLEAR_USERNAME "@eq 1" \
"phase:2,pass,nolog, \
id:'1000952', \
setvar:!IP.KEY"
SecRule ARGS:username|ARGS:user ".+" \
"t:none,t:lowercase,pass,nolog, \
id:'1000953', \
setvar:IP.username_ct=+1, \
expirevar:IP.username_ct=300"
SecMarker END_BRUTE_FORCE
|
|
From: Barry P. <bar...@ho...> - 2016-09-07 21:29:50
|
Are you seeing any errors like this in your logs?: Message: collections_remove_stale: Failed deleting collection I'm really not convinced this works using disk based collections file - especially under stress (like brute force attacks). So ultimately don't think ModSecurity is right solution for this - at least until a better, memory based collections store is available. See more details when this was discussed before on this list here: https://sourceforge.net/p/mod-security/mailman/message/34392875/ Thanks, Barry On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...<mailto:rie...@gm...>> wrote: Hello, I'm trying to write rules to track and log (not block) brute force login attempts. I've initialized the Global and IP collection in CRS_10, and have created a separate rule/config file for horizontal brute force attempts. I've included my attempt at a horizontal brute force rule below ("rule one"). The rule works exactly as I'd expect if I use Burp Intruder to simulate a brute force attack, sending POST requests to login.jsp, with "username" and "password" parameters in the request body, and changing only the username on each request. If I run the exact same 'attack,' but this time with five threads, I run into problems. My counter may equal '1' for five consecutive requests, and then jump to '6', but only show two usernames are stored in IP.username (for example). I've also tried using rules similar to the horizontal brute force rules in The Web Application Defender's Cookbook (recipe 7-2, pg 269-271), and found I had similar issues with multi-threaded attacks for those rules too. I know SDBM allows for concurrent access, and I've read (and re-read) about how collections work in Chapter 8 of the ModSecurity Handbook (including the parts about storing non-numeric values in SDBM, and race conditions). In light of that I've attempted to use only numeric values by eliminating all stored variables except the counter, as seen in "rule two" below (and completely eliminating the tracking of unique usernames). Still, I had issues where multi-threading a brute force attack resulted in the counter value not being correct for several requests in a row. So my questions: 1. Am I missing something, or doing something fundamentally wrong in how I'm using persistent storage? 2. I realize the potential for race conditions with SDBM, especially with non-numeric data, but is what I'm describing entirely expected behavior? Numeric values in persistent storage seem to be fairly unreliable on a per-request basis (in logs, my counter is wrong more than it is right). 3. Is there a better approach to accomplish what I'm trying to do, which is track horizontal (by IP addr and username), and eventually vertical (by IP addr and password hash) brute force attempts? For example, using Lua, or some other persistent storage mechanism (memcached isn't an option). If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, running on Windows. Thanks, Riemann ######################################## # 'Rule' One # ########### SecRule REQUEST_FILENAME "!@pm login.jsp" \ "phase:2,t:none,pass,nolog, \ id:999321, \ skipAfter:END_BRUTE_FORCE" SecRule REQUEST_METHOD "!@streq POST" \ "phase:2,t:none,pass,nolog, \ id:999322, \ skipAfter:END_BRUTE_FORCE" SecRule ARGS:username|ARGS:user ".+" \ "chain,phase:2,t:none,t:lowercase,pass,nolog, \ id:999323, \ skipAfter:LOGIN_TEST" SecRule ARGS:password|ARGS:pass ".+" \ "t:none,t:lowercase" SecAction \ "phase:2,pass,nolog, \ skipAfter:END_BRUTE_FORCE, \ id:'1000950'" SecMarker LOGIN_TEST SecRule IP:username_ct "@ge 5" \ "phase:5,t:none,log,pass, \ id:'1000951', \ msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \ logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted Usernames: %{IP.username}.', \ setvar:IP.clear_username=1" SecRule &IP:CLEAR_USERNAME "@eq 1" \ "phase:5,pass,nolog, \ id:'1000952', \ setvar:!IP.KEY" SecRule &IP:username_ct "@eq 0" \ "chain,phase:2,pass,nolog, \ id:'1000953'" SecRule ARGS:username|ARGS:user ".+" \ "t:none,t:lowercase, \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300, \ setvar:IP.username=%{matched_var}, \ expirevar:IP.username=300" SecRule &IP:username_ct "@ge 1" \ "chain,phase:2,pass,nolog, \ id:'1000954'" SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ "t:none,t:lowercase, \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300, \ setvar:'IP.username=%{IP.username},%{matched_var}', \ expirevar:IP.username=300" SecMarker END_BRUTE_FORCE ######################################## # 'Rule' Two # ########### SecRule REQUEST_FILENAME "!@pm login.jsp" \ "phase:2,t:none,pass,nolog, \ id:999321, \ skipAfter:END_BRUTE_FORCE" SecRule REQUEST_METHOD "!@streq POST" \ "phase:2,t:none,pass,nolog, \ id:999322, \ skipAfter:END_BRUTE_FORCE" SecRule ARGS:username|ARGS:user ".+" \ "chain,phase:2,t:none,t:lowercase,pass,nolog, \ id:999323, \ skipAfter:LOGIN_TEST" SecRule ARGS:password|ARGS:pass ".+" \ "t:none,t:lowercase" SecAction \ "phase:2,pass,nolog, \ skipAfter:END_BRUTE_FORCE, \ id:'1000950'" SecAction \ "phase:2,pass,log, \ id:'1000959', \ logdata:'Usernames Attempted Count: %{IP.username_ct}.'" SecRule IP:username_ct "@ge 7" \ "phase:2,t:none,log,pass, \ id:'1000951', \ msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \ logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ setvar:IP.clear_username=1" SecRule &IP:CLEAR_USERNAME "@eq 1" \ "phase:2,pass,nolog, \ id:'1000952', \ setvar:!IP.KEY" SecRule ARGS:username|ARGS:user ".+" \ "t:none,t:lowercase,pass,nolog, \ id:'1000953', \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300" SecMarker END_BRUTE_FORCE ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Riemann . <rie...@gm...> - 2016-09-07 23:01:36
|
Hi Barry, Thanks for your response and the link to the previous discussion. I do see the error you mentioned in my debug log. This is a bummer. In the ModSec Handbook, Ivan seems to infer the double-retrieve/write-lock mechanism should prevent race conditions with numerical data being stored (pg 129-130), but this just doesn't seem to be the case. Maybe this is a bad idea, but as an alternative to using persistent storage, would it be crazy to try to implement some of these rules by passing variables to a Lua script that reads from and writes to an instance-specific SQLite or enterprise-class database? Are there any other ideas as to how this can be acheived? Of course, it would be ideal to integrate some of these types of things directly into the application, but it's a lot faster to turn rules out with ModSec, than to try to get them fit into a release cycle that may be a few months (or more away), depending on development's other priorities. Thanks, Riemann On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...> wrote: > Are you seeing any errors like this in your logs?: > > Message: collections_remove_stale: Failed deleting collection > > I'm really not convinced this works using disk based collections file - > especially under stress (like brute force attacks). So ultimately don't > think ModSecurity is right solution for this - at least until a better, > memory based collections store is available. > > See more details when this was discussed before on this list here: > https://sourceforge.net/p/mod-security/mailman/message/34392875/ > > Thanks, > Barry > > On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...> wrote: > > Hello, > > I'm trying to write rules to track and log (not block) brute force login > attempts. I've initialized the Global and IP collection in CRS_10, and have > created a separate rule/config file for horizontal brute force attempts. > I've included my attempt at a horizontal brute force rule below ("rule > one"). The rule works exactly as I'd expect if I use Burp Intruder to > simulate a brute force attack, sending POST requests to login.jsp, with > "username" and "password" parameters in the request body, and changing only > the username on each request. > > If I run the exact same 'attack,' but this time with five threads, I run > into problems. My counter may equal '1' for five consecutive requests, and > then jump to '6', but only show two usernames are stored in IP.username > (for example). I've also tried using rules similar to the horizontal brute > force rules in The Web Application Defender's Cookbook (recipe 7-2, pg > 269-271), and found I had similar issues with multi-threaded attacks for > those rules too. > > I know SDBM allows for concurrent access, and I've read (and re-read) > about how collections work in Chapter 8 of the ModSecurity Handbook > (including the parts about storing non-numeric values in SDBM, and race > conditions). In light of that I've attempted to use only numeric values by > eliminating all stored variables except the counter, as seen in "rule two" > below (and completely eliminating the tracking of unique usernames). Still, > I had issues where multi-threading a brute force attack resulted in the > counter value not being correct for several requests in a row. > > So my questions: > > 1. Am I missing something, or doing something fundamentally wrong in > how I'm using persistent storage? > 2. I realize the potential for race conditions with SDBM, especially > with non-numeric data, but is what I'm describing entirely expected > behavior? Numeric values in persistent storage seem to be fairly unreliable > on a per-request basis (in logs, my counter is wrong more than it is right). > 3. Is there a better approach to accomplish what I'm trying to do, > which is track horizontal (by IP addr and username), and eventually > vertical (by IP addr and password hash) brute force attempts? For example, > using Lua, or some other persistent storage mechanism (memcached isn't an > option). > > If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, > running on Windows. > > Thanks, > Riemann > > ######################################## > # 'Rule' One # > ########### > > SecRule REQUEST_FILENAME "!@pm login.jsp" \ > "phase:2,t:none,pass,nolog, \ > id:999321, \ > skipAfter:END_BRUTE_FORCE" > SecRule REQUEST_METHOD "!@streq POST" \ > "phase:2,t:none,pass,nolog, \ > id:999322, \ > skipAfter:END_BRUTE_FORCE" > > SecRule ARGS:username|ARGS:user ".+" \ > "chain,phase:2,t:none,t:lowercase,pass,nolog, \ > id:999323, \ > skipAfter:LOGIN_TEST" > SecRule ARGS:password|ARGS:pass ".+" \ > "t:none,t:lowercase" > > SecAction \ > "phase:2,pass,nolog, \ > skipAfter:END_BRUTE_FORCE, \ > id:'1000950'" > > SecMarker LOGIN_TEST > SecRule IP:username_ct "@ge 5" \ > "phase:5,t:none,log,pass, \ > id:'1000951', \ > msg:'Multiple Username Violation - Too Many Usernames Submitted from > %{REMOTE_ADDR}.' \ > logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted > Usernames: %{IP.username}.', \ > setvar:IP.clear_username=1" > SecRule &IP:CLEAR_USERNAME "@eq 1" \ > "phase:5,pass,nolog, \ > id:'1000952', \ > setvar:!IP.KEY" > SecRule &IP:username_ct "@eq 0" \ > "chain,phase:2,pass,nolog, \ > id:'1000953'" > SecRule ARGS:username|ARGS:user ".+" \ > "t:none,t:lowercase, \ > setvar:IP.username_ct=+1, \ > expirevar:IP.username_ct=300, \ > setvar:IP.username=%{matched_var}, \ > expirevar:IP.username=300" > > SecRule &IP:username_ct "@ge 1" \ > "chain,phase:2,pass,nolog, \ > id:'1000954'" > SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ > "t:none,t:lowercase, \ > setvar:IP.username_ct=+1, \ > expirevar:IP.username_ct=300, \ > setvar:'IP.username=%{IP.username},%{matched_var}', \ > expirevar:IP.username=300" > SecMarker END_BRUTE_FORCE > > ######################################## > # 'Rule' Two # > ########### > > SecRule REQUEST_FILENAME "!@pm login.jsp" \ > "phase:2,t:none,pass,nolog, \ > id:999321, \ > skipAfter:END_BRUTE_FORCE" > SecRule REQUEST_METHOD "!@streq POST" \ > "phase:2,t:none,pass,nolog, \ > id:999322, \ > skipAfter:END_BRUTE_FORCE" > > SecRule ARGS:username|ARGS:user ".+" \ > "chain,phase:2,t:none,t:lowercase,pass,nolog, \ > id:999323, \ > skipAfter:LOGIN_TEST" > SecRule ARGS:password|ARGS:pass ".+" \ > "t:none,t:lowercase" > > SecAction \ > "phase:2,pass,nolog, \ > skipAfter:END_BRUTE_FORCE, \ > id:'1000950'" > > SecAction \ > "phase:2,pass,log, \ > id:'1000959', \ > logdata:'Usernames Attempted Count: %{IP.username_ct}.'" > SecRule IP:username_ct "@ge 7" \ > "phase:2,t:none,log,pass, \ > id:'1000951', \ > msg:'Multiple Username Violation - Too Many Usernames Submitted from > %{REMOTE_ADDR}.' \ > logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ > setvar:IP.clear_username=1" > SecRule &IP:CLEAR_USERNAME "@eq 1" \ > "phase:2,pass,nolog, \ > id:'1000952', \ > setvar:!IP.KEY" > SecRule ARGS:username|ARGS:user ".+" \ > "t:none,t:lowercase,pass,nolog, \ > id:'1000953', \ > setvar:IP.username_ct=+1, \ > expirevar:IP.username_ct=300" > SecMarker END_BRUTE_FORCE > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > |
|
From: Christian F. <chr...@ne...> - 2016-09-08 13:08:33
|
Hello Riemann, The SDBM access tends to become fragile under high pressure. I have not really seen the sort of behaviour which you describe, but I am by no means surprised. Have you tried it under unix? Maybe that makes difference - or maybe it's just a prejudice against the stability of the core turning on Windows. If you try out the lua option which you outlined, then please keep us posted about the progress of the project. Generally, I agree that ModSecurity on an Apache Reverse Proxy is the better option than fixing the DoS within the application. But it's still not really optimal. I think ModSec can be the sensor, but the logic is better located outside Apache. You could pull this off as follows: You have Apache write a custom log with the IP and the username. Then you have an external process monitor that log and if it finds any suspicious IP address, then you blacklist it immediately on the firewall. As immediate steps against DoS on the login: - Enforce the correct referer for the POST - Make the client eat a cookie first and check for the cookie in the POST - Temporary captcha (during the attack) Good luck! Cheers, Christian On Wed, Sep 07, 2016 at 06:01:28PM -0500, Riemann . wrote: > Hi Barry, > > Thanks for your response and the link to the previous discussion. I do see > the error you mentioned in my debug log. > > This is a bummer. In the ModSec Handbook, Ivan seems to infer the > double-retrieve/write-lock mechanism should prevent race conditions with > numerical data being stored (pg 129-130), but this just doesn't seem to be > the case. > > Maybe this is a bad idea, but as an alternative to using persistent > storage, would it be crazy to try to implement some of these rules by > passing variables to a Lua script that reads from and writes to an > instance-specific SQLite or enterprise-class database? Are there any other > ideas as to how this can be acheived? > > Of course, it would be ideal to integrate some of these types of things > directly into the application, but it's a lot faster to turn rules out with > ModSec, than to try to get them fit into a release cycle that may be a few > months (or more away), depending on development's other priorities. > > Thanks, > Riemann > > > > On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...> > wrote: > > > Are you seeing any errors like this in your logs?: > > > > Message: collections_remove_stale: Failed deleting collection > > > > I'm really not convinced this works using disk based collections file - > > especially under stress (like brute force attacks). So ultimately don't > > think ModSecurity is right solution for this - at least until a better, > > memory based collections store is available. > > > > See more details when this was discussed before on this list here: > > https://sourceforge.net/p/mod-security/mailman/message/34392875/ > > > > Thanks, > > Barry > > > > On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...> wrote: > > > > Hello, > > > > I'm trying to write rules to track and log (not block) brute force login > > attempts. I've initialized the Global and IP collection in CRS_10, and have > > created a separate rule/config file for horizontal brute force attempts. > > I've included my attempt at a horizontal brute force rule below ("rule > > one"). The rule works exactly as I'd expect if I use Burp Intruder to > > simulate a brute force attack, sending POST requests to login.jsp, with > > "username" and "password" parameters in the request body, and changing only > > the username on each request. > > > > If I run the exact same 'attack,' but this time with five threads, I run > > into problems. My counter may equal '1' for five consecutive requests, and > > then jump to '6', but only show two usernames are stored in IP.username > > (for example). I've also tried using rules similar to the horizontal brute > > force rules in The Web Application Defender's Cookbook (recipe 7-2, pg > > 269-271), and found I had similar issues with multi-threaded attacks for > > those rules too. > > > > I know SDBM allows for concurrent access, and I've read (and re-read) > > about how collections work in Chapter 8 of the ModSecurity Handbook > > (including the parts about storing non-numeric values in SDBM, and race > > conditions). In light of that I've attempted to use only numeric values by > > eliminating all stored variables except the counter, as seen in "rule two" > > below (and completely eliminating the tracking of unique usernames). Still, > > I had issues where multi-threading a brute force attack resulted in the > > counter value not being correct for several requests in a row. > > > > So my questions: > > > > 1. Am I missing something, or doing something fundamentally wrong in > > how I'm using persistent storage? > > 2. I realize the potential for race conditions with SDBM, especially > > with non-numeric data, but is what I'm describing entirely expected > > behavior? Numeric values in persistent storage seem to be fairly unreliable > > on a per-request basis (in logs, my counter is wrong more than it is right). > > 3. Is there a better approach to accomplish what I'm trying to do, > > which is track horizontal (by IP addr and username), and eventually > > vertical (by IP addr and password hash) brute force attempts? For example, > > using Lua, or some other persistent storage mechanism (memcached isn't an > > option). > > > > If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, > > running on Windows. > > > > Thanks, > > Riemann > > > > ######################################## > > # 'Rule' One # > > ########### > > > > SecRule REQUEST_FILENAME "!@pm login.jsp" \ > > "phase:2,t:none,pass,nolog, \ > > id:999321, \ > > skipAfter:END_BRUTE_FORCE" > > SecRule REQUEST_METHOD "!@streq POST" \ > > "phase:2,t:none,pass,nolog, \ > > id:999322, \ > > skipAfter:END_BRUTE_FORCE" > > > > SecRule ARGS:username|ARGS:user ".+" \ > > "chain,phase:2,t:none,t:lowercase,pass,nolog, \ > > id:999323, \ > > skipAfter:LOGIN_TEST" > > SecRule ARGS:password|ARGS:pass ".+" \ > > "t:none,t:lowercase" > > > > SecAction \ > > "phase:2,pass,nolog, \ > > skipAfter:END_BRUTE_FORCE, \ > > id:'1000950'" > > > > SecMarker LOGIN_TEST > > SecRule IP:username_ct "@ge 5" \ > > "phase:5,t:none,log,pass, \ > > id:'1000951', \ > > msg:'Multiple Username Violation - Too Many Usernames Submitted from > > %{REMOTE_ADDR}.' \ > > logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted > > Usernames: %{IP.username}.', \ > > setvar:IP.clear_username=1" > > SecRule &IP:CLEAR_USERNAME "@eq 1" \ > > "phase:5,pass,nolog, \ > > id:'1000952', \ > > setvar:!IP.KEY" > > SecRule &IP:username_ct "@eq 0" \ > > "chain,phase:2,pass,nolog, \ > > id:'1000953'" > > SecRule ARGS:username|ARGS:user ".+" \ > > "t:none,t:lowercase, \ > > setvar:IP.username_ct=+1, \ > > expirevar:IP.username_ct=300, \ > > setvar:IP.username=%{matched_var}, \ > > expirevar:IP.username=300" > > > > SecRule &IP:username_ct "@ge 1" \ > > "chain,phase:2,pass,nolog, \ > > id:'1000954'" > > SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ > > "t:none,t:lowercase, \ > > setvar:IP.username_ct=+1, \ > > expirevar:IP.username_ct=300, \ > > setvar:'IP.username=%{IP.username},%{matched_var}', \ > > expirevar:IP.username=300" > > SecMarker END_BRUTE_FORCE > > > > ######################################## > > # 'Rule' Two # > > ########### > > > > SecRule REQUEST_FILENAME "!@pm login.jsp" \ > > "phase:2,t:none,pass,nolog, \ > > id:999321, \ > > skipAfter:END_BRUTE_FORCE" > > SecRule REQUEST_METHOD "!@streq POST" \ > > "phase:2,t:none,pass,nolog, \ > > id:999322, \ > > skipAfter:END_BRUTE_FORCE" > > > > SecRule ARGS:username|ARGS:user ".+" \ > > "chain,phase:2,t:none,t:lowercase,pass,nolog, \ > > id:999323, \ > > skipAfter:LOGIN_TEST" > > SecRule ARGS:password|ARGS:pass ".+" \ > > "t:none,t:lowercase" > > > > SecAction \ > > "phase:2,pass,nolog, \ > > skipAfter:END_BRUTE_FORCE, \ > > id:'1000950'" > > > > SecAction \ > > "phase:2,pass,log, \ > > id:'1000959', \ > > logdata:'Usernames Attempted Count: %{IP.username_ct}.'" > > SecRule IP:username_ct "@ge 7" \ > > "phase:2,t:none,log,pass, \ > > id:'1000951', \ > > msg:'Multiple Username Violation - Too Many Usernames Submitted from > > %{REMOTE_ADDR}.' \ > > logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ > > setvar:IP.clear_username=1" > > SecRule &IP:CLEAR_USERNAME "@eq 1" \ > > "phase:2,pass,nolog, \ > > id:'1000952', \ > > setvar:!IP.KEY" > > SecRule ARGS:username|ARGS:user ".+" \ > > "t:none,t:lowercase,pass,nolog, \ > > id:'1000953', \ > > setvar:IP.username_ct=+1, \ > > expirevar:IP.username_ct=300" > > SecMarker END_BRUTE_FORCE > > > > ------------------------------------------------------------ > > ------------------ > > > > _______________________________________________ > > mod-security-users mailing list > > mod...@li... > > https://lists.sourceforge.net/lists/listinfo/mod-security-users > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > > http://www.modsecurity.org/projects/commercial/rules/ > > http://www.modsecurity.org/projects/commercial/support/ > > > > > > ------------------------------------------------------------ > > ------------------ > > > > _______________________________________________ > > mod-security-users mailing list > > mod...@li... > > https://lists.sourceforge.net/lists/listinfo/mod-security-users > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > > http://www.modsecurity.org/projects/commercial/rules/ > > http://www.modsecurity.org/projects/commercial/support/ > > > > > ------------------------------------------------------------------------------ > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ -- https://www.feistyduck.com/training/modsecurity-training-course mailto:chr...@ne... twitter: @ChrFolini |
|
From: Felipe C. <FC...@tr...> - 2016-09-08 13:16:29
|
Hi, There are few problems in the persistent storage for 2.x. Huge amount of disk space usage, Concurrence problems, Slow sync between the threads are example of those problems. The best way to circumvent all the problems is to reduce the usage of the storage, to a “size” that fits what a dbm can holds. The problem might not be specific on the ModSecurity implementation, but with the fact that we might be making an abuse in the way that we are using the dbm. For ModSecurity version 3 the persistent storage are modular and you can choose what fits better your infrastructure, for instance: in-memory storage, Redis, Memcache. Open tickets regarding that: Redis - https://github.com/SpiderLabs/ModSecurity/issues/1139 Memcache - https://github.com/SpiderLabs/ModSecurity/issues/1140 We already have implemented the lmdb and the in memory backends: https://github.com/SpiderLabs/ModSecurity/tree/libmodsecurity/src/collection/backend Technically it is possible to use Lua, although I am not sure if that is a good idea, you have to check how much overhead it will add into your environment. Br., Felipe “Zimmerle” Costa Security Researcher, Lead Developer ModSecurity. Trustwave | SMART SECURITY ON DEMAND www.trustwave.com<http://www.trustwave.com/> From: "Riemann ." <rie...@gm...<mailto:rie...@gm...>> Reply-To: "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>> Date: Wednesday, September 7, 2016 at 8:01 PM To: "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>> Subject: Re: [mod-security-users] Working Around Race Conditions in Persistent Storage Hi Barry, Thanks for your response and the link to the previous discussion. I do see the error you mentioned in my debug log. This is a bummer. In the ModSec Handbook, Ivan seems to infer the double-retrieve/write-lock mechanism should prevent race conditions with numerical data being stored (pg 129-130), but this just doesn't seem to be the case. Maybe this is a bad idea, but as an alternative to using persistent storage, would it be crazy to try to implement some of these rules by passing variables to a Lua script that reads from and writes to an instance-specific SQLite or enterprise-class database? Are there any other ideas as to how this can be acheived? Of course, it would be ideal to integrate some of these types of things directly into the application, but it's a lot faster to turn rules out with ModSec, than to try to get them fit into a release cycle that may be a few months (or more away), depending on development's other priorities. Thanks, Riemann On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...<mailto:bar...@ho...>> wrote: Are you seeing any errors like this in your logs?: Message: collections_remove_stale: Failed deleting collection I'm really not convinced this works using disk based collections file - especially under stress (like brute force attacks). So ultimately don't think ModSecurity is right solution for this - at least until a better, memory based collections store is available. See more details when this was discussed before on this list here: https://sourceforge.net/p/mod-security/mailman/message/34392875/<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCVF5ukQbQ&s=5&u=https%3a%2f%2fsourceforge%2enet%2fp%2fmod-security%2fmailman%2fmessage%2f34392875%2f> Thanks, Barry On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...<mailto:rie...@gm...>> wrote: Hello, I'm trying to write rules to track and log (not block) brute force login attempts. I've initialized the Global and IP collection in CRS_10, and have created a separate rule/config file for horizontal brute force attempts. I've included my attempt at a horizontal brute force rule below ("rule one"). The rule works exactly as I'd expect if I use Burp Intruder to simulate a brute force attack, sending POST requests to login.jsp, with "username" and "password" parameters in the request body, and changing only the username on each request. If I run the exact same 'attack,' but this time with five threads, I run into problems. My counter may equal '1' for five consecutive requests, and then jump to '6', but only show two usernames are stored in IP.username (for example). I've also tried using rules similar to the horizontal brute force rules in The Web Application Defender's Cookbook (recipe 7-2, pg 269-271), and found I had similar issues with multi-threaded attacks for those rules too. I know SDBM allows for concurrent access, and I've read (and re-read) about how collections work in Chapter 8 of the ModSecurity Handbook (including the parts about storing non-numeric values in SDBM, and race conditions). In light of that I've attempted to use only numeric values by eliminating all stored variables except the counter, as seen in "rule two" below (and completely eliminating the tracking of unique usernames). Still, I had issues where multi-threading a brute force attack resulted in the counter value not being correct for several requests in a row. So my questions: 1. Am I missing something, or doing something fundamentally wrong in how I'm using persistent storage? 2. I realize the potential for race conditions with SDBM, especially with non-numeric data, but is what I'm describing entirely expected behavior? Numeric values in persistent storage seem to be fairly unreliable on a per-request basis (in logs, my counter is wrong more than it is right). 3. Is there a better approach to accomplish what I'm trying to do, which is track horizontal (by IP addr and username), and eventually vertical (by IP addr and password hash) brute force attempts? For example, using Lua, or some other persistent storage mechanism (memcached isn't an option). If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, running on Windows. Thanks, Riemann ######################################## # 'Rule' One # ########### SecRule REQUEST_FILENAME "!@pm login.jsp" \ "phase:2,t:none,pass,nolog, \ id:999321, \ skipAfter:END_BRUTE_FORCE" SecRule REQUEST_METHOD "!@streq POST" \ "phase:2,t:none,pass,nolog, \ id:999322, \ skipAfter:END_BRUTE_FORCE" SecRule ARGS:username|ARGS:user ".+" \ "chain,phase:2,t:none,t:lowercase,pass,nolog, \ id:999323, \ skipAfter:LOGIN_TEST" SecRule ARGS:password|ARGS:pass ".+" \ "t:none,t:lowercase" SecAction \ "phase:2,pass,nolog, \ skipAfter:END_BRUTE_FORCE, \ id:'1000950'" SecMarker LOGIN_TEST SecRule IP:username_ct "@ge 5" \ "phase:5,t:none,log,pass, \ id:'1000951', \ msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \ logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted Usernames: %{IP.username}.', \ setvar:IP.clear_username=1" SecRule &IP:CLEAR_USERNAME "@eq 1" \ "phase:5,pass,nolog, \ id:'1000952', \ setvar:!IP.KEY" SecRule &IP:username_ct "@eq 0" \ "chain,phase:2,pass,nolog, \ id:'1000953'" SecRule ARGS:username|ARGS:user ".+" \ "t:none,t:lowercase, \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300, \ setvar:IP.username=%{matched_var}, \ expirevar:IP.username=300" SecRule &IP:username_ct "@ge 1" \ "chain,phase:2,pass,nolog, \ id:'1000954'" SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ "t:none,t:lowercase, \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300, \ setvar:'IP.username=%{IP.username},%{matched_var}', \ expirevar:IP.username=300" SecMarker END_BRUTE_FORCE ######################################## # 'Rule' Two # ########### SecRule REQUEST_FILENAME "!@pm login.jsp" \ "phase:2,t:none,pass,nolog, \ id:999321, \ skipAfter:END_BRUTE_FORCE" SecRule REQUEST_METHOD "!@streq POST" \ "phase:2,t:none,pass,nolog, \ id:999322, \ skipAfter:END_BRUTE_FORCE" SecRule ARGS:username|ARGS:user ".+" \ "chain,phase:2,t:none,t:lowercase,pass,nolog, \ id:999323, \ skipAfter:LOGIN_TEST" SecRule ARGS:password|ARGS:pass ".+" \ "t:none,t:lowercase" SecAction \ "phase:2,pass,nolog, \ skipAfter:END_BRUTE_FORCE, \ id:'1000950'" SecAction \ "phase:2,pass,log, \ id:'1000959', \ logdata:'Usernames Attempted Count: %{IP.username_ct}.'" SecRule IP:username_ct "@ge 7" \ "phase:2,t:none,log,pass, \ id:'1000951', \ msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \ logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ setvar:IP.clear_username=1" SecRule &IP:CLEAR_USERNAME "@eq 1" \ "phase:2,pass,nolog, \ id:'1000952', \ setvar:!IP.KEY" SecRule ARGS:username|ARGS:user ".+" \ "t:none,t:lowercase,pass,nolog, \ id:'1000953', \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300" SecMarker END_BRUTE_FORCE ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> http://www.modsecurity.org/projects/commercial/support/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> http://www.modsecurity.org/projects/commercial/support/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> ________________________________ This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
|
From: Riemann . <rie...@gm...> - 2016-09-08 15:09:53
|
Thanks for your response Felipe. In terms of the dbm size, I wanted to make sure I wasn't running into any sort of issues like that. I only enabled my modsec config (CRS_10) and a second config file with the rules I included in my orig post. I was seeing these issues with "rule two" (just keeping a count of logins per IP) while sending just a total of 22 requests using five threads. There was no other traffic hitting the server and no other rules were being used. I also tried it with all rules enabled, and moved some of the rules to phase 5 to try to introduce some kind of delay, but that didn't help either. Hopefully this won't be an issue in v3. I haven't done any testing with it yet since it's not yet production-ready, but it sounds like there are several positive improvements. On Thu, Sep 8, 2016 at 8:04 AM, Felipe Costa <FC...@tr...> wrote: > Hi, > > There are few problems in the persistent storage for 2.x. Huge amount of > disk space usage, > Concurrence problems, Slow sync between the threads are example of those > problems. The > best way to circumvent all the problems is to reduce the usage of the > storage, to a “size” that > fits what a dbm can holds. > > The problem might not be specific on the ModSecurity implementation, but > with the fact that > we might be making an abuse in the way that we are using the dbm. > > For ModSecurity version 3 the persistent storage are modular and you can > choose what fits > better your infrastructure, for instance: in-memory storage, Redis, > Memcache. Open tickets > regarding that: > > Redis - https://github.com/SpiderLabs/ModSecurity/issues/1139 > Memcache - https://github.com/SpiderLabs/ModSecurity/issues/1140 > > We already have implemented the lmdb and the in memory backends: > https://github.com/SpiderLabs/ModSecurity/tree/ > libmodsecurity/src/collection/backend > > Technically it is possible to use Lua, although I am not sure if that is a > good idea, you have > to check how much overhead it will add into your environment. > > > Br., > > *Felipe “Zimmerle” Costa * > > Security Researcher, Lead Developer ModSecurity. > > > > *Trustwave* | SMART SECURITY ON DEMAND > > www.trustwave.com > > > From: "Riemann ." <rie...@gm...> > Reply-To: "mod...@li..." < > mod...@li...> > Date: Wednesday, September 7, 2016 at 8:01 PM > To: "mod...@li..." <mod-security-users@lists. > sourceforge.net> > Subject: Re: [mod-security-users] Working Around Race Conditions in > Persistent Storage > > Hi Barry, > > Thanks for your response and the link to the previous discussion. I do see > the error you mentioned in my debug log. > > This is a bummer. In the ModSec Handbook, Ivan seems to infer the > double-retrieve/write-lock mechanism should prevent race conditions with > numerical data being stored (pg 129-130), but this just doesn't seem to be > the case. > > Maybe this is a bad idea, but as an alternative to using persistent > storage, would it be crazy to try to implement some of these rules by > passing variables to a Lua script that reads from and writes to an > instance-specific SQLite or enterprise-class database? Are there any other > ideas as to how this can be acheived? > > Of course, it would be ideal to integrate some of these types of things > directly into the application, but it's a lot faster to turn rules out with > ModSec, than to try to get them fit into a release cycle that may be a few > months (or more away), depending on development's other priorities. > > Thanks, > Riemann > > > > On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...> > wrote: > >> Are you seeing any errors like this in your logs?: >> >> Message: collections_remove_stale: Failed deleting collection >> >> I'm really not convinced this works using disk based collections file - >> especially under stress (like brute force attacks). So ultimately don't >> think ModSecurity is right solution for this - at least until a better, >> memory based collections store is available. >> >> See more details when this was discussed before on this list here: >> https://sourceforge.net/p/mod-security/mailman/message/34392875/ >> <https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCVF5ukQbQ&s=5&u=https%3a%2f%2fsourceforge%2enet%2fp%2fmod-security%2fmailman%2fmessage%2f34392875%2f> >> >> Thanks, >> Barry >> >> On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...> wrote: >> >> Hello, >> >> I'm trying to write rules to track and log (not block) brute force login >> attempts. I've initialized the Global and IP collection in CRS_10, and have >> created a separate rule/config file for horizontal brute force attempts. >> I've included my attempt at a horizontal brute force rule below ("rule >> one"). The rule works exactly as I'd expect if I use Burp Intruder to >> simulate a brute force attack, sending POST requests to login.jsp, with >> "username" and "password" parameters in the request body, and changing only >> the username on each request. >> >> If I run the exact same 'attack,' but this time with five threads, I run >> into problems. My counter may equal '1' for five consecutive requests, and >> then jump to '6', but only show two usernames are stored in IP.username >> (for example). I've also tried using rules similar to the horizontal brute >> force rules in The Web Application Defender's Cookbook (recipe 7-2, pg >> 269-271), and found I had similar issues with multi-threaded attacks for >> those rules too. >> >> I know SDBM allows for concurrent access, and I've read (and re-read) >> about how collections work in Chapter 8 of the ModSecurity Handbook >> (including the parts about storing non-numeric values in SDBM, and race >> conditions). In light of that I've attempted to use only numeric values by >> eliminating all stored variables except the counter, as seen in "rule two" >> below (and completely eliminating the tracking of unique usernames). Still, >> I had issues where multi-threading a brute force attack resulted in the >> counter value not being correct for several requests in a row. >> >> So my questions: >> >> 1. Am I missing something, or doing something fundamentally wrong in >> how I'm using persistent storage? >> 2. I realize the potential for race conditions with SDBM, especially >> with non-numeric data, but is what I'm describing entirely expected >> behavior? Numeric values in persistent storage seem to be fairly unreliable >> on a per-request basis (in logs, my counter is wrong more than it is right). >> 3. Is there a better approach to accomplish what I'm trying to do, >> which is track horizontal (by IP addr and username), and eventually >> vertical (by IP addr and password hash) brute force attempts? For example, >> using Lua, or some other persistent storage mechanism (memcached isn't an >> option). >> >> If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, >> running on Windows. >> >> Thanks, >> Riemann >> >> ######################################## >> # 'Rule' One # >> ########### >> >> SecRule REQUEST_FILENAME "!@pm login.jsp" \ >> "phase:2,t:none,pass,nolog, \ >> id:999321, \ >> skipAfter:END_BRUTE_FORCE" >> SecRule REQUEST_METHOD "!@streq POST" \ >> "phase:2,t:none,pass,nolog, \ >> id:999322, \ >> skipAfter:END_BRUTE_FORCE" >> >> SecRule ARGS:username|ARGS:user ".+" \ >> "chain,phase:2,t:none,t:lowercase,pass,nolog, \ >> id:999323, \ >> skipAfter:LOGIN_TEST" >> SecRule ARGS:password|ARGS:pass ".+" \ >> "t:none,t:lowercase" >> >> SecAction \ >> "phase:2,pass,nolog, \ >> skipAfter:END_BRUTE_FORCE, \ >> id:'1000950'" >> >> SecMarker LOGIN_TEST >> SecRule IP:username_ct "@ge 5" \ >> "phase:5,t:none,log,pass, \ >> id:'1000951', \ >> msg:'Multiple Username Violation - Too Many Usernames Submitted from >> %{REMOTE_ADDR}.' \ >> logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted >> Usernames: %{IP.username}.', \ >> setvar:IP.clear_username=1" >> SecRule &IP:CLEAR_USERNAME "@eq 1" \ >> "phase:5,pass,nolog, \ >> id:'1000952', \ >> setvar:!IP.KEY" >> SecRule &IP:username_ct "@eq 0" \ >> "chain,phase:2,pass,nolog, \ >> id:'1000953'" >> SecRule ARGS:username|ARGS:user ".+" \ >> "t:none,t:lowercase, \ >> setvar:IP.username_ct=+1, \ >> expirevar:IP.username_ct=300, \ >> setvar:IP.username=%{matched_var}, \ >> expirevar:IP.username=300" >> >> SecRule &IP:username_ct "@ge 1" \ >> "chain,phase:2,pass,nolog, \ >> id:'1000954'" >> SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ >> "t:none,t:lowercase, \ >> setvar:IP.username_ct=+1, \ >> expirevar:IP.username_ct=300, \ >> setvar:'IP.username=%{IP.username},%{matched_var}', \ >> expirevar:IP.username=300" >> SecMarker END_BRUTE_FORCE >> >> ######################################## >> # 'Rule' Two # >> ########### >> >> SecRule REQUEST_FILENAME "!@pm login.jsp" \ >> "phase:2,t:none,pass,nolog, \ >> id:999321, \ >> skipAfter:END_BRUTE_FORCE" >> SecRule REQUEST_METHOD "!@streq POST" \ >> "phase:2,t:none,pass,nolog, \ >> id:999322, \ >> skipAfter:END_BRUTE_FORCE" >> >> SecRule ARGS:username|ARGS:user ".+" \ >> "chain,phase:2,t:none,t:lowercase,pass,nolog, \ >> id:999323, \ >> skipAfter:LOGIN_TEST" >> SecRule ARGS:password|ARGS:pass ".+" \ >> "t:none,t:lowercase" >> >> SecAction \ >> "phase:2,pass,nolog, \ >> skipAfter:END_BRUTE_FORCE, \ >> id:'1000950'" >> >> SecAction \ >> "phase:2,pass,log, \ >> id:'1000959', \ >> logdata:'Usernames Attempted Count: %{IP.username_ct}.'" >> SecRule IP:username_ct "@ge 7" \ >> "phase:2,t:none,log,pass, \ >> id:'1000951', \ >> msg:'Multiple Username Violation - Too Many Usernames Submitted from >> %{REMOTE_ADDR}.' \ >> logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ >> setvar:IP.clear_username=1" >> SecRule &IP:CLEAR_USERNAME "@eq 1" \ >> "phase:2,pass,nolog, \ >> id:'1000952', \ >> setvar:!IP.KEY" >> SecRule ARGS:username|ARGS:user ".+" \ >> "t:none,t:lowercase,pass,nolog, \ >> id:'1000953', \ >> setvar:IP.username_ct=+1, \ >> expirevar:IP.username_ct=300" >> SecMarker END_BRUTE_FORCE >> >> ------------------------------------------------------------ >> ------------------ >> >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> <https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> >> http://www.modsecurity.org/projects/commercial/support/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> >> >> >> ------------------------------------------------------------ >> ------------------ >> >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> <https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> >> http://www.modsecurity.org/projects/commercial/support/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> >> >> > > ------------------------------ > > This transmission may contain information that is privileged, > confidential, and/or exempt from disclosure under applicable law. If you > are not the intended recipient, you are hereby notified that any > disclosure, copying, distribution, or use of the information contained > herein (including any reliance thereon) is strictly prohibited. If you > received this transmission in error, please immediately contact the sender > and destroy the material in its entirety, whether in electronic or hard > copy format. > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > |
|
From: Riemann . <rie...@gm...> - 2016-09-08 14:39:35
|
Thanks Christian. We only use Windows servers in production, so I haven't tried this under Unix. It might be that this specific behavior is specific to the Windows install (which probably isn't the most typical for ModSec users). I appreciate your suggestions--and I'll definitely look into using a custom Apache log and trying to parse and alert on that as an alternative option. On Thu, Sep 8, 2016 at 8:08 AM, Christian Folini < chr...@ne...> wrote: > Hello Riemann, > > The SDBM access tends to become fragile under high pressure. I have not > really seen the sort of behaviour which you describe, but I am by no > means surprised. Have you tried it under unix? Maybe that makes > difference - or maybe it's just a prejudice against the stability of > the core turning on Windows. > > If you try out the lua option which you outlined, then please keep us > posted about the progress of the project. > > Generally, I agree that ModSecurity on an Apache Reverse Proxy is the > better option than fixing the DoS within the application. But it's still > not really optimal. I think ModSec can be the sensor, but the logic is > better located outside Apache. > > You could pull this off as follows: You have Apache write a custom log > with the IP and the username. Then you have an external process monitor > that log and if it finds any suspicious IP address, then you blacklist > it immediately on the firewall. > > As immediate steps against DoS on the login: > - Enforce the correct referer for the POST > - Make the client eat a cookie first and check for the > cookie in the POST > - Temporary captcha (during the attack) > > Good luck! > > Cheers, > > Christian > > > On Wed, Sep 07, 2016 at 06:01:28PM -0500, Riemann . wrote: > > Hi Barry, > > > > Thanks for your response and the link to the previous discussion. I do > see > > the error you mentioned in my debug log. > > > > This is a bummer. In the ModSec Handbook, Ivan seems to infer the > > double-retrieve/write-lock mechanism should prevent race conditions with > > numerical data being stored (pg 129-130), but this just doesn't seem to > be > > the case. > > > > Maybe this is a bad idea, but as an alternative to using persistent > > storage, would it be crazy to try to implement some of these rules by > > passing variables to a Lua script that reads from and writes to an > > instance-specific SQLite or enterprise-class database? Are there any > other > > ideas as to how this can be acheived? > > > > Of course, it would be ideal to integrate some of these types of things > > directly into the application, but it's a lot faster to turn rules out > with > > ModSec, than to try to get them fit into a release cycle that may be a > few > > months (or more away), depending on development's other priorities. > > > > Thanks, > > Riemann > > > > > > > > On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho... > > > > wrote: > > > > > Are you seeing any errors like this in your logs?: > > > > > > Message: collections_remove_stale: Failed deleting collection > > > > > > I'm really not convinced this works using disk based collections file - > > > especially under stress (like brute force attacks). So ultimately don't > > > think ModSecurity is right solution for this - at least until a better, > > > memory based collections store is available. > > > > > > See more details when this was discussed before on this list here: > > > https://sourceforge.net/p/mod-security/mailman/message/34392875/ > > > > > > Thanks, > > > Barry > > > > > > On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...> wrote: > > > > > > Hello, > > > > > > I'm trying to write rules to track and log (not block) brute force > login > > > attempts. I've initialized the Global and IP collection in CRS_10, and > have > > > created a separate rule/config file for horizontal brute force > attempts. > > > I've included my attempt at a horizontal brute force rule below ("rule > > > one"). The rule works exactly as I'd expect if I use Burp Intruder to > > > simulate a brute force attack, sending POST requests to login.jsp, with > > > "username" and "password" parameters in the request body, and changing > only > > > the username on each request. > > > > > > If I run the exact same 'attack,' but this time with five threads, I > run > > > into problems. My counter may equal '1' for five consecutive requests, > and > > > then jump to '6', but only show two usernames are stored in IP.username > > > (for example). I've also tried using rules similar to the horizontal > brute > > > force rules in The Web Application Defender's Cookbook (recipe 7-2, pg > > > 269-271), and found I had similar issues with multi-threaded attacks > for > > > those rules too. > > > > > > I know SDBM allows for concurrent access, and I've read (and re-read) > > > about how collections work in Chapter 8 of the ModSecurity Handbook > > > (including the parts about storing non-numeric values in SDBM, and race > > > conditions). In light of that I've attempted to use only numeric > values by > > > eliminating all stored variables except the counter, as seen in "rule > two" > > > below (and completely eliminating the tracking of unique usernames). > Still, > > > I had issues where multi-threading a brute force attack resulted in the > > > counter value not being correct for several requests in a row. > > > > > > So my questions: > > > > > > 1. Am I missing something, or doing something fundamentally wrong in > > > how I'm using persistent storage? > > > 2. I realize the potential for race conditions with SDBM, especially > > > with non-numeric data, but is what I'm describing entirely expected > > > behavior? Numeric values in persistent storage seem to be fairly > unreliable > > > on a per-request basis (in logs, my counter is wrong more than it > is right). > > > 3. Is there a better approach to accomplish what I'm trying to do, > > > which is track horizontal (by IP addr and username), and eventually > > > vertical (by IP addr and password hash) brute force attempts? For > example, > > > using Lua, or some other persistent storage mechanism (memcached > isn't an > > > option). > > > > > > If it matters, I'm currently testing with ModSec 2.9.1 on Apache > 2.4.23, > > > running on Windows. > > > > > > Thanks, > > > Riemann > > > > > > ######################################## > > > # 'Rule' One # > > > ########### > > > > > > SecRule REQUEST_FILENAME "!@pm login.jsp" \ > > > "phase:2,t:none,pass,nolog, \ > > > id:999321, \ > > > skipAfter:END_BRUTE_FORCE" > > > SecRule REQUEST_METHOD "!@streq POST" \ > > > "phase:2,t:none,pass,nolog, \ > > > id:999322, \ > > > skipAfter:END_BRUTE_FORCE" > > > > > > SecRule ARGS:username|ARGS:user ".+" \ > > > "chain,phase:2,t:none,t:lowercase,pass,nolog, \ > > > id:999323, \ > > > skipAfter:LOGIN_TEST" > > > SecRule ARGS:password|ARGS:pass ".+" \ > > > "t:none,t:lowercase" > > > > > > SecAction \ > > > "phase:2,pass,nolog, \ > > > skipAfter:END_BRUTE_FORCE, \ > > > id:'1000950'" > > > > > > SecMarker LOGIN_TEST > > > SecRule IP:username_ct "@ge 5" \ > > > "phase:5,t:none,log,pass, \ > > > id:'1000951', \ > > > msg:'Multiple Username Violation - Too Many Usernames Submitted from > > > %{REMOTE_ADDR}.' \ > > > logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted > > > Usernames: %{IP.username}.', \ > > > setvar:IP.clear_username=1" > > > SecRule &IP:CLEAR_USERNAME "@eq 1" \ > > > "phase:5,pass,nolog, \ > > > id:'1000952', \ > > > setvar:!IP.KEY" > > > SecRule &IP:username_ct "@eq 0" \ > > > "chain,phase:2,pass,nolog, \ > > > id:'1000953'" > > > SecRule ARGS:username|ARGS:user ".+" \ > > > "t:none,t:lowercase, \ > > > setvar:IP.username_ct=+1, \ > > > expirevar:IP.username_ct=300, \ > > > setvar:IP.username=%{matched_var}, \ > > > expirevar:IP.username=300" > > > > > > SecRule &IP:username_ct "@ge 1" \ > > > "chain,phase:2,pass,nolog, \ > > > id:'1000954'" > > > SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ > > > "t:none,t:lowercase, \ > > > setvar:IP.username_ct=+1, \ > > > expirevar:IP.username_ct=300, \ > > > setvar:'IP.username=%{IP.username},%{matched_var}', \ > > > expirevar:IP.username=300" > > > SecMarker END_BRUTE_FORCE > > > > > > ######################################## > > > # 'Rule' Two # > > > ########### > > > > > > SecRule REQUEST_FILENAME "!@pm login.jsp" \ > > > "phase:2,t:none,pass,nolog, \ > > > id:999321, \ > > > skipAfter:END_BRUTE_FORCE" > > > SecRule REQUEST_METHOD "!@streq POST" \ > > > "phase:2,t:none,pass,nolog, \ > > > id:999322, \ > > > skipAfter:END_BRUTE_FORCE" > > > > > > SecRule ARGS:username|ARGS:user ".+" \ > > > "chain,phase:2,t:none,t:lowercase,pass,nolog, \ > > > id:999323, \ > > > skipAfter:LOGIN_TEST" > > > SecRule ARGS:password|ARGS:pass ".+" \ > > > "t:none,t:lowercase" > > > > > > SecAction \ > > > "phase:2,pass,nolog, \ > > > skipAfter:END_BRUTE_FORCE, \ > > > id:'1000950'" > > > > > > SecAction \ > > > "phase:2,pass,log, \ > > > id:'1000959', \ > > > logdata:'Usernames Attempted Count: %{IP.username_ct}.'" > > > SecRule IP:username_ct "@ge 7" \ > > > "phase:2,t:none,log,pass, \ > > > id:'1000951', \ > > > msg:'Multiple Username Violation - Too Many Usernames Submitted from > > > %{REMOTE_ADDR}.' \ > > > logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ > > > setvar:IP.clear_username=1" > > > SecRule &IP:CLEAR_USERNAME "@eq 1" \ > > > "phase:2,pass,nolog, \ > > > id:'1000952', \ > > > setvar:!IP.KEY" > > > SecRule ARGS:username|ARGS:user ".+" \ > > > "t:none,t:lowercase,pass,nolog, \ > > > id:'1000953', \ > > > setvar:IP.username_ct=+1, \ > > > expirevar:IP.username_ct=300" > > > SecMarker END_BRUTE_FORCE > > > > > > ------------------------------------------------------------ > > > ------------------ > > > > > > _______________________________________________ > > > mod-security-users mailing list > > > mod...@li... > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users > > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > > > http://www.modsecurity.org/projects/commercial/rules/ > > > http://www.modsecurity.org/projects/commercial/support/ > > > > > > > > > ------------------------------------------------------------ > > > ------------------ > > > > > > _______________________________________________ > > > mod-security-users mailing list > > > mod...@li... > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users > > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > > > http://www.modsecurity.org/projects/commercial/rules/ > > > http://www.modsecurity.org/projects/commercial/support/ > > > > > > > > > ------------------------------------------------------------ > ------------------ > > > _______________________________________________ > > mod-security-users mailing list > > mod...@li... > > https://lists.sourceforge.net/lists/listinfo/mod-security-users > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > > http://www.modsecurity.org/projects/commercial/rules/ > > http://www.modsecurity.org/projects/commercial/support/ > > > -- > https://www.feistyduck.com/training/modsecurity-training-course > mailto:chr...@ne... > twitter: @ChrFolini > > ------------------------------------------------------------ > ------------------ > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > |
|
From: Christian F. <chr...@ne...> - 2016-09-08 15:38:15
|
On Thu, Sep 08, 2016 at 09:39:25AM -0500, Riemann . wrote: > Thanks Christian. We only use Windows servers in production, so I haven't > tried this under Unix. It might be that this specific behavior is specific > to the Windows install (which probably isn't the most typical for ModSec > users). According do Felipe rather not. But it might be worth a try. > I appreciate your suggestions--and I'll definitely look into using a custom > Apache log and trying to parse and alert on that as an alternative option. https://www.netnea.com/cms/flying-frog-anti-ddos-script/ might bring some inspiration. It's more of a proof of concept, but I still think the idea is neat. Cheers, Christian -- There is one rule, above all others, for being a man. Whatever comes, face it on your feet. -- Robert Jordan |
|
From: Robert P. <rpa...@fe...> - 2016-09-08 18:47:09
|
ModSec's double-read delta method isn't foolproof; I've seen some similar behaviors with large (hundreds of MB to several DB) sdbm instances. Tbh until libmodsec is stable and has support for memcached and redis persistent storage engines, I think there should be much more documentation and warning about the uses of on-disk persistent storage for highly concurrent operations like this. Spinning disks simply weren't meant to handle this kind of usage, and continuing to offer up things like https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/master/experimental_rules/modsecurity_crs_11_dos_protection.conf and https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.0.0-rc1/rules/REQUEST-912-DOS-PROTECTION.conf without providing a stable, high-performance environment to track this data is dangerous, imo (and to be clear, this isn't a criticism of the DoS ruleset or the CRS team's work in this area, but the fact that its essentially marketed as stable and usable while the community acknowledges that there's no reasonable expectation of being able to support these behaviors). Also, frankly using Apache as a DoS mitigation vector feels like shoving a square peg into a round hole. Writing complex rulesets to rate limit certain request patterns seems much better served by a downstream proxy than the app server itself (and at this point, we're starting to get into "move the WAF away from the app server" territory); indeed, using Nginx as a simple proxy with libmodsec to monitor this traffic would likely be a much more performant solution. Alternatives might also include using something like OpenResty to provide flexible scripting options for login traffic analysis in the context of a downstream reverse proxy, without being locked in directly to ModSecurity's syntax and environment (but still being able to leverage libmodsecurity if desired). On Thu, Sep 8, 2016 at 8:38 AM, Christian Folini < chr...@ne...> wrote: > On Thu, Sep 08, 2016 at 09:39:25AM -0500, Riemann . wrote: > > Thanks Christian. We only use Windows servers in production, so I haven't > > tried this under Unix. It might be that this specific behavior is > specific > > to the Windows install (which probably isn't the most typical for ModSec > > users). > > According do Felipe rather not. But it might be worth a try. > > > I appreciate your suggestions--and I'll definitely look into using a > custom > > Apache log and trying to parse and alert on that as an alternative > option. > > https://www.netnea.com/cms/flying-frog-anti-ddos-script/ might bring > some inspiration. It's more of a proof of concept, but I still think the > idea is neat. > > Cheers, > > Christian > > > -- > There is one rule, above all others, for being a man. > Whatever comes, face it on your feet. > -- Robert Jordan > > ------------------------------------------------------------ > ------------------ > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > |
|
From: Barry P. <bar...@ho...> - 2016-09-08 19:19:01
|
Agree Apache (and in particular ModSecurity's current implementation) isn't the right place for this. Disagree that Nginx and libmodsecurity is any better! It should be noted that rules are also in the experimental folder though must admit was quite confused what that meant when first using ModSecurity so think so extra documentation here would help. Also DoS protection is explicitly mentioned on OWASP website (https://www.owasp.org/index.php/Category:OWASP_ModSecurity_Core_Rule_Set_Project) when it probably shouldn't be. Saying that I have been quite successful in using this against brute force attacks. Here I only initialise the collection for very specific set of URLs (rather than every URL requested) to massively reduce read/write access of the collection and if a few updates are missed its no big deal. I also have to manually delete the collection file once a day to stop it growing too big (which it shouldn't do as values should in theory be cleared down but they aren't!). This obviously clears down any current counters, but Apache/ModSecurity seems to just create the file again for next request without complaint or need to restart. Thanks, Barry On 8 Sep 2016, at 19:51, Robert Paprocki <rpa...@fe...<mailto:rpa...@fe...>> wrote: ModSec's double-read delta method isn't foolproof; I've seen some similar behaviors with large (hundreds of MB to several DB) sdbm instances. Tbh until libmodsec is stable and has support for memcached and redis persistent storage engines, I think there should be much more documentation and warning about the uses of on-disk persistent storage for highly concurrent operations like this. Spinning disks simply weren't meant to handle this kind of usage, and continuing to offer up things like https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/master/experimental_rules/modsecurity_crs_11_dos_protection.conf and https://github.com/SpiderLabs/owasp-modsecurity-crs/blob/v3.0.0-rc1/rules/REQUEST-912-DOS-PROTECTION.conf without providing a stable, high-performance environment to track this data is dangerous, imo (and to be clear, this isn't a criticism of the DoS ruleset or the CRS team's work in this area, but the fact that its essentially marketed as stable and usable while the community acknowledges that there's no reasonable expectation of being able to support these behaviors). Also, frankly using Apache as a DoS mitigation vector feels like shoving a square peg into a round hole. Writing complex rulesets to rate limit certain request patterns seems much better served by a downstream proxy than the app server itself (and at this point, we're starting to get into "move the WAF away from the app server" territory); indeed, using Nginx as a simple proxy with libmodsec to monitor this traffic would likely be a much more performant solution. Alternatives might also include using something like OpenResty to provide flexible scripting options for login traffic analysis in the context of a downstream reverse proxy, without being locked in directly to ModSecurity's syntax and environment (but still being able to leverage libmodsecurity if desired). On Thu, Sep 8, 2016 at 8:38 AM, Christian Folini <chr...@ne...<mailto:chr...@ne...>> wrote: On Thu, Sep 08, 2016 at 09:39:25AM -0500, Riemann . wrote: > Thanks Christian. We only use Windows servers in production, so I haven't > tried this under Unix. It might be that this specific behavior is specific > to the Windows install (which probably isn't the most typical for ModSec > users). According do Felipe rather not. But it might be worth a try. > I appreciate your suggestions--and I'll definitely look into using a custom > Apache log and trying to parse and alert on that as an alternative option. https://www.netnea.com/cms/flying-frog-anti-ddos-script/ might bring some inspiration. It's more of a proof of concept, but I still think the idea is neat. Cheers, Christian -- There is one rule, above all others, for being a man. Whatever comes, face it on your feet. -- Robert Jordan ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/ http://www.modsecurity.org/projects/commercial/support/ |
|
From: Reindl H. <h.r...@th...> - 2016-09-08 19:27:18
Attachments:
signature.asc
|
Am 08.09.2016 um 20:47 schrieb Robert Paprocki: > Also, frankly using Apache as a DoS mitigation vector feels like shoving > a square peg into a round hole. Writing complex rulesets to rate limit > certain request patterns seems much better served by a downstream proxy > than the app server itself (and at this point, we're starting to get > into "move the WAF away from the app server" territory); indeed, using > Nginx as a simple proxy with libmodsec to monitor this traffic would > likely be a much more performant solution. Alternatives might also > include using something like OpenResty to provide flexible scripting > options for login traffic analysis in the context of a downstream > reverse proxy, without being locked in directly to ModSecurity's syntax > and environment (but still being able to leverage libmodsecurity if > desired) the idea protect a application from DOS attacks by some code inside the attacked application is simply a pervert one - is, was and ever will be for most type sof DOS attacks iptables on the host itself is much better (if you can't or don#t want have some layer *before* the target machine at all) |
|
From: Barry P. <bar...@ho...> - 2016-09-08 19:49:06
|
> the idea protect a application from DOS attacks by some code inside the attacked application is simply a pervert one - is, was and ever will be The idea of protecting the webserver itself from DoS attacks from within the webserver is indeed wrong. However the idea of protecting the application (which typically has a much lower DoS threshold) using the webserver that sits in front of it is not so crazy... |
|
From: Christian F. <chr...@ne...> - 2016-09-08 19:55:46
|
On Thu, Sep 08, 2016 at 07:48:51PM +0000, Barry Pollard wrote: > > the idea protect a application from DOS attacks by some code inside > > the attacked application is simply a pervert one - is, was and ever > > will be > > The idea of protecting the webserver itself from DoS attacks from > within the webserver is indeed wrong. > > However the idea of protecting the application (which typically has a > much lower DoS threshold) using the webserver that sits in front of it > is not so crazy... I think the self-defence is a very natural thought. It's just that it does not really work with DoS. But again, it's perfectly OK to have Apache/ModSecurity as sensor. Just don't let them do the blocking. This frees resources which helps them detect the remaining attackers. Ahoj, Christian -- Do what you can, with what you have, where you are! -- Theodore Roosevelt |
|
From: Reindl H. <h.r...@th...> - 2016-09-08 19:58:58
Attachments:
signature.asc
|
Am 08.09.2016 um 21:48 schrieb Barry Pollard:
>> the idea protect a application from DOS attacks by some code inside the attacked application is simply a pervert one - is, was and ever will be
>
> The idea of protecting the webserver itself from DoS attacks from within the webserver is indeed wrong.
>
> However the idea of protecting the application (which typically has a much lower DoS threshold) using the webserver that sits in front of it is not so crazy...
not really - i need to protect my server from accept more requests and
connections that it can handle, independent of the single application
playing around with some "dos prevention for the poor" inside the
application *will lower* that numbers by the added overhead
yes, you may have some applications in a mixed load which are more
problematic, but that won't change the problem bringing persistent
collections for rate-controls insidne the application layer will have a
negtaive impact for the whole server
___________________________
most of your problems are solved by rules like below on the application
host *combined* with a firewall in front with higher bounds to keep away
real attacks from the server at all
0 0 DROP tcp -- !lo * 0.0.0.0/0
0.0.0.0/0 ctstate NEW recent: UPDATE seconds: 2 hit_count:
100 name: DEFAULT side: source mask: 255.255.255.255
0 0 DROP tcp -- * * !192.168.196.0/24
0.0.0.0/0 tcp flags:0x17/0x02 #conn src/32 > 50
0 0 DROP tcp -- * * !192.168.196.0/24
0.0.0.0/0 tcp flags:0x17/0x02 #conn src/24 > 150
0 0 DROP tcp -- * * !192.168.196.0/24
0.0.0.0/0 tcp flags:0x17/0x02 #conn src/16 > 250
0 0 DROP tcp -- * * !192.168.196.0/24
0.0.0.0/0 tcp flags:0x17/0x02 #conn src/8 > 500
|
|
From: Jose P. V. L. <pab...@gm...> - 2016-09-08 20:15:56
|
Hi all. Sorry for not replying during all this time but I was fired from my job after years of mobbing from work companions. There are scripts that protect your server from ddos attacks like this one: http://www.fractalizer.ru/frpost_25/linux-installing-automatic-protection-from-dos-and-ddos-attacks-to-your-server/ I know that you are talking about dos attacks that are different from ddos attacks. Anyway time ago you could set in iptables how many times a ip could reach your server in a frame time and also you can set iptables rules forcing correct packet receiving so you can avoid fragmentation attacks and also syn flood attacks. Also there are firewalls able to check ip spoofing and fase ips, so with all these "screening" like juniper firewalls screening that you can configure in firewall you can add additional protection against dos attacks and another attack types. You can search for those attack types in iptables and add those rules to iptables because probably those rules are already written (or in firewall-cmd in redhat 7) Greetings El jue., 8 de septiembre de 2016 22:03, Reindl Harald < h.r...@th...> escribió: > > > Am 08.09.2016 um 21:48 schrieb Barry Pollard: > >> the idea protect a application from DOS attacks by some code inside the > attacked application is simply a pervert one - is, was and ever will be > > > > The idea of protecting the webserver itself from DoS attacks from within > the webserver is indeed wrong. > > > > However the idea of protecting the application (which typically has a > much lower DoS threshold) using the webserver that sits in front of it is > not so crazy... > > not really - i need to protect my server from accept more requests and > connections that it can handle, independent of the single application > > playing around with some "dos prevention for the poor" inside the > application *will lower* that numbers by the added overhead > > yes, you may have some applications in a mixed load which are more > problematic, but that won't change the problem bringing persistent > collections for rate-controls insidne the application layer will have a > negtaive impact for the whole server > ___________________________ > > most of your problems are solved by rules like below on the application > host *combined* with a firewall in front with higher bounds to keep away > real attacks from the server at all > > 0 0 DROP tcp -- !lo * 0.0.0.0/0 > 0.0.0.0/0 ctstate NEW recent: UPDATE seconds: 2 hit_count: > 100 name: DEFAULT side: source mask: 255.255.255.255 > 0 0 DROP tcp -- * * !192.168.196.0/24 > 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/32 > 50 > 0 0 DROP tcp -- * * !192.168.196.0/24 > 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/24 > 150 > 0 0 DROP tcp -- * * !192.168.196.0/24 > 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/16 > 250 > 0 0 DROP tcp -- * * !192.168.196.0/24 > 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/8 > 500 > > > ------------------------------------------------------------------------------ > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > |
|
From: Jose P. V. L. <pab...@gm...> - 2016-09-08 22:56:47
|
In this link you will find a iptables rule for dos prevention: http://blog.bodhizazen.net/linux/prevent-dos-with-iptables/ iptables -A INPUT -p tcp --dport 80 -m state --state NEW -m limit --limit 50/minute --limit-burst 200 -j ACCEPT "Lets break that rule down into intelligible chunks. -p tcp --dport 80 => Specifies traffic on port 80 (Normally Apache, but as you can see here I am using nginx). -m state NEW => This rule applies to NEW connections. -m limit --limit 50/minute --limit-burst 200 -j ACCEPT =>This is the essence of preventing DOS. “--limit-burst” is a bit confusing, but in a nutshell 200 new connections (packets really) are allowed before the limit of 50 NEW connections (packets) per minute is applied." Kind regards El jue., 8 de septiembre de 2016 22:15, Jose Pablo Valcárcel Lázaro < pab...@gm...> escribió: > Hi all. Sorry for not replying during all this time but I was fired from > my job after years of mobbing from work companions. > > There are scripts that protect your server from ddos attacks like this > one: > http://www.fractalizer.ru/frpost_25/linux-installing-automatic-protection-from-dos-and-ddos-attacks-to-your-server/ > > I know that you are talking about dos attacks that are different from ddos > attacks. Anyway time ago you could set in iptables how many times a ip > could reach your server in a frame time and also you can set iptables rules > forcing correct packet receiving so you can avoid fragmentation attacks and > also syn flood attacks. Also there are firewalls able to check ip spoofing > and fase ips, so with all these "screening" like juniper firewalls > screening that you can configure in firewall you can add additional > protection against dos attacks and another attack types. You can search for > those attack types in iptables and add those rules to iptables because > probably those rules are already written (or in firewall-cmd in redhat 7) > > Greetings > > El jue., 8 de septiembre de 2016 22:03, Reindl Harald < > h.r...@th...> escribió: > >> >> >> Am 08.09.2016 um 21:48 schrieb Barry Pollard: >> >> the idea protect a application from DOS attacks by some code inside >> the attacked application is simply a pervert one - is, was and ever will be >> > >> > The idea of protecting the webserver itself from DoS attacks from >> within the webserver is indeed wrong. >> > >> > However the idea of protecting the application (which typically has a >> much lower DoS threshold) using the webserver that sits in front of it is >> not so crazy... >> >> not really - i need to protect my server from accept more requests and >> connections that it can handle, independent of the single application >> >> playing around with some "dos prevention for the poor" inside the >> application *will lower* that numbers by the added overhead >> >> yes, you may have some applications in a mixed load which are more >> problematic, but that won't change the problem bringing persistent >> collections for rate-controls insidne the application layer will have a >> negtaive impact for the whole server >> ___________________________ >> >> most of your problems are solved by rules like below on the application >> host *combined* with a firewall in front with higher bounds to keep away >> real attacks from the server at all >> >> 0 0 DROP tcp -- !lo * 0.0.0.0/0 >> 0.0.0.0/0 ctstate NEW recent: UPDATE seconds: 2 hit_count: >> 100 name: DEFAULT side: source mask: 255.255.255.255 >> 0 0 DROP tcp -- * * !192.168.196.0/24 >> 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/32 > 50 >> 0 0 DROP tcp -- * * !192.168.196.0/24 >> 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/24 > 150 >> 0 0 DROP tcp -- * * !192.168.196.0/24 >> 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/16 > 250 >> 0 0 DROP tcp -- * * !192.168.196.0/24 >> 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/8 > 500 >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> http://www.modsecurity.org/projects/commercial/support/ >> > |
|
From: Reindl H. <h.r...@th...> - 2016-09-09 08:17:39
Attachments:
signature.asc
|
Am 09.09.2016 um 00:56 schrieb Jose Pablo Valcárcel Lázaro: > In this link you will find a iptables rule for dos prevention: > http://blog.bodhizazen.net/linux/prevent-dos-with-iptables/ > > iptables -A INPUT -p tcp --dport 80 -m state --state NEW -m limit > --limit 50/minute --limit-burst 200 -j ACCEPT > > "Lets break that rule down into intelligible chunks. > > -p tcp --dport 80 => Specifies traffic on port 80 (Normally Apache, but > as you can see here I am using nginx). > > -m state NEW => This rule applies to NEW connections. > > -m limit --limit 50/minute --limit-burst 200 -j ACCEPT =>This is the > essence of preventing DOS. > > “--limit-burst” is a bit confusing, but in a nutshell 200 new > connections (packets really) are allowed before the limit of 50 NEW > connections (packets) per minute is applied." well 50 connections per minute on a webserver is a joke since a typical site these days has a lot of scripts, images and different people are coming with the same IP trhugh carrier grade NAT on mobile networks :-) frankly there where a reason why i posted *values from production servers* and not soemthing completly unuseable, 50/minute is pretty sure copied from a inbound mailserver and 25 replaced by 80 since there is hardly a service which has a real workload with more connections within a timeframe it also makes no sense to apply that specificly to port 80 0 0 DROP tcp -- !lo * 0.0.0.0/0 0.0.0.0/0 ctstate NEW recent: UPDATE seconds: 2 hit_count: 100 name: DEFAULT side: source mask: 255.255.255.255 0 0 DROP tcp -- * * !192.168.196.0/24 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/32 > 50 0 0 DROP tcp -- * * !192.168.196.0/24 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/24 > 150 0 0 DROP tcp -- * * !192.168.196.0/24 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/16 > 250 0 0 DROP tcp -- * * !192.168.196.0/24 0.0.0.0/0 tcp flags:0x17/0x02 #conn src/8 > 500 echo "DOS-PROTECTION: Nicht mehr als 100 NEUE Verbindungen pro 2-Sekunden/Client-IP" iptables -I INPUT -p tcp ! -i lo -m conntrack --ctstate NEW -m recent --set iptables -I INPUT -p tcp ! -i lo -m conntrack --ctstate NEW -m recent --update --seconds 2 --hitcount 100 -j DROP iptables -I INPUT -p tcp ! -i lo -m conntrack --ctstate NEW -m recent --update --seconds 2 --hitcount 100 -m limit --limit 50/h -j LOG --log-level debug --log-prefix "Firewall Rate-Control: " iptables -I INPUT -p udp ! -i lo -m conntrack --ctstate NEW -m recent --name udpflood --set iptables -I INPUT -p udp ! -i lo -m conntrack --ctstate NEW -m recent --name udpflood --update --seconds 2 --hitcount 100 -j DROP iptables -I INPUT -p udp ! -i lo -m conntrack --ctstate NEW -m recent --name udpflood --update --seconds 2 --hitcount 100 -m limit --limit 50/h -j LOG --log-level debug --log-prefix "Firewall Rate-Control: " echo "DOS-PROTECTION: Nicht mehr als 50 GLEICHZEITIGE Verbindungen pro IP (Slowloris)" iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 50 --connlimit-mask 32 -m limit --limit 50/h -j LOG --log-level debug --log-prefix "Firewall Slowloris mask 32: " iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 50 --connlimit-mask 32 -j DROP echo "DOS-PROTECTION: Nicht mehr als 150 GLEICHZEITIGE Verbindungen pro /24-Netzwerk" iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 150 --connlimit-mask 24 -m limit --limit 50/h -j LOG --log-level debug --log-prefix "Firewall Slowloris mask 24: " iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 150 --connlimit-mask 24 -j DROP echo "DOS-PROTECTION: Nicht mehr als 250 GLEICHZEITIGE Verbindungen pro /16-Netzwerk" iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 250 --connlimit-mask 16 -m limit --limit 50/h -j LOG --log-level debug --log-prefix "Firewall Slowloris mask 16: " iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 250 --connlimit-mask 16 -j DROP echo "DOS-PROTECTION: Nicht mehr als 500 GLEICHZEITIGE Verbindungen pro /8-Netzwerk" iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 500 --connlimit-mask 8 -m limit --limit 50/h -j LOG --log-level debug --log-prefix "Firewall Slowloris mask 8: " iptables -A INPUT -p tcp--syn -m connlimit --connlimit-above 500 --connlimit-mask 8 -j DROP |
