Re: [mod-security-users] Working Around Race Conditions in Persistent Storage

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi,

There are few problems in the persistent storage for 2.x. Huge amount of disk space usage,
Concurrence problems, Slow sync between the threads are example of those problems. The
best way to circumvent all the problems is to reduce the usage of the storage, to a “size” that
fits what a dbm can holds.

The problem might not be specific on the ModSecurity implementation, but with the fact that
we might be making an abuse in the way that we are using the dbm.

For ModSecurity version 3 the persistent storage are modular and you can choose what fits
better your infrastructure, for instance: in-memory storage, Redis, Memcache. Open tickets
regarding that:

Redis - https://github.com/SpiderLabs/ModSecurity/issues/1139
Memcache - https://github.com/SpiderLabs/ModSecurity/issues/1140

We already have implemented the lmdb and the in memory backends:
https://github.com/SpiderLabs/ModSecurity/tree/libmodsecurity/src/collection/backend

Technically it is possible to use Lua, although I am not sure if that is a good idea, you have
to check how much overhead it will add into your environment.

Br.,
Felipe “Zimmerle” Costa
Security Researcher, Lead Developer ModSecurity.

Trustwave | SMART SECURITY ON DEMAND
www.trustwave.com<http://www.trustwave.com/>

From: "Riemann ." <rie...@gm...<mailto:rie...@gm...>>
Reply-To: "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>>
Date: Wednesday, September 7, 2016 at 8:01 PM
To: "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>>
Subject: Re: [mod-security-users] Working Around Race Conditions in Persistent Storage

Hi Barry,

Thanks for your response and the link to the previous discussion. I do see the error you mentioned in my debug log.

This is a bummer. In the ModSec Handbook, Ivan seems to infer the double-retrieve/write-lock mechanism should prevent race conditions with numerical data being stored (pg 129-130), but this just doesn't seem to be the case.

Maybe this is a bad idea, but as an alternative to using persistent storage, would it be crazy to try to implement some of these rules by passing variables to a Lua script that reads from and writes to an instance-specific SQLite or enterprise-class database? Are there any other ideas as to how this can be acheived?

Of course, it would be ideal to integrate some of these types of things directly into the application, but it's a lot faster to turn rules out with ModSec, than to try to get them fit into a release cycle that may be a few months (or more away), depending on development's other priorities.

Thanks,
Riemann

On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...<mailto:bar...@ho...>> wrote:
Are you seeing any errors like this in your logs?:

Message: collections_remove_stale: Failed deleting collection

I'm really not convinced this works using disk based collections file - especially under stress (like brute force attacks). So ultimately don't think ModSecurity is right solution for this - at least until a better, memory based collections store is available.

See more details when this was discussed before on this list here:
https://sourceforge.net/p/mod-security/mailman/message/34392875/<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCVF5ukQbQ&s=5&u=https%3a%2f%2fsourceforge%2enet%2fp%2fmod-security%2fmailman%2fmessage%2f34392875%2f>

Thanks,
Barry

On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...<mailto:rie...@gm...>> wrote:

Hello,

I'm trying to write rules to track and log (not block) brute force login attempts. I've initialized the Global and IP collection in CRS_10, and have created a separate rule/config file for horizontal brute force attempts. I've included my attempt at a horizontal brute force rule below ("rule one"). The rule works exactly as I'd expect if I use Burp Intruder to simulate a brute force attack, sending POST requests to login.jsp, with "username" and "password" parameters in the request body, and changing only the username on each request.

If I run the exact same 'attack,' but this time with five threads, I run into problems. My counter may equal '1' for five consecutive requests, and then jump to '6', but only show two usernames are stored in IP.username (for example). I've also tried using rules similar to the horizontal brute force rules in The Web Application Defender's Cookbook (recipe 7-2, pg 269-271), and found I had similar issues with multi-threaded attacks for those rules too.

I know SDBM allows for concurrent access, and I've read (and re-read) about how collections work in Chapter 8 of the ModSecurity Handbook (including the parts about storing non-numeric values in SDBM, and race conditions). In light of that I've attempted to use only numeric values by eliminating all stored variables except the counter, as seen in "rule two" below (and completely eliminating the tracking of unique usernames). Still, I had issues where multi-threading a brute force attack resulted in the counter value not being correct for several requests in a row.

So my questions:

  1.  Am I missing something, or doing something fundamentally wrong in how I'm using persistent storage?
  2.  I realize the potential for race conditions with SDBM, especially with non-numeric data, but is what I'm describing entirely expected behavior? Numeric values in persistent storage seem to be fairly unreliable on a per-request basis (in logs, my counter is wrong more than it is right).
  3.  Is there a better approach to accomplish what I'm trying to do, which is track horizontal (by IP addr and username), and eventually vertical (by IP addr and password hash) brute force attempts? For example, using Lua, or some other persistent storage mechanism (memcached isn't an option).

If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, running on Windows.

Thanks,
Riemann

########################################
# 'Rule' One #
###########

SecRule REQUEST_FILENAME "!@pm login.jsp" \
"phase:2,t:none,pass,nolog, \
id:999321, \
skipAfter:END_BRUTE_FORCE"
SecRule REQUEST_METHOD "!@streq POST" \
"phase:2,t:none,pass,nolog, \
id:999322, \
skipAfter:END_BRUTE_FORCE"

SecRule ARGS:username|ARGS:user ".+" \
"chain,phase:2,t:none,t:lowercase,pass,nolog, \
id:999323, \
skipAfter:LOGIN_TEST"
SecRule ARGS:password|ARGS:pass ".+" \
"t:none,t:lowercase"

SecAction \
"phase:2,pass,nolog, \
skipAfter:END_BRUTE_FORCE, \
id:'1000950'"

SecMarker LOGIN_TEST
SecRule IP:username_ct "@ge 5" \
"phase:5,t:none,log,pass, \
id:'1000951', \
msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \
logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted Usernames: %{IP.username}.', \
setvar:IP.clear_username=1"
SecRule &IP:CLEAR_USERNAME "@eq 1" \
"phase:5,pass,nolog, \
id:'1000952', \
setvar:!IP.KEY"
SecRule &IP:username_ct "@eq 0" \
"chain,phase:2,pass,nolog, \
id:'1000953'"
SecRule ARGS:username|ARGS:user ".+" \
"t:none,t:lowercase, \
setvar:IP.username_ct=+1, \
expirevar:IP.username_ct=300, \
setvar:IP.username=%{matched_var}, \
expirevar:IP.username=300"

SecRule &IP:username_ct "@ge 1" \
"chain,phase:2,pass,nolog, \
id:'1000954'"
SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \
"t:none,t:lowercase, \
setvar:IP.username_ct=+1, \
expirevar:IP.username_ct=300, \
setvar:'IP.username=%{IP.username},%{matched_var}', \
expirevar:IP.username=300"
SecMarker END_BRUTE_FORCE

########################################
# 'Rule' Two #
###########

SecRule REQUEST_FILENAME "!@pm login.jsp" \
"phase:2,t:none,pass,nolog, \
id:999321, \
skipAfter:END_BRUTE_FORCE"
SecRule REQUEST_METHOD "!@streq POST" \
"phase:2,t:none,pass,nolog, \
id:999322, \
skipAfter:END_BRUTE_FORCE"

SecRule ARGS:username|ARGS:user ".+" \
"chain,phase:2,t:none,t:lowercase,pass,nolog, \
id:999323, \
skipAfter:LOGIN_TEST"
SecRule ARGS:password|ARGS:pass ".+" \
"t:none,t:lowercase"

SecAction \
"phase:2,pass,nolog, \
skipAfter:END_BRUTE_FORCE, \
id:'1000950'"

SecAction \
"phase:2,pass,log, \
id:'1000959', \
logdata:'Usernames Attempted Count: %{IP.username_ct}.'"
SecRule IP:username_ct "@ge 7" \
"phase:2,t:none,log,pass, \
id:'1000951', \
msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \
logdata:'Usernames Attempted Count: %{IP.username_ct}.', \
setvar:IP.clear_username=1"
SecRule &IP:CLEAR_USERNAME "@eq 1" \
"phase:2,pass,nolog, \
id:'1000952', \
setvar:!IP.KEY"
SecRule ARGS:username|ARGS:user ".+" \
"t:none,t:lowercase,pass,nolog, \
id:'1000953', \
setvar:IP.username_ct=+1, \
expirevar:IP.username_ct=300"
SecMarker END_BRUTE_FORCE
------------------------------------------------------------------------------
_______________________________________________
mod-security-users mailing list
mod...@li...<mailto:mod...@li...>
https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users>
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f>
http://www.modsecurity.org/projects/commercial/support/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f>

------------------------------------------------------------------------------

_______________________________________________
mod-security-users mailing list
mod...@li...<mailto:mod...@li...>
https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users>
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f>
http://www.modsecurity.org/projects/commercial/support/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f>

________________________________

This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.