Re: [mod-security-users] Working Around Race Conditions in Persistent Storage
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
Re: [mod-security-users] Working Around Race Conditions in Persistent Storage
|
From: Riemann . <rie...@gm...> - 2016-09-08 15:09:53
|
Thanks for your response Felipe. In terms of the dbm size, I wanted to make sure I wasn't running into any sort of issues like that. I only enabled my modsec config (CRS_10) and a second config file with the rules I included in my orig post. I was seeing these issues with "rule two" (just keeping a count of logins per IP) while sending just a total of 22 requests using five threads. There was no other traffic hitting the server and no other rules were being used. I also tried it with all rules enabled, and moved some of the rules to phase 5 to try to introduce some kind of delay, but that didn't help either. Hopefully this won't be an issue in v3. I haven't done any testing with it yet since it's not yet production-ready, but it sounds like there are several positive improvements. On Thu, Sep 8, 2016 at 8:04 AM, Felipe Costa <FC...@tr...> wrote: > Hi, > > There are few problems in the persistent storage for 2.x. Huge amount of > disk space usage, > Concurrence problems, Slow sync between the threads are example of those > problems. The > best way to circumvent all the problems is to reduce the usage of the > storage, to a “size” that > fits what a dbm can holds. > > The problem might not be specific on the ModSecurity implementation, but > with the fact that > we might be making an abuse in the way that we are using the dbm. > > For ModSecurity version 3 the persistent storage are modular and you can > choose what fits > better your infrastructure, for instance: in-memory storage, Redis, > Memcache. Open tickets > regarding that: > > Redis - https://github.com/SpiderLabs/ModSecurity/issues/1139 > Memcache - https://github.com/SpiderLabs/ModSecurity/issues/1140 > > We already have implemented the lmdb and the in memory backends: > https://github.com/SpiderLabs/ModSecurity/tree/ > libmodsecurity/src/collection/backend > > Technically it is possible to use Lua, although I am not sure if that is a > good idea, you have > to check how much overhead it will add into your environment. > > > Br., > > *Felipe “Zimmerle” Costa * > > Security Researcher, Lead Developer ModSecurity. > > > > *Trustwave* | SMART SECURITY ON DEMAND > > www.trustwave.com > > > From: "Riemann ." <rie...@gm...> > Reply-To: "mod...@li..." < > mod...@li...> > Date: Wednesday, September 7, 2016 at 8:01 PM > To: "mod...@li..." <mod-security-users@lists. > sourceforge.net> > Subject: Re: [mod-security-users] Working Around Race Conditions in > Persistent Storage > > Hi Barry, > > Thanks for your response and the link to the previous discussion. I do see > the error you mentioned in my debug log. > > This is a bummer. In the ModSec Handbook, Ivan seems to infer the > double-retrieve/write-lock mechanism should prevent race conditions with > numerical data being stored (pg 129-130), but this just doesn't seem to be > the case. > > Maybe this is a bad idea, but as an alternative to using persistent > storage, would it be crazy to try to implement some of these rules by > passing variables to a Lua script that reads from and writes to an > instance-specific SQLite or enterprise-class database? Are there any other > ideas as to how this can be acheived? > > Of course, it would be ideal to integrate some of these types of things > directly into the application, but it's a lot faster to turn rules out with > ModSec, than to try to get them fit into a release cycle that may be a few > months (or more away), depending on development's other priorities. > > Thanks, > Riemann > > > > On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...> > wrote: > >> Are you seeing any errors like this in your logs?: >> >> Message: collections_remove_stale: Failed deleting collection >> >> I'm really not convinced this works using disk based collections file - >> especially under stress (like brute force attacks). So ultimately don't >> think ModSecurity is right solution for this - at least until a better, >> memory based collections store is available. >> >> See more details when this was discussed before on this list here: >> https://sourceforge.net/p/mod-security/mailman/message/34392875/ >> <https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCVF5ukQbQ&s=5&u=https%3a%2f%2fsourceforge%2enet%2fp%2fmod-security%2fmailman%2fmessage%2f34392875%2f> >> >> Thanks, >> Barry >> >> On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...> wrote: >> >> Hello, >> >> I'm trying to write rules to track and log (not block) brute force login >> attempts. I've initialized the Global and IP collection in CRS_10, and have >> created a separate rule/config file for horizontal brute force attempts. >> I've included my attempt at a horizontal brute force rule below ("rule >> one"). The rule works exactly as I'd expect if I use Burp Intruder to >> simulate a brute force attack, sending POST requests to login.jsp, with >> "username" and "password" parameters in the request body, and changing only >> the username on each request. >> >> If I run the exact same 'attack,' but this time with five threads, I run >> into problems. My counter may equal '1' for five consecutive requests, and >> then jump to '6', but only show two usernames are stored in IP.username >> (for example). I've also tried using rules similar to the horizontal brute >> force rules in The Web Application Defender's Cookbook (recipe 7-2, pg >> 269-271), and found I had similar issues with multi-threaded attacks for >> those rules too. >> >> I know SDBM allows for concurrent access, and I've read (and re-read) >> about how collections work in Chapter 8 of the ModSecurity Handbook >> (including the parts about storing non-numeric values in SDBM, and race >> conditions). In light of that I've attempted to use only numeric values by >> eliminating all stored variables except the counter, as seen in "rule two" >> below (and completely eliminating the tracking of unique usernames). Still, >> I had issues where multi-threading a brute force attack resulted in the >> counter value not being correct for several requests in a row. >> >> So my questions: >> >> 1. Am I missing something, or doing something fundamentally wrong in >> how I'm using persistent storage? >> 2. I realize the potential for race conditions with SDBM, especially >> with non-numeric data, but is what I'm describing entirely expected >> behavior? Numeric values in persistent storage seem to be fairly unreliable >> on a per-request basis (in logs, my counter is wrong more than it is right). >> 3. Is there a better approach to accomplish what I'm trying to do, >> which is track horizontal (by IP addr and username), and eventually >> vertical (by IP addr and password hash) brute force attempts? For example, >> using Lua, or some other persistent storage mechanism (memcached isn't an >> option). >> >> If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, >> running on Windows. >> >> Thanks, >> Riemann >> >> ######################################## >> # 'Rule' One # >> ########### >> >> SecRule REQUEST_FILENAME "!@pm login.jsp" \ >> "phase:2,t:none,pass,nolog, \ >> id:999321, \ >> skipAfter:END_BRUTE_FORCE" >> SecRule REQUEST_METHOD "!@streq POST" \ >> "phase:2,t:none,pass,nolog, \ >> id:999322, \ >> skipAfter:END_BRUTE_FORCE" >> >> SecRule ARGS:username|ARGS:user ".+" \ >> "chain,phase:2,t:none,t:lowercase,pass,nolog, \ >> id:999323, \ >> skipAfter:LOGIN_TEST" >> SecRule ARGS:password|ARGS:pass ".+" \ >> "t:none,t:lowercase" >> >> SecAction \ >> "phase:2,pass,nolog, \ >> skipAfter:END_BRUTE_FORCE, \ >> id:'1000950'" >> >> SecMarker LOGIN_TEST >> SecRule IP:username_ct "@ge 5" \ >> "phase:5,t:none,log,pass, \ >> id:'1000951', \ >> msg:'Multiple Username Violation - Too Many Usernames Submitted from >> %{REMOTE_ADDR}.' \ >> logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted >> Usernames: %{IP.username}.', \ >> setvar:IP.clear_username=1" >> SecRule &IP:CLEAR_USERNAME "@eq 1" \ >> "phase:5,pass,nolog, \ >> id:'1000952', \ >> setvar:!IP.KEY" >> SecRule &IP:username_ct "@eq 0" \ >> "chain,phase:2,pass,nolog, \ >> id:'1000953'" >> SecRule ARGS:username|ARGS:user ".+" \ >> "t:none,t:lowercase, \ >> setvar:IP.username_ct=+1, \ >> expirevar:IP.username_ct=300, \ >> setvar:IP.username=%{matched_var}, \ >> expirevar:IP.username=300" >> >> SecRule &IP:username_ct "@ge 1" \ >> "chain,phase:2,pass,nolog, \ >> id:'1000954'" >> SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ >> "t:none,t:lowercase, \ >> setvar:IP.username_ct=+1, \ >> expirevar:IP.username_ct=300, \ >> setvar:'IP.username=%{IP.username},%{matched_var}', \ >> expirevar:IP.username=300" >> SecMarker END_BRUTE_FORCE >> >> ######################################## >> # 'Rule' Two # >> ########### >> >> SecRule REQUEST_FILENAME "!@pm login.jsp" \ >> "phase:2,t:none,pass,nolog, \ >> id:999321, \ >> skipAfter:END_BRUTE_FORCE" >> SecRule REQUEST_METHOD "!@streq POST" \ >> "phase:2,t:none,pass,nolog, \ >> id:999322, \ >> skipAfter:END_BRUTE_FORCE" >> >> SecRule ARGS:username|ARGS:user ".+" \ >> "chain,phase:2,t:none,t:lowercase,pass,nolog, \ >> id:999323, \ >> skipAfter:LOGIN_TEST" >> SecRule ARGS:password|ARGS:pass ".+" \ >> "t:none,t:lowercase" >> >> SecAction \ >> "phase:2,pass,nolog, \ >> skipAfter:END_BRUTE_FORCE, \ >> id:'1000950'" >> >> SecAction \ >> "phase:2,pass,log, \ >> id:'1000959', \ >> logdata:'Usernames Attempted Count: %{IP.username_ct}.'" >> SecRule IP:username_ct "@ge 7" \ >> "phase:2,t:none,log,pass, \ >> id:'1000951', \ >> msg:'Multiple Username Violation - Too Many Usernames Submitted from >> %{REMOTE_ADDR}.' \ >> logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ >> setvar:IP.clear_username=1" >> SecRule &IP:CLEAR_USERNAME "@eq 1" \ >> "phase:2,pass,nolog, \ >> id:'1000952', \ >> setvar:!IP.KEY" >> SecRule ARGS:username|ARGS:user ".+" \ >> "t:none,t:lowercase,pass,nolog, \ >> id:'1000953', \ >> setvar:IP.username_ct=+1, \ >> expirevar:IP.username_ct=300" >> SecMarker END_BRUTE_FORCE >> >> ------------------------------------------------------------ >> ------------------ >> >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> <https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> >> http://www.modsecurity.org/projects/commercial/support/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> >> >> >> ------------------------------------------------------------ >> ------------------ >> >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> <https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> >> http://www.modsecurity.org/projects/commercial/support/ >> <http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> >> >> > > ------------------------------ > > This transmission may contain information that is privileged, > confidential, and/or exempt from disclosure under applicable law. If you > are not the intended recipient, you are hereby notified that any > disclosure, copying, distribution, or use of the information contained > herein (including any reliance thereon) is strictly prohibited. If you > received this transmission in error, please immediately contact the sender > and destroy the material in its entirety, whether in electronic or hard > copy format. > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > |
