Re: [mod-security-users] Working Around Race Conditions in Persistent Storage
Brought to you by:
victorhora,
zimmerletw
From: Felipe C. <FC...@tr...> - 2016-09-08 13:16:29
|
Hi, There are few problems in the persistent storage for 2.x. Huge amount of disk space usage, Concurrence problems, Slow sync between the threads are example of those problems. The best way to circumvent all the problems is to reduce the usage of the storage, to a “size” that fits what a dbm can holds. The problem might not be specific on the ModSecurity implementation, but with the fact that we might be making an abuse in the way that we are using the dbm. For ModSecurity version 3 the persistent storage are modular and you can choose what fits better your infrastructure, for instance: in-memory storage, Redis, Memcache. Open tickets regarding that: Redis - https://github.com/SpiderLabs/ModSecurity/issues/1139 Memcache - https://github.com/SpiderLabs/ModSecurity/issues/1140 We already have implemented the lmdb and the in memory backends: https://github.com/SpiderLabs/ModSecurity/tree/libmodsecurity/src/collection/backend Technically it is possible to use Lua, although I am not sure if that is a good idea, you have to check how much overhead it will add into your environment. Br., Felipe “Zimmerle” Costa Security Researcher, Lead Developer ModSecurity. Trustwave | SMART SECURITY ON DEMAND www.trustwave.com<http://www.trustwave.com/> From: "Riemann ." <rie...@gm...<mailto:rie...@gm...>> Reply-To: "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>> Date: Wednesday, September 7, 2016 at 8:01 PM To: "mod...@li...<mailto:mod...@li...>" <mod...@li...<mailto:mod...@li...>> Subject: Re: [mod-security-users] Working Around Race Conditions in Persistent Storage Hi Barry, Thanks for your response and the link to the previous discussion. I do see the error you mentioned in my debug log. This is a bummer. In the ModSec Handbook, Ivan seems to infer the double-retrieve/write-lock mechanism should prevent race conditions with numerical data being stored (pg 129-130), but this just doesn't seem to be the case. Maybe this is a bad idea, but as an alternative to using persistent storage, would it be crazy to try to implement some of these rules by passing variables to a Lua script that reads from and writes to an instance-specific SQLite or enterprise-class database? Are there any other ideas as to how this can be acheived? Of course, it would be ideal to integrate some of these types of things directly into the application, but it's a lot faster to turn rules out with ModSec, than to try to get them fit into a release cycle that may be a few months (or more away), depending on development's other priorities. Thanks, Riemann On Wed, Sep 7, 2016 at 4:29 PM, Barry Pollard <bar...@ho...<mailto:bar...@ho...>> wrote: Are you seeing any errors like this in your logs?: Message: collections_remove_stale: Failed deleting collection I'm really not convinced this works using disk based collections file - especially under stress (like brute force attacks). So ultimately don't think ModSecurity is right solution for this - at least until a better, memory based collections store is available. See more details when this was discussed before on this list here: https://sourceforge.net/p/mod-security/mailman/message/34392875/<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCVF5ukQbQ&s=5&u=https%3a%2f%2fsourceforge%2enet%2fp%2fmod-security%2fmailman%2fmessage%2f34392875%2f> Thanks, Barry On 7 Sep 2016, at 22:05, Riemann . <rie...@gm...<mailto:rie...@gm...>> wrote: Hello, I'm trying to write rules to track and log (not block) brute force login attempts. I've initialized the Global and IP collection in CRS_10, and have created a separate rule/config file for horizontal brute force attempts. I've included my attempt at a horizontal brute force rule below ("rule one"). The rule works exactly as I'd expect if I use Burp Intruder to simulate a brute force attack, sending POST requests to login.jsp, with "username" and "password" parameters in the request body, and changing only the username on each request. If I run the exact same 'attack,' but this time with five threads, I run into problems. My counter may equal '1' for five consecutive requests, and then jump to '6', but only show two usernames are stored in IP.username (for example). I've also tried using rules similar to the horizontal brute force rules in The Web Application Defender's Cookbook (recipe 7-2, pg 269-271), and found I had similar issues with multi-threaded attacks for those rules too. I know SDBM allows for concurrent access, and I've read (and re-read) about how collections work in Chapter 8 of the ModSecurity Handbook (including the parts about storing non-numeric values in SDBM, and race conditions). In light of that I've attempted to use only numeric values by eliminating all stored variables except the counter, as seen in "rule two" below (and completely eliminating the tracking of unique usernames). Still, I had issues where multi-threading a brute force attack resulted in the counter value not being correct for several requests in a row. So my questions: 1. Am I missing something, or doing something fundamentally wrong in how I'm using persistent storage? 2. I realize the potential for race conditions with SDBM, especially with non-numeric data, but is what I'm describing entirely expected behavior? Numeric values in persistent storage seem to be fairly unreliable on a per-request basis (in logs, my counter is wrong more than it is right). 3. Is there a better approach to accomplish what I'm trying to do, which is track horizontal (by IP addr and username), and eventually vertical (by IP addr and password hash) brute force attempts? For example, using Lua, or some other persistent storage mechanism (memcached isn't an option). If it matters, I'm currently testing with ModSec 2.9.1 on Apache 2.4.23, running on Windows. Thanks, Riemann ######################################## # 'Rule' One # ########### SecRule REQUEST_FILENAME "!@pm login.jsp" \ "phase:2,t:none,pass,nolog, \ id:999321, \ skipAfter:END_BRUTE_FORCE" SecRule REQUEST_METHOD "!@streq POST" \ "phase:2,t:none,pass,nolog, \ id:999322, \ skipAfter:END_BRUTE_FORCE" SecRule ARGS:username|ARGS:user ".+" \ "chain,phase:2,t:none,t:lowercase,pass,nolog, \ id:999323, \ skipAfter:LOGIN_TEST" SecRule ARGS:password|ARGS:pass ".+" \ "t:none,t:lowercase" SecAction \ "phase:2,pass,nolog, \ skipAfter:END_BRUTE_FORCE, \ id:'1000950'" SecMarker LOGIN_TEST SecRule IP:username_ct "@ge 5" \ "phase:5,t:none,log,pass, \ id:'1000951', \ msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \ logdata:'Usernames Attempted Count: %{IP.username_ct}, Attempted Usernames: %{IP.username}.', \ setvar:IP.clear_username=1" SecRule &IP:CLEAR_USERNAME "@eq 1" \ "phase:5,pass,nolog, \ id:'1000952', \ setvar:!IP.KEY" SecRule &IP:username_ct "@eq 0" \ "chain,phase:2,pass,nolog, \ id:'1000953'" SecRule ARGS:username|ARGS:user ".+" \ "t:none,t:lowercase, \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300, \ setvar:IP.username=%{matched_var}, \ expirevar:IP.username=300" SecRule &IP:username_ct "@ge 1" \ "chain,phase:2,pass,nolog, \ id:'1000954'" SecRule ARGS:username|ARGS:user "!@within %{IP.username}" \ "t:none,t:lowercase, \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300, \ setvar:'IP.username=%{IP.username},%{matched_var}', \ expirevar:IP.username=300" SecMarker END_BRUTE_FORCE ######################################## # 'Rule' Two # ########### SecRule REQUEST_FILENAME "!@pm login.jsp" \ "phase:2,t:none,pass,nolog, \ id:999321, \ skipAfter:END_BRUTE_FORCE" SecRule REQUEST_METHOD "!@streq POST" \ "phase:2,t:none,pass,nolog, \ id:999322, \ skipAfter:END_BRUTE_FORCE" SecRule ARGS:username|ARGS:user ".+" \ "chain,phase:2,t:none,t:lowercase,pass,nolog, \ id:999323, \ skipAfter:LOGIN_TEST" SecRule ARGS:password|ARGS:pass ".+" \ "t:none,t:lowercase" SecAction \ "phase:2,pass,nolog, \ skipAfter:END_BRUTE_FORCE, \ id:'1000950'" SecAction \ "phase:2,pass,log, \ id:'1000959', \ logdata:'Usernames Attempted Count: %{IP.username_ct}.'" SecRule IP:username_ct "@ge 7" \ "phase:2,t:none,log,pass, \ id:'1000951', \ msg:'Multiple Username Violation - Too Many Usernames Submitted from %{REMOTE_ADDR}.' \ logdata:'Usernames Attempted Count: %{IP.username_ct}.', \ setvar:IP.clear_username=1" SecRule &IP:CLEAR_USERNAME "@eq 1" \ "phase:2,pass,nolog, \ id:'1000952', \ setvar:!IP.KEY" SecRule ARGS:username|ARGS:user ".+" \ "t:none,t:lowercase,pass,nolog, \ id:'1000953', \ setvar:IP.username_ct=+1, \ expirevar:IP.username_ct=300" SecMarker END_BRUTE_FORCE ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> http://www.modsecurity.org/projects/commercial/support/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> ------------------------------------------------------------------------------ _______________________________________________ mod-security-users mailing list mod...@li...<mailto:mod...@li...> https://lists.sourceforge.net/lists/listinfo/mod-security-users<https://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSHdH57FHYw&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo%2fmod-security-users> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: http://www.modsecurity.org/projects/commercial/rules/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCUVs-gTZw&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2frules%2f> http://www.modsecurity.org/projects/commercial/support/<http://scanmail.trustwave.com/?c=4062&d=lJ3Q14dFu_P-rdUvQgwejJ8wYTorQhgkSCQR4eRHMg&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercial%2fsupport%2f> ________________________________ This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |