Re: [mod-security-users] rule performance
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
Re: [mod-security-users] rule performance
|
From: Bruno de A. <br...@sa...> - 2015-03-26 18:07:24
|
Hi Felipe, Apache. I was only trialing nginx a few months back, we've been using apache + mod_sec for a few years now. I am trying to find the slow rules using the debug log now, but that's time consuming. Bruno On 26 March 2015 at 13:21, Felipe Costa <FC...@tr...> wrote: > Hi Bruno, > > are you still using nginx? Or did you moved to Apache? > > > Felipe ³Zimmerle² Costa > Security Researcher, SpiderLabs > > Trustwave | SMART SECURITY ON DEMAND > www.trustwave.com <http://www.trustwave.com/> > > > > > > > On 3/25/15, 1:48 PM, "Christian Folini" <chr...@ti...> > wrote: > > >Hey Bruno, > > > >I had hoped firing a broadside at your problem might solve it, > >but I see a more selective approach is needed. > > > >Unfortunately, I am not that much acquainted with the > >Stopwatch2 info to tell you what it really represents. > >The audit-log format is underdocumented in my eyes. > > > >What I would do in your case would be to work with the > >output in the DebugLog. If you raise the loglevel to > >a sufficiently high value (5? 9?), the performance of > >every rule executed is reported as it's being executed. > >This should help to nail down the culprit. > > > >Your perf problem is not unheard off, but usually there > >is a solution. > > > >Ahoj, > > > >Christian > > > > > >On Wed, Mar 25, 2015 at 12:35:49PM -0400, Bruno de Almeida wrote: > >> Hi Christian, > >> > >> Thanks for the link, but it looks like all the information generated > >>from > >> that tutorial is available in the Stopwatch2 field. > >> > >> The problem I am having are with rules that are not triggering, but > >>still > >> being executed. > >> > >> >From what I understand, the information on Stopwatch2 includes the > >>combined > >> performance of ALL the rules that analysed that specific request and not > >> only the triggered rule. I'd like to know the Phase2 times for each > >>rule, > >> so I can tell which ones are causing the times to go up. > >> > >> Basically, I am upgrading my owasp crs rules from a very old version to > >>the > >> latest and I found that the new ones are a LOT slower and apache uses a > >> lot more cpu. > >> > >> Using the example I gave above of one of my custom authentication rule, > >> with the old owasp crs installed, the avg Phase 2 time is 4100usec, with > >> owasp crs 2.2.9, the avg goes up to 12000usec. > >> > >> Hope I am making sense.. > >> > >> > >> Bruno > >> > >> On 25 March 2015 at 00:32, Christian Folini < > >> chr...@ti...> wrote: > >> > >> > Hi Bruno, > >> > > >> > There is a (German) tutorial at > >> > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Q2bjxUfyQ&s=5&u=http%3a%2f%2fwww%2enetnea%2ecom%2fcms%2fapache-tutorial-6 > >>-modsecurity-einbinden%2f > >> > which brings a complete apache/modsec configuration with an > >> > extensive performance log that covers all the phases seperately > >> > and which can be switched on and off based on a SecRule->Env-Variable. > >> > > >> > Even if you do not read German, you should be able to get it working > >> > for your. > >> > > >> > Let me know, if you need any help. > >> > > >> > Ahoj, > >> > > >> > Christian > >> > > >> > > >> > On Tue, Mar 24, 2015 at 06:03:33PM -0400, Bruno de Almeida wrote: > >> > > Hi All, > >> > > > >> > > I am trying to find out the most expensive rules I have on my setup > >>using > >> > > SecRulePerfTime and PERF_RULES but the data is not making much > >>sense. > >> > > > >> > > > >> > > For example, I have this one custom rule that is tracking > >>authentication > >> > > events, it basically inspects POST to a specific URL and grabs some > >>info. > >> > > > >> > > SecRule REQUEST_FILENAME "@strmatch j_spring_security_check" > >> > > > >> > > >>"chain,phase:3,id:'7002',t:none,pass,nolog,auditlog,severity:6,msg:'Succe > >>ssful > >> > > Authentication',logdata:'email=%{args.j_username}'" > >> > > SecRule REQUEST_METHOD "@streq POST" "chain,t:none" > >> > > SecRule RESPONSE_STATUS "@streq 302" "chain,t:none" > >> > > SecRule RESPONSE_HEADERS:Location "!@strmatch signin" "chain,t:none" > >> > > SecRule WEBAPPID "@strmatch www" "chain,t:none" > >> > > SecRule ARGS:j_username ".*" "t:none" > >> > > > >> > > > >> > > The combined time for this rule varies between 10000 and 40000, 99% > >>of > >> > this > >> > > time is in Phase 2, as you can see below > >> > > > >> > > Stopwatch2: 1427233612156631 1027209; combined=30177, p1=420, > >>p2=29315, > >> > > p3=329, p4=51, p5=60, sr=49, sw=2, l=0, gc=0 > >> > > > >> > > > >> > > I have added SecRulePerfTime 1 to my config so I can see all rules > >>taking > >> > > 1+ usec. > >> > > > >> > > The problem is, according to Rules-Performance-Info, the sum of all > >>rules > >> > > taking more than 1 usec is nowhere near p2=29315, and, still > >>acording > >> > > to Rules-Performance-Info, the Authentication Tracking rule is only > >> > taking > >> > > "2 usecs" ("7002=2") > >> > > > >> > > The sum of all the processing times below is *850*, a lot less than > >>the > >> > > combined=30177 > >> > > > >> > > Rules-Performance-Info: "900012=1", "900018=2", "900019=4", > >>"1000=4", > >> > > "900020=1", "900021=2", "5001=2", "5006=1", "5009=1", "5010=1", > >>"4001=1", > >> > > "4002=1", "960911=6", "960016=1", "960012=1", "960342=1", > >>"960032=3", > >> > > "950012=1", "10001=6", " > >> > > 10002=1", "11003=1", "4003=3", "900040=1", "960912=2", "960914=1", > >> > > "960915=1", "958295=1", "950108=4", "950116=1", "960901=15", > >>"960008=1", > >> > > "960006=1", "960017=1", "960209=1", "960208=1", "960335=1", > >>"960341=1", > >> > > "981078=3", "960034=4", "9 > >> > > 60035=1", "960038=36", "990002=7", "990901=7", "990902=1", > >>"990012=7", > >> > > "950907=4", "950018=1", "950019=2", "950910=39", "950911=43", > >>"950117=3", > >> > > "950118=2", "950119=2", "950120=1", "981133=20", "981134=1", > >>"950009=36", > >> > > "950003=3", "950000 > >> > > =3", "950005=4", "950002=5", "950006=6", "981231=3", "981260=5", > >> > > "981318=6", "981319=9", "950901=7", "981320=3", "981300=10", > >>"981303=1", > >> > > "981304=1", "981306=1", "981307=1", "981308=1", "981309=1", > >>"981311=1", > >> > > "981312=1", "981313=1", "981 > >> > > 314=1", "950007=6", "950001=16", "959070=8", "959071=2", "959072=1", > >> > > "950908=2", "959073=23", "981272=2", "981244=8", "981255=7", > >>"981257=6", > >> > > "981248=10", "981277=3", "981250=4", "981241=3", "981252=4", > >>"981256=4", > >> > > "981245=9", "981276=2", > >> > > "981254=2", "981270=1", "981240=6", "981249=7", "981253=4", > >>"981242=11", > >> > > "981246=7", "981251=4", "981247=8", "981243=6", "973336=1", > >>"973337=3", > >> > > "973338=6", "981136=14", "981018=1", "973300=3", "973301=1", > >>"973302=29", > >> > > "973303=5", "97330 > >> > > 4=4", "973305=2", "973306=3", "973307=2", "973308=2", "973309=3", > >> > > "973310=1", "973311=3", "973312=2", "973313=3", "973314=3", > >>"973331=2", > >> > > "973315=1", "973330=4", "973327=4", "973326=5", "973346=8", > >>"973345=4", > >> > > "973324=2", "973323=3", "973 > >> > > 322=2", "973348=2", "973321=3", "973320=1", "973318=1", "973317=2", > >> > > "973347=2", "973335=1", "973334=3", "973333=3", "973332=3", > >>"973329=1", > >> > > "973328=1", "973316=1", "973325=1", "973319=2", "950103=22", > >>"950110=1", > >> > > "981020=1", "981022=2", " > >> > > 981175=1", "1001=1", "2010=1", "2011=1", "3001=12", "9001=1", > >>"200003=1", > >> > > "200004=1", "7002=2", "7003=1", "7004=1", "7006=2", "7009=1", > >>"8002=2", > >> > > "8003=1", "1234=66", "981080=3", "970118=1", "981177=1", "981004=1", > >> > > "981007=1", "981200=2", > >> > > "981201=2", "981204=3", "981205=13", "7001=1". > >> > > > >> > > Does anyone know if Rules-Performance-Info takes into account all > >>phases? > >> > > If not, which phase is it reporting? > >> > > > >> > > Thanks, > >> > > > >> > > > >> > > -- > >> > > - Bruno > >> > > >> > > > >> > > >>------------------------------------------------------------------------- > >>----- > >> > > Dive into the World of Parallel Programming The Go Parallel Website, > >> > sponsored > >> > > by Intel and developed in partnership with Slashdot Media, is your > >>hub > >> > for all > >> > > things parallel software development, from weekly thought leadership > >> > blogs to > >> > > news, videos, case studies, tutorials and more. Take a look and > >>join the > >> > > conversation now. > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>VXL30ZBmg&s=5&u=http%3a%2f%2fgoparallel%2esourceforge%2enet%2f > >> > > >> > > _______________________________________________ > >> > > mod-security-users mailing list > >> > > mod...@li... > >> > > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>QnJiRIdyA&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinf > >>o%2fmod-security-users > >> > > Commercial ModSecurity Rules and Support from Trustwave's > >>SpiderLabs: > >> > > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Vub3UtJzA&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommerc > >>ial%2frules%2f > >> > > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Vqfj0cdmQ&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommerc > >>ial%2fsupport%2f > >> > > >> > > >> > > >> > > >>------------------------------------------------------------------------- > >>----- > >> > Dive into the World of Parallel Programming The Go Parallel Website, > >> > sponsored > >> > by Intel and developed in partnership with Slashdot Media, is your > >>hub for > >> > all > >> > things parallel software development, from weekly thought leadership > >>blogs > >> > to > >> > news, videos, case studies, tutorials and more. Take a look and join > >>the > >> > conversation now. > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>VXL30ZBmg&s=5&u=http%3a%2f%2fgoparallel%2esourceforge%2enet%2f > >> > _______________________________________________ > >> > mod-security-users mailing list > >> > mod...@li... > >> > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>QnJiRIdyA&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinf > >>o%2fmod-security-users > >> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > >> > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Vub3UtJzA&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommerc > >>ial%2frules%2f > >> > > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Vqfj0cdmQ&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommerc > >>ial%2fsupport%2f > >> > > >> > >> > >> > >> -- > >> - Bruno > > > >> > >>------------------------------------------------------------------------- > >>----- > >> Dive into the World of Parallel Programming The Go Parallel Website, > >>sponsored > >> by Intel and developed in partnership with Slashdot Media, is your hub > >>for all > >> things parallel software development, from weekly thought leadership > >>blogs to > >> news, videos, case studies, tutorials and more. Take a look and join > >>the > >> conversation now. > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>VXL30ZBmg&s=5&u=http%3a%2f%2fgoparallel%2esourceforge%2enet%2f > > > >> _______________________________________________ > >> mod-security-users mailing list > >> mod...@li... > >> > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>QnJiRIdyA&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinf > >>o%2fmod-security-users > >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > >> > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Vub3UtJzA&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommerc > >>ial%2frules%2f > >> > >> > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7 > >>Vqfj0cdmQ&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommerc > >>ial%2fsupport%2f > > > > > >-------------------------------------------------------------------------- > >---- > >Dive into the World of Parallel Programming The Go Parallel Website, > >sponsored > >by Intel and developed in partnership with Slashdot Media, is your hub > >for all > >things parallel software development, from weekly thought leadership > >blogs to > >news, videos, case studies, tutorials and more. Take a look and join the > >conversation now. > > > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7V > >XL30ZBmg&s=5&u=http%3a%2f%2fgoparallel%2esourceforge%2enet%2f > >_______________________________________________ > >mod-security-users mailing list > >mod...@li... > > > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7Q > >nJiRIdyA&s=5&u=https%3a%2f%2flists%2esourceforge%2enet%2flists%2flistinfo% > >2fmod-security-users > >Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > > > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7V > >ub3UtJzA&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercia > >l%2frules%2f > > > http://scanmail.trustwave.com/?c=4062&d=vOeS1Z1rnm8cYx9uRXjjrHn2EQREOKjs7V > >qfj0cdmQ&s=5&u=http%3a%2f%2fwww%2emodsecurity%2eorg%2fprojects%2fcommercia > >l%2fsupport%2f > > > ________________________________ > > This transmission may contain information that is privileged, > confidential, and/or exempt from disclosure under applicable law. If you > are not the intended recipient, you are hereby notified that any > disclosure, copying, distribution, or use of the information contained > herein (including any reliance thereon) is strictly prohibited. If you > received this transmission in error, please immediately contact the sender > and destroy the material in its entirety, whether in electronic or hard > copy format. > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for > all > things parallel software development, from weekly thought leadership blogs > to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > -- - Bruno |
