Thread: [Mod-security-developers] Advanced Slow DoS Mitigation
Brought to you by:
victorhora,
zimmerletw
From: Christian F. <chr...@ti...> - 2011-07-05 18:42:30
|
Hi there, ModSecurity always had a few nice options to help with request delaying mitigation. The combination with mod_reqtimeout is a good strategy as explained by Ryan at http://blog.spiderlabs.com/2010/11/advanced-topic-of-the-week-mitigating-slow-http-dos-attacks.html In May, I did a presentation on the defense against Request Delaying DDoS aka as Slow DoS or Slowloris type attacks. That Swiss Cyberstorm talk is now online at http://www.youtube.com/watch?v=svN49PIbcks Around 32:30, I mention some advanced ideas on how to identify attackers very easily. As you know, a POST request to /index.html is perfectly okay with Apache. You can prevent it with ModSecurity, but not immediately when the server receives the requestline. Ideally, you should be able to drop a request trying this immediately. I would like to write rules that trigger as soon as the request line has been received. A phase 0 somehow. This would also be handy to drop requests that try to upload files before the user has been authenticated. Apache does not mind large uploads from unauthenticated users until it has received the whole blob. Not even mod_reqtimeout is of big help if you need to allow big file uploads. Now I doubt that Apache allows for a phase 0 (I am not an apache developer and as you know not even a ModSecurity developer) as there seems to be no hook at that moment and if I get it right, the whole request record is not being prepared until post-read-request. But maybe I am wrong. So what do you guys think? Cheers, Christian -- I think IT projects are about supporting social systems - about communications between people and machines. They tend to fail due to cultural issues. -- Tim Berners-Lee |
From: Ryan B. <RBa...@tr...> - 2011-07-06 12:43:03
|
On 7/5/11 2:42 PM, "Christian Folini" <chr...@ti...> wrote: >Hi there, Hey Christian! Application (D)DoS detection and mitigation is a very challenging and important topic and merits more discussion so thank you for the email. > >ModSecurity always had a few nice options to help with request delaying >mitigation. The combination with mod_reqtimeout is a good strategy as >explained by Ryan at >http://blog.spiderlabs.com/2010/11/advanced-topic-of-the-week-mitigating-s >low-http-dos-attacks.html Did you see that Breno recently added SecWriteStateLimit as well to help mitigate Slow POST Attacks? http://sourceforge.net/apps/mediawiki/mod-security/index.php?title=Referenc e_Manual#SecWriteStateLimit > >In May, I did a presentation on the defense against Request Delaying DDoS >aka as Slow DoS or Slowloris type attacks. That Swiss Cyberstorm talk is >now online at http://www.youtube.com/watch?v=svN49PIbcks Great preso and really highlights the threat. I was wondering what percentage of WikiLeaks DoS attacks were utilizing Slowloris-type techniques. > >Around 32:30, I mention some advanced ideas on how to identify attackers >very easily. > >As you know, a POST request to /index.html is perfectly okay with >Apache. You can prevent it with ModSecurity, but not immediately >when the server receives the requestline. > >Ideally, you should be able to drop a request trying this immediately. >I would like to write rules that trigger as soon as the request line >has been received. A phase 0 somehow. We are currently having a discussion about ModSecurity's phases in this Jira ticket - https://www.modsecurity.org/tracker/browse/MODSEC-98 Specifically, phase:1 was moved by Ivan awhile ago to be the same as phase:2 (instead of Apache post-read-request) due to many users wanting to use phase:1 rules inside Apache scope directives like <Location>. I personally do not agree with this change and we are reviewing a potential change back. Regardless - I believe that we should consider a "phase:0" option that would essentially work at the Apache Filter level hook. So, this would not be parsed like the other variables but could give basic access to src IP data and the entire request payload as perhaps a new variable - THE_REQUEST. > >This would also be handy to drop requests that try to upload files >before the user has been authenticated. Apache does not mind large >uploads from unauthenticated users until it has received the whole >blob. Not even mod_reqtimeout is of big help if you need to allow >big file uploads. > >Now I doubt that Apache allows for a phase 0 (I am not an apache >developer and >as you know not even a ModSecurity developer) as there seems to be no >hook at that moment and if I get it right, the whole request record >is not being prepared until post-read-request. But maybe I am wrong. The main issue that I see with a Filter level hook is that mod_uniqueid is not yet available and that is used by ModSecurity for proper logging. I will let Breno comment on some ideas. Cheers, Ryan > >So what do you guys think? > >Cheers, > >Christian > > >-- >I think IT projects are about supporting social systems - about >communications between people and machines. They tend to fail due to >cultural issues. >-- Tim Berners-Lee > > >-------------------------------------------------------------------------- >---- >All of the data generated in your IT infrastructure is seriously valuable. >Why? It contains a definitive record of application performance, security >threats, fraudulent activity, and more. Splunk takes this data and makes >sense of it. IT sense. And common sense. >http://p.sf.net/sfu/splunk-d2d-c2 >_______________________________________________ >mod-security-developers mailing list >mod...@li... >https://lists.sourceforge.net/lists/listinfo/mod-security-developers >ModSecurity Services from Trustwave's SpiderLabs: >https://www.trustwave.com/spiderLabs.php > This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. |
From: Christian F. <chr...@ti...> - 2011-07-07 05:34:12
|
Hi there, On Wed, Jul 06, 2011 at 07:42:50AM -0500, Ryan Barnett wrote: > Great preso and really highlights the threat. I was wondering what > percentage of WikiLeaks DoS attacks were utilizing Slowloris-type > techniques. Me2. ;) To be honest there was too much noise to do any sort of measurements. We have seen a lot of things, also a lot of vanilla slowloris, but we also must have missed a lot of other interesting attacks. > Specifically, phase:1 was moved by Ivan awhile ago to be the same as > phase:2 (instead of Apache post-read-request) due to many users wanting to > use phase:1 rules inside Apache scope directives like <Location>. I > personally do not agree with this change and we are reviewing a potential > change back. I bumped into the old phase:1 / Location issue before so I understand the motivation. But I thought it would have been the better countermeasure to have apache refuse to start with a phase:1 rule inside a location. > Regardless - I believe that we should consider a "phase:0" option that > would essentially work at the Apache Filter level hook. So, this would > not be parsed like the other variables but could give basic access to src > IP data and the entire request payload as perhaps a new variable - > THE_REQUEST. That sounds nice. > The main issue that I see with a Filter level hook is that mod_uniqueid is > not yet available and that is used by ModSecurity for proper logging. That does not sound very nice, though. Was not there a discussion on the Apache ML to hand over mod_uniqueid (functionality) to ModSecurity? I think that would be wrong, but maybe it is possible to introduce a patch to have mod_uniqueid run at this early hook too (and before phase:0). Cheers, Christian -- If we could read the secret history of our enemies, we should find in each man's life sorrow and suffering enough to disarm all hostility. -- Henry Wadsworth Longfellow |
From: Oleg G. <ole...@ya...> - 2012-08-18 04:27:36
|
Hello, Has anyone had a chance to compare efficiency of mod_sec vs. QoS when it comes to mitigating against slow HTTP attacks? I've heard different opinions, e.g. that mod_sec might work well for slow reads, but not writes, while QoS is good for both. Are there any research papers or just objective data that can confirm that or prove opposite? The other question is related to performance impact in both solutions when it comes to high volume systems. Any pointers are highly appreciated. --- On Thu, 7/7/11, Christian Folini <chr...@ti...> wrote: From: Christian Folini <chr...@ti...> Subject: Re: [Mod-security-developers] Advanced Slow DoS Mitigation To: mod...@li... Date: Thursday, July 7, 2011, 1:34 AM Hi there, On Wed, Jul 06, 2011 at 07:42:50AM -0500, Ryan Barnett wrote: > Great preso and really highlights the threat. I was wondering what > percentage of WikiLeaks DoS attacks were utilizing Slowloris-type > techniques. Me2. ;) To be honest there was too much noise to do any sort of measurements. We have seen a lot of things, also a lot of vanilla slowloris, but we also must have missed a lot of other interesting attacks. > Specifically, phase:1 was moved by Ivan awhile ago to be the same as > phase:2 (instead of Apache post-read-request) due to many users wanting to > use phase:1 rules inside Apache scope directives like <Location>. I > personally do not agree with this change and we are reviewing a potential > change back. I bumped into the old phase:1 / Location issue before so I understand the motivation. But I thought it would have been the better countermeasure to have apache refuse to start with a phase:1 rule inside a location. > Regardless - I believe that we should consider a "phase:0" option that > would essentially work at the Apache Filter level hook. So, this would > not be parsed like the other variables but could give basic access to src > IP data and the entire request payload as perhaps a new variable - > THE_REQUEST. That sounds nice. > The main issue that I see with a Filter level hook is that mod_uniqueid is > not yet available and that is used by ModSecurity for proper logging. That does not sound very nice, though. Was not there a discussion on the Apache ML to hand over mod_uniqueid (functionality) to ModSecurity? I think that would be wrong, but maybe it is possible to introduce a patch to have mod_uniqueid run at this early hook too (and before phase:0). Cheers, Christian -- If we could read the secret history of our enemies, we should find in each man's life sorrow and suffering enough to disarm all hostility. -- Henry Wadsworth Longfellow ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 _______________________________________________ mod-security-developers mailing list mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-security-developers ModSecurity Services from Trustwave's SpiderLabs: https://www.trustwave.com/spiderLabs.php |
From: Christian F. <chr...@ti...> - 2011-07-07 05:25:01
|
Hi Ryan, Thank you for your extensive comments. I agree with almost all. Let me just quickly say a few words about SecWriteStateLimit. On Wed, Jul 06, 2011 at 07:42:50AM -0500, Ryan Barnett wrote: > Did you see that Breno recently added SecWriteStateLimit as well to help > mitigate Slow POST Attacks? > http://sourceforge.net/apps/mediawiki/mod-security/index.php?title=Referenc > e_Manual#SecWriteStateLimit I have seen it immediately when it came out and it is a must-have feature. But it is limited to single IP attackers and I am not really afraid of those. Otherwise SecWriteStateLimit interferes with HTTP Proxies. My real world experience tells me that a legitimate Proxy can easily have 50 active connections to my server. Not all of those will be in SERVER_BUSY_WRITE but somehow SecWriteStateLimit treats all connections equally and I would need some way to tweak with that. Mod_qos has a notion of VIP connections (via a list of predefined IP ranges). I do not really think that this mechanism is very elegant, but whatever you do with DDoS defense, it gets hairy very fast. SecWriteStateLimit is elegant, but very limited. Best, Christian -- It is not power that corrupts but fear. Fear of losing power corrupts those who wield it and fear of the scourge of power corrupts those who are subject to it. -- Aung San Suu Kyi |
From: Breno S. <bre...@gm...> - 2011-07-07 14:31:52
|
Hi Christian, Did you try to SecWriteStateLimit ? I think we can use a value like 150-250 and detect the attacks and maybe you will not see FPs. When you say "active connections" if i understand well the term you are using ... it is a established connections right ? But it is not necessary a simultaneous SERVER_BUSY threads. So don't think in SecWriteStateLimit as a counter for connections... but for simultaneous threads in that state. Also you can have active 200 threads .. but a few in SERVER_BUSY state. I recommend you test (if you didn't ) it with the range of value i said here. Thanks Breno On Thu, Jul 7, 2011 at 12:24 AM, Christian Folini < chr...@ti...> wrote: > Hi Ryan, > > Thank you for your extensive comments. I agree with almost all. > Let me just quickly say a few words about SecWriteStateLimit. > > On Wed, Jul 06, 2011 at 07:42:50AM -0500, Ryan Barnett wrote: > > Did you see that Breno recently added SecWriteStateLimit as well to help > > mitigate Slow POST Attacks? > > > http://sourceforge.net/apps/mediawiki/mod-security/index.php?title=Referenc > > e_Manual#SecWriteStateLimit > > I have seen it immediately when it came out and it is a must-have > feature. But it is limited to single IP attackers and I am > not really afraid of those. > > Otherwise SecWriteStateLimit interferes with HTTP Proxies. My > real world experience tells me that a legitimate Proxy can easily > have 50 active connections to my server. Not all of those > will be in SERVER_BUSY_WRITE but somehow SecWriteStateLimit > treats all connections equally and I would need some way to > tweak with that. Mod_qos has a notion of VIP connections > (via a list of predefined IP ranges). I do not really > think that this mechanism is very elegant, but whatever > you do with DDoS defense, it gets hairy very fast. > SecWriteStateLimit is elegant, but very limited. > > Best, > > Christian > > > -- > It is not power that corrupts but fear. Fear of losing power corrupts > those who wield it and fear of the scourge of power corrupts those who > are subject to it. > -- Aung San Suu Kyi > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > _______________________________________________ > mod-security-developers mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-developers > ModSecurity Services from Trustwave's SpiderLabs: > https://www.trustwave.com/spiderLabs.php > |
From: Breno S. <bre...@gm...> - 2011-07-07 15:17:09
|
FYI. I'm adding a small check in SecWriteStateLimit to only check for POST connections (2.6.1-stable) thanks Breno On Thu, Jul 7, 2011 at 9:31 AM, Breno Silva <bre...@gm...> wrote: > Hi Christian, > > Did you try to SecWriteStateLimit ? I think we can use a value like 150-250 > and detect the attacks and maybe you will not see FPs. > > When you say "active connections" if i understand well the term you are > using ... it is a established connections right ? But it is not necessary a > simultaneous SERVER_BUSY threads. > > So don't think in SecWriteStateLimit as a counter for connections... but > for simultaneous threads in that state. Also you can have active 200 threads > .. but a few in SERVER_BUSY state. > > I recommend you test (if you didn't ) it with the range of value i said > here. > > Thanks > > Breno > > > On Thu, Jul 7, 2011 at 12:24 AM, Christian Folini < > chr...@ti...> wrote: > >> Hi Ryan, >> >> Thank you for your extensive comments. I agree with almost all. >> Let me just quickly say a few words about SecWriteStateLimit. >> >> On Wed, Jul 06, 2011 at 07:42:50AM -0500, Ryan Barnett wrote: >> > Did you see that Breno recently added SecWriteStateLimit as well to help >> > mitigate Slow POST Attacks? >> > >> http://sourceforge.net/apps/mediawiki/mod-security/index.php?title=Referenc >> > e_Manual#SecWriteStateLimit >> >> I have seen it immediately when it came out and it is a must-have >> feature. But it is limited to single IP attackers and I am >> not really afraid of those. >> >> Otherwise SecWriteStateLimit interferes with HTTP Proxies. My >> real world experience tells me that a legitimate Proxy can easily >> have 50 active connections to my server. Not all of those >> will be in SERVER_BUSY_WRITE but somehow SecWriteStateLimit >> treats all connections equally and I would need some way to >> tweak with that. Mod_qos has a notion of VIP connections >> (via a list of predefined IP ranges). I do not really >> think that this mechanism is very elegant, but whatever >> you do with DDoS defense, it gets hairy very fast. >> SecWriteStateLimit is elegant, but very limited. >> >> Best, >> >> Christian >> >> >> -- >> It is not power that corrupts but fear. Fear of losing power corrupts >> those who wield it and fear of the scourge of power corrupts those who >> are subject to it. >> -- Aung San Suu Kyi >> >> >> ------------------------------------------------------------------------------ >> All of the data generated in your IT infrastructure is seriously valuable. >> Why? It contains a definitive record of application performance, security >> threats, fraudulent activity, and more. Splunk takes this data and makes >> sense of it. IT sense. And common sense. >> http://p.sf.net/sfu/splunk-d2d-c2 >> _______________________________________________ >> mod-security-developers mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers >> ModSecurity Services from Trustwave's SpiderLabs: >> https://www.trustwave.com/spiderLabs.php >> > > |
From: Christian F. <chr...@ti...> - 2011-07-08 12:15:02
|
Hi Breno, SecWriteStateLimit was like one week to late for real world use when we came under attack by anonymous. It would have helped a big deal and I think it is a very good defense mechanism against generic DoS attack scripts. If I am able to set the limit as high as 150-250 then I am sure I will be free of collateral damage (at least in our case). The DDoS case asks for different tool though. Especially on servers which accept big file uploads. On Thu, Jul 07, 2011 at 09:31:46AM -0500, Breno Silva wrote: > When you say "active connections" if i understand well the term you are > using ... it is a established connections right ? But it is not necessary a > simultaneous SERVER_BUSY threads. Yes, I meant "established" on the tcp level. Some of them can be in SERVER_BUSY in Apache. (Sorry for being inexact in the previous message). And from the other message: > FYI. I'm adding a small check in SecWriteStateLimit to only check for POST > connections (2.6.1-stable) How about the other methods? Don't a few of the less frequently used methods like PUT enter the SERVER_BUSY state? Best, Christian -- Everyone is a prisoner of his own experiences. No one can eliminate prejudices - just recognize them. --- Edward R. Murrow |