Hi Breno,
> BTW... what are u thinking about the SSL stuff ?
As mentioned, I do not know enough about the internals of the SSL protocol. However, I see various ways to
delay a HTTPS request. More and more thoughts and tools are being published. But AFAIK, so far there was nothing
regarding the delaying of an SSL handshake. I mentioned this a few years back on the ModSec mailinglist but
got no response.
So either it is not really possible - or it is just complicated to achieve and thus potentially difficult to defend against.
Maybe Ivan has something to say.
Cheers,
Christian
Von: Breno Silva [mailto:breno.silva@...]
Gesendet: Dienstag, 30. November 2010 21:21
An: Folini Christian, IT222 extern
Cc: mod-security-users@...
Betreff: Re: [mod-security-users] [Owasp-modsecurity-core-rule-set] Advanced Topic of the Week: Mitigating Slow HTTP DoS Attacks
The other msg was to another mail :)
BTW... what are u thinking about the SSL stuff ?
On Tue, Nov 30, 2010 at 12:59 AM, <christian.folini@...>> wrote:
Hi Breno,
Thank you for your extensive efforts and research on real-world behavior of your new feature.
> As you can see we have the two side of the coin... nothing is perfect in the life :-).
> If you don't turn on the SecReadStateLImit the attacker may cause a problem from all the world that is
> trying to contact your server and nobody will be able to connect to your site.In another hand if you enable
> the defense you will lost some connections from the attacker IP address (lets suppose he is behind a NAT for example),
> but the rest of the world still will be able to connect to your site.
That is more or less what I expected from this feature. I suggest you make this clear in the documentation.
As such I believe it is a defense utility which you enable in case of attack, but not in the base config,
since the negative side-effect comes into play immediately, also in absence of any ongoing attack.
> During my tests with slowloris it required hundreds (sometimes thousands) of sockets available on the attacker
> side to have success, which means that a single ip will generate a lot of concurrent connections. So i understand
> that a number of 100 to SecReadStateLImit will be good for many environments, however people may spending
> some time tuning it to their environment.
> Also this feature can be a way to detect the ip address of an attacker, because without this apache will only
> print a lot of request headers error msgs without any source. And as it is a low bandwidth attack if very difficult
> to detect using a network device.
You can get the info from netstat as long as it is DoS. If you face a DDoS attack, then the defense gets
tricky - and SecReadStateLImit won't help you anymore.
> Keep it in your sleeve... maybe it can help you sometime.
Exactly. Thanks for another tool in that case. They are desperately needed.
And from the other message:
> An improvement for it will be to find a way to identify the larger upload connections and do not count it.
I do not understand you here. What do you mean exactly?
A sidenote: Has anybody ever thought of a request delaying / slowloris attack targeting the SSL handshake?
Is that technically feasible - and how would one defend against it? Sorry if this leads away from the ModSec
discussion, but I think it's an intriguing idea and I never saw it discussed anywhere.
Fjodor used to have a blog post online touching on other very evil request delaying techniques. Unfortunately
he has pulled it back. Maybe there is a copy somewhere in the web.
Regs,
Christian
Von: Breno Silva [mailto:breno.silva@...>]
Gesendet: Donnerstag, 25. November 2010 01:39
An: Folini Christian, IT222 extern
Cc: RBarnett@...>; mod-security-users@...>
Betreff: Re: [mod-security-users] [Owasp-modsecurity-core-rule-set] Advanced Topic of the Week: Mitigating Slow HTTP DoS Attacks
Hi Christian,
I spent some time doing some tests to share with you. I setup a webserver 192.168.0.102 and two clients (192.168.0.101 and 192.168.0.100)
- First, let's see what happens when we send some requests. The server is under attack (192.168.0.100) without the defense:
< From the client 192.168.0.101 >
Server Software: Apache
Server Hostname: 192.168.0.102
Server Port: 80
Document Path: /www/discr/disc.zip
Document Length: 696374 bytes
Concurrency Level: 10
Time taken for tests: 405.342 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 696623000 bytes
HTML transferred: 696374000 bytes
Requests per second: 2.47 [#/sec] (mean)
Time per request: 4053.420 [ms] (mean)
Time per request: 405.342 [ms] (mean, across all concurrent requests)
Transfer rate: 1678.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 25 249.8 3 3000
Processing: 69 4025 4196.3 1576 12466
Waiting: 0 3762 4191.1 1340 11985
Total: 71 4050 4232.2 1578 13559
Percentage of the requests served within a certain time (ms)
50% 1578
66% 5268
75% 8896
80% 9902
90% 10716
95% 11127
98% 12028
99% 12193
100% 13559 (longest request)
<From the attacker 192.168.0.100 >
Server Software: Apache
Server Hostname: 192.168.0.102
Server Port: 80
Document Path: /www/disc/disc.zip
Document Length: 696374 bytes
Concurrency Level: 10
Time taken for tests: 438.080 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 696623000 bytes
HTML transferred: 696374000 bytes
Requests per second: 2.28 [#/sec] (mean)
Time per request: 4380.795 [ms] (mean)
Time per request: 438.080 [ms] (mean, across all concurrent requests)
Transfer rate: 1552.90 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 30.2 1 955
Processing: 35 4366 4277.3 2039 12568
Waiting: 2 4023 4289.1 1642 12064
Total: 35 4369 4276.6 2044 12570
Percentage of the requests served within a certain time (ms)
50% 2044
66% 6185
75% 9538
80% 10040
90% 10755
95% 11171
98% 11642
99% 12103
100% 12570 (longest request)
- Now, i set SecReadStateLimit to 100. Let's see the results:
<From the client 192.168.0.101>
Server Software: Apache
Server Hostname: 192.168.0.102
Server Port: 80
Document Path: /www.limao.com.br/discador/discador_limao.zip<http://www.limao.com.br/discador/discador_limao.zip>
Document Length: 696374 bytes
Concurrency Level: 10
Time taken for tests: 56.275 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 696623000 bytes
HTML transferred: 696374000 bytes
Requests per second: 17.77 [#/sec] (mean)
Time per request: 562.745 [ms] (mean)
Time per request: 56.275 [ms] (mean, across all concurrent requests)
Transfer rate: 12088.88 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 31 283.2 3 3005
Processing: 40 521 491.5 386 3822
Waiting: 2 161 388.7 17 2989
Total: 41 552 572.8 391 4122
Percentage of the requests served within a certain time (ms)
50% 391
66% 534
75% 654
80% 756
90% 1142
95% 1582
98% 2413
99% 3467
100% 4122 (longest request)
<From the attacker 192.168.0.100>
Server Software: Apache
Server Hostname: 192.168.0.102
Server Port: 80
Document Path: /www.limao.com.br/discador/discador_limao.zip<http://www.limao.com.br/discador/discador_limao.zip>
Document Length: 696374 bytes
Concurrency Level: 10
Time taken for tests: 13.877 seconds
Complete requests: 1000
Failed requests: 572
(Connect: 0, Receive: 0, Length: 872, Exceptions: 0)
Write errors: 0
Total transferred: 89167744 bytes
HTML transferred: 89135872 bytes
Requests per second: 72.06 [#/sec] (mean)
Time per request: 138.772 [ms] (mean)
Time per request: 13.877 [ms] (mean, across all concurrent requests)
Transfer rate: 6274.90 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 28.8 1 910
Processing: 1 136 379.7 3 2404
Waiting: 0 28 188.1 0 2205
Total: 1 139 384.3 5 2433
Percentage of the requests served within a certain time (ms)
50% 5
66% 7
75% 22
80% 75
90% 451
95% 1061
98% 1672
99% 1899
100% 2433 (longest request)
As you can see we have the two side of the coin... nothing is perfect in the life :-). If you don't turn on the SecReadStateLImit the attacker may cause a problem from all the world that is trying to contact your server and nobody will be able to connect to your site.In another hand if you enable the defense you will lost some connections from the attacker IP address (lets suppose he is behind a NAT for example), but the rest of the world still will be able to connect to your site.
The trick is to set a good value to SecReadStateLimit which is the number of threads that can remain in READ_BUSY_STATE (before request_rec is valid for apache) per ip address. I suggest for almost all environments to understand it as the number of concurrent connections per IP address.
During my tests with slowloris it required hundreds (sometimes thousands) of sockets available on the attacker side to have success, which means that a single ip will generate a lot of concurrent connections. So i understand that a number of 100 to SecReadStateLImit will be good for many environments, however people may spending some time tuning it to their environment.
Also this feature can be a way to detect the ip address of an attacker, because without this apache will only print a lot of request headers error msgs without any source. And as it is a low bandwidth attack if very difficult to detect using a network device.
So... want to say that it is an option. Not a mandatory feature... and have pros and cons... in my point of view as it can save some heads in a company ... there are more pros :-)
Keep it in your sleeve... maybe it can help you sometime.
Also if you have an high speed environment and want to try and share with us good values to SecReadStateLimit
Cheers and thanks for the discussion
Breno
On Wed, Nov 24, 2010 at 9:22 AM, Breno Silva <breno.silva@...>> wrote:
If you are using the apache-prefork and receive this kind of attack ... it can kill all your operation system forking too many process. So a good reason to enable this feature :)
Note that this is not a definitive solution, but can help you a lot during this kind of attack saving a big part of legit connections.
Thanks
Breno
On Wed, Nov 24, 2010 at 7:47 AM, Breno Silva <breno.silva@...>> wrote:
Christian,
Yes and No :)
Keep it mind that with small number of threads can handle many requests ... so you can also have a large number of long (BUSY_STATE) legit connections with low threads per ipaddress.
However you u have behind your proxy a lot of user doing a LOT of big uploads to a specific server... and the SecReadStateLimit is very small ... it can reject the connection.
On Wed, Nov 24, 2010 at 7:22 AM, <christian.folini@...>> wrote:
Thanks Breno,
I think I got it.
So clients will be save for GET requests, but for long POST requests the mileage may vary.
Best,
Christian
Von: Breno Silva [mailto:breno.silva@...>]
Gesendet: Mittwoch, 24. November 2010 14:18
An: Folini Christian, IT222 extern
Cc: RBarnett@...>; mod-security-users@...>
Betreff: Re: [mod-security-users] [Owasp-modsecurity-core-rule-set] Advanced Topic of the Week: Mitigating Slow HTTP DoS Attacks
This an example
I sent an attack from my PC to my apache server in another machine. And without the protection i sent in the same time using ab ligit connections:
Concurrency Level: 100
Time taken for tests: 32.842 seconds
Complete requests: 500
As you can see it spent too much time to answer my connections
Now i did the same test, but turn on the defense:
Concurrency Level: 100
Time taken for tests: 0.122 seconds
Complete requests: 500
As you can see the legit connections was accepted normally during the attack from the same ip address.
Breno
On Wed, Nov 24, 2010 at 7:10 AM, Breno Silva <breno.silva@...>> wrote:
Hi Christian,
Internally into the modsec you are checking how many threads are in SERVER_BUSY_STATE and limiting the number of this threads per ip address using the SecReadStateLimit. The large part of the proxied connections will leave this state very quickly and the connection will be not counted to be rejected. So a few number of threads should be enough to handle your legit SERVER_BUSY_STATE connections.
Thanks
Breno
On Wed, Nov 24, 2010 at 6:31 AM, <christian.folini@...>> wrote:
Hi Breno,
Could explain the term "bad connection" a bit? Ryan's blog post implies a client IP is considered bad
when it has too many connections in read state. Your entry in the CHANGES document reads,
"Add SecReadStateLimit to limit the number of BUSY connections".
I can't see why a proxy can't have a lot of legitimate connections
in read state. AFAIK Request Body reading is also considered "read".
So uploads can remain in READ for a certain time - depending on service.
I do not want to pester you too much, but I just want to make sure I
get this correctly - and people are aware that telling good from bad
connections is very tricky. Especially when it comes to request delaying and
you want to make sure you are not locking legitimate users.
Best Regs,
Christian
Von: Breno Silva [mailto:breno.silva@...>]
Gesendet: Mittwoch, 24. November 2010 13:20
An: Folini Christian, IT222 extern
Cc: RBarnett@...>; mod-security-users@...>; owasp-modsecurity-core-rule-set@...>
Betreff: Re: [mod-security-users] [Owasp-modsecurity-core-rule-set] Advanced Topic of the Week: Mitigating Slow HTTP DoS Attacks
Hi Christian,
The SecReadStateLimit is not only a threshold for ip address. It is looking for an "anomaly" in connection process. So if you are behind a proxy or a NAT only the bad connections will be dropped. The good ones will pass normally. So legit connections behind the proxy will works fine.
Thanks
Breno
On Wed, Nov 24, 2010 at 1:17 AM, <christian.folini@...>> wrote:
Hi Ryan,
Nice post. Thanks. Especially the combination of mod_reqtimeout and ModS
is very elegant in my eyes.
I am not so happy with SecReadStateLimit looking only at the IP address.
How do protect proxies from your countermeasures? A proxy might share multiple
hundred legitimate connections with your server for multiple hundred legitimate
clients, all appearing to come from the same IP address.
Regs,
Christian
-----Ursprüngliche Nachricht-----
Von: owasp-modsecurity-core-rule-set-bounces@...> [mailto:owasp-modsecurity-core-rule-set-bounces@...>] Im Auftrag von Ryan Barnett
Gesendet: Mittwoch, 24. November 2010 02:45
An: mod-security-users@...>; owasp-modsecurity-core-rule-set@...>
Betreff: [Owasp-modsecurity-core-rule-set] Advanced Topic of the Week: Mitigating Slow HTTP DoS Attacks
This week's blog post -
http://blog.spiderlabs.com/2010/11/advanced-topic-of-the-week-mitigating-slow-http-dos-attacks.html
--
Ryan Barnett
Senior Security Researcher
Trustwave - SpiderLabs
_______________________________________________
Owasp-modsecurity-core-rule-set mailing list
Owasp-modsecurity-core-rule-set@...>
https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
mod-security-users mailing list
mod-security-users@...>
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Appliances, Rule Sets and Support:
http://www.modsecurity.org/breach/index.html
|