mod-security-developers

[Mod-security-developers] mod_security test suite

[Kurt S. <kse...@re...>]

Stupid question but is there a simple and good test suite for mod_secuirty,
e.g. I've whipped up a few tiny php scripts with wget to trigger them to do
things like echo back GET/POST/JSON so I can confirm if @detectXSS blocking
is behaving in general or not.


--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: sec...@re...

Re: [Mod-security-developers] mod_security test suite

[Felipe C. <FC...@tr...>]

Hi Kurt,

At this point that particular question have too many answers :) There are
few ways to test if an operator in behaving in the way you expect. The
methods are described bellow.

1) Unit tests.

The unit test suite, contains tests for specific operators, as you can see
here:
https://github.com/SpiderLabs/ModSecurity/tree/master/tests/op

If you have compiled ModSecurity by your own, you should have a binary
inside the test directory that you can run to execute the tests described
in the ³.t" files.

This method does not involve execute the web server, rather it tests the
operator hooking the method that will be used by Apache, so it produce
very fast results.


2) Regression tests.

If you want to test it inside Apache (or nginx) you can write a regression
test, as demonstrated here:
https://github.com/SpiderLabs/ModSecurity/blob/master/tests/regression/misc
/00-multipart-parser.t


There is a perl script that you can execute to run those regression tests.
Those start an instance of the web server and analyze the output of the
*logs* to considere a test valid or not.

The regression tests are slower than the unit tests, but necessary if you
want to play with the variables.


3) Using the libmodsecurity

For the libmodsecurity [1] we also have the ³unit tests² and
³regression tests², the big difference is that the regression tests does
not demand a server to be started, so it produce faster results. The
request can be mimic inside a JSON:

https://github.com/SpiderLabs/ModSecurity/blob/libmodsecurity/test/test-cas
es/regression/variable-REQUEST_BODY.json


I have my own dirty script to convert from Wireshark to this JSON format
(will be published soon, as soon as I polish it a little bit)


4) Using libinjection test suite

For this particular operator you can use the libinjection test suite:

https://github.com/client9/libinjection/tree/master/tests


Not sure if Nick Galbreath is on the list, if so, he can give you more
Details about it.




[1] https://github.com/SpiderLabs/ModSecurity/tree/libmodsecurity




Br,
Felipe ³Zimmerle² Costa
Security Researcher, SpiderLabs

Trustwave | SMART SECURITY ON DEMAND
www.trustwave.com <http://www.trustwave.com/>





From:  Kurt Seifried <kse...@re...>
Reply-To:  "mod...@li..."
<mod...@li...>
Date:  Wednesday, September 30, 2015 at 11:49 PM
To:  "mod...@li..."
<mod...@li...>
Subject:  [Mod-security-developers] mod_security test suite


Stupid question but is there a simple and good test suite for
mod_secuirty, e.g. I've whipped up a few tiny php scripts with wget to
trigger them to do things like echo back GET/POST/JSON so I can confirm if
@detectXSS blocking is behaving in
 general or not.


--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: sec...@re...


________________________________

This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.

Re: [Mod-security-developers] mod_security test suite

[Kurt S. <kse...@re...>]

On Thu, Oct 1, 2015 at 8:11 AM, Felipe Costa <FC...@tr...> wrote:

> Hi Kurt,
>
> At this point that particular question have too many answers :) There are
> few ways to test if an operator in behaving in the way you expect. The
> methods are described bellow.
>


Sorry I should have been more explicit, my concern is integration testing
more, e.g. we have a layered product sitting on top of Red Hat Enterprise
Linux (e.g. Satellite Server, CloudForms, OpenStack, whatever), if I put
mod_security in front of it I need to test that the whole stack works
together, obviously one aspect of that is "Real world" testing, but in
order to avoid false negatives (e.g. a test passes, but maybe mod_security
didn't block it properly and some nasty data managed to get through to the
backend) I need some simple ways to ensure I can test for false negatives
and false positives (e.g. a highly simplistic app that echoes what it got
directly back so there is no doubt if mod_security blocked it or not). In
theory this kind of stuff shouldn't happen with proper testing methodology,
but my job is not to plan for the happy scenario, it's to plan for the
failure scenario =). Belt AND suspenders and all that.

--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: sec...@re...


________________________________

This transmission may contain information that is privileged, confidential,
and/or exempt from disclosure under applicable law. If you are not the
intended recipient, you are hereby notified that any disclosure, copying,
distribution, or use of the information contained herein (including any
reliance thereon) is strictly prohibited. If you received this transmission
in error, please immediately contact the sender and destroy the material in
its entirety, whether in electronic or hard copy format.

------------------------------------------------------------------------------
_______________________________________________
mod-security-developers mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-developers
ModSecurity Services from Trustwave's SpiderLabs:
https://www.trustwave.com/spiderLabs.php



-- 

--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: sec...@re...

Re: [Mod-security-developers] mod_security test suite

[Christian F. <chr...@ti...>]

Kurt,

On Thu, Oct 01, 2015 at 08:52:48AM -0600, Kurt Seifried wrote:
> Sorry I should have been more explicit, my concern is integration testing
> more, e.g. we have a layered product sitting on top of Red Hat Enterprise
> Linux (e.g. Satellite Server, CloudForms, OpenStack, whatever), if I put
> mod_security in front of it I need to test that the whole stack works
> together, obviously one aspect of that is "Real world" testing, but in
> order to avoid false negatives (e.g. a test passes, but maybe mod_security
> didn't block it properly and some nasty data managed to get through to the
> backend) I need some simple ways to ensure I can test for false negatives
> and false positives

Through the years I used various approaches to achieve this. In the
end I returned to a self-written testsuite that uses ruby unit-testing
to fire requests at Apache/ModSec and expects certain responses.

I use this to test anything from simple requests to multi-step logins 
(expecting a session cookie at the end of the process); legitimate
requests and expected blockings.

Whenever, I write custom rules for the service I add more test
requests. Those that should pass, and those that should be blocked.

When a pen-test finds a weakness and we need to block an exploit in
ModSecurity, the exact vector of the pen-test is added to the
test suite. This makes sure the virtual patch is not disappearing
in a future release of the service.

I am not sure how I would scale this for a larger installation,
but for small standalone services, it works nicely, and I can
give my customers a ring, if their service suddenly reintroduces
old bugs.

Just my 2 cents.

Ahoj,

Christian


-- 
I have always observed that to succeed in the world one should appear
like a fool but be wise.
-- Charles de Montesquieu