Re: [mod-security-users] Possible to remove rules by multiple tags?

[Jamie B. <ja...@ib...>]

Thanks for your replies and time Christian, I appreciate it.

I'm very new to the world of ModSecurity and CRS, so at this stage I have more questions than answers unfortunately! I've realised that actually this is extremely complicated and that the more you look at it, the more nuanced it is and I'm sporting a semi-permanent expression which is somewhere between wonderment and confusion.

I initially just wanted to create a custom rate limiting rule to protect a few endpoints and also enable a specific PL2 rule for blocking scripting agents. I have come at this a bit backwards probably. Anyway, these rules have worked well, so I've been looking at enabling more of the CRS and reading through the documentation and also your tutorials on dealing with and filtering out false positives.

You are spot on about not wanting to see things that you do. I enabled the rules to detect, for example, requests without an Accept header and without a User agent and these have now filled up the log, so it's almost impossible to "see past them". I want to log them, I want them to count towards the anomaly score, but the log is now unmanageable with them in place. For now I removed those rules. I started looking for some sort of GUI that could somehow sift through and summarise the entries that could then be drilled down in to, but have come up short, at least using the basic serial log. I have seen some of your posts where you summarise on the command line, these are helpful.

The web server I'm experimenting on actually houses a few completely different virtual hosts, some web apps that are written by me and some third-party, so the setup is complex and of course now I'm faced with a huge log of false positives (and some true) from each different app, particularly in respect of XSS and SQLi against the admin area/back-end forms. And there are just too many places this sort of thing can crop up. For now, I started turning off the rule engine completely for the back-ends, which are IP protected, but that's almost certainly not the best way to handle things.

Cheers
Jamie

> On 17 Jun 2020, at 09:50, Christian Folini <chr...@ne...> wrote:
> 
> You have a good way of asking the relevant questions Jamie.
> 
> No, there is no established best practice for this very problem. I've been
> thinking about it quite some time, but I have not yet found an elegant
> solution (like the "executing paranoia level" solving the problem how to take
> a well-tuned system to a higher PL without also diluting your anomaly
> threshold setting).
> 
> If you have something in mind, then please share.
> 
> For me, the problem is related to the paradox, that when you write a rule
> exclusion against a false positive, you go blind on the false positive. And
> neither you, nor Schroedinger's cat can easily tell if wether the rule
> exclusion is still needed. It's like you want to see things that you do
> not want to see.
> 
> Best,
> 
> Christian
> 
> 
>> On Wed, Jun 17, 2020 at 08:58:26AM +0100, Jamie Burchell wrote:
>> OK, thank you. Is there a particular strategy or best practise for dealing
>> with new untested rules that are added to the CRS?
>> 
>> I'm thinking if you have tuned the rules, are running at PL2 and new ones
>> are added in an update. It seems that any new rules would need to be
>> identified and carefully managed in an existing setup, short of putting the
>> whole install back in detection only mode and remonitoring.
>> 
>> Thanks Jamie
>> 
>>> On 17 Jun 2020, at 05:37, Christian Folini <chr...@ne...>
>>> wrote:
>>> 
>>> Hello Jamie,
>>> 
>>> I have never throught about this, but it sounds like a cool idea.
>>> 
>>> Unfortunately, it is not possible.
>>> 
>>>> On Tue, Jun 16, 2020 at 11:34:53PM +0100, Jamie Burchell wrote: of rules
>>>> should not be at PL2. I was looking at doing this by ID range instead,
>>>> but the IDs don’t seem facilitate ranges based on PL.
>>> 
>>> The ids and the PL have no connection, that's correct.
>>> 
>>> (And they can't since we are adding new rules from time to time and the
>>> concept of strict siblings would no longer work the way it does with the
>>> ID namespace)
>>> 
>>> Ahoj,
>>> 
>>> Christian
>>> 
>>>> 
>>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Jamie
>>> 
>>> 
>>>> _______________________________________________ mod-security-users
>>>> mailing list mod...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>> http://www.modsecurity.org/projects/commercial/support/
>>> 
>>> 
>>> 
>>> _______________________________________________ mod-security-users mailing
>>> list mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial
>>> ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>> 
>> 
>> _______________________________________________ mod-security-users mailing
>> list mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users Commercial
>> ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
> 
>> On Wed, Jun 17, 2020 at 08:58:26AM +0100, Jamie Burchell wrote:
>> OK, thank you. Is there a particular strategy or best practise for dealing with new untested rules that are added to the CRS?
>> 
>> I'm thinking if you have tuned the rules, are running at PL2 and new ones are added in an update. It seems that any new rules would need to be identified and carefully managed in an existing setup, short of putting the whole install back in detection only mode and remonitoring.
>> 
>> Thanks
>> Jamie
>> 
>>>> On 17 Jun 2020, at 05:37, Christian Folini <chr...@ne...> wrote:
>>> 
>>> Hello Jamie,
>>> 
>>> I have never throught about this, but it sounds like a cool idea.
>>> 
>>> Unfortunately, it is not possible.
>>> 
>>>> On Tue, Jun 16, 2020 at 11:34:53PM +0100, Jamie Burchell wrote:
>>>> of rules should not be at PL2. I was looking at doing this by ID range
>>>> instead, but the IDs don’t seem facilitate ranges based on PL.
>>> 
>>> The ids and the PL have no connection, that's correct.
>>> 
>>> (And they can't since we are adding new rules from time to time and the
>>> concept of strict siblings would no longer work the way it does with the
>>> ID namespace)
>>> 
>>> Ahoj,
>>> 
>>> Christian
>>> 
>>>> 
>>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Jamie
>>> 
>>> 
>>>> _______________________________________________
>>>> mod-security-users mailing list
>>>> mod...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>>> http://www.modsecurity.org/projects/commercial/rules/
>>>> http://www.modsecurity.org/projects/commercial/support/
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> mod-security-users mailing list
>>> mod...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>>> http://www.modsecurity.org/projects/commercial/rules/
>>> http://www.modsecurity.org/projects/commercial/support/
>> 
>> 
>> _______________________________________________
>> mod-security-users mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
> 
> 
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/