Re: [mod-security-users] Collections_remove_stale: Failed deleting collection

[Barry P. <bar...@ho...>]

There's a much better solution than either the access log or the audit log.

ModSecurity rules which are fired, are logged in the Apache error log (on one line) as well as in the Audit Log (in multiple lines so more difficult to report on and more meant to help you investigate). The error log by default shows the client IP address, though you can configure it as much as you want in Apache.

Here's two example errors from my test system (with some data changed to remove details of my system):

SecRuleEngine On:
[Tue Aug 25 18:04:15.767701 2015] [:error] [pid 123:tid 13456] [client 10.40.123.456] ModSecurity: Access denied with code 403 (phase 2). Invalid URL Encoding: Not enough characters at the end of input at REQUEST_URI. [file "blahblah/base_rules/modsecurity_crs_20_protocol_violations.conf"] [line "461"] [id "950107"] [rev "2"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"] [ver "OWASP_CRS/2.2.9"] [maturity "6"] [accuracy "8"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/EVASION"] [hostname "server.example.com"] [uri "/index.html"] [unique_id "VdygDwooaRUAAEU8FUYAAACW"]

SecRuleEngine DetectionOnly:
[Tue Aug 25 18:05:26.156386 2015] [:error] [pid 124:tid 123456] [client 10.40.123.456] ModSecurity: Warning. Invalid URL Encoding: Not enough characters at the end of input at REQUEST_URI. [file "blahblah/base_rules/modsecurity_crs_20_protocol_violations.conf"] [line "461"] [id "950107"] [rev "2"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"] [ver "OWASP_CRS/2.2.9"] [maturity "6"] [accuracy "8"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/EVASION"] [hostname "www.example.com"] [uri "/index.html"] [unique_id "VdygVgooaRUAAASs0ZoAAACD"]

I believe that gives all the information you need (the client ip address, the rule that fired, whether access was denied (and with what return code) or it's just a Warning...etc.). Now the exact logging depends on the rule definition, and as you can see the OWASP rules have lots of information.

As to your other questions:

Be vary of LocationMatch for ModSecurity rule filtering as that runs AFTER phase 1 rules, so any filters in there won't work for phase 1 rules. See my answer here for more details on that: http://stackoverflow.com/questions/32041482/apache-locationmatch-wildcard-for-modsecurity-on-wordpress-site/32067850#32067850

In regards to reducing requests that need logged into collections, if you're are SOAP requests then probably not much you can do here, since they will all be application requests and so shouldn't bypass ModSecurity as presumably you put this in place to protect that application. If you were running a web server, then there are rules you can use to weed out static requests and reduce load on ModSecurity and some rules are in modsecurity_crs10_ignore_static.conf but, as I mentioned in my first reply, I think they could use a little tweaking and happy to explain how if anyone wants to know but don't want to divert from this thread too much.

Thanks,
Barry



----------------------------------------
> To: mod...@li...
> From: rai...@ki...
> Date: Tue, 25 Aug 2015 14:38:47 +0200
> Subject: Re: [mod-security-users] Collections_remove_stale: Failed deleting collection
>
> Am 25.08.2015 um 12:02 schrieb Sophie Loewenthal:
>> Hi Rainer ,
>>
>>>> Since 509 is very specific, why not taking the IP from the normal
>> access log of the web server?
>> Mod_Sec is running in Detection Only mode. Would this be logged in the
>> access log?
>
> Any web server allows to activate an access log. Typically it is already
> active by default. The contents of the log are adjustable but again the
> IP and the HTTP status code should be there by default.
>
> The access log contains one line for each http request that was handled
> by the web server. So you'd have to find the log or identify how to
> activate it for your web server and identify the configured or default
> format of its content. It will then be easy to filter the log for all
> requests that were answered with a 509 and extract the IP addresses.
>
> All of this access log stuff is independent of your mod_security config.
>
> Regards,
>
> Rainer
>
>> On 8/25/2015 11:27 AM, Rainer Jung wrote:
>>> Am 25.08.2015 um 10:47 schrieb Sophie Loewenthal:
>>>> Hi Barry,
>>>>
>>>> Thank you for your well penned reply.
>>>>
>>>> For an quick fix, I have put the directory into a ram disc, and
>>>> shall run some pruning methods from cron.
>>>>
>>>> Your other suggestion require I spend more time on how I should
>>>> differentiate between static and dynamic content, although I doubt any
>>>> static is requested because this is a soap gateway. Looking anyway :)
>>>>
>>>> How else could I reduce modsec traffic? If I could wrap this code
>>>> up into a LocationMatch and place inside a vhost entry, maybe this could
>>>> help.
>>>>
>>>> Management would like a list of potential offenders by IP.
>>>> How could I adapt this code to add logging of IP and or request into a
>>>> file? I looked at SecAuditLogParts and enabling everything I could not
>>>> see of it hit a rule. Currently this runs in detection mode.
>>>> I have logging enabled on this rule:
>>>> SecRule IP:SOMEPATHCOUNTER "@gt 120"
>>>> "phase:2,pause:100,deny,status:509,setenv:RATELIMITED,skip:1,log,id:10000003,msg:Sophie_10000003"
>>>> But would like an IP address logged when it was sent a 509 status message.
>>>> Still reading
>>>> https://www.feistyduck.com/library/modsecurity-handbook-free/online/ch04-logging.html
>>>> !
>>> Since 509 is very specific, why not taking the IP from the normal access
>>> log of the web server?
>>>
>>> Regardsco
>>>
>>> Rainer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/