Hi Felipe,

thanks for the fast response.

Regarding SecCollectionTimeout... I've already set the value to 240, as I read that some people were
having trouble with too high settings. Also we've tried a different solution, where we set up a cron job
that ran every 45 minutes to delete the ip.pag and ip.dir files. As the ip.pag and .dir files are created 
automatically, this didn't seem to be an issue- still it didn't change anything for us, neither.

Regarding the debugging tool. I don't have any current IP collection file (due to the cronjob ;-) ). But
I'll have one server running with IP collection enable again, to get a new file. I'll provide you with 
the details, once the issue occured again.


Thanks
Winfried


From: "Felipe Costa" <FCosta@trustwave.com>
To: "<mod-security-users@lists.sourceforge.net>" <mod-security-users@lists.sourceforge.net>, "Winfried Neessen" <neessen@cleverbridge.com>
Sent: Wednesday, April 16, 2014 7:39:12 PM
Subject: Re: [mod-security-users] Locking issue with enabled persistent        collection

Hi Winfried,

On Apr 16, 2014, at 5:41 AM, Winfried Neessen <neessen@cleverbridge.com> wrote:
 Once the IP collection is enabled in the ruleset (in addtion to the blocking rules), the server still runs
fineā€¦ at least for a couple of hours. But after approx. 5-12 hours the logs begin to throw messages

There are some issues opened on GitHub related to similar problems. It seems that this problem is trigged in specific scenarios, so thanks for your detailed report, it is valuable. 

Low values for SecCollectionTimeout (https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#SecCollectionTimeout) may help you to minimize it.

While debugging those issues we have created the modsec-sdbm-util, which is able to open the collection file and interpret its content as ModSecurity does. 
The utility can be downloaded here: https://github.com/SpiderLabs/modsec-sdbm-util. The full list of functionalities: https://github.com/SpiderLabs/modsec-sdbm-util/blob/master/README.md

Similar problems have been reported in the following issues:
https://github.com/SpiderLabs/ModSecurity/issues/558
https://github.com/SpiderLabs/ModSecurity/issues/314
https://github.com/SpiderLabs/ModSecurity/issues/574
https://github.com/SpiderLabs/ModSecurity/issues/473

We are also working in alternatives to SDBM:
https://github.com/SpiderLabs/ModSecurity/issues/378

Br.,
Felipe "Zimmerle" Costa
Security Researcher, SpiderLabs

Trustwave | SMART SECURITY ON DEMAND





This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.