Thread: [mod-security-users] Failed deleting collection error
Brought to you by:
victorhora,
zimmerletw
From: Nick H. <dar...@gm...> - 2013-04-24 19:28:14
|
I'm in the process of setting up mod_security for the first time. Things seem to be working OK so far, but I'm periodically getting a strange message in the audit_log: Message: Failed deleting collection (name "ip", key "123.123.123.123"): Internal error Apache-Handler: php5-script Stopwatch: 1366821230906385 183614 (- - -) Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); OWASP_CRS/2.2.5. Server: Apache I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and using the CRS version 2.2.5. I'm using modsecurity_crs_11_dos_protection.conf which I believe to be the origin of this message. The message seems to appear upon a request from a totally different IP address, and happens an hour after the IP address referred to in the error message made a request. The IP address in the error message never got flagged for something by mod_security. Without knowing much about mod_security's code, I'm guessing that as mod_security records the IP addresses of who is requesting into its collection, it must expire them out an hour later. I'm just not sure why it is sometimes failing to delete them out, as most of the time it must be successful because this log doesn't happen very often. Or am I supposed to ignore this error? Thanks. |
From: Breno S. <bre...@gm...> - 2013-04-24 20:33:05
|
Hello Nick, This is an old issue that we never had a chance to reproduce and debug. Looks like this error message is from a apr function. I would like to ask you to set: SecCollectionTimeout 100 Please monitor the the ip file in your SecDataDir and after some days check if the entry related to a failed ip was deleted. Maybe it is failing for some reason but has success to delete in the next time. Thanks, Please give us a feedback after some days. Breno On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...> wrote: > I'm in the process of setting up mod_security for the first time. Things > seem to be working OK so far, but I'm periodically getting a strange > message in the audit_log: > > Message: Failed deleting collection (name "ip", key "123.123.123.123"): > Internal error > Apache-Handler: php5-script > Stopwatch: 1366821230906385 183614 (- - -) > Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, > p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 > Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); > OWASP_CRS/2.2.5. > Server: Apache > > I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and > using the CRS version 2.2.5. I'm > using modsecurity_crs_11_dos_protection.conf which I believe to be the > origin of this message. > > The message seems to appear upon a request from a totally different IP > address, and happens an hour after the IP address referred to in the error > message made a request. The IP address in the error message never got > flagged for something by mod_security. > > Without knowing much about mod_security's code, I'm guessing that as > mod_security records the IP addresses of who is requesting into its > collection, it must expire them out an hour later. I'm just not sure why it > is sometimes failing to delete them out, as most of the time it must be > successful because this log doesn't happen very often. Or am I supposed to > ignore this error? Thanks. > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > |
From: Bernard C. <be...@go...> - 2013-04-25 02:35:09
|
Hi Breno, I referred to a similar problem written several weeks ago on the same issue unfortunately for which there were no replies to date. In my case I have already set SecCollectionTimeout to 180 but still got plenty of these errors in the audit logs and the db files still pretty big (several hundred MBs). I confirmed using jwall's collection viewer tool and handpicked some of the keys in question that they could not be found, so I expect has somehow been cleaned up already. At least, can we prevent such errors from showing up in audit log and in turn to AuditConsole, since otherwise valid requests got logged with the only message being this error appearing in an unrelated transaction? Thank you. On Thu, Apr 25, 2013 at 4:32 AM, Breno Silva <bre...@gm...> wrote: > Hello Nick, > > This is an old issue that we never had a chance to reproduce and debug. > Looks like this error message is from a apr function. > > I would like to ask you to set: > > SecCollectionTimeout 100 > > Please monitor the the ip file in your SecDataDir and after some days > check if the entry related to a failed ip was deleted. Maybe it is failing > for some reason but has success to delete in the next time. > > Thanks, > > Please give us a feedback after some days. > > Breno > > > On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...> wrote: > >> I'm in the process of setting up mod_security for the first time. Things >> seem to be working OK so far, but I'm periodically getting a strange >> message in the audit_log: >> >> Message: Failed deleting collection (name "ip", key "123.123.123.123"): >> Internal error >> Apache-Handler: php5-script >> Stopwatch: 1366821230906385 183614 (- - -) >> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >> OWASP_CRS/2.2.5. >> Server: Apache >> >> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and >> using the CRS version 2.2.5. I'm >> using modsecurity_crs_11_dos_protection.conf which I believe to be the >> origin of this message. >> >> The message seems to appear upon a request from a totally different IP >> address, and happens an hour after the IP address referred to in the error >> message made a request. The IP address in the error message never got >> flagged for something by mod_security. >> >> Without knowing much about mod_security's code, I'm guessing that as >> mod_security records the IP addresses of who is requesting into its >> collection, it must expire them out an hour later. I'm just not sure why it >> is sometimes failing to delete them out, as most of the time it must be >> successful because this log doesn't happen very often. Or am I supposed to >> ignore this error? Thanks. >> >> >> ------------------------------------------------------------------------------ >> Try New Relic Now & We'll Send You this Cool Shirt >> New Relic is the only SaaS-based application performance monitoring >> service >> that delivers powerful full stack analytics. Optimize and monitor your >> browser, app, & servers with just a few lines of code. Try New Relic >> and get this awesome Nerd Life shirt! >> http://p.sf.net/sfu/newrelic_d2d_apr >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> http://www.modsecurity.org/projects/commercial/support/ >> >> > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > -- Regards, Bernard Chan. GoAnimate. |
From: Breno S. <bre...@gm...> - 2013-04-25 12:43:04
|
Hello Bernard, So, at least the entries are being removed in the next execution of apr delete function. We don't have a feature to remove specific fields from a audit log part. However this is listed in our roadmap. Now you need to remove part H from your logs. Thanks On Wed, Apr 24, 2013 at 10:27 PM, Bernard Chan <be...@go...>wrote: > > Hi Breno, > > I referred to a similar problem written several weeks ago on the same > issue unfortunately for which there were no replies to date. In my case I > have already set SecCollectionTimeout to 180 but still got plenty of these > errors in the audit logs and the db files still pretty big (several hundred > MBs). I confirmed using jwall's collection viewer tool and handpicked some > of the keys in question that they could not be found, so I expect has > somehow been cleaned up already. > > At least, can we prevent such errors from showing up in audit log and in > turn to AuditConsole, since otherwise valid requests got logged with the > only message being this error appearing in an unrelated transaction? Thank > you. > > > On Thu, Apr 25, 2013 at 4:32 AM, Breno Silva <bre...@gm...>wrote: > >> Hello Nick, >> >> This is an old issue that we never had a chance to reproduce and debug. >> Looks like this error message is from a apr function. >> >> I would like to ask you to set: >> >> SecCollectionTimeout 100 >> >> Please monitor the the ip file in your SecDataDir and after some days >> check if the entry related to a failed ip was deleted. Maybe it is failing >> for some reason but has success to delete in the next time. >> >> Thanks, >> >> Please give us a feedback after some days. >> >> Breno >> >> >> On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...>wrote: >> >>> I'm in the process of setting up mod_security for the first time. >>> Things seem to be working OK so far, but I'm periodically getting a strange >>> message in the audit_log: >>> >>> Message: Failed deleting collection (name "ip", key "123.123.123.123"): >>> Internal error >>> Apache-Handler: php5-script >>> Stopwatch: 1366821230906385 183614 (- - -) >>> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >>> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >>> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >>> OWASP_CRS/2.2.5. >>> Server: Apache >>> >>> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and >>> using the CRS version 2.2.5. I'm >>> using modsecurity_crs_11_dos_protection.conf which I believe to be the >>> origin of this message. >>> >>> The message seems to appear upon a request from a totally different IP >>> address, and happens an hour after the IP address referred to in the error >>> message made a request. The IP address in the error message never got >>> flagged for something by mod_security. >>> >>> Without knowing much about mod_security's code, I'm guessing that as >>> mod_security records the IP addresses of who is requesting into its >>> collection, it must expire them out an hour later. I'm just not sure why it >>> is sometimes failing to delete them out, as most of the time it must be >>> successful because this log doesn't happen very often. Or am I supposed to >>> ignore this error? Thanks. >>> >>> >>> ------------------------------------------------------------------------------ >>> Try New Relic Now & We'll Send You this Cool Shirt >>> New Relic is the only SaaS-based application performance monitoring >>> service >>> that delivers powerful full stack analytics. Optimize and monitor your >>> browser, app, & servers with just a few lines of code. Try New Relic >>> and get this awesome Nerd Life shirt! >>> http://p.sf.net/sfu/newrelic_d2d_apr >>> _______________________________________________ >>> mod-security-users mailing list >>> mod...@li... >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>> http://www.modsecurity.org/projects/commercial/rules/ >>> http://www.modsecurity.org/projects/commercial/support/ >>> >>> >> >> >> ------------------------------------------------------------------------------ >> Try New Relic Now & We'll Send You this Cool Shirt >> New Relic is the only SaaS-based application performance monitoring >> service >> that delivers powerful full stack analytics. Optimize and monitor your >> browser, app, & servers with just a few lines of code. Try New Relic >> and get this awesome Nerd Life shirt! >> http://p.sf.net/sfu/newrelic_d2d_apr >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> http://www.modsecurity.org/projects/commercial/support/ >> >> > > > -- > > Regards, > Bernard Chan. > GoAnimate. > |
From: Bernard C. <be...@go...> - 2013-04-26 02:36:16
|
Hi Breno, I was not looking for as specific as removing individual fields from audit log. Basically, can we have a rule that if a transaction does not trigger other rules (say those generated by CRS or our own those that uses the log action), then trigger ctl:auditLogParts on it in phase 5 early on to prevent those transactions from even appearing in audit log, perhaps by matching against HIGHEST_SEVERITY? Since it is just a nuisance to browse through lots of transactions with a rather large proportion of valid transactions just with that specific error message due to the piggyback cleanup ... On Thu, Apr 25, 2013 at 8:42 PM, Breno Silva <bre...@gm...> wrote: > Hello Bernard, > > So, at least the entries are being removed in the next execution of apr > delete function. > We don't have a feature to remove specific fields from a audit log part. > However this is listed in our roadmap. Now you need to remove part H from > your logs. > > Thanks > > > On Wed, Apr 24, 2013 at 10:27 PM, Bernard Chan <be...@go...>wrote: > >> >> Hi Breno, >> >> I referred to a similar problem written several weeks ago on the same >> issue unfortunately for which there were no replies to date. In my case I >> have already set SecCollectionTimeout to 180 but still got plenty of these >> errors in the audit logs and the db files still pretty big (several hundred >> MBs). I confirmed using jwall's collection viewer tool and handpicked some >> of the keys in question that they could not be found, so I expect has >> somehow been cleaned up already. >> >> At least, can we prevent such errors from showing up in audit log and in >> turn to AuditConsole, since otherwise valid requests got logged with the >> only message being this error appearing in an unrelated transaction? Thank >> you. >> >> >> On Thu, Apr 25, 2013 at 4:32 AM, Breno Silva <bre...@gm...>wrote: >> >>> Hello Nick, >>> >>> This is an old issue that we never had a chance to reproduce and debug. >>> Looks like this error message is from a apr function. >>> >>> I would like to ask you to set: >>> >>> SecCollectionTimeout 100 >>> >>> Please monitor the the ip file in your SecDataDir and after some days >>> check if the entry related to a failed ip was deleted. Maybe it is failing >>> for some reason but has success to delete in the next time. >>> >>> Thanks, >>> >>> Please give us a feedback after some days. >>> >>> Breno >>> >>> >>> On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...>wrote: >>> >>>> I'm in the process of setting up mod_security for the first time. >>>> Things seem to be working OK so far, but I'm periodically getting a strange >>>> message in the audit_log: >>>> >>>> Message: Failed deleting collection (name "ip", key "123.123.123.123"): >>>> Internal error >>>> Apache-Handler: php5-script >>>> Stopwatch: 1366821230906385 183614 (- - -) >>>> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >>>> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >>>> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >>>> OWASP_CRS/2.2.5. >>>> Server: Apache >>>> >>>> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and >>>> using the CRS version 2.2.5. I'm >>>> using modsecurity_crs_11_dos_protection.conf which I believe to be the >>>> origin of this message. >>>> >>>> The message seems to appear upon a request from a totally different IP >>>> address, and happens an hour after the IP address referred to in the error >>>> message made a request. The IP address in the error message never got >>>> flagged for something by mod_security. >>>> >>>> Without knowing much about mod_security's code, I'm guessing that as >>>> mod_security records the IP addresses of who is requesting into its >>>> collection, it must expire them out an hour later. I'm just not sure why it >>>> is sometimes failing to delete them out, as most of the time it must be >>>> successful because this log doesn't happen very often. Or am I supposed to >>>> ignore this error? Thanks. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Try New Relic Now & We'll Send You this Cool Shirt >>>> New Relic is the only SaaS-based application performance monitoring >>>> service >>>> that delivers powerful full stack analytics. Optimize and monitor your >>>> browser, app, & servers with just a few lines of code. Try New Relic >>>> and get this awesome Nerd Life shirt! >>>> http://p.sf.net/sfu/newrelic_d2d_apr >>>> _______________________________________________ >>>> mod-security-users mailing list >>>> mod...@li... >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>>> http://www.modsecurity.org/projects/commercial/rules/ >>>> http://www.modsecurity.org/projects/commercial/support/ >>>> >>>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Try New Relic Now & We'll Send You this Cool Shirt >>> New Relic is the only SaaS-based application performance monitoring >>> service >>> that delivers powerful full stack analytics. Optimize and monitor your >>> browser, app, & servers with just a few lines of code. Try New Relic >>> and get this awesome Nerd Life shirt! >>> http://p.sf.net/sfu/newrelic_d2d_apr >>> _______________________________________________ >>> mod-security-users mailing list >>> mod...@li... >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>> http://www.modsecurity.org/projects/commercial/rules/ >>> http://www.modsecurity.org/projects/commercial/support/ >>> >>> >> >> >> -- >> >> Regards, >> Bernard Chan. >> GoAnimate. >> > > -- Regards, Bernard Chan. GoAnimate. |
From: Breno S. <bre...@gm...> - 2013-04-26 02:37:53
|
Maybe i can add a FAILED_DELETE_DBM_ERROR variable or something similar. Then people can check if it is set. Breno On Thu, Apr 25, 2013 at 11:29 PM, Bernard Chan <be...@go...>wrote: > > Hi Breno, > > I was not looking for as specific as removing individual fields from audit > log. Basically, can we have a rule that if a transaction does not trigger > other rules (say those generated by CRS or our own those that uses the log > action), then trigger ctl:auditLogParts on it in phase 5 early on to > prevent those transactions from even appearing in audit log, perhaps by > matching against HIGHEST_SEVERITY? Since it is just a nuisance to browse > through lots of transactions with a rather large proportion of valid > transactions just with that specific error message due to the piggyback > cleanup ... > > > > On Thu, Apr 25, 2013 at 8:42 PM, Breno Silva <bre...@gm...>wrote: > >> Hello Bernard, >> >> So, at least the entries are being removed in the next execution of apr >> delete function. >> We don't have a feature to remove specific fields from a audit log part. >> However this is listed in our roadmap. Now you need to remove part H from >> your logs. >> >> Thanks >> >> >> On Wed, Apr 24, 2013 at 10:27 PM, Bernard Chan <be...@go...>wrote: >> >>> >>> Hi Breno, >>> >>> I referred to a similar problem written several weeks ago on the same >>> issue unfortunately for which there were no replies to date. In my case I >>> have already set SecCollectionTimeout to 180 but still got plenty of these >>> errors in the audit logs and the db files still pretty big (several hundred >>> MBs). I confirmed using jwall's collection viewer tool and handpicked some >>> of the keys in question that they could not be found, so I expect has >>> somehow been cleaned up already. >>> >>> At least, can we prevent such errors from showing up in audit log and in >>> turn to AuditConsole, since otherwise valid requests got logged with the >>> only message being this error appearing in an unrelated transaction? Thank >>> you. >>> >>> >>> On Thu, Apr 25, 2013 at 4:32 AM, Breno Silva <bre...@gm...>wrote: >>> >>>> Hello Nick, >>>> >>>> This is an old issue that we never had a chance to reproduce and debug. >>>> Looks like this error message is from a apr function. >>>> >>>> I would like to ask you to set: >>>> >>>> SecCollectionTimeout 100 >>>> >>>> Please monitor the the ip file in your SecDataDir and after some days >>>> check if the entry related to a failed ip was deleted. Maybe it is failing >>>> for some reason but has success to delete in the next time. >>>> >>>> Thanks, >>>> >>>> Please give us a feedback after some days. >>>> >>>> Breno >>>> >>>> >>>> On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...>wrote: >>>> >>>>> I'm in the process of setting up mod_security for the first time. >>>>> Things seem to be working OK so far, but I'm periodically getting a strange >>>>> message in the audit_log: >>>>> >>>>> Message: Failed deleting collection (name "ip", key >>>>> "123.123.123.123"): Internal error >>>>> Apache-Handler: php5-script >>>>> Stopwatch: 1366821230906385 183614 (- - -) >>>>> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >>>>> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >>>>> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >>>>> OWASP_CRS/2.2.5. >>>>> Server: Apache >>>>> >>>>> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and >>>>> using the CRS version 2.2.5. I'm >>>>> using modsecurity_crs_11_dos_protection.conf which I believe to be the >>>>> origin of this message. >>>>> >>>>> The message seems to appear upon a request from a totally different IP >>>>> address, and happens an hour after the IP address referred to in the error >>>>> message made a request. The IP address in the error message never got >>>>> flagged for something by mod_security. >>>>> >>>>> Without knowing much about mod_security's code, I'm guessing that as >>>>> mod_security records the IP addresses of who is requesting into its >>>>> collection, it must expire them out an hour later. I'm just not sure why it >>>>> is sometimes failing to delete them out, as most of the time it must be >>>>> successful because this log doesn't happen very often. Or am I supposed to >>>>> ignore this error? Thanks. >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Try New Relic Now & We'll Send You this Cool Shirt >>>>> New Relic is the only SaaS-based application performance monitoring >>>>> service >>>>> that delivers powerful full stack analytics. Optimize and monitor your >>>>> browser, app, & servers with just a few lines of code. Try New Relic >>>>> and get this awesome Nerd Life shirt! >>>>> http://p.sf.net/sfu/newrelic_d2d_apr >>>>> _______________________________________________ >>>>> mod-security-users mailing list >>>>> mod...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>>>> http://www.modsecurity.org/projects/commercial/rules/ >>>>> http://www.modsecurity.org/projects/commercial/support/ >>>>> >>>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Try New Relic Now & We'll Send You this Cool Shirt >>>> New Relic is the only SaaS-based application performance monitoring >>>> service >>>> that delivers powerful full stack analytics. Optimize and monitor your >>>> browser, app, & servers with just a few lines of code. Try New Relic >>>> and get this awesome Nerd Life shirt! >>>> http://p.sf.net/sfu/newrelic_d2d_apr >>>> _______________________________________________ >>>> mod-security-users mailing list >>>> mod...@li... >>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>>> http://www.modsecurity.org/projects/commercial/rules/ >>>> http://www.modsecurity.org/projects/commercial/support/ >>>> >>>> >>> >>> >>> -- >>> >>> Regards, >>> Bernard Chan. >>> GoAnimate. >>> >> >> > > > -- > > Regards, > Bernard Chan. > GoAnimate. > |
From: Bernard C. <be...@go...> - 2013-04-26 02:47:51
|
That could be helpful, thank you. On Fri, Apr 26, 2013 at 10:37 AM, Breno Silva <bre...@gm...> wrote: > Maybe i can add a FAILED_DELETE_DBM_ERROR variable or something similar. > Then people can check if it is set. > Breno > > > On Thu, Apr 25, 2013 at 11:29 PM, Bernard Chan <be...@go...>wrote: > >> >> Hi Breno, >> >> I was not looking for as specific as removing individual fields from >> audit log. Basically, can we have a rule that if a transaction does not >> trigger other rules (say those generated by CRS or our own those that uses >> the log action), then trigger ctl:auditLogParts on it in phase 5 early on >> to prevent those transactions from even appearing in audit log, perhaps by >> matching against HIGHEST_SEVERITY? Since it is just a nuisance to browse >> through lots of transactions with a rather large proportion of valid >> transactions just with that specific error message due to the piggyback >> cleanup ... >> >> >> >> On Thu, Apr 25, 2013 at 8:42 PM, Breno Silva <bre...@gm...>wrote: >> >>> Hello Bernard, >>> >>> So, at least the entries are being removed in the next execution of apr >>> delete function. >>> We don't have a feature to remove specific fields from a audit log part. >>> However this is listed in our roadmap. Now you need to remove part H from >>> your logs. >>> >>> Thanks >>> >>> >>> On Wed, Apr 24, 2013 at 10:27 PM, Bernard Chan <be...@go...>wrote: >>> >>>> >>>> Hi Breno, >>>> >>>> I referred to a similar problem written several weeks ago on the same >>>> issue unfortunately for which there were no replies to date. In my case I >>>> have already set SecCollectionTimeout to 180 but still got plenty of these >>>> errors in the audit logs and the db files still pretty big (several hundred >>>> MBs). I confirmed using jwall's collection viewer tool and handpicked some >>>> of the keys in question that they could not be found, so I expect has >>>> somehow been cleaned up already. >>>> >>>> At least, can we prevent such errors from showing up in audit log and >>>> in turn to AuditConsole, since otherwise valid requests got logged with the >>>> only message being this error appearing in an unrelated transaction? Thank >>>> you. >>>> >>>> >>>> On Thu, Apr 25, 2013 at 4:32 AM, Breno Silva <bre...@gm...>wrote: >>>> >>>>> Hello Nick, >>>>> >>>>> This is an old issue that we never had a chance to reproduce and >>>>> debug. Looks like this error message is from a apr function. >>>>> >>>>> I would like to ask you to set: >>>>> >>>>> SecCollectionTimeout 100 >>>>> >>>>> Please monitor the the ip file in your SecDataDir and after some days >>>>> check if the entry related to a failed ip was deleted. Maybe it is failing >>>>> for some reason but has success to delete in the next time. >>>>> >>>>> Thanks, >>>>> >>>>> Please give us a feedback after some days. >>>>> >>>>> Breno >>>>> >>>>> >>>>> On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...>wrote: >>>>> >>>>>> I'm in the process of setting up mod_security for the first time. >>>>>> Things seem to be working OK so far, but I'm periodically getting a strange >>>>>> message in the audit_log: >>>>>> >>>>>> Message: Failed deleting collection (name "ip", key >>>>>> "123.123.123.123"): Internal error >>>>>> Apache-Handler: php5-script >>>>>> Stopwatch: 1366821230906385 183614 (- - -) >>>>>> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >>>>>> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >>>>>> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >>>>>> OWASP_CRS/2.2.5. >>>>>> Server: Apache >>>>>> >>>>>> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) >>>>>> and using the CRS version 2.2.5. I'm >>>>>> using modsecurity_crs_11_dos_protection.conf which I believe to be the >>>>>> origin of this message. >>>>>> >>>>>> The message seems to appear upon a request from a totally different >>>>>> IP address, and happens an hour after the IP address referred to in the >>>>>> error message made a request. The IP address in the error message never got >>>>>> flagged for something by mod_security. >>>>>> >>>>>> Without knowing much about mod_security's code, I'm guessing that as >>>>>> mod_security records the IP addresses of who is requesting into its >>>>>> collection, it must expire them out an hour later. I'm just not sure why it >>>>>> is sometimes failing to delete them out, as most of the time it must be >>>>>> successful because this log doesn't happen very often. Or am I supposed to >>>>>> ignore this error? Thanks. >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> Try New Relic Now & We'll Send You this Cool Shirt >>>>>> New Relic is the only SaaS-based application performance monitoring >>>>>> service >>>>>> that delivers powerful full stack analytics. Optimize and monitor your >>>>>> browser, app, & servers with just a few lines of code. Try New Relic >>>>>> and get this awesome Nerd Life shirt! >>>>>> http://p.sf.net/sfu/newrelic_d2d_apr >>>>>> _______________________________________________ >>>>>> mod-security-users mailing list >>>>>> mod...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>>>>> http://www.modsecurity.org/projects/commercial/rules/ >>>>>> http://www.modsecurity.org/projects/commercial/support/ >>>>>> >>>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Try New Relic Now & We'll Send You this Cool Shirt >>>>> New Relic is the only SaaS-based application performance monitoring >>>>> service >>>>> that delivers powerful full stack analytics. Optimize and monitor your >>>>> browser, app, & servers with just a few lines of code. Try New Relic >>>>> and get this awesome Nerd Life shirt! >>>>> http://p.sf.net/sfu/newrelic_d2d_apr >>>>> _______________________________________________ >>>>> mod-security-users mailing list >>>>> mod...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>>>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>>>> http://www.modsecurity.org/projects/commercial/rules/ >>>>> http://www.modsecurity.org/projects/commercial/support/ >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Regards, >>>> Bernard Chan. >>>> GoAnimate. >>>> >>> >>> >> >> >> -- >> >> Regards, >> Bernard Chan. >> GoAnimate. >> > > -- Regards, Bernard Chan. GoAnimate. |
From: Nick <dar...@gm...> - 2013-04-26 17:19:47
|
Hello, I've been running with this setting and upon examination, the IP addresses that are being logged are being cleaned out of the IP collection eventually. Hopefully that helps you track down the bug. I guess for now, my knowledge about mod_security is pretty limited, is there a problem with this happening occasionally other than the annoying message in the log? Also, should I leave the setting SecCollectionTimeout set to 100, or remove that line? Thanks, Nick On Wed, Apr 24, 2013 at 3:32 PM, Breno Silva <bre...@gm...> wrote: > Hello Nick, > > This is an old issue that we never had a chance to reproduce and debug. > Looks like this error message is from a apr function. > > I would like to ask you to set: > > SecCollectionTimeout 100 > > Please monitor the the ip file in your SecDataDir and after some days > check if the entry related to a failed ip was deleted. Maybe it is failing > for some reason but has success to delete in the next time. > > Thanks, > > Please give us a feedback after some days. > > Breno > > > On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...> wrote: > >> I'm in the process of setting up mod_security for the first time. Things >> seem to be working OK so far, but I'm periodically getting a strange >> message in the audit_log: >> >> Message: Failed deleting collection (name "ip", key "123.123.123.123"): >> Internal error >> Apache-Handler: php5-script >> Stopwatch: 1366821230906385 183614 (- - -) >> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >> OWASP_CRS/2.2.5. >> Server: Apache >> >> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and >> using the CRS version 2.2.5. I'm >> using modsecurity_crs_11_dos_protection.conf which I believe to be the >> origin of this message. >> >> The message seems to appear upon a request from a totally different IP >> address, and happens an hour after the IP address referred to in the error >> message made a request. The IP address in the error message never got >> flagged for something by mod_security. >> >> Without knowing much about mod_security's code, I'm guessing that as >> mod_security records the IP addresses of who is requesting into its >> collection, it must expire them out an hour later. I'm just not sure why it >> is sometimes failing to delete them out, as most of the time it must be >> successful because this log doesn't happen very often. Or am I supposed to >> ignore this error? Thanks. >> >> >> ------------------------------------------------------------------------------ >> Try New Relic Now & We'll Send You this Cool Shirt >> New Relic is the only SaaS-based application performance monitoring >> service >> that delivers powerful full stack analytics. Optimize and monitor your >> browser, app, & servers with just a few lines of code. Try New Relic >> and get this awesome Nerd Life shirt! >> http://p.sf.net/sfu/newrelic_d2d_apr >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >> http://www.modsecurity.org/projects/commercial/rules/ >> http://www.modsecurity.org/projects/commercial/support/ >> >> > |
From: Breno S. <bre...@gm...> - 2013-05-02 12:11:51
|
Hello guys, Just added the new variable SDBM_DELETE_ERROR. Could you download the trunk, test and give a feedback if it is OK ? Thanks On Fri, Apr 26, 2013 at 2:19 PM, Nick <dar...@gm...> wrote: > Hello, > > I've been running with this setting and upon examination, the IP addresses > that are being logged are being cleaned out of the IP collection > eventually. Hopefully that helps you track down the bug. I guess for now, > my knowledge about mod_security is pretty limited, is there a problem with > this happening occasionally other than the annoying message in the log? > Also, should I leave the setting SecCollectionTimeout set to 100, or remove > that line? Thanks, > > Nick > > > > On Wed, Apr 24, 2013 at 3:32 PM, Breno Silva <bre...@gm...>wrote: > >> Hello Nick, >> >> This is an old issue that we never had a chance to reproduce and debug. >> Looks like this error message is from a apr function. >> >> I would like to ask you to set: >> >> SecCollectionTimeout 100 >> >> Please monitor the the ip file in your SecDataDir and after some days >> check if the entry related to a failed ip was deleted. Maybe it is failing >> for some reason but has success to delete in the next time. >> >> Thanks, >> >> Please give us a feedback after some days. >> >> Breno >> >> >> On Wed, Apr 24, 2013 at 3:28 PM, Nick Hall <dar...@gm...>wrote: >> >>> I'm in the process of setting up mod_security for the first time. >>> Things seem to be working OK so far, but I'm periodically getting a strange >>> message in the audit_log: >>> >>> Message: Failed deleting collection (name "ip", key "123.123.123.123"): >>> Internal error >>> Apache-Handler: php5-script >>> Stopwatch: 1366821230906385 183614 (- - -) >>> Stopwatch2: 1366821230906385 183614; combined=39557, p1=106, p2=1166, >>> p3=0, p4=0, p5=19164, sr=28, sw=49, l=0, gc=19072 >>> Producer: ModSecurity for Apache/2.6.8 (http://www.modsecurity.org/); >>> OWASP_CRS/2.2.5. >>> Server: Apache >>> >>> I am running mod_security 2.6.8 (installed on RedHat EL5 via EPEL) and >>> using the CRS version 2.2.5. I'm >>> using modsecurity_crs_11_dos_protection.conf which I believe to be the >>> origin of this message. >>> >>> The message seems to appear upon a request from a totally different IP >>> address, and happens an hour after the IP address referred to in the error >>> message made a request. The IP address in the error message never got >>> flagged for something by mod_security. >>> >>> Without knowing much about mod_security's code, I'm guessing that as >>> mod_security records the IP addresses of who is requesting into its >>> collection, it must expire them out an hour later. I'm just not sure why it >>> is sometimes failing to delete them out, as most of the time it must be >>> successful because this log doesn't happen very often. Or am I supposed to >>> ignore this error? Thanks. >>> >>> >>> ------------------------------------------------------------------------------ >>> Try New Relic Now & We'll Send You this Cool Shirt >>> New Relic is the only SaaS-based application performance monitoring >>> service >>> that delivers powerful full stack analytics. Optimize and monitor your >>> browser, app, & servers with just a few lines of code. Try New Relic >>> and get this awesome Nerd Life shirt! >>> http://p.sf.net/sfu/newrelic_d2d_apr >>> _______________________________________________ >>> mod-security-users mailing list >>> mod...@li... >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users >>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: >>> http://www.modsecurity.org/projects/commercial/rules/ >>> http://www.modsecurity.org/projects/commercial/support/ >>> >>> >> > |