[mod-security-users] strange behavior of blacklist with modsecurity 2.5.12

[R.A. I. <li...@fl...>]

I am experiencing erratic behavior of the blacklist mechanism with Modsecurity 2.5.12 and crs 2.0.9 after migrating a root server from Debian 4 to Ubuntu 10.04

In general everything works as expected, but about 25% of IPs that get blacklisted have absolutely no entry in the Apache and Modsec log files and don't occur in a consistent manner: calling some url may succeed one time and get blacklisted another time.

Since nothing shows in the logs for those IPs (except for blacklist_web.pl saying it blacklisted such and such IP), I am not having any luck tracing the cause.

One thought was since the new version of Virtualmin that is being used to configure the various virtual hosts has a some older site running with mod_php and others with FCGId, and since blacklist.c picks up the remote IP from an environment variable, that maybe somewhere the environment variables of the different virtual hosts running simultaneously get mixed up ?  I guess that's not likely...

Or, since my Modsecurity initialization directives have accumulated since quite a few back versions, maybe there is some inconsistency in the global initialization?
I now have the below, combining some older selfcontained rules for obvious hacks like calls to phpmyadmin that lead to immediate blacklisting, together with the new crs 2.

For example, are the directives for SecData, SecTmpDir etc still needed?

Finally, on a secondary question, is the rule marked as ##A below the correct way to blacklist more serious anomalies in the context of anomaly scoring/collaborative detection? (the idea being that the rule present in modsecurity_crs_49_inbound_blocking blocks requests with anomalies above 5, but for example with a total score above 20 the IP should be blacklisted)


#initialization (modsecurity_crs_10_config.conf and base_rules unmodified from CRS 2.09):

SecGuardianLog |/sbin/httpd-guardian.pl

Include /etc/apache2/modsecurity_crs_10_config.conf

SecDataDir /tmp
SecTmpDir /tmp

SecDebugLog /var/log/apache2/modsec_debug.log
SecDebugLogLevel 3

SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4\d[^4])"
SecAuditLogType Serial
SecAuditLog /var/log/apache2/modsec_audit.log
SecAuditLogParts "ABFHZ"

Include /etc/apache2/base_rules/*.conf

## A:
SecRule TX:ANOMALY_SCORE "@gt 20" "phase:2,t:none,log,auditlog,exec:/sbin/blacklist_web,drop,id:'432023',msg:'blacklisted (Total Score: %{TX.ANOMALY_SCORE}, SQLi=%{TX.SQL_INJECTION_SCORE}, XSS=%{TX.XSS_SCORE}): %{tx.msg}',setvar:tx.inbound_tx_msg=%{tx.msg},setvar:tx.inbound_anomaly_score=%{tx.anomaly_score}"

SecRule REQUEST_URI "@pm install.txt vbulletin manager.php backend korff .dll db_config roundcube rcube ftp: :// ../ /// zencart" "drop,exec:/sbin/blacklist_web,msg:'hacks 1 - blacklisted',severity:'2',id:'789073',tag:'blacklisted'"

.... etc.

Many thanks in advance for any input !