[mod-security-users] v1.9 memory usage problem

[Jim <st...@cl...>]

Hi all,

First time post here and newsgroup newbie in general so I hope I do this correctly :)

-----

Distro: CentOS 3.6 (cPanel control panel installed)
RAM: 2 GB
Swap: 4 GB

-----

root@xxxx [/usr/src]# uname -a
Linux xxxx.xxxx.xxxx 2.4.21-40.ELsmp #1 SMP Wed Mar 15 14:21:45 EST 2006 i686 i686 i386 GNU/Linux

-----

Apache (v1.3.34) loaded modules:

Compiled-in modules:
  http_core.c
  mod_env.c
  mod_log_config.c
  mod_mime.c
  mod_negotiation.c
  mod_status.c
  mod_include.c
  mod_autoindex.c
  mod_dir.c
  mod_cgi.c
  mod_asis.c
  mod_imap.c
  mod_actions.c
  mod_userdir.c
  mod_alias.c
  mod_access.c
  mod_auth.c
  mod_so.c
  mod_setenvif.c
  mod_ssl.c
  mod_frontpage.c
suexec: enabled; valid wrapper /usr/local/apache/bin/suexec

-----

PHP (v4.4.2) compile line:

'./configure' '--with-apxs=/usr/local/apache/bin/apxs' '--with-xml' '--enable-bcmath' '--enable-calendar' '--with-curl' '--with-dom' '--with-dom-xslt' '--with-dom-exslt' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr' '--with-xpm-dir=/usr/X11R6' '--with-imap' '--with-imap-ssl' '--with-kerberos' '--enable-mbstring' '--enable-mbstr-enc-trans' '--enable-mbregex' '--with-mcrypt' '--with-ming=../ming-0.2a' '--enable-magic-quotes' '--with-mysqli' '--with-mysql=/usr' '--with-openssl' '--enable-discard-path' '--with-pear' '--enable-sockets' '--enable-track-vars' '--with-ttf' '--with-freetype-dir=/usr' '--enable-gd-native-ttf' '--enable-versioning' '--with-xmlrpc' '--with-zlib' '--enable-xslt' '--with-xslt-sablot' '--with-sablot-js=/usr'

-----


The issue I am posting about has previously been discussed here: http://www.gotroot.com/tiki-view_forum_thread.php?forumId=35&comments_threshold=0&comments_parentId=658&comments_offset=0&comments_maxComments=20&comments_style=commentStyle_threaded

The problem we (and others from the look of that thread) have seen is that mod_security is causing used RAM to go extremely high resulting in swapping.

We first saw this when upgrading from mod_security v1.8.7 to version 1.9.1.  After being unable to find the cause, we rolled back to 1.8 and all has been fine.  Recently, the upgrade has been tried again (this time to v1.9.3) with exactly the same problem.

This only happens on 2 of our servers out of about 45-50 which is very strange.  When Apache is started things are fine with each httpd process showing around 20-30 MB usage in 'top'.  Within approx 15 minutes, one httpd process (a parent) can then be seen to be using 1.0-1.1 GB of RAM in 'top'.  Soon after, things go downhill.

I have been using trial and error to try and find the cause of this but it's definitely related to mod_security.  When no rules are in use (but module still loaded in httpd.conf) there is no problem.  In fact, I have narrowed it down to the 'SecFilterScanPOST' setting - when this is enabled the problem occurs, when disabled/Off there is no problem.

Our ruleset is a custom ruleset comprising of around 500-600 rules.  We are continuing to troubleshoot this to see if we can find common factors between the very few servers affected.

This post is mainly to see if anyone has experienced similar problems with the 1.9 branch of mod_security?  Maybe the data in this post will be of use to anyone troubleshooting the same issue.

If any additional info is needed (or suggestions to try and post back), feel free to ask.

Thanks,

Jim