Re: [mod-security-users] v1.9 memory usage problem
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
Re: [mod-security-users] v1.9 memory usage problem
|
From: Jim <st...@cl...> - 2006-04-23 11:33:57
|
Hello Ivan,
I had planned to begin troubleshooting properly tomorrow but while I was here I started some things and thought I'd post back as the early results are interesting.
I upgraded to mod_sec 1.9.3 again and first tried the suggestion of adding the 'DynamicOnly' and no post buffer settings and got the same problem.
Next, I started with the removal of all rules and adding them back block by block to see if this turns anything up. I removed everything and got the same problem within 10 minutes of the apache restart.
mod_sec config:
root@xxxx [~]# cat /etc/httpd/conf/mod_security.conf
# Turn the filtering engine On or Off
SecFilterEngine DynamicOnly
# Make sure that URL encoding is valid
SecFilterCheckURLEncoding Off
# Unicode encoding check
SecFilterCheckUnicodeEncoding Off
# Only allow bytes from this range
SecFilterForceByteRange 0 255
# Only log suspicious requests
SecAuditEngine RelevantOnly
# The name of the audit log file
SecAuditLog /var/log/httpd/audit_log
# Debug level set to a minimum
SecFilterDebugLog /var/log/httpd/modsec_debug_log
SecFilterDebugLevel 0
# Should mod_security inspect POST payloads
SecFilterScanPOST On
# By default log and deny suspicious requests
# with HTTP status 500
SecFilterDefaultAction "deny,log,status:406"
# no ban to localhost
SecFilterSelective REMOTE_ADDR "^127.0.0.1$" nolog,allow
# suggestion by mod_sec devs
SetEnvIfNoCase Content-Type "^multipart/form-data;" \
"MODSEC_NOPOSTBUFFERING=Do not buffer file uploads"
SecFilterSelective THE_REQUEST "_vti_bin" allow,nolog
SecFilterSelective THE_REQUEST "/fpsrvadm\.exe" pass
SecFilterSelective THE_REQUEST "/fpremadm\.exe" pass
SecFilterSelective THE_REQUEST "/admisapi/fpadmin\.htm" pass
SecFilterSelective THE_REQUEST "/scripts/Fpadmcgi\.exe" pass
SecFilterSelective THE_REQUEST "/_private/orders\.txt" pass
SecFilterSelective THE_REQUEST "/_private/form_results\.txt" pass
SecFilterSelective THE_REQUEST "/_private/registrations\.htm" pass
SecFilterSelective THE_REQUEST "/cfgwiz\.exe" pass
SecFilterSelective THE_REQUEST "/authors\.pwd" pass
SecFilterSelective THE_REQUEST "/_vti_bin/_vti_aut/author\.exe" pass
SecFilterSelective THE_REQUEST "/administrators\.pwd" pass
SecFilterSelective THE_REQUEST "/_private/form_results\.htm" pass
SecFilterSelective THE_REQUEST "/_vti_pvt/access\.cnf" pass
SecFilterSelective THE_REQUEST "/_private/register\.txt" pass
SecFilterSelective THE_REQUEST "/_private/registrations\.txt" pass
SecFilterSelective THE_REQUEST "/_vti_pvt/service\.cnf" pass
SecFilterSelective THE_REQUEST "/service\.pwd" pass
SecFilterSelective THE_REQUEST "/_vti_pvt/service\.stp" pass
SecFilterSelective THE_REQUEST "/_vti_pvt/services\.cnf" pass
SecFilterSelective THE_REQUEST "/_vti_bin/shtml\.exe" pass
SecFilterSelective THE_REQUEST "/_vti_pvt/svcacl\.cnf" pass
SecFilterSelective THE_REQUEST "/users\.pwd" pass
SecFilterSelective THE_REQUEST "/_vti_pvt/writeto\.cnf" pass
SecFilterSelective THE_REQUEST "/dvwssr\.dll" pass
SecFilterSelective THE_REQUEST "/_private/register\.htm" pass
SecFilterSelective THE_REQUEST "/_vti_bin/" pass
httpd processes in top when problem starts (see highlighted process which is the parent httpd process previously mentioned):
root@xxxx [~]# top -b | grep httpd
2088 nobody 15 0 24756 24M 5352 S 0.4 1.2 0:00 0 httpd
1479 root 15 0 24072 23M 4944 S 0.0 1.1 0:01 0 httpd
1497 nobody 15 0 28128 27M 5488 S 0.0 1.3 0:01 1 httpd
1498 nobody 15 0 31420 30M 5664 S 0.0 1.5 0:02 0 httpd
1499 nobody 15 0 28688 28M 5668 S 0.0 1.3 0:01 0 httpd
1500 nobody 15 0 29768 29M 5652 S 0.0 1.4 0:01 0 httpd
1501 nobody 15 0 34308 33M 5676 S 0.0 1.6 0:01 1 httpd
1502 nobody 15 0 29076 28M 5676 S 0.0 1.4 0:02 0 httpd
1503 nobody 15 0 31460 30M 5664 S 0.0 1.5 0:02 1 httpd
********
1504 nobody 15 0 1052M 1.0G 5492 S 0.0 52.4 0:03 1 httpd
********
1505 nobody 15 0 31636 30M 5668 S 0.0 1.5 0:02 0 httpd
1506 nobody 15 0 37040 36M 5464 S 0.0 1.8 0:01 0 httpd
2086 nobody 15 0 27888 27M 5412 S 0.0 1.3 0:00 0 httpd
2087 nobody 15 0 24684 24M 5320 S 0.0 1.2 0:00 0 httpd
2104 nobody 16 0 24628 24M 5316 S 0.0 1.1 0:00 1 httpd
Output of free:
root@xxxx [~]# free -m
total used free shared buffers cached
Mem: 2006 1983 23 0 61 515
-/+ buffers/cache: 1406 600
Swap: 4094 148 3945
================================================
================================================
Next, I kept things *exactly* the same except I changed one line of the mod_sec config:
root@xxxx [~]# grep SecFilterScanPOST /etc/httpd/conf/mod_security.conf
SecFilterScanPOST Off
I sat and watched top for 30-40 minutes (the big httpd process has usually reared its head by this time) and all is fine:
root@xxxx [~]# top -b | grep httpd
2322 nobody 15 0 35492 34M 5496 S 1.4 1.7 0:02 1 httpd
2325 nobody 15 0 36964 36M 5496 S 0.9 1.7 0:01 0 httpd
2330 nobody 15 0 36112 35M 5504 S 0.9 1.7 0:03 0 httpd
2327 nobody 15 0 37252 36M 5672 S 0.4 1.8 0:02 1 httpd
2329 nobody 15 0 29296 28M 5504 S 0.4 1.4 0:02 1 httpd
2311 root 15 0 24076 23M 4944 S 0.0 1.1 0:01 0 httpd
2321 nobody 15 0 31444 30M 5496 S 0.0 1.5 0:02 0 httpd
2323 nobody 15 0 31592 30M 5664 S 0.0 1.5 0:02 0 httpd
2324 nobody 15 0 30424 29M 5508 S 0.0 1.4 0:02 1 httpd
2326 nobody 15 0 35908 35M 5504 S 0.0 1.7 0:03 1 httpd
2328 nobody 15 0 37556 36M 5492 S 0.0 1.8 0:03 1 httpd
root@xxxx [~]# free -m
total used free shared buffers cached
Mem: 2006 1125 881 0 92 603
-/+ buffers/cache: 428 1577
Swap: 4094 151 3943
================================================
================================================
Finally, I rolled back to version 1.8.7 (this is a live server and I can't leave it on 1.9) and put our full ruleset back into place (including SecFilterScanPOST On). As it has been doing for months on this version, everything is still fine.
root@xxxx [~]# top -b | grep httpd
4321 nobody 15 0 36732 35M 5620 S 0.9 1.7 0:02 0 httpd
4325 nobody 15 0 30792 30M 5636 S 0.9 1.4 0:03 0 httpd
4327 nobody 15 0 54432 53M 5628 S 0.9 2.6 0:10 1 httpd
4328 nobody 16 0 26240 25M 5428 S 0.9 1.2 0:01 1 httpd
4315 nobody 15 0 29888 29M 5464 S 0.4 1.4 0:01 1 httpd
4317 nobody 15 0 27272 26M 5444 S 0.4 1.3 0:01 1 httpd
4308 root 15 0 24552 23M 4928 S 0.0 1.1 0:01 0 httpd
4318 nobody 15 0 30404 29M 5460 S 0.0 1.4 0:02 1 httpd
4319 nobody 15 0 29752 29M 5480 S 0.0 1.4 0:02 0 httpd
4322 nobody 15 0 36556 35M 5444 S 0.0 1.7 0:02 0 httpd
4326 nobody 15 0 28120 27M 5448 S 0.0 1.3 0:01 0 httpd
4357 nobody 15 0 30624 29M 5444 S 0.0 1.4 0:02 1 httpd
4537 nobody 15 0 27656 27M 5364 S 0.0 1.3 0:00 0 httpd
4538 nobody 15 0 25100 24M 5196 S 0.0 1.2 0:00 0 httpd
4539 nobody 15 0 25216 24M 5344 S 0.0 1.2 0:00 1 httpd
root@xxxx [/var/log/httpd]# free -m
total used free shared buffers cached
Mem: 2006 1167 838 0 98 654
-/+ buffers/cache: 415 1590
Swap: 4094 151 3943
================================================
================================================
One thing I forgot to mention earlier which I don't think should make a difference but not sure is that we're using the -DDISABLE_HTACCESS_CONFIG flag on our mod_security compiles to disable the addition/alteration of rulesets via user htaccess files.
|
