Thread: [mod-security-users] Performance using ModSecurity with Apache 1.3.x
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
mod-security-users
|
From: Ivan R. <iv...@we...> - 2005-12-15 22:06:22
|
Some information for those of you using ModSecurity with Apache 1.3.x: I have just completed a round of performance tests. As some of you already know, the regular expression engine that comes with Apache 1.3.x is much slower than the one that comes with Apache 2.x (PCRE). When I say slower I mean *several times* slower for non-trivial requests. However, today I tried something else: I compiled ModSecurity for Apache 1.3.x against PCRE instead of the built-in regex library. I only had a very brief time to test the result but it appears that everything works well and the regex execution speed is equal to that of Apache 2.x. Chances are I will officially support compilation against PCRE in the forthcoming 1.9.2. -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: K. C. L. <li...@la...> - 2005-12-16 00:07:57
|
On Thu, 15 Dec 2005, Ivan Ristic wrote:
> However, today I tried something else: I compiled
> ModSecurity for Apache 1.3.x against PCRE instead of the
> built-in regex library. I only had a very brief time to
> test the result but it appears that everything works
> well and the regex execution speed is equal to that of
> Apache 2.x.
That sounds interesting. How does one compile Apache 1.3.x with PCRE
instead of the built-in regex engine please?
While on the subject of response time, we deployed mod_security on one of
our Apache 1.3.33 servers (PHP-4.4.1, OpenSSL-0.9.8, Apache_SSL and
mmcache-2.4.4) running Linux 2.2.26. It worked well for anything between a
few hours to a day before two, and only two, of the Apache child processes
start eating up CPU time. eg. 45% and 49%. Apache would eventually become
unresponsive and had to be restarted. Recompiling Apache without
mod_security would restore it to it's former steady running state. Any
pointers as what might be causing the high CPU consumption please?
Please see the Apache configuration section of mod_security at the end.
Regards,
Kwong Li
London
httpsd.conf mod_security section:
<IfModule mod_security.c>
SecFilterEngine DynamicOnly
SecFilterScanPOST On
SecFilterSelective HTTP_Transfer-Encoding "!^$"
SecFilterDefaultAction "deny,log,status:403"
SecFilterInheritance On
SecFilterCheckURLEncoding On
SecFilterCheckUnicodeEncoding Off
SecFilterForceByteRange 1 255
SecFilter "\.\./"
SecFilter /etc/password
SecFilter /bin/ls
SecFilter "<(.|\n)+>"
SecFilter "<[[:space:]]*script"
SecFilter "delete[[:space:]]+from"
SecFilter "insert[[:space:]]+into"
SecFilter "drop[[:space:]]table"
SecFilter "select.+from"
SecFilterSelective REQUEST_METHOD "!^(GET|HEAD)$" chain
SecFilterSelective HTTP_Content-Type "!(^application/x-www-form-urlencoded$|^multipart/form-data;)"
SecFilterSelective REQUEST_METHOD "^(GET|HEAD)$" chain
SecFilterSelective HTTP_Content-Length "!^$"
SecFilterSelective REQUEST_METHOD "^POST$" chain
SecFilterDebugLog logs/modsec_debug_log
SecFilterDebugLevel 0
SecAuditEngine RelevantOnly
SecAuditLog logs/audit_log
</IfModule>
|
|
From: Ivan R. <iv...@we...> - 2005-12-16 15:23:22
|
K. C. Li wrote: > > That sounds interesting. How does one compile Apache 1.3.x with PCRE > instead of the built-in regex engine please? That's not what I did. I simply changed ModSecurity to use PCRE directly, ignoring the regex library that comes with Apache 1.3.x. > While on the subject of response time, we deployed mod_security on one of > our Apache 1.3.33 servers (PHP-4.4.1, OpenSSL-0.9.8, Apache_SSL and > mmcache-2.4.4) running Linux 2.2.26. It worked well for anything between a > few hours to a day before two, and only two, of the Apache child processes > start eating up CPU time. eg. 45% and 49%. Apache would eventually become > unresponsive and had to be restarted. Recompiling Apache without > mod_security would restore it to it's former steady running state. Any > pointers as what might be causing the high CPU consumption please? Hmmm, yours is the first report to mention ModSecurity malfunctioning like that. (And, for the record, I don't get many bug reports either ;) Is that something that happened to you once, or is it something that happens every time you turn ModSecurity on? Also, which version of ModSecurity did you use? If you are up for it (I am) maybe you can do some of the things listed in the Apache debugging guide: http://httpd.apache.org/dev/debugging.html Starting with strace, for example. > Please see the Apache configuration section of mod_security at the end. > > ... > > SecFilterSelective REQUEST_METHOD "^POST$" chain ^ Another directive is supposed to come after this one. (It's not something that would have brought a process down, though.) -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: K. C. L. <li...@la...> - 2005-12-16 16:50:06
|
On Fri, 16 Dec 2005, Ivan Ristic wrote: > That's not what I did. I simply changed ModSecurity to > use PCRE directly, ignoring the regex library that comes > with Apache 1.3.x. Ah, I see. No wonder I couldn't see any option to select which regex engine to use in Apache or PHP. > Hmmm, yours is the first report to mention ModSecurity malfunctioning > like that. (And, for the record, I don't get many bug reports either ;) It is probably our configuration rather than mod_security that is causing the problem. It is a great idea and we couldn't wait to have it running properly. Some popular applications such as phpNuke are so full of security holes that it is a nightmare for any sysadmin. > Is that something that happened to you once, or is it something that > happens every time you turn ModSecurity on? Also, which version of We switched it on twice. The first time the high CPU usage occurred a few hours into the running. The second time the high CPU usage occurred about a day later. But when it occurs, Apache would effectively hang until restarted. > ModSecurity did you use? Sorry, I should have given that information initially. It is version 1.9.1. > If you are up for it (I am) maybe you can do some of the things > listed in the Apache debugging guide: > > http://httpd.apache.org/dev/debugging.html > > Starting with strace, for example. The Apache server is quite busy so would probably produce a large amount of strace or debugging information. I'll have a read and see what are practical to do. Thanks. > > SecFilterSelective REQUEST_METHOD "^POST$" chain > > ^ Another directive is supposed to come after this one. (It's not > something that would have brought a process down, though.) Sorry, I have omitted it by mistake on copying. It should be followed by: SecFilterSelective HTTP_Content-Length "^$" Regards, Kwong Li London |
|
From: Ivan R. <iv...@we...> - 2005-12-16 18:23:12
|
K. C. Li wrote: > >> Is that something that happened to you once, or is it something that >> happens every time you turn ModSecurity on? Also, which version of > > We switched it on twice. The first time the high CPU usage occurred a few > hours into the running. The second time the high CPU usage occurred about > a day later. But when it occurs, Apache would effectively hang until > restarted. > > ... > > The Apache server is quite busy so would probably produce a large amount > of strace or debugging information. I'll have a read and see what are > practical to do. Thanks. In my experience symptoms like the ones you reported occur when the web server experiences very high load (with or without ModSecurity). Since running ModSecurity requires additional resources it may be that it's what tipped the system over the edge so to speak. -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: K. C. L. <li...@la...> - 2005-12-16 19:24:53
|
On Fri, 16 Dec 2005, Ivan Ristic wrote: > In my experience symptoms like the ones you reported occur > when the web server experiences very high load (with or without > ModSecurity). Since running ModSecurity requires additional resources > it may be that it's what tipped the system over the edge so to speak. Our web server is indeed subjected to quite high load from time to time. I have just compiled mod_security with Apache v1.3.34 and the high CPU on one Apache instance occurred about an hour into the restart. Below is a section of ps showing that PID 29759 was continuously running: 29755 ? S 0:00 /usr/local/apache/bin/httpsd 29756 ? S 0:00 /usr/local/apache/bin/gcache 505 /usr/local/apache/logs/gcache_port 29757 ? S 1:08 /usr/local/apache/bin/httpsd 29758 ? S 0:52 /usr/local/apache/bin/httpsd 29759 ? R 7:18 /usr/local/apache/bin/httpsd 29760 ? S 0:49 /usr/local/apache/bin/httpsd 29761 ? S 0:56 /usr/local/apache/bin/httpsd 29762 ? S 1:04 /usr/local/apache/bin/httpsd 29763 ? S 1:33 /usr/local/apache/bin/httpsd 29764 ? S 0:43 /usr/local/apache/bin/httpsd 29765 ? S 1:43 /usr/local/apache/bin/httpsd 29766 ? S 0:50 /usr/local/apache/bin/httpsd 29768 ? S 1:21 /usr/local/apache/bin/httpsd 29771 ? S 1:34 /usr/local/apache/bin/httpsd ruby:~# /usr/local/apache/bin/httpsd -l Compiled-in modules: http_core.c mod_env.c mod_log_config.c mod_mime.c mod_negotiation.c mod_status.c mod_include.c mod_autoindex.c mod_dir.c mod_cgi.c mod_asis.c mod_imap.c mod_actions.c mod_userdir.c mod_alias.c mod_rewrite.c mod_access.c mod_auth.c mod_setenvif.c apache_ssl.c mod_php4.c mod_security.c suexec: enabled; valid wrapper /usr/local/apache/bin/suexec Apart from reducing the high load, are there anything that could be done to rectify the problem? Regards, Kwong Li London |
|
From: Zach R. <ad...@li...> - 2005-12-16 02:31:34
|
This sounds like a very good idea. Some of us that use mod_security do so with cPanel, Plesk, or other commercial control panels in shared hosting environments and cannot switch to Apache 2 since it is not supported. Keep up the great work Ivan. :) Zach Ivan Ristic wrote: >Some information for those of you using ModSecurity with >Apache 1.3.x: > >I have just completed a round of performance tests. As some of >you already know, the regular expression engine that comes >with Apache 1.3.x is much slower than the one that comes with >Apache 2.x (PCRE). When I say slower I mean *several times* >slower for non-trivial requests. > >However, today I tried something else: I compiled >ModSecurity for Apache 1.3.x against PCRE instead of the >built-in regex library. I only had a very brief time to >test the result but it appears that everything works >well and the regex execution speed is equal to that of >Apache 2.x. > >Chances are I will officially support compilation against >PCRE in the forthcoming 1.9.2. > > > |
|
From: Justin G. <web...@sw...> - 2005-12-16 08:13:35
|
I'm on h-sphere, same problem, specially on busy servers. Using gotroot's rules was always a pain, specially their blacklists and badips, if I turn these on, server skyrockets :) Ivan, please keep us posted on your findings. thanks, Justin Zach Roberts wrote: > This sounds like a very good idea. Some of us that use mod_security do > so with cPanel, Plesk, or other commercial control panels in shared > hosting environments and cannot switch to Apache 2 since it is not > supported. > > Keep up the great work Ivan. :) > > Zach > > Ivan Ristic wrote: > >> Some information for those of you using ModSecurity with >> Apache 1.3.x: >> >> I have just completed a round of performance tests. As some of >> you already know, the regular expression engine that comes >> with Apache 1.3.x is much slower than the one that comes with >> Apache 2.x (PCRE). When I say slower I mean *several times* >> slower for non-trivial requests. >> >> However, today I tried something else: I compiled >> ModSecurity for Apache 1.3.x against PCRE instead of the >> built-in regex library. I only had a very brief time to >> test the result but it appears that everything works >> well and the regex execution speed is equal to that of >> Apache 2.x. >> >> Chances are I will officially support compilation against >> PCRE in the forthcoming 1.9.2. >> >> >> > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users |
|
From: Michael S. <mi...@go...> - 2005-12-18 19:10:47
|
Zach Roberts wrote: > This sounds like a very good idea. Some of us that use mod_security do > so with cPanel, Plesk, or other commercial control panels in shared > hosting environments and cannot switch to Apache 2 since it is not > supported. Apache 2.x is supported by Plesk (and has been for years). > > Keep up the great work Ivan. :) > > Zach > > Ivan Ristic wrote: > >> Some information for those of you using ModSecurity with >> Apache 1.3.x: >> >> I have just completed a round of performance tests. As some of >> you already know, the regular expression engine that comes >> with Apache 1.3.x is much slower than the one that comes with >> Apache 2.x (PCRE). When I say slower I mean *several times* >> slower for non-trivial requests. >> >> However, today I tried something else: I compiled >> ModSecurity for Apache 1.3.x against PCRE instead of the >> built-in regex library. I only had a very brief time to >> test the result but it appears that everything works >> well and the regex execution speed is equal to that of >> Apache 2.x. >> >> Chances are I will officially support compilation against >> PCRE in the forthcoming 1.9.2. >> >> >> > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users |
|
From: Zach R. <ad...@li...> - 2005-12-19 08:32:31
|
I've been known to be wrong about Plesk from time to time since I don't work with it much. At any rate for those of us that use Apache 1.3 for one reason or another would benefit from this. Any ETA on a release or at least a beta? Zach Michael Shinn wrote: > Zach Roberts wrote: > >> This sounds like a very good idea. Some of us that use mod_security >> do so with cPanel, Plesk, or other commercial control panels in >> shared hosting environments and cannot switch to Apache 2 since it is >> not supported. > > > > Apache 2.x is supported by Plesk (and has been for years). > >> >> Keep up the great work Ivan. :) >> >> Zach >> >> Ivan Ristic wrote: >> >>> Some information for those of you using ModSecurity with >>> Apache 1.3.x: >>> >>> I have just completed a round of performance tests. As some of >>> you already know, the regular expression engine that comes >>> with Apache 1.3.x is much slower than the one that comes with >>> Apache 2.x (PCRE). When I say slower I mean *several times* >>> slower for non-trivial requests. >>> >>> However, today I tried something else: I compiled >>> ModSecurity for Apache 1.3.x against PCRE instead of the >>> built-in regex library. I only had a very brief time to >>> test the result but it appears that everything works >>> well and the regex execution speed is equal to that of >>> Apache 2.x. >>> >>> Chances are I will officially support compilation against >>> PCRE in the forthcoming 1.9.2. >>> >>> >>> >> >> >> >> ------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. Do you grep through >> log files >> for problems? Stop! Download the new AJAX search engine that makes >> searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! >> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click >> _______________________________________________ >> mod-security-users mailing list >> mod...@li... >> https://lists.sourceforge.net/lists/listinfo/mod-security-users > > > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users |
|
From: Ivan R. <iv...@we...> - 2005-12-19 16:52:29
|
Zach Roberts wrote: > I've been known to be wrong about Plesk from time to time since I don't > work with it much. > > At any rate for those of us that use Apache 1.3 for one reason or > another would benefit from this. > > Any ETA on a release or at least a beta? I've made the changes. I will release 1.9.2-rc1 either tonight or tomorrow morning. I am hoping you will give me feedback on the PCRE compilation prior to final 1.9.2. -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: Zach R. <ad...@li...> - 2005-12-19 22:16:40
|
I would definitely let you know if there are any problems. Zach Ivan Ristic wrote: >Zach Roberts wrote: > > >>I've been known to be wrong about Plesk from time to time since I don't >>work with it much. >> >>At any rate for those of us that use Apache 1.3 for one reason or >>another would benefit from this. >> >>Any ETA on a release or at least a beta? >> >> > > I've made the changes. I will release 1.9.2-rc1 either tonight or > tomorrow morning. I am hoping you will give me feedback on the > PCRE compilation prior to final 1.9.2. > > > |
|
From: Ivan R. <iv...@we...> - 2005-12-20 12:20:15
|
I have released 1.9.2-rc1: http://www.modsecurity.org/download/modsecurity-apache-1.9.2-rc1.tar.gz The instructions how to compile against PCRE are in the manual. It's pretty straightforward. Everything works as expected on my systems. Please give it a go on yours. This release also introduces the DISABLE_SUEXEC switch. Again, it appears to work fine here. Those of you who have complained about suEXEC please test it :) -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: Justin G. <web...@sw...> - 2005-12-20 14:31:24
|
hi, I will install it on a quite busy server tonight. Have you checked performance, specially when using gotroot's rules, when compiled against PCRE? Hope we'll get apache2 performance :) About the upload script - Is it possible to return a predefined page explaining for example a virus was found in the attachment, instead of the general 500 error? thanks, Justin ps - This mailing list has a weird setup - When replying a message, the default is to reply to the original poster, not the list. I always have to change the To: address manually. Please see if it can be changed. Ivan Ristic wrote: > I have released 1.9.2-rc1: > > http://www.modsecurity.org/download/modsecurity-apache-1.9.2-rc1.tar.gz > > The instructions how to compile against PCRE are in > the manual. It's pretty straightforward. Everything works > as expected on my systems. Please give it a go on yours. > > This release also introduces the DISABLE_SUEXEC switch. Again, > it appears to work fine here. Those of you who have complained > about suEXEC please test it :) > |
|
From: Ivan R. <iv...@we...> - 2005-12-20 15:01:27
|
Justin Grindea wrote: > hi, > > I will install it on a quite busy server tonight. > Have you checked performance, specially when using gotroot's rules, when > compiled against PCRE? > Hope we'll get apache2 performance :) Yes, you should. > About the upload script - Is it possible to return a predefined page > explaining for example a virus > was found in the attachment, instead of the general 500 error? No. But you can use some other (rarely used) error code with the ErrorDocument directive to achieve a similar effect. -- Ivan Ristic Apache Security (O'Reilly) - http://www.apachesecurity.net Open source web application firewall - http://www.modsecurity.org |
|
From: Steffen <in...@ap...> - 2005-12-22 10:49:47
|
Is anyone using Chrooting under Windows ? I set the SecChrootDir to all kind of paths, even not a Apache path. But still no indication it is working. Also in the Debug log, no indication that Chrooting is doing some. The docs says "Only the root user can escape the jail", when I understand that, when you run Apache under an Administrator account then it does not work. Maybe I have to run Apache under a user account. Afterall I am not sure if it is running with Windows and maybe it is a Linux thingy. Steffen http://www.apachelounge.com |
|
From: Ivan R. <iv...@we...> - 2005-12-22 11:55:07
|
On 12/22/05, Steffen <in...@ap...> wrote: > Is anyone using Chrooting under Windows ? > > ... > > Afterall I am not sure if it is running with Windows and maybe it is a Li= nux > thingy. Windows does not support the chroot facility. It's an UNIX-only thing. Ivan |
Thanks for helping keep SourceForge clean.
X