Re: [mod-security-users] Ideas for future features..

[Zach R. <ad...@li...>]

I apologize for being absent for most of the discussion. My schedule has 
been quite full lately.

I have been using a forked mod_access_rbl for about a year now. While I 
don't use it to scan every request that comes in I do use it to control 
access to two or three files that are accessed quite a bit. For these 
three files I am using seven different blacklists and I've noticed no 
drop in performance. I don't think DNS lookups are all that heavy in 
terms of resource usage when compared to PHP/MySQL being run for every 
spambot request but, using dnsrbls to deny access to an entire website 
could be fairly resource intensive.

The idea of an internal cache was mostly to save additional DNS lookups 
to the local DNS server but, it isn't necessary. The performance of the 
existing modules with no internal cache is enough that it shouldn't be a 
problem for protecting a few files but, if this was going to be deployed 
to scan every request or every argument sent to the server it could be a 
problem.

----

The issue of IPs in a firewall that I mentioned wasn't really directed 
at 5000 - 10000 IPs but, more along the lines of 40000 - 50000 IPs. It 
scales for now but, we need a better solution for the future.

---

  As a matter of fact, ModSecurity 1.8.x-dev was able to interface
  with external spam checkers. I announced it on the list (I think)
  but since no one used it I removed it prior to 1.9 final.

I believe this sort of checking needs to be internal. Accessing an 
external Perl script for example would be far too resource intensive if 
it were used to scan a very large number of incoming connections.

I can see you guys have a good handle on the situation. The future 
features of 2.0.0 look very promising with functionality similar to 
mod_evasive. If the functionality works with Frontpage too (mod_evasive 
does not) it will be all that much better.

Zach

Michael Shinn wrote:

>On Fri, 2006-02-24 at 15:53 +0000, Ivan Ristic wrote:
>  
>
>>Michael Shinn wrote:
>>    
>>
>>>attackers, owned boxes, etc.).  This will use a modified mod_access,
>>>which will allow for real time lookups of the IPs.  rsync access will
>>>also be available to the zones for secondaries, and sites that wish to
>>>use the IPs for firewalling purposes.
>>>      
>>>
>>  How long do the lookups take to complete when the information
>>  is cached in a local DNS?
>>    
>>
>
>Good question, I'll have to generate some official stats to quantify the
>performance, but I've been running a modified mod_access doing lookups
>against the spamhaus.org RBL for about a year and I've never noticed a
>measurable change in performance as a user.  In short, a local DNS (in
>my case bind) seems to do an good job of caching the lookups, in the
>same manner that a local DNS seems to do a good enough job with SMTP
>RBLs.
>
>Based on this anecdotal experience,I don't think that an internal cache
>would be necessary for mod_security to support RBL lookups.  It may not
>even be optimal in some cases, as TTLs may require one lookup to be
>retried in 1 minute, another in 3600, etc. further complicating the code
>in mod_security to age these out differently.  But, I definitely could
>see some users not being aware of the need to setup a local DNS and
>experiencing a significant performance problem and then perceiving that
>as a mod_security issue.  
>
>Regardless, a local DNS in my experience does seem to do an adequate
>job, so my vote would be to add the RBL lookup capability to
>mod_security minus the cache for the initial testing release, and if the
>performance seems suboptimal with a local DNS, then add in a cache for
>the next phase of testing.  
>
>  
>