Yes.. i will need! I will contact you.
Thanks

Breno

On Thu, Sep 8, 2011 at 9:57 AM, Michael Haas <michael.haas10@gmail.com> wrote:
Hi,

that's great, if you need assistance in testing some new versions or
the old, i can help with this.

Thanks
Michael

2011/9/8 Breno Silva <breno.silva@gmail.com>:
> Hey Michel,
>
> I'm going to change it for 2.6.3 version. So i can investigate it sooner.
>
> Thanks
>
> Breno
>
> On Thu, Sep 8, 2011 at 9:40 AM, Michael Haas <michael.haas10@gmail.com>
> wrote:
>>
>> Found also this bug
>> https://www.modsecurity.org/tracker/browse/MODSEC-160 seems that i
>> need to disable the session rules because the file size is only
>> increasing.
>> Is there a schedule for the availability of 2.7.0.
>>
>> Thanks
>> Michael
>>
>> 2011/9/8 Michael Haas <michael.haas10@gmail.com>:
>> > Hello,
>> >
>> > the size is also a problem for the session collection, i need for the
>> > persistence of 15000 sessions around 1GB Storage (around 64k per
>> > session).
>> > Is it possible to reduce this or is this really needed?
>> > I use a modified version of modsecurity_crs_16_session_hijacking.conf
>> > from crs 2.2.1 where i also save the data to a backend and there the
>> > size is not that big.
>> >
>> > Thanks in advance
>> > Michael
>> >
>> >
>> > 2011/9/7 Breno Silva <breno.silva@gmail.com>:
>> >> Hi Kevin,
>> >>
>> >> We are planning to implement  a DDoS learning algorithm for
>> >> ModSecurity. I
>> >> know using data collection in heavy systems can cause performance
>> >> issues.
>> >> Most of Apache modules which works for  bandwidth limitation also
>> >> stores
>> >> data into temporary files, and it will also cause performance problems.
>> >> So
>> >> we need some code to store network information only in memory, persist
>> >> only
>> >> the learned data for recovery. It will also requires a time-to-time
>> >> cleanup
>> >> of internal modsecurity data structs. So i think in the future we will
>> >> have
>> >> a good solution.
>> >>
>> >> Until we do not have a better solution, maybe you could try limit per
>> >> ip
>> >> connections in a another network device (firewall, sw) ?
>> >>
>> >> Thanks
>> >>
>> >> Breno
>> >>
>> >> On Wed, Sep 7, 2011 at 10:42 AM, Kevin Jackson
>> >> <kevin@linuxservices.co.uk>
>> >> wrote:
>> >>>
>> >>> Dear all,
>> >>> It's been a while since I posted this issue and I want to share with
>> >>> you some of my experiences as I'm seeing issues when the IP collection
>> >>> grows in size.
>> >>>
>> >>> My setup is Apache 2.2.20, ModSecurity 2.6.1 and CRS 2.2.1.
>> >>> After a lot of performance testing and configuring ModSecurity to do 1
>> >>> job (protect against a DoS against the web servers [which in turn
>> >>> would protect against the real assets that ModSecurity setup is
>> >>> proxying for]) on a "2CPU", 7Gb box (in EC2, m1.large) I get somewhere
>> >>> between 200 and 300 hits per second against a static resource served
>> >>> from Apache.  Not great, but acceptable.  What isn't acceptable is the
>> >>> wildly fluctuating figures I get and load hitting 200+ when the IP
>> >>> data collection grows in size.
>> >>>
>> >>> My performance tests are replaying existing logs with real IP and URIs
>> >>> in, but I use the IP to set an X-Forwarded-For header (so ModSec sees
>> >>> it as a new IP), it uses the hash of the UA to give a level of
>> >>> uniqueness to the IP as per the CRS configs and I rewrite all URLs to
>> >>> a static file (to ensure I'm not running stuff against live in any
>> >>> shape or form).
>> >>>
>> >>> All is good at the beginning of the run and my IP collection database
>> >>> is at 0Mb.  I get around 300 hits per second at a load avg of around
>> >>> 0.4.  Over a short period of time (at the rate mentioned above), this
>> >>> IP collection builds up and when it hits a certain size, its game
>> >>> over.  High load starts around 5-10 minutes.  Leaving this running for
>> >>> 20 mins sees load shoot up to 200+.  This isn't a gradual increase, it
>> >>> hits a point and fails.  This seems to be file size as interrogating
>> >>> the SDBM file lists for the IP collections it is a small figure.  For
>> >>> example, I have a 700Mb SDBM file and inside are only 28 IP
>> >>> collections.
>> >>>
>> >>> I've played with TIMEOUT values to see if keeping less is helpful and
>> >>> won't affect things too much, but ideally for DoS protection I'd hope
>> >>> that keeping the IP in the database for as long as possible will be
>> >>> more beneficial.
>> >>>
>> >>> Any help is appreciated - as a potential mitigation step against a
>> >>> Layer 7 DDoS (network and other good stuff in place to protect against
>> >>> this aside) - what can I do from here?
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Kev
>> >>>
>> >>> On 19 August 2011 13:37, Ryan Barnett <RBarnett@trustwave.com> wrote:
>> >>> >
>> >>> > On 8/19/11 6:55 AM, "Kevin Jackson" <kevin@linuxservices.co.uk>
>> >>> > wrote:
>> >>> >
>> >>> >>Dear all,
>> >>> >>After a number of weeks fine tuning the CRS rules appropriate for a
>> >>> >>busy website, we did a full live test pushing traffic through Apache
>> >>> >>with ModSecurity 2.6.1 proxying requests to evaluate the performance
>> >>> >>of ModSecurity through the backend service.
>> >>> >>
>> >>> >>Traffic is around 1k/second and we had Apache running on 18 servers
>> >>> >>sitting behind a load balancer and Apache coping without issue.
>> >>> >>
>> >>> >>ModSecurity on the other hand didn't fare so well - it struggled to
>> >>> >>keep pace causing resource deadlock issues on the ip database.  The
>> >>> >>upshot was incredibly high load averages for obvious reasons and
>> >>> >> poor
>> >>> >>user experience.
>> >>> >>
>> >>> >>--ba66410d-H--
>> >>> >>Message: Failed to access DBM file "/usr/local/apache/data/ip":
>> >>> >>Resource deadlock avoided
>> >>> >>Apache-Handler: proxy-server
>> >>> >>Stopwatch: 1313660329409820 283469 (- - -)
>> >>> >>Stopwatch2: 1313660329409820 283469; combined=99526, p1=3446,
>> >>> >>p2=12838, p3=3, p4=0, p5=41666, sr=3272, sw=41573, l=0, gc=0
>> >>> >>Producer: ModSecurity for Apache/2.6.1
>> >>> >> (http://www.modsecurity.org/);
>> >>> >>core ruleset/2.2.1.
>> >>> >>Server: Apache
>> >>> >>
>> >>> >>--ba66410d-Z--
>> >>> >>
>> >>> >>My question is what to suggest here? People must be running
>> >>> >>ModSecurity on high volume websites? Have you encountered this
>> >>> >> issue?
>> >>> >>This traffic is a fraction of our peak traffic at a quiet time
>> >>> >> during
>> >>> >>the day and the environment only coped for about 10 minutes before
>> >>> >>keeling over.
>> >>> >
>> >>> > Hey Kevin,
>> >>> > Question for you - are you using the IP persistent collection to
>> >>> > track
>> >>> > data about clients across requests?  If not, then you may want to
>> >>> > disable
>> >>> > the initcol rules at the end of the modsecurity_crs_10_config.conf
>> >>> > file.
>> >>> >
>> >>> > See if that helps.
>> >>> >
>> >>> > -Ryan
>> >>> >
>> >>> >>
>> >>> >>Cheers,
>> >>> >>
>> >>> >>Kev
>> >>> >>
>> >>>
>> >>> >>
>> >>> >> >> >>--------------------------------------------------------------------------
>> >>> >>----
>> >>> >>Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>> >>> >>user administration capabilities and model configuration. Take
>> >>> >>the hassle out of deploying and managing Subversion and the
>> >>> >>tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
>> >>> >>_______________________________________________
>> >>> >>mod-security-users mailing list
>> >>> >>mod-security-users@lists.sourceforge.net
>> >>> >>https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>> >>ModSecurity Services from Trustwave's SpiderLabs:
>> >>> >>https://www.trustwave.com/application-security.php
>> >>> >>
>> >>> >
>> >>> >
>> >>> > This transmission may contain information that is privileged,
>> >>> > confidential, and/or exempt from disclosure under applicable law. If
>> >>> > you are
>> >>> > not the intended recipient, you are hereby notified that any
>> >>> > disclosure,
>> >>> > copying, distribution, or use of the information contained herein
>> >>> > (including
>> >>> > any reliance thereon) is STRICTLY PROHIBITED. If you received this
>> >>> > transmission in error, please immediately contact the sender and
>> >>> > destroy the
>> >>> > material in its entirety, whether in electronic or hard copy format.
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> ------------------------------------------------------------------------------
>> >>> Using storage to extend the benefits of virtualization and iSCSI
>> >>> Virtualization increases hardware utilization and delivers a new level
>> >>> of
>> >>> agility. Learn what those decisions are and how to modernize your
>> >>> storage
>> >>> and backup environments for virtualization.
>> >>> http://www.accelacomm.com/jaw/sfnl/114/51434361/
>> >>> _______________________________________________
>> >>> mod-security-users mailing list
>> >>> mod-security-users@lists.sourceforge.net
>> >>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >>> ModSecurity Services from Trustwave's SpiderLabs:
>> >>> https://www.trustwave.com/application-security.php
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Using storage to extend the benefits of virtualization and iSCSI
>> >> Virtualization increases hardware utilization and delivers a new level
>> >> of
>> >> agility. Learn what those decisions are and how to modernize your
>> >> storage
>> >> and backup environments for virtualization.
>> >> http://www.accelacomm.com/jaw/sfnl/114/51434361/
>> >> _______________________________________________
>> >> mod-security-users mailing list
>> >> mod-security-users@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> >> ModSecurity Services from Trustwave's SpiderLabs:
>> >> https://www.trustwave.com/application-security.php
>> >>
>> >>
>> >
>
>