MantisBT

Without disclosing details about our instance, let's check we're up to
date in terms of patches applied.

See for instance http://www.securityfocus.com/bid/38857/info

Robert

On Sat, Sep 29, 2012 at 9:51 AM, Damien Regad
<damien.regad@...> wrote:
> FYI I got an alert at 07:13 CET that the site was down.
>
> In the linode console, I saw that the server had been rebooted twice earlier in
> the night (at 02:32 and 02:33), and was listed as up and running although there
> seemed to be no activity (CPU and traffic graphs flat).
>
> Since I could not connect by SSH, I checked on the console and saw this -
>
> [linode137251@... lish]
>  [<c0103fa5>] ? pte_pfn_to_mfn+0xb5/0xd0
>  [<c019e111>] ? unmap_page_range+0x111/0x190
>  [<c019e2bb>] ? unmap_vmas+0x12b/0x1e0
>  [<c01a0481>] ? exit_mmap+0x91/0x140
>  [<c01324eb>] ? mmput+0x2b/0xc0
>  [<c0135ddf>] ? exit_mm+0xef/0x120
>  [<c068fb50>] ? _raw_spin_lock_irq+0x10/0x20
>  [<c01379d5>] ? do_exit+0x125/0x350
>  [<c0137c3c>] ? do_group_exit+0x3c/0xa0
>  [<c0142b17>] ? get_signal_to_deliver+0x1a7/0x3d0
>  [<c012f9f7>] ? finish_task_switch+0x47/0xb0
>  [<c0108cbf>] ? do_signal+0x8f/0x150
>  [<c01062c4>] ? check_events+0x8/0xc
>  [<c0105ae7>] ? xen_force_evtchn_callback+0x17/0x30
>  [<c01062c4>] ? check_events+0x8/0xc
>  [<c068faf5>] ? _raw_spin_lock+0x5/0x10
>  [<c01ebc20>] ? locks_delete_block+0x40/0x50
>  [<c0105ae7>] ? xen_force_evtchn_callback+0x17/0x30
>  [<c01062c4>] ? check_events+0x8/0xc
>  [<c01062bb>] ? xen_restore_fl_direct_reloc+0x4/0x4
>  [<c01af1d1>] ? kmem_cache_free+0x41/0x60
>  [<c0108dc8>] ? do_notify_resume+0x48/0x50
>  [<c068ff12>] ? work_notifysig+0x9/0xf
>  [<c0680000>] ? sctp_rcv_ootb+0x50/0xf0
>
> So I rebooted again, and now the situation seems normal. Site is up since 08:30
> (downtime 1 hr 17 min 29 secs).
>
> I have no idea what happened, but have no time to do any more detailed
> investigation at the moment.
>
> Damien
>
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev



-- 
Sent from my (old) computer

that's a 1 year old xen kernel bug

Paul

On Sat, Sep 29, 2012 at 8:38 AM, Robert Munteanu
<robert.munteanu@...:

> Without disclosing details about our instance, let's check we're up to
> date in terms of patches applied.
>
> See for instance http://www.securityfocus.com/bid/38857/info
>
> Robert
>
> On Sat, Sep 29, 2012 at 9:51 AM, Damien Regad
> <damien.regad@...> wrote:
> > FYI I got an alert at 07:13 CET that the site was down.
> >
> > In the linode console, I saw that the server had been rebooted twice
> earlier in
> > the night (at 02:32 and 02:33), and was listed as up and running
> although there
> > seemed to be no activity (CPU and traffic graphs flat).
> >
> > Since I could not connect by SSH, I checked on the console and saw this -
> >
> > [linode137251@... lish]
> >  [<c0103fa5>] ? pte_pfn_to_mfn+0xb5/0xd0
> >  [<c019e111>] ? unmap_page_range+0x111/0x190
> >  [<c019e2bb>] ? unmap_vmas+0x12b/0x1e0
> >  [<c01a0481>] ? exit_mmap+0x91/0x140
> >  [<c01324eb>] ? mmput+0x2b/0xc0
> >  [<c0135ddf>] ? exit_mm+0xef/0x120
> >  [<c068fb50>] ? _raw_spin_lock_irq+0x10/0x20
> >  [<c01379d5>] ? do_exit+0x125/0x350
> >  [<c0137c3c>] ? do_group_exit+0x3c/0xa0
> >  [<c0142b17>] ? get_signal_to_deliver+0x1a7/0x3d0
> >  [<c012f9f7>] ? finish_task_switch+0x47/0xb0
> >  [<c0108cbf>] ? do_signal+0x8f/0x150
> >  [<c01062c4>] ? check_events+0x8/0xc
> >  [<c0105ae7>] ? xen_force_evtchn_callback+0x17/0x30
> >  [<c01062c4>] ? check_events+0x8/0xc
> >  [<c068faf5>] ? _raw_spin_lock+0x5/0x10
> >  [<c01ebc20>] ? locks_delete_block+0x40/0x50
> >  [<c0105ae7>] ? xen_force_evtchn_callback+0x17/0x30
> >  [<c01062c4>] ? check_events+0x8/0xc
> >  [<c01062bb>] ? xen_restore_fl_direct_reloc+0x4/0x4
> >  [<c01af1d1>] ? kmem_cache_free+0x41/0x60
> >  [<c0108dc8>] ? do_notify_resume+0x48/0x50
> >  [<c068ff12>] ? work_notifysig+0x9/0xf
> >  [<c0680000>] ? sctp_rcv_ootb+0x50/0xf0
> >
> > So I rebooted again, and now the situation seems normal. Site is up
> since 08:30
> > (downtime 1 hr 17 min 29 secs).
> >
> > I have no idea what happened, but have no time to do any more detailed
> > investigation at the moment.
> >
> > Damien
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > How fast is your code?
> > 3 out of 4 devs don\\\'t know how their code performs in production.
> > Find out how slow your code is with AppDynamics Lite.
> > http://ad.doubleclick.net/clk;262219672;13503038;z?
> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> > _______________________________________________
> > mantisbt-dev mailing list
> > mantisbt-dev@...
> > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>
>
>
> --
> Sent from my (old) computer
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>

And that's why I suggested that someone with server access looks into
it. Are we xen-based ( I don't know that) ? Are we at a recent enough
kernel version( I don't know that)?

Robert

On Sat, Sep 29, 2012 at 12:29 PM, Paul Richards <paul@...> wrote:
> that's a 1 year old xen kernel bug
>
> Paul
>
>
> On Sat, Sep 29, 2012 at 8:38 AM, Robert Munteanu <robert.munteanu@...>
> wrote:
>>
>> Without disclosing details about our instance, let's check we're up to
>> date in terms of patches applied.
>>
>> See for instance http://www.securityfocus.com/bid/38857/info
>>
>> Robert
>>
>> On Sat, Sep 29, 2012 at 9:51 AM, Damien Regad
>> <damien.regad@...> wrote:
>> > FYI I got an alert at 07:13 CET that the site was down.
>> >
>> > In the linode console, I saw that the server had been rebooted twice
>> > earlier in
>> > the night (at 02:32 and 02:33), and was listed as up and running
>> > although there
>> > seemed to be no activity (CPU and traffic graphs flat).
>> >
>> > Since I could not connect by SSH, I checked on the console and saw this
>> > -
>> >
>> > [linode137251@... lish]
>> >  [<c0103fa5>] ? pte_pfn_to_mfn+0xb5/0xd0
>> >  [<c019e111>] ? unmap_page_range+0x111/0x190
>> >  [<c019e2bb>] ? unmap_vmas+0x12b/0x1e0
>> >  [<c01a0481>] ? exit_mmap+0x91/0x140
>> >  [<c01324eb>] ? mmput+0x2b/0xc0
>> >  [<c0135ddf>] ? exit_mm+0xef/0x120
>> >  [<c068fb50>] ? _raw_spin_lock_irq+0x10/0x20
>> >  [<c01379d5>] ? do_exit+0x125/0x350
>> >  [<c0137c3c>] ? do_group_exit+0x3c/0xa0
>> >  [<c0142b17>] ? get_signal_to_deliver+0x1a7/0x3d0
>> >  [<c012f9f7>] ? finish_task_switch+0x47/0xb0
>> >  [<c0108cbf>] ? do_signal+0x8f/0x150
>> >  [<c01062c4>] ? check_events+0x8/0xc
>> >  [<c0105ae7>] ? xen_force_evtchn_callback+0x17/0x30
>> >  [<c01062c4>] ? check_events+0x8/0xc
>> >  [<c068faf5>] ? _raw_spin_lock+0x5/0x10
>> >  [<c01ebc20>] ? locks_delete_block+0x40/0x50
>> >  [<c0105ae7>] ? xen_force_evtchn_callback+0x17/0x30
>> >  [<c01062c4>] ? check_events+0x8/0xc
>> >  [<c01062bb>] ? xen_restore_fl_direct_reloc+0x4/0x4
>> >  [<c01af1d1>] ? kmem_cache_free+0x41/0x60
>> >  [<c0108dc8>] ? do_notify_resume+0x48/0x50
>> >  [<c068ff12>] ? work_notifysig+0x9/0xf
>> >  [<c0680000>] ? sctp_rcv_ootb+0x50/0xf0
>> >
>> > So I rebooted again, and now the situation seems normal. Site is up
>> > since 08:30
>> > (downtime 1 hr 17 min 29 secs).
>> >
>> > I have no idea what happened, but have no time to do any more detailed
>> > investigation at the moment.
>> >
>> > Damien
>> >
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > How fast is your code?
>> > 3 out of 4 devs don\\\'t know how their code performs in production.
>> > Find out how slow your code is with AppDynamics Lite.
>> > http://ad.doubleclick.net/clk;262219672;13503038;z?
>> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> > _______________________________________________
>> > mantisbt-dev mailing list
>> > mantisbt-dev@...
>> > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>>
>>
>>
>> --
>> Sent from my (old) computer
>>
>>
>> ------------------------------------------------------------------------------
>> How fast is your code?
>> 3 out of 4 devs don\\\'t know how their code performs in production.
>> Find out how slow your code is with AppDynamics Lite.
>> http://ad.doubleclick.net/clk;262219672;13503038;z?
>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> _______________________________________________
>> mantisbt-dev mailing list
>> mantisbt-dev@...
>> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>



-- 
Sent from my (old) computer

Server is fully patched and up-to-date as far as I can tell.

Site went down again for a few minutes (but no crash this time), apparently due
to out-of-memory issues (some apache/mysql processes were killed).

The average CPU usage as well as network IO for the last 24 hrs is much higher
than the last 30 days average (about 200-250%) but I have no idea why. It's back
to normal now.

Noticed in auth.log over 4200 failed ssh connections attempts since 25-Sep from
a single IP address (which is now banned, just in case).

D

I thought we had something like fail2ban installed to automatically
ban repeated SSH login attempts.

On Sat, Sep 29, 2012 at 6:33 AM, Damien Regad
<damien.regad@...> wrote:
> Server is fully patched and up-to-date as far as I can tell.
>
> Site went down again for a few minutes (but no crash this time), apparently due
> to out-of-memory issues (some apache/mysql processes were killed).
>
> The average CPU usage as well as network IO for the last 24 hrs is much higher
> than the last 30 days average (about 200-250%) but I have no idea why. It's back
> to normal now.
>
> Noticed in auth.log over 4200 failed ssh connections attempts since 25-Sep from
> a single IP address (which is now banned, just in case).
>
> D
>
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev



-- 
John Reese
noswap.com

On Sat, Sep 29, 2012 at 4:33 PM, Damien Regad
<damien.regad@...> wrote:
>
> Noticed in auth.log over 4200 failed ssh connections attempts since 25-Sep from
> a single IP address (which is now banned, just in case).

I suggest:

- moving ssh to a non-standard port
- allowing only public key authentication

This has dropped the level of noise in my ssh logs to zero.

Robert

-- 
Sent from my (old) computer

I disagree on changing the port.  That's just annoying to legitimate
users, and only stops the people that wouldn't be getting in with
random login attempts.  If you really want to cut out noise from the
logs, use something like Fail2Ban or SSHGuard or one of the other
systems that will ban IPs that fail more than X login attempts in a
given period of time.

As far as I know, SSH should already be set up on the server to only
allow public key authentication anyways, and if it's not, then it
certainly should be.

On Sat, Sep 29, 2012 at 11:59 AM, Robert Munteanu
<robert.munteanu@...> wrote:
> On Sat, Sep 29, 2012 at 4:33 PM, Damien Regad
> <damien.regad@...> wrote:
>>
>> Noticed in auth.log over 4200 failed ssh connections attempts since 25-Sep from
>> a single IP address (which is now banned, just in case).
>
> I suggest:
>
> - moving ssh to a non-standard port
> - allowing only public key authentication
>
> This has dropped the level of noise in my ssh logs to zero.
>
> Robert
>
> --
> Sent from my (old) computer
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev



-- 
John Reese
noswap.com

On Sat, Sep 29, 2012 at 10:09 PM, John Reese <john@...> wrote:
> I disagree on changing the port.  That's just annoying to legitimate
> users,

That's never annoying to me ; I just add the port to ~/.ssh/config and
that's that.

Robert
-- 
Sent from my (old) computer

John Reese <john@...> writes:
> I thought we had something like fail2ban installed to automatically
> ban repeated SSH login attempts.

Thanks for the hint on fail2ban John, I did not know about this tool. Indeed it
is running on the server. 

I did some research and it turns out that the specific entries I noticed in
auth.log are not covered by existing rules in the fail2ban config files. Other
people experienced this as well, see for example [1], but debian package
maintainer closed as won't fix.

Considering the very large number failed attempts in the log (>4200) and the
fact that peaks in occurence seem to coincide with high CPU/network activity, I
added a rule to ban these anyway, and removed the static entries I previously
added to hosts.deny.

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=588431

Thanks for the follow-up; I'm not holding my breath on whether this
fixes our ongoing problems, but it would be really nice if it did...
:P  I'll have to look at my fail2ban setups on my own machines
(running Arch) and see if I have the same issue with regex mismatch.

Cheers

On Sat, Sep 29, 2012 at 4:11 PM, Damien Regad
<damien.regad@...> wrote:
> John Reese <john@...> writes:
>> I thought we had something like fail2ban installed to automatically
>> ban repeated SSH login attempts.
>
> Thanks for the hint on fail2ban John, I did not know about this tool. Indeed it
> is running on the server.
>
> I did some research and it turns out that the specific entries I noticed in
> auth.log are not covered by existing rules in the fail2ban config files. Other
> people experienced this as well, see for example [1], but debian package
> maintainer closed as won't fix.
>
> Considering the very large number failed attempts in the log (>4200) and the
> fact that peaks in occurence seem to coincide with high CPU/network activity, I
> added a rule to ban these anyway, and removed the static entries I previously
> added to hosts.deny.
>
> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=588431
>
>
> ------------------------------------------------------------------------------
> How fast is your code?
> 3 out of 4 devs don\\\'t know how their code performs in production.
> Find out how slow your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219672;13503038;z?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev



-- 
John Reese
noswap.com

I rebooted the server this morning at 07:16 CET (it went down at 06:35).

After a quick analysis, the crash was caused by the server running out of
memory, apparently due to spawning a huge number of apache2 processes (239); at
that point (06:16 CET), kernel killed mysqld.

I don't have time to go into details now, but a quick look in apache error log,
shows a very large number of errors, originating from several 180.76.x.x ip's
requesting non-existent files. This resolves to 

Details on IP address 180.76.6.15

% [whois.apnic.net node-2]
% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html

inetnum:        180.76.0.0 - 180.76.255.255
netname:        Baidu
descr:          Beijing Baidu Netcom Science and Technology Co., Ltd.
descr:          Baidu Plaza, No.10, Shangdi 10th street,Haidian District
Beijing,100080
country:        CN
admin-c:        WN141-AP
tech-c:         JC2179-AP
mnt-by:         MAINT-CNNIC-AP
mnt-lower:      MAINT-CNNIC-AP
mnt-routes:     MAINT-CNNIC-AP
status:         ALLOCATED PORTABLE
changed:        hm-changed@... 20090715
source:         APNIC

Not sure if that's the root cause or not.

Damien

Thanks for the fix, investigation and writeup.
On Feb 8, 2013 10:54 PM, "Damien Regad" <damien.regad@...> wrote:

> I rebooted the server this morning at 07:16 CET (it went down at 06:35).
>
> After a quick analysis, the crash was caused by the server running out of
> memory, apparently due to spawning a huge number of apache2 processes
> (239); at
> that point (06:16 CET), kernel killed mysqld.
>
> I don't have time to go into details now, but a quick look in apache error
> log,
> shows a very large number of errors, originating from several 180.76.x.x
> ip's
> requesting non-existent files. This resolves to
>
> Details on IP address 180.76.6.15
>
> % [whois.apnic.net node-2]
> % Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html
>
> inetnum:        180.76.0.0 - 180.76.255.255
> netname:        Baidu
> descr:          Beijing Baidu Netcom Science and Technology Co., Ltd.
> descr:          Baidu Plaza, No.10, Shangdi 10th street,Haidian District
> Beijing,100080
> country:        CN
> admin-c:        WN141-AP
> tech-c:         JC2179-AP
> mnt-by:         MAINT-CNNIC-AP
> mnt-lower:      MAINT-CNNIC-AP
> mnt-routes:     MAINT-CNNIC-AP
> status:         ALLOCATED PORTABLE
> changed:        hm-changed@... 20090715
> source:         APNIC
>
> Not sure if that's the root cause or not.
>
> Damien
>
>
>
>
>
> ------------------------------------------------------------------------------
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>

>From the concept panel, I typically notice that hangs happen after high
load. The two scenarios I think of are some attack or crawling.  I expect
that crawlers are written in a way that doesn't overload the servers being
crawled. I suspect there are tools that would limit the # of connections or
load from a source IP.


On Fri, Feb 8, 2013 at 10:58 PM, John Reese <john@...> wrote:

> Thanks for the fix, investigation and writeup.
> On Feb 8, 2013 10:54 PM, "Damien Regad" <damien.regad@...>
> wrote:
>
>> I rebooted the server this morning at 07:16 CET (it went down at 06:35).
>>
>> After a quick analysis, the crash was caused by the server running out of
>> memory, apparently due to spawning a huge number of apache2 processes
>> (239); at
>> that point (06:16 CET), kernel killed mysqld.
>>
>> I don't have time to go into details now, but a quick look in apache
>> error log,
>> shows a very large number of errors, originating from several 180.76.x.x
>> ip's
>> requesting non-existent files. This resolves to
>>
>> Details on IP address 180.76.6.15
>>
>> % [whois.apnic.net node-2]
>> % Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html
>>
>> inetnum:        180.76.0.0 - 180.76.255.255
>> netname:        Baidu
>> descr:          Beijing Baidu Netcom Science and Technology Co., Ltd.
>> descr:          Baidu Plaza, No.10, Shangdi 10th street,Haidian District
>> Beijing,100080
>> country:        CN
>> admin-c:        WN141-AP
>> tech-c:         JC2179-AP
>> mnt-by:         MAINT-CNNIC-AP
>> mnt-lower:      MAINT-CNNIC-AP
>> mnt-routes:     MAINT-CNNIC-AP
>> status:         ALLOCATED PORTABLE
>> changed:        hm-changed@... 20090715
>> source:         APNIC
>>
>> Not sure if that's the root cause or not.
>>
>> Damien
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Free Next-Gen Firewall Hardware Offer
>> Buy your Sophos next-gen firewall before the end March 2013
>> and get the hardware for free! Learn more.
>> http://p.sf.net/sfu/sophos-d2d-feb
>> _______________________________________________
>> mantisbt-dev mailing list
>> mantisbt-dev@...
>> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>>
>
>
> ------------------------------------------------------------------------------
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev
>
>

Tamás Gulácsi <gt-dev@...> writes:
> Sorry to sugggest without ask

No problem, your input is appreciated. I assume your message was in response to 

> Victor Boctor <vboctor@...> writes:
>> The two scenarios I think of are some attack or crawling.  I expect that 
>> crawlers are written in a way that doesn't overload the servers being 
>> crawled. I suspect there are tools that would limit the # of connections 
>> or load from a source IP.

> but can't we use fail2ban for banning ips based on mantis/apache/nginx/php 
> error log, nginx with php5-cgi as fastcgi, thus limiting the running php5 
> processes to a known safe limit?
> 
> (Nginx does not fork).

We do have fail2ban running on the server, and it is working fine as far as I
can tell. In this case though, the errors in the apache log I mentioned in my
initial e-mails came from several IP addresses from the mentioned subnet, and I
suppose that's why they were not captured and blocked.

I don't know enough about the tool to configure it as you suggest. If you know
how, advice is welcome.

That said, I am not even sure these errors are in fact the root cause of the 
problem, i.e. the reason why apache started forking so many processes.

Damien

On Sun, Feb 10, 2013 at 3:57 PM, Damien Regad
<damien.regad@...> wrote:
> Tamás Gulácsi <gt-dev@...> writes:
>> Sorry to sugggest without ask
>
> No problem, your input is appreciated. I assume your message was in response to
>
>> Victor Boctor <vboctor@...> writes:
>>> The two scenarios I think of are some attack or crawling.  I expect that
>>> crawlers are written in a way that doesn't overload the servers being
>>> crawled. I suspect there are tools that would limit the # of connections
>>> or load from a source IP.
>
>> but can't we use fail2ban for banning ips based on mantis/apache/nginx/php
>> error log, nginx with php5-cgi as fastcgi, thus limiting the running php5
>> processes to a known safe limit?
>>
>> (Nginx does not fork).
>
> We do have fail2ban running on the server, and it is working fine as far as I
> can tell. In this case though, the errors in the apache log I mentioned in my
> initial e-mails came from several IP addresses from the mentioned subnet, and I
> suppose that's why they were not captured and blocked.
>
> I don't know enough about the tool to configure it as you suggest. If you know
> how, advice is welcome.
>
> That said, I am not even sure these errors are in fact the root cause of the
> problem, i.e. the reason why apache started forking so many processes.

Can't we simply limit the number of apache processes to something we
know is safe, like 150 or 200?

Robert

>
> Damien
>
>
>
> ------------------------------------------------------------------------------
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> _______________________________________________
> mantisbt-dev mailing list
> mantisbt-dev@...
> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev



-- 
Sent from my (old) computer