Re: [VuFind-General] DoS / bots issue

[Demian K. <dem...@vi...>]

Interesting -- we did some experimentation with mod_evasive over here, but we had trouble with the way it interacted (or rather, failed to interact) with mod_rewrite, so that requests that involved rewrites would still swamp our server because mod_rewrite used up CPU cycles before mod_evasive had a chance to block the request. A bit of research suggested that there was no solution to the problem, so we gave up... but maybe that's only a problem in the particular edge case we were testing (a DOS targeted on a URL that triggered a redirect), but it's still helpful in more general use. It's also possible I'm not quite remembering all of the details correctly!

- Demian

-----Original Message-----
From: tu...@yo... [mailto:tu...@yo...] 
Sent: Thursday, September 17, 2015 12:54 PM
To: Tim Fletcher
Cc: vuf...@li...
Subject: Re: [VuFind-General] DoS / bots issue

Hi Tim

We use mod_evasive for Apache to slow them down, it is very easy to setup.

https://www.digitalocean.com/community/tutorials/how-to-protect-against-dos-and-ddos-with-mod_evasive-for-apache-on-centos-7


On Sep 17, 2015, at 12:03 PM, Tim Fletcher <T.F...@bb...> wrote:

> Hi,
> 
> Our VuFind is currently unusable due to it being hit with repeated 
> queries from Ahrefs Bot. We have put them in robots.txt but this isn't 
> having an effect at the moment.
> 
> Has anyone had any experience of this, or other bots, and can give us 
> any tips?
> 
> Thanks,
> Tim
> 
> ---------------
> Tim Fletcher
> 
> Library Technology Innovation Manager
> Birkbeck, University of London
> Malet Street
> London WC1E 7HX
> 
> t.f...@bb...
> 
> Tel: 020 7631 6219 (as from 27 July 2015)
> Fax: 020 7631 6066
> http://www.bbk.ac.uk/lib/
> 
> 
> 
> ----------------------------------------------------------------------
> -------- Monitor Your Dynamic Infrastructure at Any Scale With 
> Datadog!
> Get real-time metrics from all of your servers, apps and tools in one 
> place.
> SourceForge users - Click here to start your Free Trial of Datadog now!
> http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
> _______________________________________________
> VuFind-General mailing list
> VuF...@li...
> https://lists.sourceforge.net/lists/listinfo/vufind-general


------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
_______________________________________________
VuFind-General mailing list
VuF...@li...
https://lists.sourceforge.net/lists/listinfo/vufind-general