Re: [Fail2ban-users] fail2ban no match with regex

[Fabian W. <fa...@we...>]

Hello Alex

On 09.03.2013 16:40, al...@ma... wrote:
> That's been going on for years now, how do you think I should feel about it?
> Blocking bots on your own servers is one thing, but making false abuse
> accusations is totally different, yet nobody seems to recognise it on here
> and I did not see any effort made to change fail2ban software to avoid such
> side effects. :(

Why does this come up again?

As it already has been told to you, the default install of 
fail2ban does not send out abuse notices. Also there is no jail 
enabled. The filters included in fail2ban, which do match on 
error 404, do this on quite limited special cases. See the thread 
"Trying to contact maintainers/developers of fail2ban" [1] in 
mailing list archive for details.

   [1] http://sourceforge.net/mailarchive/message.php?msg_id=29671920

If an user (aka SysAdmin) does enable the stuff which is already 
there in fail2ban, it will not create abuse notices because of 
missing robots.txt. You can not blame the software developers for 
mistakes clearly done by the end user who does adjust the 
configuration in such a way that it does match on all error 404 
and also sending out abuse notices. For this to get working the 
user needs to create their own filter with own regex.

I do understand the troubles you have with this abuse notices. As 
it has been discussed back in August 2012, it seems that you get 
this probably from a single user / organisation as the subject 
line of those e-mails could not been found anywhere else. This 
clearly indicates that it is custom made.

Eventually you need to rethink your business model for your 
search engine and e.g. doing the crawling out of your own IP 
networks, which are registered to your company. Then you will get 
such abuse notices directly and are able to better deal with 
them. As long as you are depending on volunteers running your 
crawler from their home internet connection, the abuse notices 
will go to their ISP, which then will forward it to their end 
user, probably without disclosing the original sender.


bye
Fabian