Re: [TWiki-Dev] I've got blacklisted

[Kenneth L. <ke...@la...>]

Michael Daum wrote:
>
> Please disable the _blackist_ feature while still checking content for spam.
>
> 	Micha.
>
>   
You are blacklisted both when you save a topic with spam that was 
already there and when you score too many points.

Removing the point system and banning the IP address is a necessary 
protection.

It happens OFTEN that some idiot tries to mirrow a TWiki. They start 
some program that follow ALL LINKS and download and download. and put a 
permanent load on the site for hours and hours.

The most popular can blocked using apache config protection but there 
are always new "site sucking" software that comes out that we do not 
know or does not leave an "agent type fingerprint" we can use to protect 
ourselves.

There are FIVE things we can do to make the BlackListPlugin a little 
less pain in the butt.

1. Anyone with a fixed IP address who is a regular developer of TWiki 
can go on the whitelist. Some of you that have fallen into the trap 
already have access to do it yourself. And others can have their IPs 
added as long as you are a known regular user. If you have dynamic IP 
this solution will not work naturally.

2. The blacklisting of IP addresses is efficient in stopping a spammer 
or site sucker the minute he is doing something wrong. But I have one 
observation about spammers in general.
They change IP address all the time.
The IP address they use is never the IP that the spam points to.
They probably use various relays (anonymizers which change IP all the 
time) and compromised computers.
So I do not think there is any need to permanently blacklist an IP address.
The blacklist plugin could expire a blacklisted IP address after maybe 2 
hours or 12 hours without practical loss of any protection.

3. The TRAP people fall into when saving a topic that contains a string 
that already contains spam added before this particular word got added 
to the signature file is very annoying. You can open a topic containing 
spam - add a "hello" - save it - and get blacklisted. The way to avoid 
this situation is to let the Blacklist plugin scan the topic also when 
you hit the EDIT button and then warn the user that the topic contains 
spam and which word is the spam word. Then you can either cancel out or 
make sure you remove the offending URL before you save. This will make 
the EDIT function a little slower but normal view will not be affected.

4. I would like to see an enhancement of BlackListPlugin which can scan 
the entire site. It would be a script in the tools directory which scans 
through all topics looking for spam URLs. Each topic found with spam is 
listed with web.topic name and the spam pattern found. The result is 
emailed to the admin.
This script can be run by cron only once per 24 hours. The admin can 
then remove spam that was added before spam pattern was added to the 
list. This will both help removing spam and prevent some of the 
incidents where people are blacklisted from saving a topic that somebody 
else added a spamming URL to.

5. The blacklist plugin could have an additional white list feature. 
WikiName based white list. It could look in a WhiteListGroup and never 
blacklist any members of this group. It should be allowed that this 
group can consist of both individuals and groups. This whitelist feature 
will only be used when you save topics so normal view performance should 
not be affected.

Kenneth