[Tiki-devel] Convert your site to Tiki and increase your traffic...

[Tom J. <tom...@ag...>]

Unfortunately it isn't the kind of traffic anybody would want.

There is a Tiki register bot that goes to Tiki sites and tries to 
register on them.  There are lots of malicious bots out there that 
search for things like WordPress, Drupal, and Tiki sites, but this one 
targets just Tiki sites.  It doesn't go randomly to all the sites on the 
server, it has a list of the Tiki sites and in some cases specific pages 
on those Tiki sites.

You can recognize this bot by a couple of things.  Since it isn't a 
human using a browser with an agent string, it has a set of 16 agent 
strings that it randomly uses, even changing between the 16 agent 
strings during the same visit.  (I wish there was something I could add 
to .htaccess to block and IP address if it changes agent strings during 
its visit.)
And even the very first time that IP address visits your site it has a 
referrer string that says it is being referred from your site. (Note 
that most search engine spiders do this too.)
Plus, if you check the IP addresses against the Stop Forum Spam 
(http://www.stopforumspam.com/) database of reported forum spammers, you 
will find a very high percentage of them are the IP addresses of 
reported spammers.

The bots usual visit goes:
GET /          (Always with your site as the referrer)
GET /HomePage
GET /tiki-register.php

And sometimes it goes:
GET /OneOfYourTikiPages      Always with your site as the referrer
GET /tiki-register.php

And sometimes it goes:
GET /          Always with your site as the referrer
GET /HomePage
GET /tiki-login.php

It goes for the tiki-register and tiki-login even if there are no links 
on any of your pages to those Tiki pages.

On Sunday afternoon I converted a website from using just html, to use 
Tiki 9.5.  It had been an html site since 2007.  The site has never had 
many visitors, the average number of visitors per day in May was 10 
(that count includes search engine spiders).

While I set up Tiki, I routed visitors to a "under maintenance" page.  
Once it was basically ready, I opened the site to visitors and the first 
visitor that got the Tiki version of the site was at 16/Jun/2013:17:03:00

The first access by the tiki register bot was at 18/Jun/2013:03:49:53.  
Just 34 hours and 46 minutes after the Tiki site was available.

And between then (18/Jun/2013:03:49:53) and now (19/Jun/2013:23:28:07) 
there have been 210 accesses of tiki-login.php, every single one of them 
by an IP address using one of those 16 agent strings I mentioned.

Since there were visits by only 29 different IP addresses between the 
time the site was an active Tiki site and the first visit by the 
spammers bot, I figured that one of those IP addresses was the IP 
address of the search engine the spammer uses to find Tiki sites.  And I 
think I have identified which one it was.

54.226.103.57 - - [17/Jun/2013:00:46:07 -0500] "GET / HTTP/1.1" 302 - 
"-" "Mozilla/5.0 (compatible; URLAppendBot/1.0; 
+http://www.profound.net/urlappendbot.html)"

and then the next 2 URLs it went for, after being redirected to the Tiki 
home page, were:

54.226.103.57 - - [17/Jun/2013:00:46:49 -0500] "GET 
/tiki-remind_password.php HTTP/1.1" 200 17414 "-" "Mozilla/5.0 
(compatible; URLAppendBot/1.0; +http://www.profound.net/urlappendbot.html)"
54.226.103.57 - - [17/Jun/2013:00:46:52 -0500] "GET /tiki-register.php 
HTTP/1.1" 200 24258 "-" "Mozilla/5.0 (compatible; URLAppendBot/1.0; 
+http://www.profound.net/urlappendbot.html)"

Even though there were not links on the home page to either of those.  I 
did not have registration turned on and I had deleted the Login module 
before I allowed the first visitor to the site.

If you check the link from the agent string 
(http://www.profound.net/urlappendbot.html), it does not exist.  This IP 
address is from Amazon EC2.

I am going to add:

RewriteCond %{HTTP_USER_AGENT}  URLAppendBot [NC]
RewriteRule .*  - [F]

to all of my .htaccess files.

I already have this in the .htaccess files of most of my Tiki sites:
### Stop certain Agent Strings from accessing tiki-register.php
RewriteCond %{THE_REQUEST}      /tiki-register.php
RewriteCond %{HTTP_USER_AGENT}  ... one for each of the 16 agent strings 
[OR]
RewriteCond %{HTTP_USER_AGENT}  ... last one
RewriteRule .*  - [F]

And, while I am on the subject, I have written a plugin that uses the 
Stop Forum Spam API to check IP addresses to see if they are of reported 
spammers.

If you want to use it, save the code below in 
/lib/wiki-plugins/wikiplugin_checkipwithstopforumspam.php

<?php
function wikiplugin_checkipwithstopforumspam($data, $params)
{
         global $user, $prefs, $tikilib, $smarty;
         extract($params, EXTR_SKIP);
   if (!isset($msg) && !isset($page)) {$msg="Sorry, you may not register.";}
  //       Only check Stop Forum Spam if the group is Anonymous
   $userGroups = $tikilib->get_user_groups($user);
   $sfschk=1;
  //       This will check all the groups, if any are not Anonymous, do 
not do SFS check
   foreach ($userGroups as $key=>$grp) { if ($grp != 'Anonymous') 
{$sfschk=0;} }

   if ($sfschk) {
     // ***  CHECK IP against SFS
     $addr = $_SERVER['REMOTE_ADDR'];
     #   $addr="222.187.222.66"; # for testing, bad IP address
     $response = 
file_get_contents('http://www.stopforumspam.com/api?ip='.$addr);
     $pattern = '/<appears>yes<\/appears>/';
     if (preg_match($pattern, $response))
        {$SFS=1;} else {$SFS=0;}  // SFS set to 1 if known spammer IP 
address
     // *** End of SFS check

     if ($SFS) {
         if (isset($page)) {
            header("Location: tiki-index.php?page=$page");         // 
redirect to page if SFS known spammer
            exit;
         } else {
              header("Location: ./tiki-information.php?msg=$msg");   // 
leave and display message if SFS known spammer
            exit;
         }
     }
   }
return "";
}


To use this plugin, set up a User Tracker to collect information about 
your members, select that the tracker description should be wiki parsed, 
and then add the plugin to your User Tracker description:

{CHECKIPWITHSTOPFORUMSPAM()/}

There are 2 optional parameters, page= and msg=.  If neither of them are 
set, and the IP address is found in Stop Forum Spam's database, the 
visitor gets sent to a page that displays the default message: "Sorry, 
you may not register".

You can change the message with msg="your message", or send them to a 
wiki page with page=PageName.  If a page= is specified, the msg= is 
ignored.

It does not check the IP address with Stop Forum Spam of someone that is 
logged in, so it will not call Stop Forum Spam for your IP address when 
you list your trackers and it shows the description.

You can test that it is working by un commenting (removing the first 
"#") the line:

#   $addr="222.187.222.66"; # for testing, bad IP address

and logging out (or use a different browser) and accessing your 
tiki-register.php.  Be sure to add the "#" back in when you are done 
testing, or delete that line altogether.

Tom Jarvis