[Lxr-dev] [ lxr-Bugs-3220139 ] Security fix: robot snooping

[SourceForge.net <no...@so...>]

Bugs item #3220139, was opened at 2011-03-17 15:08
Message generated for change (Comment added) made by mbox
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3220139&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Security fix: robot snooping

Initial Comment:
Within 24 hours after clicking on "html 4 verified" logo (and having received a positive check from W3C), Microsoft's robot Bingbot tried to crawl and index my machine. My test tree lies in a very private part of my computer and is connected to the Net on an unusual port (in the private range). The only explanation for this intrusion is that W3C validation sites are compromised.

To prevent such intrusion, comment out the link under HTML logo.

If production site admins want the validation feature, they can uncomment the link.

Does it sound sane?

----------------------------------------------------------------------

>Comment By: Malcolm Box (mbox)
Date: 2011-03-17 18:01

Message:
That doesn't seem like a good fix since it removes an easy way for our
users to check that we really are compliant.

If your machine is on the internet and serving files, then it *will* be
discovered and crawled by someone.  There's enough places where traces of
activity will be left.

LXR could include a robots.txt file to block access to (well behaved)
robots unless the admin deliberately turns the block off.  That would seem
like a good thing to do, especially since crawling a LXR tree is likely to
go wrong in interesting ways.

We could also add a warning note in the install instructions not to run
LXR with a public webserver unless you want your source code exported to
the world.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3220139&group_id=27350