I have had an extra ordinary amount of attacks bothy imbedded URL's and some new "search" engines in August. My bandwidth usage has trippled but the site has remained robust and running.
Special thanks to the developers
You could put a robots.txt in the root of your website and configure like this:
Thats not required. Perhaps you missed wdm001's point - DESPITE the attacks PGV was untouched.
PGV already has very extensive protection against unwanted robots (not all are unwanted), and hackers / spammers etc of all kinds. These measures are upgraded with pretty well every new release, which is one reason we always recommend everyone upgrades to the latest version at every opportunity.