Menu
▾
▴
#82 Add no index no follow meta tag to error pages
In seeing how much spam can hang around as well as contact information that is mistakenly added to a sitemaker site, I am wondering if it would not be a good idea to add a:
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
meta tag to the header of an "access denied" as well as "not found" page.
This would help clear up search engine listings as well as point bots away from those pages, especially after they have been removed. Since results of pages that are deleted or access is removed have a tendency to hang around a long time, the error pages are not alerting bots that content is modified enough. That would be a simple fix, but a larger fix would be getting the pages to appropriately give 403 and 404 errors.