From: Johnny L. W. <jo...@ma...> - 2002-10-18 19:44:47
|
I was looking around the sourceforge page and noticed that there's an open task to write a robots.txt file which will prevent a few pages from being indexed. Maybe instead, we should include tags like this on pages we don't want indexed: <META NAME="ROBOTS" CONTENT="NOINDEX"> And, if you want the robot to stop following links on this page, you add this to it: <META NAME="ROBOTS" CONTENT="NOFOLLOW"> That should get everything you need to do done, right? -- Johnny Wales Book Systems, Inc. |