Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- how to disallow feed in robot.txt readable by all bots

tangor - 5:56 am on Aug 16, 2011 (gmt 0)


Either are correct, but bad bots will ignore... and will also use those "hints" as to what to rip.

feed all by itself will work for bots that honor...

Thread source:: http://www.webmasterworld.com/robots_txt/4351875.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com