Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Really necessary to add to robots.txt
coolshop - 9:06 pm on Jul 4, 2003 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/92.htm
Is it really necessary to add the two lines to robots.txt like the AdSense FAQ says? As long as you are not explicitly banning their bot, why is this necessary?
Brought to you by WebmasterWorld: http://www.webmasterworld.com