Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Do you have robot.txt on your site?
Edge - 6:56 pm on Apr 12, 2010 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/4113942.htm
Each Googlebot crawls different data/media. There are places I don't want a particualr bot and other places that are OK.
Different rules for each google bot.
Brought to you by WebmasterWorld: http://www.webmasterworld.com