Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Robots.txt - Disallow: or Disallow: /


lixsmk1 - 8:38 pm on Jul 22, 2009 (gmt 0)


Hi g1smd what would this say about the a robot.txt file,is it good enough? when it contains the following syntax

User-Agent: *
Allow: /

and if we want to specify the sitemap location would it help spiderability and indexing. If yes then how would we include the sitemap if you could tell the proper syntax for it please,thanks


Thread source:: http://www.webmasterworld.com/robots_txt/3900674.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com