Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- how to ban Google from indexing ALL .php files
Dijkgraaf - 3:48 am on Jun 2, 2005 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/622.htm
I wonder if "Disallow: /*?" actually means that they just won't follow URL's with query strings, rather than they won't spider .php, .asp etc.
I know what their FAQ says, but is it accurate?
Brought to you by WebmasterWorld: http://www.webmasterworld.com