Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- How to get just Homepage to be indexed, and not any other pages?

g1smd - 1:07 pm on May 11, 2009 (gmt 0)


I'd say that syntax is risky, as for any bot not understanding "allow" it blocks everything: OK I see you specify it only for Google.

I'd guess the OP does also want to consider Yahoo, Live, Ask, SearchMe, etc...

Thread source:: http://www.webmasterworld.com/robots_txt/3931680.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com