Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Is this robots.txt blocking anybody


ncw164x - 9:35 am on Feb 1, 2004 (gmt 0)


MarkHutch is right you robots.txt file is not blocking any bots so your problem must be elsewhere

This indicates that nothing is disallowed and the spider can follow all links
User-agent: *
Disallow:

To allow a single robot complete access and exclude all others
User-agent: Googlebot/1.0
Disallow:
User-agent: *
Disallow: /

This would prevent your entire web site from being indexed
User-agent: *
Disallow: /

ncw164x


Thread source:: http://www.webmasterworld.com/robots_txt/261.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com