Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- robots.txt syntex


jdMorgan - 2:08 am on Oct 30, 2003 (gmt 0)


jdancing,

You probably want to keep that "User-agent: *" at the end -- Remember that good robots will obey the first record containing either a match on their user-agent name or "*" whichever comes first.

Check your file for extraneous characters - such as spaces at the end of lines, etc.

More info:
Learn: [robotstxt.org...]
Validate: [searchengineworld.com...]

Jim


Thread source:: http://www.webmasterworld.com/robots_txt/180.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com