Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- robots.txt syntex


jdMorgan - 4:28 am on Oct 30, 2003 (gmt 0)


jdancing,

An issue? What, with respect to 'regular' GoogleBot? No, that's not an issue.

But you do want your specific, per-robot stuff first, and then either allow or disallow the rest with the
"User-agent: *" record at the end.

Also, since you say you have about 100 bad-bot disallows, you might want to peruse this old thread: [webmasterworld.com...]

Jim


Thread source:: http://www.webmasterworld.com/robots_txt/180.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com