Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Yahoo! Slurp Now Supports Wildcards in robots.txt


Asia_Expat - 4:01 am on Nov 13, 2006 (gmt 0)


bouncybunny, I thought your question was quite clear... and I have the same question but it appears to me that if you want to creat a single set of rules for all bots, then there is no need to specify for each bott, as the wildcard rules/protocols appear to be the same.
Someone please tell me if I'm wrong.


Thread source:: http://www.webmasterworld.com/robots_txt/3144662.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com