Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Yahoo! Slurp Now Supports Wildcards in robots.txt


incrediBILL - 12:37 am on Nov 8, 2006 (gmt 0)


Woo hoo!

Everything we've not wanted and MORE!

Can Yahoo say CACHE server?

Can Yahoo direct all their stinking bots to use the ONE copy of my page they just downloaded?

Now THAT would be an improvement!

Funny, Google just did this for all their bots, but we wouldn't want to hold Google up as an example to Yahoo how to do something right as that would just be mean and unproductive...

[edited by: incrediBILL at 12:39 am (utc) on Nov. 8, 2006]


Thread source:: http://www.webmasterworld.com/robots_txt/3144662.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com