Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Yahoo now violating robots.txt
keyplyr - 3:20 am on Dec 31, 2007 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/3536187.htm
I've added Slurp/3.0 to denies and will thus see how it effects my listings and other crawls - wilderness
You're a better man than I Gunga Din.
Brought to you by WebmasterWorld: http://www.webmasterworld.com