Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Yahoo! Slurp/3.0 released on new IPs


incrediBILL - 2:51 am on Apr 16, 2008 (gmt 0)


So far today Slurp/3.0 only crawled 46 pages out of 21K total Slurped pages today.

Googlebot came in 3rd with only got 8K pages and msnbot claimed 2nd with 11K pages, making Slurp the biggest crawler and it's been this way for many weeks now.

The bot that crawls the least sends the most traffic, the irony.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3626881.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com