Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Yahoo! Slurp


Pfui - 3:50 am on Sep 20, 2011 (gmt 0)


Apropos of I dunno what, 'plain' Slurp just cruised through and logged six same-dir index page referrers (out of 20 files hit throughout the day).

I don't recall any Yahoo bot EVER using (or appearing to use) referrers.

b3091154.crawl.yahoo.net
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

IP: 67.195.112.189


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4360952.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com