Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- New Yahoo China bot range


Mokita - 1:35 am on Dec 3, 2010 (gmt 0)


wilderness wrote:
If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges?


Most of our sites are APNIC based. I don't think I have ever seen a verifiable, googlebot or bingbot/msnbot crawling from anything other than ARIN based IPs. Blocking those would be incredibly self-defeating (unless you don't want any traffic).

Yahoo only crawls from ARIN based IPs or Chinese - and I have been blocking Slurp China (APNIC based) for a very long time, with no apparent effect on our listings in Yahoo.

HTH.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4237913.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com