Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Google scraping through China?


keyplyr - 7:07 am on Dec 20, 2012 (gmt 0)


I guess I just don't see what G has to gain by this, nor why they'd spend the effort when they can just access your files normally. OTOH a user up to no good would have an advantage hiding in this manner, however it still seems like a lot of work when they could just use an anonymous proxy with a spoofed UA.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4529439.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com