Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Does amazon have a crawler or are all Amazon AWS IPs fair game?
Bewenched - 9:35 pm on Nov 5, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4514970.htm
I've busted several amazon "bots" scraping our site for images.... makes me wonder if they are actually stealing product images from sites for their own use.
Brought to you by WebmasterWorld: http://www.webmasterworld.com