Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Are there ANY legitimate visits from AWS?


motorhaven - 3:09 am on Mar 4, 2013 (gmt 0)


If I weren't displaying Adsense ads I'd certainly block Proximic. They don't support crawl delay, serve no useful propose other than targeting for advertsers and can be resource intensive during peak times, and follow up user page fetches a minute or so after the Adsense bot so each HTML page results in 3 hits.

When I first started seeing them and noticing the pattern of their crawler fetching a page shortly after a legit user did, I knew it had to be connected to Adsense. I managed to get in touch with someone there on the phone.

Based on the discussion I had not only do they crawl on behalf of direct clients, but also for other ad networks who work with Adsense. On a test site I have which has extremely steady Adsense income I saw daily drops between 10-15%, and this site normally sees 2-5% differences. The drop occurred a few days after blocking the bot.

Of course this is going to vary by site, some sites are going to get different proportions of Google's Adword ads verses Google's affiliated ad networks.

[edited by: motorhaven at 3:12 am (utc) on Mar 4, 2013]


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4550485.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com