dstiles - 7:22 pm on Jun 12, 2012 (gmt 0)
I've been seeing a LOT of accesses from a yahoo IP in the range 98.139.241.nnn (resolves to ycar2.mobile.bf1.yahoo.com).
I recently spent some time removing this range from an auto-ban based on its header, working on the idea that it was a proxy for mobile devices and as such I was killing customers.
I am now seeing this IP range access sites that range from often-visited to almost never. Action is always the same: load default home page, load favicon (the real one, held in a subdirectory and only known via a meta tag in page headers).
Frequency has now built up to over 1000 hits per day across a dozen or so domains (one server, several IPs).
In each case the access is through a double proxy that resolves to (typically) inktomi and a broadband IP from one of the UK ISPs. In a few cases the combination is Yahoo SG and an SG broadband IP. There are probably other combinations but too many to analyze in a short time. There are no other hits from either the broadband proxy or the yahoo IPs (at least, not on the seldom-accessed sites).
The UA is always (at least, this month):
Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3
I am looking at this now as some kind of cache rather than a real person. Something along the lines of G's web preview. The hits are far too frequent on one of our seldom-visited sites - almost the whole log for such a site today consisted of these hits.
I am thinking of pushing the IP range back into the "bad bots" section with a 403 or similar. Any comments on this ploy? Is anyone seeing real traffic on the back of such yahoo hits?