Welcome to WebmasterWorld Guest from 54.159.12.17

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Yahoo Mobile access with Safari

Every sign of being a bot not a proxy

     

dstiles

7:22 pm on Jun 12, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



I've been seeing a LOT of accesses from a yahoo IP in the range 98.139.241.nnn (resolves to ycar2.mobile.bf1.yahoo.com).

I recently spent some time removing this range from an auto-ban based on its header, working on the idea that it was a proxy for mobile devices and as such I was killing customers.

I am now seeing this IP range access sites that range from often-visited to almost never. Action is always the same: load default home page, load favicon (the real one, held in a subdirectory and only known via a meta tag in page headers).

Frequency has now built up to over 1000 hits per day across a dozen or so domains (one server, several IPs).

In each case the access is through a double proxy that resolves to (typically) inktomi and a broadband IP from one of the UK ISPs. In a few cases the combination is Yahoo SG and an SG broadband IP. There are probably other combinations but too many to analyze in a short time. There are no other hits from either the broadband proxy or the yahoo IPs (at least, not on the seldom-accessed sites).

The UA is always (at least, this month):
Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3

I am looking at this now as some kind of cache rather than a real person. Something along the lines of G's web preview. The hits are far too frequent on one of our seldom-visited sites - almost the whole log for such a site today consisted of these hits.

I am thinking of pushing the IP range back into the "bad bots" section with a 403 or similar. Any comments on this ploy? Is anyone seeing real traffic on the back of such yahoo hits?

santapaws

7:53 am on Jun 27, 2012 (gmt 0)

5+ Year Member



just wondering if this ended up in your ban list or not? I am seeing the same pattern.

dstiles

8:54 pm on Jun 27, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Yes, I blocked the range a few hours later, there being no report in here to counter my findings. Still getting loads of hits through the proxied range. There still seems no logical reason for it.

wilderness

3:04 am on Jun 28, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



There's another thread or two on "YahooCacheSystem"

I saved this the other day because the BlackBerry requests were unique to me.

98.139.241.245 - - [24/Jun/2012:17:01:51 +0100] "GET / HTTP/1.1" 403 559 "-" "YahooCacheSystem"
98.139.241.241 - - [24/Jun/2012:22:57:22 +0100] "GET / HTTP/1.1" 403 559 "-" "YahooCacheSystem"
98.139.241.245 - - [25/Jun/2012:02:49:01 +0100] "GET / HTTP/1.1" 200 6010 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9650; en-US) AppleWebKit/534.8+ (KHTML, like Gecko) Version/6.0.0.524 Mobile Safari/534.8+"
98.139.241.245 - - [25/Jun/2012:02:49:02 +0100] "GET /favicon.ico HTTP/1.1" 200 318 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9650; en-US) AppleWebKit/534.8+ (KHTML, like Gecko) Version/6.0.0.524 Mobile Safari/534.8+"
98.139.241.243 - - [25/Jun/2012:04:22:21 +0100] "GET / HTTP/1.1" 200 6010 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9670; en-US) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.407 Mobile Safari/534.1+"
98.139.241.243 - - [25/Jun/2012:04:22:22 +0100] "GET /favicon.ico HTTP/1.1" 200 318 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9670; en-US) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.407 Mobile Safari/534.1+"

dstiles

8:42 pm on Jun 28, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Did it make sense to you at the time?

Had you blocked 98.139.241/24 would you have seen the 403 or would your BB still have loaded the proper requested page? And if the latter, from a previous cache or from your site?

wilderness

9:23 pm on Jun 28, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Did it make sense to you at the time?


No it did not.

Don't believe I've the Ip range denied, rather the UA YahooCacheSystem

or perhaps even portions of same UA.

dstiles

8:50 pm on Jun 29, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Thanks. I was wondering what effect my blocking the IP range had on what people saw on mobile devices. Not being able to check that kind of thing is about the only drawback of not having a mobile device. :)

wilderness

1:14 am on Oct 11, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



FWIW

98.139.241.241 - - [11/Oct/2012:00:23:03 +0100] "GET / HTTP/1.1" 403 559 "-" "YahooCacheSystem"
98.139.241.248 - - [11/Oct/2012:01:25:05 +0100] "GET / HTTP/1.1" 403 559 "-" "Mozilla/5.0 (Linux; U; Android 4.0.4; en-ca; SGH-I747M Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"
 

Featured Threads

Hot Threads This Week

Hot Threads This Month