Welcome to WebmasterWorld Guest from 54.147.63.124

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Yahoo Mobile access with Safari

Every sign of being a bot not a proxy

     
7:22 pm on Jun 12, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


I've been seeing a LOT of accesses from a yahoo IP in the range 98.139.241.nnn (resolves to ycar2.mobile.bf1.yahoo.com).

I recently spent some time removing this range from an auto-ban based on its header, working on the idea that it was a proxy for mobile devices and as such I was killing customers.

I am now seeing this IP range access sites that range from often-visited to almost never. Action is always the same: load default home page, load favicon (the real one, held in a subdirectory and only known via a meta tag in page headers).

Frequency has now built up to over 1000 hits per day across a dozen or so domains (one server, several IPs).

In each case the access is through a double proxy that resolves to (typically) inktomi and a broadband IP from one of the UK ISPs. In a few cases the combination is Yahoo SG and an SG broadband IP. There are probably other combinations but too many to analyze in a short time. There are no other hits from either the broadband proxy or the yahoo IPs (at least, not on the seldom-accessed sites).

The UA is always (at least, this month):
Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3

I am looking at this now as some kind of cache rather than a real person. Something along the lines of G's web preview. The hits are far too frequent on one of our seldom-visited sites - almost the whole log for such a site today consisted of these hits.

I am thinking of pushing the IP range back into the "bad bots" section with a 403 or similar. Any comments on this ploy? Is anyone seeing real traffic on the back of such yahoo hits?
7:53 am on June 27, 2012 (gmt 0)

Preferred Member

5+ Year Member

joined:Dec 19, 2007
posts:404
votes: 0


just wondering if this ended up in your ban list or not? I am seeing the same pattern.
8:54 pm on June 27, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


Yes, I blocked the range a few hours later, there being no report in here to counter my findings. Still getting loads of hits through the proxied range. There still seems no logical reason for it.
3:04 am on June 28, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5408
votes: 2


There's another thread or two on "YahooCacheSystem"

I saved this the other day because the BlackBerry requests were unique to me.

98.139.241.245 - - [24/Jun/2012:17:01:51 +0100] "GET / HTTP/1.1" 403 559 "-" "YahooCacheSystem"
98.139.241.241 - - [24/Jun/2012:22:57:22 +0100] "GET / HTTP/1.1" 403 559 "-" "YahooCacheSystem"
98.139.241.245 - - [25/Jun/2012:02:49:01 +0100] "GET / HTTP/1.1" 200 6010 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9650; en-US) AppleWebKit/534.8+ (KHTML, like Gecko) Version/6.0.0.524 Mobile Safari/534.8+"
98.139.241.245 - - [25/Jun/2012:02:49:02 +0100] "GET /favicon.ico HTTP/1.1" 200 318 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9650; en-US) AppleWebKit/534.8+ (KHTML, like Gecko) Version/6.0.0.524 Mobile Safari/534.8+"
98.139.241.243 - - [25/Jun/2012:04:22:21 +0100] "GET / HTTP/1.1" 200 6010 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9670; en-US) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.407 Mobile Safari/534.1+"
98.139.241.243 - - [25/Jun/2012:04:22:22 +0100] "GET /favicon.ico HTTP/1.1" 200 318 "-" "Mozilla/5.0 (BlackBerry; U; BlackBerry 9670; en-US) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.407 Mobile Safari/534.1+"
8:42 pm on June 28, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


Did it make sense to you at the time?

Had you blocked 98.139.241/24 would you have seen the 403 or would your BB still have loaded the proper requested page? And if the latter, from a previous cache or from your site?
9:23 pm on June 28, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5408
votes: 2


Did it make sense to you at the time?


No it did not.

Don't believe I've the Ip range denied, rather the UA YahooCacheSystem

or perhaps even portions of same UA.
8:50 pm on June 29, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


Thanks. I was wondering what effect my blocking the IP range had on what people saw on mobile devices. Not being able to check that kind of thing is about the only drawback of not having a mobile device. :)
1:14 am on Oct 11, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5408
votes: 2


FWIW

98.139.241.241 - - [11/Oct/2012:00:23:03 +0100] "GET / HTTP/1.1" 403 559 "-" "YahooCacheSystem"
98.139.241.248 - - [11/Oct/2012:01:25:05 +0100] "GET / HTTP/1.1" 403 559 "-" "Mozilla/5.0 (Linux; U; Android 4.0.4; en-ca; SGH-I747M Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members