New Yahoo China bot range

[dstiles]

Not seen anything serious from this IP range before but today there was a flurry of yahoo slurp china from it.

UA is: Yahoo! Slurp China

Which is not the usual Yahoo bot so it may be faked, although if true why so many IPs? Haven't check if it's true bot behaviour, robots.txt etc.

There may be more IPs within the Alisoft range running this bot but this is all I've seen so far. No Yahoo rDNS: all I've checked have been UNKNOWN-(ip)-aliyun.com. There may also be non-bot IPs within the given range - I haven't made substantial checks.

"Yahoo slurp" range (approx): 110.75.171.0 - 110.75.176.255

Host rangeAlisoft Singapore: 110.75.160.0 - 110.75.191.255

Part of China CNNIC range.

[wilderness]

In the past there has not been any penalty by the major SE's if your website was North American-based and access was denied to same SE's from non-NA IP's.

I'm curious to know if the reverse is true?

If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges?
Anybody have any insight on this?

[dstiles]

Our server is UK based with mostly either UK or COM domains.

I block Chinese bots such as baidu and yahoo whilst letting in their Japanese versions because one of my clients has Japanese customers and one of our own sites features well in Japan.

Can't read the Japanese baidu site (eg) but his site comes up in it, although ours seems not to. Both come up in Japanese yahoo. The sites do not feature in the Chinese baidu or yahoo SEs.

I think (although not proven) that blocking (eg) bing or google here would be a big mistake. :)

[Mokita]

wilderness wrote:

If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges?

Most of our sites are APNIC based. I don't think I have ever seen a verifiable, googlebot or bingbot/msnbot crawling from anything other than ARIN based IPs. Blocking those would be incredibly self-defeating (unless you don't want any traffic).

Yahoo only crawls from ARIN based IPs or Chinese - and I have been blocking Slurp China (APNIC based) for a very long time, with no apparent effect on our listings in Yahoo.

HTH.

[dstiles]

Looking longer at the IP ranges and at the UA, I'm now inclined to think this may have been a bad bot, not really Yahoo after all. But that's only my opinion. If it was I think it was probably from a compromised server.

Alisoft is a subsidiary of Alibaba and is apparently involved in a Chinese character input software project. but there seems to be several other sites in the group - can't say more 'cause I never learnt Chinese. :(

[MxAngel]

I had them for the first time in my logs today.

Host: 110.75.172.109
/
Http Code: 200 Date: Dec 14 05:41:44 Http Version: HTTP/1.1 Size in Bytes: 56260
Referer: -
Agent: Yahoo! Slurp China

I block all Chinese spiders so my intention was to apply it to this one too.

When looking up the IP I noticed that they are located in a Chinese Cloud. I already block any bot from the Amazon Cloud so I wont certainly allow a Chinese Cloud access to my website.

[linkedin.com...]

From: [en.wikipedia.org...]

China Yahoo
In October 2005, Alibaba Group formed a strategic partnership with Yahoo! Inc and acquired China Yahoo! (www.yahoo.com.cn), which is a Chinese portal offering search, email, and an enhanced focus on entertainment content.

# Alibaba Cloud Computing
deny from 110.75.160.0/19