Msg#: 4237913 posted 4:16 pm on Dec 2, 2010 (gmt 0)
Not seen anything serious from this IP range before but today there was a flurry of yahoo slurp china from it.
UA is: Yahoo! Slurp China
Which is not the usual Yahoo bot so it may be faked, although if true why so many IPs? Haven't check if it's true bot behaviour, robots.txt etc.
There may be more IPs within the Alisoft range running this bot but this is all I've seen so far. No Yahoo rDNS: all I've checked have been UNKNOWN-(ip)-aliyun.com. There may also be non-bot IPs within the given range - I haven't made substantial checks.
"Yahoo slurp" range (approx): 220.127.116.11 - 18.104.22.168
Msg#: 4237913 posted 5:56 pm on Dec 2, 2010 (gmt 0)
In the past there has not been any penalty by the major SE's if your website was North American-based and access was denied to same SE's from non-NA IP's.
I'm curious to know if the reverse is true?
If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges? Anybody have any insight on this?
Msg#: 4237913 posted 1:35 am on Dec 3, 2010 (gmt 0)
If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges?
Most of our sites are APNIC based. I don't think I have ever seen a verifiable, googlebot or bingbot/msnbot crawling from anything other than ARIN based IPs. Blocking those would be incredibly self-defeating (unless you don't want any traffic).
Yahoo only crawls from ARIN based IPs or Chinese - and I have been blocking Slurp China (APNIC based) for a very long time, with no apparent effect on our listings in Yahoo.
Msg#: 4237913 posted 9:55 pm on Dec 3, 2010 (gmt 0)
Looking longer at the IP ranges and at the UA, I'm now inclined to think this may have been a bad bot, not really Yahoo after all. But that's only my opinion. If it was I think it was probably from a compromised server.
Alisoft is a subsidiary of Alibaba and is apparently involved in a Chinese character input software project. but there seems to be several other sites in the group - can't say more 'cause I never learnt Chinese. :(
Msg#: 4237913 posted 1:53 pm on Dec 14, 2010 (gmt 0)
I had them for the first time in my logs today.
Host: 22.214.171.124 / Http Code: 200 Date: Dec 14 05:41:44 Http Version: HTTP/1.1 Size in Bytes: 56260 Referer: - Agent: Yahoo! Slurp China
I block all Chinese spiders so my intention was to apply it to this one too.
When looking up the IP I noticed that they are located in a Chinese Cloud. I already block any bot from the Amazon Cloud so I wont certainly allow a Chinese Cloud access to my website.
China Yahoo In October 2005, Alibaba Group formed a strategic partnership with Yahoo! Inc and acquired China Yahoo! (www.yahoo.com.cn), which is a Chinese portal offering search, email, and an enhanced focus on entertainment content.
# Alibaba Cloud Computing deny from 126.96.36.199/19