wilderness
msg:4237956 | 5:56 pm on Dec 2, 2010 (gmt 0) |
In the past there has not been any penalty by the major SE's if your website was North American-based and access was denied to same SE's from non-NA IP's.
I'm curious to know if the reverse is true?
If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges?
Anybody have any insight on this?
|
dstiles
msg:4238060 | 10:26 pm on Dec 2, 2010 (gmt 0) |
Our server is UK based with mostly either UK or COM domains.
I block Chinese bots such as baidu and yahoo whilst letting in their Japanese versions because one of my clients has Japanese customers and one of our own sites features well in Japan.
Can't read the Japanese baidu site (eg) but his site comes up in it, although ours seems not to. Both come up in Japanese yahoo. The sites do not feature in the Chinese baidu or yahoo SEs.
I think (although not proven) that blocking (eg) bing or google here would be a big mistake. :)
|
Mokita
msg:4238130 | 1:35 am on Dec 3, 2010 (gmt 0) |
wilderness wrote:
| If your website is either RIPE or APNIC based and you denied access to North American-based-major-SE's while allowing their bots to spider from their perspective RIPE or APNIC ranges? |
|
Most of our sites are APNIC based. I don't think I have ever seen a verifiable, googlebot or bingbot/msnbot crawling from anything other than ARIN based IPs. Blocking those would be incredibly self-defeating (unless you don't want any traffic).
Yahoo only crawls from ARIN based IPs or Chinese - and I have been blocking Slurp China (APNIC based) for a very long time, with no apparent effect on our listings in Yahoo.
HTH.
|
dstiles
msg:4238503 | 9:55 pm on Dec 3, 2010 (gmt 0) |
Looking longer at the IP ranges and at the UA, I'm now inclined to think this may have been a bad bot, not really Yahoo after all. But that's only my opinion. If it was I think it was probably from a compromised server.
Alisoft is a subsidiary of Alibaba and is apparently involved in a Chinese character input software project. but there seems to be several other sites in the group - can't say more 'cause I never learnt Chinese. :(
|
MxAngel
msg:4242277 | 1:53 pm on Dec 14, 2010 (gmt 0) |
I had them for the first time in my logs today.
Host: 110.75.172.109
/
Http Code: 200 Date: Dec 14 05:41:44 Http Version: HTTP/1.1 Size in Bytes: 56260
Referer: -
Agent: Yahoo! Slurp China
I block all Chinese spiders so my intention was to apply it to this one too.
When looking up the IP I noticed that they are located in a Chinese Cloud. I already block any bot from the Amazon Cloud so I wont certainly allow a Chinese Cloud access to my website.
[linkedin.com...]
From: [en.wikipedia.org...]
China Yahoo
In October 2005, Alibaba Group formed a strategic partnership with Yahoo! Inc and acquired China Yahoo! (www.yahoo.com.cn), which is a Chinese portal offering search, email, and an enhanced focus on entertainment content.
# Alibaba Cloud Computing
deny from 110.75.160.0/19
|
|
