Forum Moderators: open

Message Too Old, No Replies

Baidu Image Spider

Is it valid?

         

GaryK

5:16 pm on Feb 22, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



BaiduImagespider ( [baidu.jp...]
122.152.129.178
ip-122-152-129-178.asianetcom.net
% [whois.apnic.net node-1]
% Whois data copyright terms [apnic.net...]
inetnum: 122.152.128.0 - 122.152.129.255
netname: BAIDU-NRT-NETBLK01
descr: Baidu Kabushiki Gaisha

This is the first time I've seen this UA. Normally I like Baidu and let it crawl my sites. Everything about this UA seems valid. However it did not read robots.txt.

Has anyone else seen this UA and if so what's your opinion of it?

Thanks. :)

wilderness

10:58 pm on Feb 22, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Gary,
I've had it denied since 2003.
Fisrt time I've seen the "images" portion added.

Don

GaryK

9:00 pm on Feb 23, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks, Don. I like the normal Baidu bot because one of my sites gets a lot of traffic from Baidu. Same thing with Yandex in Russia. But I hate image bots in general. Especially ones that don't even bother to read robots.txt. I suppose it might have used the robots.txt that the regular Baidu bot uses expect it took images and they're all disallowed in robots.txt. I sent a complaint to Baidu and banned the bot.

wilderness

2:27 am on Feb 24, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Gary,
I suppose. . .there are specific instances where webmasters (photographers or artists especially) would prefer to have their images spidered and archived outside of their own website?

Personally, I can relate a story which was responsible for all my websites images (around 5k) being numbered, rather than named and being placed in an image folder which bots are not tolerated. (even non-bots spidering images are easily spotted today).

Before the aforementioned solution implementation, I had a a nothing image (however unique) named widgets.
Many folks searched for widgets.jpg and lo and behold thousands of visitor arrived daily to view this image. NOT my page content and not ANY page content.
As a result, what possible benefit was it to myself or my websites to make this image available? (rhetorical)

Don

keyplyr

11:02 am on Feb 24, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I suppose. . .there are specific instances where webmasters (photographers or artists especially) would prefer to have their images spidered and archived outside of their own website?

It can generate significant traffic for some types of sites, especially if one were to have frame-buster scripts installed :)