Forum Moderators: open

Message Too Old, No Replies

Baidu bot bad behavior

Baidu is sending a barrage of requests to my Chinese language home page

         

digrimes

3:45 pm on Jul 28, 2006 (gmt 0)

10+ Year Member



6 times in the past 3 days I have received a barrage of requests from Baidu bot on my 2 Chinese language websites (.hk/.cn)

The behavior is the same on both sites - baidu is crawling deep pages on the site and then sends 80 - 100 requests to my home page over a 30 second span.

I don't usually have complaints about Baidu's behavior, and I don't want to restrict them from crawling the home page or any other page, but the behavior is rediculous. I don't mind them sending many requests, but there is no point to hitting the home page like this.

I don't currently use if-modified-since on the home page - does anyone know if Baidu sends that header?

Any advice is appreciated.

bsbaidu

7:40 am on Aug 5, 2006 (gmt 0)

10+ Year Member



Since baidu bot doesn't comply with any robot guidelines, the only possible solution is to rank your site #1 by excellent SEO skills. Then baidu salesperson will call and ask you to buy their paid ads. If you reject, your site will be removed at once from their index database. No baidu bot issue anymore then:)

Rendezvous

2:59 pm on Aug 5, 2006 (gmt 0)

10+ Year Member



I have the opposite problem. I have some 40,000 odd pages on one of my sites and Baidu, despite it's often reckless crawling behaviour, can't seem to keep up, and worse, it keeps crawling the same sections over and over again even though there has been no changes.