Forum Moderators: goodroi
For the last several days running (that I've noticed), my Apache access logs are full of thousands of visits from Baiduspider. I realize it's a legitimate Chinese search engine spider, but I want to block it - I have no use for Chinese traffic.
In my logs I see NO access of robots.txt by Baiduspider (IP 61.135.145.205).
If I'm right that it doesn't use robots.txt, can anyone suggest a good way to block it entirely?
Thanks!
Me:
Win2k Pro SP4, Apache 2.0.54, MySQL 4,1,11, PHP 4.3.11
A new .htaccess file at the site root:
SetEnvIfNoCase User-Agent "^Baiduspider" bad_bot
<Limit GET POST>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
... so I'll see if this works first.
Previous method didn't work. But now I'm finally getting somewhere with this:
SetEnvIfNoCase User-Agent "^Baidu" bad_bot
<Directory />
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Directory>
Directly in my Apache httpd.conf
Now every request Baiduspider makes is getting a 403 error. I just wish I could make it go away altogether, coz it's still hitting me thousands of times a day :(