I'm getting Baiduspider+ (and less frequently Baiduspider) pummeling my sites and I would like to disallow it. I've added this to my robots.txt two days ago:
User-agent: Baiduspider
Disallow: /
They've read it 5 times already, still haven't stopped. The user agent string leads to an error page:
"Baiduspider+(+http://www.baidu.com/search/spider.htm)"
And of course it's in Chinese (Mandarin?) so I can't tell what it says.
So, do you guys know if they just don't obey robots.txt or they just take their sweet time to adjust to a robots.txt change? I'm tempted to just firewall them out but that would mean that they won't read any robots.txt changes and will keep pounding.
So, anyone knows how best to stop Baiduspider ?
P.S. I've added this today, will see what happens:
User-agent: Baiduspider+
Disallow: /