-- Search Engine Spider and User Agent Identification
---- Baidu's IPs - Which Are Legit?
rowan194 - 3:28 pm on Aug 15, 2012 (gmt 0)
I understand that robots.txt doesn't block a site. What I meant was that Baidu don't seem to respect my requests in that file for them to not go anywhere on my site. Thus, I moved towards blocking at the IP level.