hot off the presses:
Started seeing this new robot a couple days ago.
185.10.104.194 - - [date] "GET /ebooks/perez/PerezEsp.html HTTP/1.0" 200 12948 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)"
Looking at the IP you would guess it's a brand-new Ukrainian robot, and you would be half right. In fact it's Baidu Hong Kong
185.10.104.0/22
Now, it's possible they asked one of their sister Baidus for a copy of robots.txt and therefore didn't need to ask on their own behalf, but...
Naah.
If you guessed from all those directory slashes that the requested file is not directly linked from the front page (which, in any case, they didn't ask for), you would be right.
If you guessed that the content of this particular page is in the public domain, you would also be right-- but the same does not apply to the robot's subsequent requests.
Let's stick with the first guess: Shoot to kill.
:: now back to wondering what the ### the PiplBot wants with my favicon ::
