-- Search Engine Spider and User Agent Identification
lucy24 - 10:14 pm on Dec 13, 2012 (gmt 0)
Yet another reason to block all translators!
Note however that the word "translate" or "translator" never appears anywhere in the logs so you can't block it by name. And that's assuming for the sake of discussion that the plainclothes MSNbot really is a translator, which I'm not at all sure of. I'm keeping it blocked pending solid proof. (I don't object to translators.)
This in turn reminds me that That Other Search Engine has taken to omitting the word "translate" from the part added to the UA string. I've been getting visits with only the "gzip" component added. Referer info confirms that it's still translation, not some suspicious new activity.
I've made non-English versions of my two most commonly translated pages. (A specific language for each.) But frankly it felt suicidal, since it basically means cutting each page's search numbers in half :(