Forum Moderators: open
I've decided to ban anything with AppEngine-Google in it as none of them has ever provided anything useful in exchange for access to my sites.
64.233.172.18
AppEngine-Google; (+http://code.google.com/appengine; appid: mapthislink)
robots.txt? NO
(The file requested was a recent Twitter mention. But thus far, "mapthislink" doesn't appear to be Twitter-specific.)
FWIW, I've yet to see any ill effect from blocking pretty much ALL Twitter fellow travelers for one reason or another, e.g., amazonaws.com-hosted; no-robots.txt; HEAD requests; bad or no UAs; ad nauseam hits, etc.
I've decided to ban anything with AppEngine-Google in it as none of them has ever provided anything useful in exchange for access to my sites.
I have the same conclusion. This method seems to be faddish at the moment amongst those who enjoy the novelty of masquerading as "Google approved" via a Google IP range.
For example, the UA in the OP would automatically get caught by a no-space-after-left-paren rule. But because a key bit is missing, replaced by a space, the rule wouldn't kick in and absent other rules, the bot would get in. So anyway, TIA if you can include unaltered UAs:)