Msg#: 4033691 posted 10:07 pm on Nov 29, 2009 (gmt 0)
AppEngine-Google; ( [code.google.com...] appid: mapthislink) 64.233.172.nn No PTR ----- OrgName: Google Inc. OrgID: GOGL Address: 1600 Amphitheatre Parkway City: Mountain View StateProv: CA NetRange: 220.127.116.11 - 18.104.22.168 ----- READ ROBOTS.TXT? No OBEYED ROBOTS.TXT? No ----- We've discussed Google's AppEngine before, but this appears to be a new user agent.
I've decided to ban anything with AppEngine-Google in it as none of them has ever provided anything useful in exchange for access to my sites.
Msg#: 4033691 posted 7:39 pm on Dec 4, 2009 (gmt 0)
Two more hits today simultaneous with swarms of Twitter-related UAs confirm mapthislink's connection.
FWIW, I've yet to see any ill effect from blocking pretty much ALL Twitter fellow travelers for one reason or another, e.g., amazonaws.com-hosted; no-robots.txt; HEAD requests; bad or no UAs; ad nauseam hits, etc.
Msg#: 4033691 posted 7:43 am on Dec 5, 2009 (gmt 0)
@Gary: When UAs you report are in any way altered by your software, might it be possible for you to please also/instead post the original, actual strings? It's really difficult to confidently block on specifics otherwise.
For example, the UA in the OP would automatically get caught by a no-space-after-left-paren rule. But because a key bit is missing, replaced by a space, the rule wouldn't kick in and absent other rules, the bot would get in. So anyway, TIA if you can include unaltered UAs:)