Forum Moderators: open
Mozilla/5.0 (compatible; Diffbot/0.1; [diffbot.com)...]
I sent it packing cause it took pages too quickly and didn't bother reading robots.txt.
So, it came back the other day with a new UA and the same version number:
Mozilla/5.0 (compatible; MSIE 7.0; Diffbot/0.1; [diffbot.com;...] http://www.example.com/tutorials/tutorial.asp?TutorialID=48&CurPage=1)
64.71.190.132
mango.diffbot.com
-----
OrgName: Hurricane Electric, Inc.
I won't make any comment about the host, otherwise I'd be a hypocrite. But you know what I'm thinking, right? ;)
I am seeing more UAs with my domain name in them. That's one of my domains up there that's been examplified. Except the URL they're using is not generally visible cause I use extension-less URLs and rewriting. But it is an accurate URL. It got sent packing too.
Just thought you all would like to know about it cause I've seen no mention of it anywhere on WebmasterWorld.