Forum Moderators: open

Message Too Old, No Replies

Diffbot

I sent this packing a few months ago. Now it's back with a new UA

         

GaryK

2:15 am on Apr 21, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Back in January I got heavy hits from:

Mozilla/5.0 (compatible; Diffbot/0.1; [diffbot.com)...]

I sent it packing cause it took pages too quickly and didn't bother reading robots.txt.

So, it came back the other day with a new UA and the same version number:

Mozilla/5.0 (compatible; MSIE 7.0; Diffbot/0.1; [diffbot.com;...] http://www.example.com/tutorials/tutorial.asp?TutorialID=48&CurPage=1)
64.71.190.132
mango.diffbot.com
-----
OrgName: Hurricane Electric, Inc.

I won't make any comment about the host, otherwise I'd be a hypocrite. But you know what I'm thinking, right? ;)

I am seeing more UAs with my domain name in them. That's one of my domains up there that's been examplified. Except the URL they're using is not generally visible cause I use extension-less URLs and rewriting. But it is an accurate URL. It got sent packing too.

Just thought you all would like to know about it cause I've seen no mention of it anywhere on WebmasterWorld.

keyplyr

9:01 am on Apr 21, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I won't make any comment about the host, otherwise I'd be a hypocrite. But you know what I'm thinking, right? ;)

;) Thanks for the heads-up

dstiles

8:51 pm on Apr 21, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've seen a few UAs with my own domains in. In some cases I've added them to the UA blocklist.

Meenakshi Mosaic

9:38 am on Apr 22, 2009 (gmt 0)

10+ Year Member



I am unable to see the URL. It is not working...

keyplyr

5:06 pm on Apr 22, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I am unable to see the URL. It is not working...

Just remove the semi-colon at the end.