Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

CrawlDaddy abot Site Harvester

7:00 pm on Sep 28, 2013 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 23, 2003
posts: 77
votes: 0

Did not request robots.txt file.
Banned itself by disobeying robots.txt standard directives.

Sample access log entry: - - [27/Sep/2013:23:59:14 -0400] "GET / HTTP/1.1" 403 362 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; CrawlDaddy v0.3.0 abot v1.2.0.0 http*://code.google.com/p/abot)"

Note: Asterisk (*) added by me to disable hot linking the above URL in this message.

Came from two remote host IPs:

The remote host IPs are owned by GoDaddy.com, LLC (GODAD).

The user of the two remote host IPs also attempted to find WordPress vulnerabilities. Here is a sample access log entry: - - [27/Sep/2013:23:59:12 -0400] "GET /wp-login.php HTTP/1.1" 403 366 "-" "User-Agent\tMozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

HTH . . .

10:56 pm on Sept 28, 2013 (gmt 0)

Moderator from US 

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
votes: 51

GoDaddy is a Hosting company, they can never send any visitors as they don't provide Internet Services like, say, Verizon does so it is generally safe to block Hosting companies' full range unless some script services you use are hosted there.

I do have them blocked across all sites. From your observations on what they were doing, I'd just block the whole GODaddy range.
11:37 pm on Sept 28, 2013 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
votes: 75

Report their behavior to GoDaddy.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members