Welcome to WebmasterWorld Guest from 54.80.219.236

Forum Moderators: Ocean10000 & keyplyr

IndustryIndexBot

     
3:58 am on Jun 13, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12557
votes: 839


UA: IndustryIndexBot/1.0 (+http://industryindex.com/bot/)
Protocol: HTTP/1.1
Robots.txt: No
Host: AWS
54.144.0.0 - 54.159.255.255
54.144.0.0/12

Marketing - 1st request was for "/" then for "ads.txt" then for "vendors.json"

Note: the URL in their UA string returns a 404 after several 301s (very poor webmastering.)
3:17 am on June 30, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12557
votes: 839


Also faking a browser...

54.84.66.** - - [29/Jun/2018:15:06:39 -0700] "GET / HTTP/1.1" 200 32130 "-" "IndustryIndexBot/1.0 (+http://industryindex.com/bot/)"

54.84.66.** - - [29/Jun/2018:15:06:40 -0700] "GET / HTTP/1.1" 403 4799 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"

54.84.66.** - - [29/Jun/2018:15:06:41 -0700] "GET /favicon HTTP/1.1" 403 1338 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"

I allow this marketing bot because I publish ads, but I do not allow sneaky covert behavior.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members