Welcome to WebmasterWorld Guest from 54.159.50.111

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

BeetleBot

188.207.200.116

     
10:15 pm on Nov 13, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:July 26, 2006
posts:1619
votes: 0


Mozilla/5.0+(compatible;+BeetleBot;+)

188.207.200.116
10:47 pm on Nov 14, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3092
votes: 2


49 hits on that IP (and no other in the range 188.207.0.0/16) since April.
12:08 am on Nov 15, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:5808
votes: 64


Bewenched could you supply more info? Did it ask for robots.txt? If so did it obey? What type files did it retrieve? How many?
9:19 am on Nov 15, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5408
votes: 2


188.207.200.116 - - [12/Nov/2012:09:16:34 +0000] "GET /robots.txt HTTP/1.0" 200 2719 "-" "Mozilla/5.0 (compatible; BeetleBot; )"
188.207.200.116 - - [12/Nov/2012:09:16:35 +0000] "GET /MyFolder/MySub/MyPage.html HTTP/1.0" 403 559 "-" "Mozilla/5.0 (compatible; BeetleBot; )"

keyplr,
note the trailing broken space.

FWIW, this thing appeared a very short while after I added the denial for Inktomi, and requesting the same page that Inktomi requested.
It's likely just a coincidence, however the requests were close enough to through up a flag.
This particular page is plentiful with links leading to it, and diverse widget keywords.
4:02 pm on Nov 17, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 26, 2002
posts:3292
votes: 6


Oops!
1:47 am on Nov 18, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:July 26, 2006
posts:1619
votes: 0


Interesting that it showed up after inktomi for you... I haven't blocked inktomi at all, but this thing showed up, did request robots.... I've blocked it now... it will be interesting if it complies with robots.txt.