Welcome to WebmasterWorld Guest from 54.144.77.26

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

BeetleBot

188.207.200.116

     

Bewenched

10:15 pm on Nov 13, 2012 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Mozilla/5.0+(compatible;+BeetleBot;+)

188.207.200.116

dstiles

10:47 pm on Nov 14, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



49 hits on that IP (and no other in the range 188.207.0.0/16) since April.

keyplyr

12:08 am on Nov 15, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Bewenched could you supply more info? Did it ask for robots.txt? If so did it obey? What type files did it retrieve? How many?

wilderness

9:19 am on Nov 15, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



188.207.200.116 - - [12/Nov/2012:09:16:34 +0000] "GET /robots.txt HTTP/1.0" 200 2719 "-" "Mozilla/5.0 (compatible; BeetleBot; )"
188.207.200.116 - - [12/Nov/2012:09:16:35 +0000] "GET /MyFolder/MySub/MyPage.html HTTP/1.0" 403 559 "-" "Mozilla/5.0 (compatible; BeetleBot; )"

keyplr,
note the trailing broken space.

FWIW, this thing appeared a very short while after I added the denial for Inktomi, and requesting the same page that Inktomi requested.
It's likely just a coincidence, however the requests were close enough to through up a flag.
This particular page is plentiful with links leading to it, and diverse widget keywords.

jimbeetle

4:02 pm on Nov 17, 2012 (gmt 0)

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Oops!

Bewenched

1:47 am on Nov 18, 2012 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Interesting that it showed up after inktomi for you... I haven't blocked inktomi at all, but this thing showed up, did request robots.... I've blocked it now... it will be interesting if it complies with robots.txt.