homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum


 5:31 am on Apr 10, 2013 (gmt 0)

I flushed all my old bot info and started new DB in December.

I don't remember seeing this so I am posting it.

ScreenerBot Crawler Beta 2.0 (+http://www.ScreenerBot.com)
Came from IP:

Webpage reported it will come from the following IP's

No Robots.txt

No idea what it does, or what it will do with its data.

added "/?dgcr=1" to my homepage url.



 5:52 am on Apr 10, 2013 (gmt 0)

You need to update your data center lists as that's good old LeaseWeb which has a decent list of IP ranges.

I used to think they really had leased the whole web.


 8:28 pm on Apr 10, 2013 (gmt 0)

Ah ha! Thought that name sounded familiar. I was looking up some undesirable activity from that selfsame 46.165 neighborhood and one of the perps was ScreenerBot. There's also a goodish bit of MJ12. I don't block them by name, but if they choose to crawl from bad neighborhoods they can jolly well deal with the consequences. Besides, three-quarters of their hits are 301 or 404-- not because my site has changed or my internal links are bad, but because they can't get the names right.

They sneakily tried to say it was only 192.0/21 but nuh-uh, I'm not falling for that one. Checked the bottom half of .128. as well since it's the same country, but that seems to be assorted other entities.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved