homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/)
Receptional Andy

 1:10 pm on Jun 30, 2008 (gmt 0)

Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/)

Straight into a basic bot trap. No real info at site (which was created a few days ago). Anonymised whois.



 6:56 pm on Jun 30, 2008 (gmt 0)

I did a search on this one last night, as well. Came across an interesting article (Google "WebDataCentreBot") that connected the dots between WebDataCentreBot and Midphase:
"Midphase.*** is the generic PTR record of a Softlayer reseller... Therefore, firewalling this particular range seems an appropriate solution to me..."

It did the same thing to me, got trapped, and drew attention to itself. Once I saw the cheesy temporary site for it, and its reluctance to share any meaningful information about its purpose, I banned it. No soup for them.


 7:11 pm on Jun 30, 2008 (gmt 0)

I've had the Class B denied for some time.

Receptional Andy

 2:56 pm on Jul 6, 2008 (gmt 0)

Came back today on 66.150.224.x


 6:48 pm on Jul 6, 2008 (gmt 0)

As far as I can see there is no stopping this bot. It doesn't respect robots.txt and comes slinking through like a hungry dog every time you turn around. It will gladly eat 403's for eternity until you firewall them out.


 8:15 pm on Jul 6, 2008 (gmt 0)

Since I whitelist Google, Yahoo, MSN, they go through first and then anything with "http://" in the UA is blocked so stuff like that is easily stopped.

Of course it has also blocked a couple of browser plug-ins in the past which in one case caused a small riot and got the "http://" removed from the UA of that plug-in ;)


 2:18 pm on Jul 10, 2008 (gmt 0)

Today from 82.96.19.**
No robots.txt


 8:04 pm on Jul 10, 2008 (gmt 0)

Appears to have added a new IP from Sweden (82.96.19.*). Its cheesy intro page says "driven by a small team of researchers from different parts of the world". Well, now it's officially International.

For such persistent and rude little bot it sure doesn't tell you much about itself.


 10:07 pm on Nov 5, 2008 (gmt 0)

Just got a visit from them also with the same UA as the original poster:
Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/)

It went through 50+ pages with an average of 2 seconds per page. I was also disappointed by their web site. I'm thinking of blocking it by UA... but at least it has one. Not sure.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved