Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- WebDataCentreBot


koan - 10:07 pm on Nov 5, 2008 (gmt 0)


Just got a visit from them also with the same UA as the original poster:
Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/)

It went through 50+ pages with an average of 2 seconds per page. I was also disappointed by their web site. I'm thinking of blocking it by UA... but at least it has one. Not sure.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3686831.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com