Forum Moderators: open
Its a dream come tru for people in need of a quality whois solution.
The amount of people who are using it and also recommending it is growing at a rapid rate from what I can gather.
I think its going to be 1 of those gems which will really take off (a bit like google actually)
Well done Lisa
Shak
[site name gets *'ed out]/info/webmasters/surveybot.html
Let's start with their "robots.txt" material, specifically point A which is then conflicted by point C - their bot does not read (and therefore honor) robots.txt meaning that sites have no method of opting-out of their business plan.
Generally (IMHO) that alone is enough to get it a ban - a quick browse through the site shows nothing which is unique enough to explain the need to deploy a bot...
Netcraft are *the* definitive source of webserver OS/uptime information for the moment so that isn't a unique/viable selling point (obviously IMO).
Secondly there are a million+1 domain search & sale websites (swhois.com etc) who don't need to "probe" sites to do their job, so again not a unique/viable selling point.
Aside from those two points I don't really see what that site actually does!
However the big stick against which all issues are measured for myself is how much traffic they bring in and at the moment these guys seem to bring in next to no traffic while still having the nerve to crawl my sites with no opt-out method which got them a ban.
- Tony
The worse offenders are those that send fake referers to ad pages. IMO, only sites with no future would resort to such a move.