Welcome to WebmasterWorld Guest from 50.16.24.12

Forum Moderators: Ocean10000 & incrediBILL

Whois.sc bot goes undercover

No way to opt out?

   
6:32 am on Feb 12, 2003 (gmt 0)

10+ Year Member



I know that this has been discussed before.
[webmasterworld.com...]

But -Now they are back to spidering without identifying themselves. They have stated on their info page that their bot does not read robots.txt and that there is no way to opt out of being probed by them. So for those of us that would like to "opt out".

We see two different user agents for these visits. Sometimes, the bot shows us
user agent = "SurveyBot/2.2" followed by a cheap link to the site complete with anchor text (just in case your log files getlisted in GG). Other times the bot attempts to pretend to be a browser with this user agent:
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

The referrer will be "http://www(link to the site in question).sc" or "http://www.virtual-private-server.com/"

This bot uses these ip addresses:

216.145.11.94
38.116.0.122
198.78.175.190
198.78.175.42
38.116.0.6
66.228.210.10
66.228.210.50
66.228.202.22

11:19 am on Feb 12, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Ever notice GoogleBot following that bot? (just wondering).
4:36 pm on Feb 12, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Do you have any evidence this bot and Googlebot might be related, Brett?
4:41 pm on Feb 12, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



no. just asking the question is all. (fishing expedition based on several fish in the boat).
4:51 pm on Feb 12, 2003 (gmt 0)

10+ Year Member



No I haven't noticed that. Of course I have had them banned since before they began to identify themselves. Call me stubborn but I refuse to play the game with any copyright bot, plagarism bot, server survey bot, etc.

We reserve the right to refuse service to anyone - as it were.

What a great way to start a rumor and get the ban lifted by some people out of fear though.

9:40 pm on Feb 20, 2003 (gmt 0)

10+ Year Member



You can add 216.145.5.42 to the list.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month