Welcome to WebmasterWorld Guest from 34.238.189.171

Forum Moderators: Ocean10000

Message Too Old, No Replies

List of bad/spam bots out there?

     
1:56 pm on Aug 12, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 6, 2004
posts:84
votes: 0


Is there a known list of bad,spam, malicious bots out there that someone has compiled? If so can you sticky me the url unless its ok to post it here.

Thanks

8:11 pm on Aug 12, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 21, 2005
posts:379
votes: 0


... list of bad,spam, malicious bots

Caution: the list in the link above is just some UAs that have been mentioned in this forum. They are not necessarily all "bad" bots.

Also, there are many new ones that have surfaced since that list was last updated nearly two years ago.

8:27 pm on Aug 12, 2006 (gmt 0)

System Operator from US 

incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


A lot of bots today look more like this:
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
8:36 pm on Aug 12, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5506
votes: 5


Also, there are many new ones that have surfaced since that list was last updated nearly two years ago.

And almost all of them primarily from the 60 Class ;)

Sorry Mokita, I just couldn't resist ;)

Don

There are also some very old threads that provide some insights and methods as well.
However . . . . as Mokita stated these are EVER-CHANGING:

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

There are numerous efforts by other to provide functional lists:
[joseluis.pellicer.org...]
[psychedelix.com...]
[botspot.com...]
[clearwaterbeachcam.com...]
[jafsoft.com...]
[projecthoneypot.org...]

9:07 pm on Aug 12, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 21, 2005
posts:379
votes: 0


Sorry Mokita, I just couldn't resist

:)

4:57 pm on Aug 14, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


Just a note: I deleted my link to a previous thread above, so Mokita's comments look out of place. They are still relevent however, because any list, once posted, immediately becomes out of date.

Don: thanks for posting those links.

6:06 pm on Aug 14, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 28, 2002
posts:505
votes: 0


A starter could be to visit search.yahoo.com or www.google.com and search for "bad bots".
Browse through the first 10 or 15 hits -- should be lot of info there.

Kind regards,
R.

8:06 pm on Aug 14, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5506
votes: 5


I wish I understood Brett's logic for not allowing my list of bad bots to be linked to from WW.

Gary,
I haven't a clue as to whether the logic was Brett's or Dan's.
One thought is that it's a Webmaster World policy not to allow us to hawk or own websites.
Dan provides a website of SE and ranges and he may not provide that link, however you or I may (its a handy tool and effort at IDing the major SE bot ranges):

[iplists.com...]

In any event. . .many times in these forums the use of broken links or obscured IP ranges are permissible.

http ://www.microsoft.com
or
123.456.789.#*$!

Hope this helps

Don

8:15 pm on Aug 14, 2006 (gmt 0)

System Operator from US 

incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


Um, Gary, the link is in your profile and we can all see it there ;)
9:42 pm on Aug 14, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 17, 2002
posts:2251
votes: 0


Dan, if you'd rather we not discuss this here please let me know and I'll drop it.

Don, The logic was Brett's. I asked him if my site could be considered an authoritative link and he told me no. I don't want to hawk my own site. I'm not sure if it's against WW's rules but it violates my own set of ethics and it's part of the TOS for every forum I run. It would be nice if someone else would think of the hard work I do once in awhile. But right now even if someone posted the link it would technically violate Brett's decision; although how anyone would know that I'm not sure. :)

Bill, you know to look at my profile. But what about everyone else who doesn't know I maintain all these lists? Why would they even consider looking at my profile?

10:25 pm on Aug 14, 2006 (gmt 0)

System Operator from US 

incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


Why would they even consider looking at my profile?

Because they're lonely and you're so hot?

Heck if I know...

Trust me, lots of people look at profiles as I get the WebmasterWorld profile as a referrer all the time.

10:31 pm on Aug 14, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 10, 2006
posts:627
votes: 0


Gray,
Although it might not look "fair" , I do support WebmasterWorld's decision (or in general terms TOS) regarding links. It is very subjective decision who is an authoritative site on a topic, what is hard work, etc (and I can already hear “Well you allowed GaryK’s site, why can’t you mine” argument).This by no means implies that you didn’t put in hard work….

I do want to tell you that once I saw your post I just clicked on your profile and found your site. Then I went back to the WebmasterWorld thread and saw that incrediBILL already mentioned this. Point of all this is that people will find it – I would guess that for the most part people on this forum are not general audience, and have sufficient skill and knowledge to find it. Give it a few days and, check your logs and see how many referrals you get from WebmasterWorld :)

EDIT:


G: Why would they even consider looking at my profile?

iB: Because they're lonely and you're so hot?

Heck if I know...

:))))

[edited by: Tastatura at 10:33 pm (utc) on Aug. 14, 2006]

6:39 pm on Aug 15, 2006 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


Please don't post lists peoples/sites/systems full ip address in this thread like that. RR.com is the perfect example. You don't know what is commercial, what is leased, and what is owned rr space. They mingle consumer, commercial, private, and govt under rr.com ip space.

Posting an ip can be construed as an attempt at a ddos attack - which has happened several times here and required legal intervention to resolve. Nothing generates a c&d faster than doing that in this forum. If it weren't for Dan, this forum would have been gone permenantly when we closed it for those reasons a few years ago. posting guidelines were not made in a vacum...

A list of agent names mostly useless. If you are looking for the later, then check the old htaccess ban lists as 95% of them are still accurate [webmasterworld.com...]

<added>
clarified
</added>

[edited by: Brett_Tabke at 3:16 pm (utc) on Aug. 16, 2006]

3:14 pm on Aug 16, 2006 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38254
votes: 113


No - I mean in this thread only. That is the same posting guidelines that we have had in this forum for several years. The way dan is handling the rest of the forum is great. We obfuscate the d block on ips. 123.123.123.* There were suggested ips in this thread that were clearly to individual users and probably on dynamic ips too boot. It is why this forum is PreModerated. We are not going to post a big list of isp users that happen to run bots. That would be a c&d and a week of grief waiting to happen. That does no one any good.
4:50 pm on Aug 16, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


Let me further clarify this issue (hopefully)

I do not obfuscate the IP addresses of visitors that are clearly search engine spiders.

I do obfuscate the IP addresses of visitors that are not clearly search engine spiders.

The reasoning behind this is that this forum is primarily about identifying search engine spiders. Its secondary purpose is discussing and identifying other bots and spiders.

Posting the IP addresses of various website visitors carries some legal liabilities for Webmasterworld, as Brett explained. We are attempting to reduce liability exposure for WebmasterWorld by only posting the IP addresses of search engine spiders.

User agents of any kind are welcome.

When starting a new thread, if you are unsure whether you should post a complete IP address, go ahead and do so. I will edit/obfuscate as necessary.

I'm sorry this thread sort of exploded into a policy discussion. Hopefully, the original question was answered in a satisfactory manner.

8:36 pm on Aug 16, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 17, 2002
posts:2251
votes: 0


Thank you both for the clarifications. :)
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members