List of bad/spam bots out there?

[Grinler]

Is there a known list of bad,spam, malicious bots out there that someone has compiled? If so can you sticky me the url unless its ok to post it here.

Thanks

[Mokita]

... list of bad,spam, malicious bots

Caution: the list in the link above is just some UAs that have been mentioned in this forum. They are not necessarily all "bad" bots.

Also, there are many new ones that have surfaced since that list was last updated nearly two years ago.

[incrediBILL]

A lot of bots today look more like this:
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

[wilderness]

Also, there are many new ones that have surfaced since that list was last updated nearly two years ago.

And almost all of them primarily from the 60 Class ;)

Sorry Mokita, I just couldn't resist ;)

Don

There are also some very old threads that provide some insights and methods as well.
However . . . . as Mokita stated these are EVER-CHANGING:

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

There are numerous efforts by other to provide functional lists:
[joseluis.pellicer.org...]
[psychedelix.com...]
[botspot.com...]
[clearwaterbeachcam.com...]
[jafsoft.com...]
[projecthoneypot.org...]

[Mokita]

Sorry Mokita, I just couldn't resist

:)

[volatilegx]

Just a note: I deleted my link to a previous thread above, so Mokita's comments look out of place. They are still relevent however, because any list, once posted, immediately becomes out of date.

Don: thanks for posting those links.

[Romeo]

A starter could be to visit search.yahoo.com or www.google.com and search for "bad bots".
Browse through the first 10 or 15 hits -- should be lot of info there.

Kind regards,
R.

[wilderness]

I wish I understood Brett's logic for not allowing my list of bad bots to be linked to from WW.

Gary,
I haven't a clue as to whether the logic was Brett's or Dan's.
One thought is that it's a Webmaster World policy not to allow us to hawk or own websites.
Dan provides a website of SE and ranges and he may not provide that link, however you or I may (its a handy tool and effort at IDing the major SE bot ranges):

[iplists.com...]

In any event. . .many times in these forums the use of broken links or obscured IP ranges are permissible.

http ://www.microsoft.com
or
123.456.789.#*$!

Hope this helps

Don

[incrediBILL]

Um, Gary, the link is in your profile and we can all see it there ;)

[GaryK]

Dan, if you'd rather we not discuss this here please let me know and I'll drop it.

Don, The logic was Brett's. I asked him if my site could be considered an authoritative link and he told me no. I don't want to hawk my own site. I'm not sure if it's against WW's rules but it violates my own set of ethics and it's part of the TOS for every forum I run. It would be nice if someone else would think of the hard work I do once in awhile. But right now even if someone posted the link it would technically violate Brett's decision; although how anyone would know that I'm not sure. :)

Bill, you know to look at my profile. But what about everyone else who doesn't know I maintain all these lists? Why would they even consider looking at my profile?

[incrediBILL]

Why would they even consider looking at my profile?

Because they're lonely and you're so hot?

Heck if I know...

Trust me, lots of people look at profiles as I get the WebmasterWorld profile as a referrer all the time.

[Tastatura]

Gray,
Although it might not look "fair" , I do support WebmasterWorld's decision (or in general terms TOS) regarding links. It is very subjective decision who is an authoritative site on a topic, what is hard work, etc (and I can already hear “Well you allowed GaryK’s site, why can’t you mine” argument).This by no means implies that you didn’t put in hard work….

I do want to tell you that once I saw your post I just clicked on your profile and found your site. Then I went back to the WebmasterWorld thread and saw that incrediBILL already mentioned this. Point of all this is that people will find it – I would guess that for the most part people on this forum are not general audience, and have sufficient skill and knowledge to find it. Give it a few days and, check your logs and see how many referrals you get from WebmasterWorld :)

EDIT:

G: Why would they even consider looking at my profile?

iB: Because they're lonely and you're so hot?

Heck if I know...

:))))

[edited by: Tastatura at 10:33 pm (utc) on Aug. 14, 2006]

[Brett_Tabke]

Please don't post lists peoples/sites/systems full ip address in this thread like that. RR.com is the perfect example. You don't know what is commercial, what is leased, and what is owned rr space. They mingle consumer, commercial, private, and govt under rr.com ip space.

Posting an ip can be construed as an attempt at a ddos attack - which has happened several times here and required legal intervention to resolve. Nothing generates a c&d faster than doing that in this forum. If it weren't for Dan, this forum would have been gone permenantly when we closed it for those reasons a few years ago. posting guidelines were not made in a vacum...

A list of agent names mostly useless. If you are looking for the later, then check the old htaccess ban lists as 95% of them are still accurate [webmasterworld.com...]

<added>
clarified
</added>

[edited by: Brett_Tabke at 3:16 pm (utc) on Aug. 16, 2006]

[Brett_Tabke]

No - I mean in this thread only. That is the same posting guidelines that we have had in this forum for several years. The way dan is handling the rest of the forum is great. We obfuscate the d block on ips. 123.123.123.* There were suggested ips in this thread that were clearly to individual users and probably on dynamic ips too boot. It is why this forum is PreModerated. We are not going to post a big list of isp users that happen to run bots. That would be a c&d and a week of grief waiting to happen. That does no one any good.

[volatilegx]

Let me further clarify this issue (hopefully)

I do not obfuscate the IP addresses of visitors that are clearly search engine spiders.

I do obfuscate the IP addresses of visitors that are not clearly search engine spiders.

The reasoning behind this is that this forum is primarily about identifying search engine spiders. Its secondary purpose is discussing and identifying other bots and spiders.

Posting the IP addresses of various website visitors carries some legal liabilities for Webmasterworld, as Brett explained. We are attempting to reduce liability exposure for WebmasterWorld by only posting the IP addresses of search engine spiders.

User agents of any kind are welcome.

When starting a new thread, if you are unsure whether you should post a complete IP address, go ahead and do so. I will edit/obfuscate as necessary.

I'm sorry this thread sort of exploded into a policy discussion. Hopefully, the original question was answered in a satisfactory manner.

[GaryK]

Thank you both for the clarifications. :)