Welcome to WebmasterWorld Guest from 54.80.58.121

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

List of bad bot/spiders

Requesting a list of known bad bots

     
12:29 am on Sep 7, 2005 (gmt 0)

New User

10+ Year Member

joined:June 23, 2005
posts:5
votes: 0


Does anyone have a current list of bad bots to block. I am seeing more and more bots which I have no idea whether should be blocked. For example:

Libby_1.1/libwww-perl/5.76
ichiro/1.0_(ichiro@nttr.co.jp)
dlman
Wildsoft_Surfer
Mozilla_blah
nicebot
Sleipnir_Version_1.60

I tried looking around the forums and googled them, but can't decide which ones should be blocked.

Any help would be appreciated (either on the specific ones listed above, or someones current list of bad bots).

Thanks!

2:59 am on Sept 7, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 27, 2001
posts:2548
votes: 0


Check out A Close to Perfect .htaccess Ban List [webmasterworld.com] and the children of that thread.
11:19 am on Sept 7, 2005 (gmt 0)

New User

10+ Year Member

joined:June 23, 2005
posts:5
votes: 0


Thank you for the reference. I actually did see that thread before but thought it may be a little out of date (it ranges from 2000 to 2003). Does it still apply or is there a newer list available?
8:35 pm on Sept 8, 2005 (gmt 0)

Administrator

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 24, 2001
posts:15756
votes: 0


the child threads go all the way up to 2004 and, for the most part, those threads are pretty inclusive and current.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members