Welcome to WebmasterWorld Guest from 54.166.172.33

This page requires javascript to function properly. It seems that your browser does not have Javascript enabled. Please enable Javascript and press the Reload/Refresh button on your browser.

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

List of bad bot/spiders

Requesting a list of known bad bots

clembot

12:29 am on Sep 7, 2005 (gmt 0)

New User

joined:June 23, 2005
posts:5
votes: 0

Does anyone have a current list of bad bots to block. I am seeing more and more bots which I have no idea whether should be blocked. For example:

Libby_1.1/libwww-perl/5.76
ichiro/1.0_(ichiro@nttr.co.jp)
dlman
Wildsoft_Surfer
Mozilla_blah
nicebot
Sleipnir_Version_1.60

I tried looking around the forums and googled them, but can't decide which ones should be blocked.

Any help would be appreciated (either on the specific ones listed above, or someones current list of bad bots).

Thanks!

physics

2:59 am on Sept 7, 2005 (gmt 0)

Senior Member

joined:Feb 27, 2001
posts:2548
votes: 0

Check out A Close to Perfect .htaccess Ban List [webmasterworld.com] and the children of that thread.

clembot

11:19 am on Sept 7, 2005 (gmt 0)

New User

joined:June 23, 2005
posts:5
votes: 0

Thank you for the reference. I actually did see that thread before but thought it may be a little out of date (it ranges from 2000 to 2003). Does it still apply or is there a newer list available?