Welcome to WebmasterWorld Guest from 54.163.49.19

Forum Moderators: phranque

Message Too Old, No Replies

Bad Bot List

     
3:35 pm on Feb 23, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12166
votes: 51


I'm looking to see if there is an updated (as of 2008) Bad Bot List around. I'm aware of the "Close to perfect htaccess ban" that resides on WebmasterWorld but that hasn't been updated in quite some time. Is there a new rendition of that somewhere here? Or do we need one?
4:48 pm on Feb 26, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Apr 28, 2006
posts:1043
votes: 1


I think that would be a great idea, I have the bad bot list myself. Unfortunately, I think its going to be a copy and paste session from hell. Much like what g1smd did with the various google datacenter IP ranges.
I'd almost take it a step further, and maybe include in the thread of how to set up a bad bot trap, as well as including the reverse dns lookup abilitys to catch bots using spoofed user-agents. Maybe end up being a stick at the top of the various subfourms.
5:02 pm on Feb 26, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12166
votes: 51


Since I started this topic, I've come to find out there are products on the market that may fulfill our requirements. Some are only for Linux at the moment, others are in beta and getting ready to hit the market. There's a new cottage industry in the mix and it all has to do with locking down your servers from rogue visitors. Instead of battling this in the trenches, we're going to move up to the firewall level and work there. We're on a .NET platform so we have our work cut out for us. It's in process. :)
5:09 pm on Feb 26, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Apr 28, 2006
posts:1043
votes: 1


Ouch, .NET hurts. I wouldn't touch that with a 10 foot pole. I wish you all the luck in that endeavor.
12:22 pm on Mar 31, 2008 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 7, 2005
posts:636
votes: 0


I'm getting hit hard with a range of IP's from Sweden just now. Can you point me in the direction of some webmasterworld tutorials with dealing with this. It's making my forum look like there are many hundreds more visitors online than there actually are in reality, not to mention the added load on the server.