Forum Moderators: phranque
I was wondering if that thread died on part 3 back in April or am I missing part 4? Is there a better faster way to block spambots, spybots, etc, these days?
Can someone like jdMorgan or the like provide a final example as to what they use so us greenhorns can look at it?
Thanks in advance.
Welcome to WebmasterWorld [webmasterworld.com]!
That thread had a *very* long and happy life, though. A few comments:
My 'ban list' would be a disappointment compared to those posted in the thread -- it's very short. Checking the user-agents that actually visit my sites against those listed in the 'perfect list', I eliminated all those which only visited a few times a year (or never) and so cut my list to about 30 entries. Interestingly, I ended up with different lists for different sites. My approach was to include only those user-agents that were part of the top 95th percentile (by number of requests) of unwelcome vistitors.
The rest are handled using a derivative of a bad-bot trap script [webmasterworld.com] originally posted here on WebmasterWorld by member Key_Master. Member Birdman has also posted a PHP version [webmasterworld.com] of this script.
Member xlcus added another PHP bad-bot script [webmasterworld.com] to the arsenal, a time-based script that limits visitors to a certain number of requests per time interval.
These scripts block user-agents by IP address based on their behaviour; However, if you find a certain user-agent being blocked repeatedly from different IP addresses, you can "promote" that user-agent to your static user-agent block list.
Jim
