incrediBILL - 5:29 pm on Jun 26, 2012 (gmt 0)

please excuse a newbie question, but how do you use headers to block bots?

Browsers send certain things in headers that most bots do not.

For instance, I can send you a 100% exact user agent string that matches Firefox 13 and you'll happily let it access your site.

However, had you bothered to also examine the headers being sent, you'll notice that one or two things that Firefox 13 always sends are not present or are incorrectly presented when the bot faking Firefox 13 sends a request.

Simple to check, simple to block.

Nice cup of 403 forbidden served steaming hot.