Forum Moderators: phranque
you should read through this extensive WebmasterWorld thread on blocking bots [webmasterworld.com].
Sure, I appreciate that it's not foolproof, but I just wondered if anyone had done this and what method they used? I have used .htaccess to block bots before, but have never used it to only *allow* certain user agents, rather than *disallow*.
Any help appreciated.
I tried the method here [webmasterworld.com...]
but it keeps giving me internal server errors.
[*EDIT] Found a solution, all is well again ;)