I just banned this with SetEnvIf User-Agent "id\-search" keep_out
There is another bot called GridBot: RewriteCond %{HTTP_USER_AGENT} ^.*GridBot [OR]
Do you usually ban through User Agent or IP ranges?
keyplyr
12:34 am on Feb 27, 2017 (gmt 0)
Each webmaster determines if a bot or user agent is beneficial or not to their interests. This is different for each webmaster.
I generally block by UA if the bot is distributed, comes from many IP ranges or ISP ranges and/or doesn't get caught by my other filters (IP range, headers, method & behavior.)
I generally block by IP range if I *only* see the bot come from IP ranges I already have blocked.
Note: I personally block all server farm ranges, then allow exception for UAs I determine are beneficial to my interests.
lucy24
2:15 am on Feb 27, 2017 (gmt 0)
^.*
This element is never necessary. Depending on how your server handles string matching, it may even add a picosecond or so to rule execution.
TorontoBoy
2:39 am on Feb 27, 2017 (gmt 0)
Thank you both. I guess I have built up bad habits with the ^.* prefix, still in my old RewriteCond's. I've not used them in my SetEnvIfs.
