JD_Toims - 11:57 pm on Aug 1, 2013 (gmt 0)
I agree with blekko not necessarily being "bad" as a bot, but I don't like their practices and the way they ignore / misinterpret some onpage directives, so they've got a good bot, but their practices get them blocked from my sites.
what's a bad agent for one site may be beneficial for another site, so it's pointless.
It's easy to edit and the one I posted was not only built from right here in this forum, it's more robust than most, so rather than people needing to "find and code a list" as a "first line of defense" I posted one they can remove from easily to tailor to their specific needs.
This is quite clever and potentially speeds up the scanning vs. big long monolithic lists.
.htaccess speed is one of my "things" and you're correct, those rules "break" much more quickly than many of the lists I've seen here, because as soon as a letter does not match it moves on, where some of the lists here have 4 or 5 lines (or more) all beginning with W, even though the "W agents" could all be checked on a single line instead.
Also, .htaccess processing of mod_rewrite restarts from the top every time a rule is matched, so I try to "find as much speed" as I can to keep the sites I work on fast for real visitors.