-- Search Engine Spider and User Agent Identification
---- Best method of blocking?
lucy24 - 9:56 pm on Sep 4, 2013 (gmt 0)
I have found the vast majority of directories are garbage, with all due respect to IncrediBill
The bare fact that his directories are actively maintained-- which was the point of his post --surely puts him in the 5% :)
Bill, do you have a recognizable UA that isn't subject to blocking by an ordinary responsible webmaster? An obvious example is the w3 link checker. Every time I run it I have to comment-out one line of UA blocks.
:: wandering off to figure out if I can code an exception within mod_setenvif ::