- Search Engines
-- Search Engine Spider and User Agent Identification
---- Stale bad bot lists
- 1:14 pm on Mar 25, 2012
The question was, how can your htaccess (acting as bouncer) identify a new robot that it hasn't met before?
Under the scheme of white-listing your not required to ID every newcomer as you have all the doors closed.
(similar to a new colo customer coming from a colo range that you previously denied. Why waste your time attempting to add them to your most wanted list?)
Brought to you by