Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Server Farms - March 2013
keyplyr - 11:43 pm on Apr 7, 2013 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4552055.htm
Just FYI to all (especially to cut'n pasters): It is wise to check & recheck all ranges in block or allow lists periodically and conflate where needed. Often times there are overlaps. I know I've found a few.
Brought to you by WebmasterWorld: http://www.webmasterworld.com