Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- msnbot-media


wilderness - 1:49 am on Jan 11, 2013 (gmt 0)


My own "system"* is pretty much the same except that it also uses mod_setenvif and mod_auth-whatever-it-is. And I get along better with CIDR ranges, on account of having learned binary at an early age.


FWIW, I converted some of my mod_rewrite IP's to mod_auth (deny from) and CIDR's, despite the CIDR looking like pure gibberish and requiring six calculators and four website to convert to a comprehensible ten-finger-ten-toe format.

The reduction in file size thus far was 25k+ or an approximate 20%.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4470273.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com