Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- msnbot-media


lucy24 - 1:01 am on Jan 11, 2013 (gmt 0)


My post was actually a response to wilderness, right above yours. You just slipped in and typed faster :)

As I understand it, wilderness's system is:

Me user. You mod_rewrite.
Me make rules. You obey.

I think he's even got mod_rewrite taking out the garbage and cleaning up the widget droppings.

My own "system"* is pretty much the same except that it also uses mod_setenvif and mod_auth-whatever-it-is. And I get along better with CIDR ranges, on account of having learned binary at an early age.


* You can apply sarcastic quotes to yourself. Who's going to be offended? ;)


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4470273.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com