Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- MSNBot posing as Internet Explorer 7 and not reading robots.txt?
wilderness - 2:01 am on Jun 26, 2013 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4587076.htm
|SetEnvIf User-Agent "; " keep_out |
FWIW, the correct syntax is with two trailing spaces.
SetEnvIf User-Agent "; " keep_out
Brought to you by WebmasterWorld: http://www.webmasterworld.com