Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- above the law? ignores ROBOTS.TXT
keyplyr - 10:41 am on May 17, 2013 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4573203.htm
One somewhat inebriated night I did a drive-by on a couple of people I know keep an eye on their log files and there was ZERO mention of the wacky user agents and whatnot in the forum.
So you're the one!
Brought to you by WebmasterWorld: http://www.webmasterworld.com