Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- LoneStarBot Doesn't Honor Robots.txt
dstiles - 6:58 pm on Jun 2, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4460658.htm
Thanks. I didn't have all of those ranges.
I have a note that 22.214.171.124 - 126.96.36.199 is bluehost but 188.8.131.52 - 184.108.40.206 is Strategic Systems Consulting (also blocked) - you include 69.89.15 in the rewrite.
Brought to you by WebmasterWorld: http://www.webmasterworld.com