Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- LoneStarBot Doesn't Honor Robots.txt
dstiles - 6:58 pm on Jun 2, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4460658.htm
Thanks. I didn't have all of those ranges.
I have a note that 18.104.22.168 - 22.214.171.124 is bluehost but 126.96.36.199 - 188.8.131.52 is Strategic Systems Consulting (also blocked) - you include 69.89.15 in the rewrite.
Brought to you by WebmasterWorld: http://www.webmasterworld.com