Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- What's the Best Way to Keep All Spiders/Bots Out?

inbound - 12:26 am on Oct 3, 2011 (gmt 0)


Thanks Lucy, that helps by effectively making it a certainty that certain things are out of the question. In the UK I had to choose a specific broadband provider to make sure I got a static IP.

Thread source:: http://www.webmasterworld.com/search_engine_spiders/4369764.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com