Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- deny all
wilderness - 3:50 pm on May 14, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4422323.htm
Many thanks again.
I put this in place last night.
Tested and added a page, and will begin adding a few preliminary pages to see how the SE's deal with them.
It seems to be functioning as intended.
Brought to you by WebmasterWorld: http://www.webmasterworld.com