Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Controlling Crawler Access - different options
goodroi - 6:45 pm on Aug 7, 2010 (gmt 0)
its what i would do. there are some different approaches you could take but they all have some downsides.
Thread source:: http://www.webmasterworld.com/robots_txt/4180920.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com