I was wondering if you all know of any risks with the "no index, follow" in terms of other search bots ignoring the instruction.
as far as i understand, google, yahoo and msn wont index the pages - but what about those other bots that could potentially ignore your request.
Is there a way to redirect people but allow the search engines to crawl the page...or is that considered cloaking?
the pages i want to do a "no index, follow" are very basic non-important parts of the site that has some links on them.
Your help would be appreciated.