Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Pages are ranking, in spite of robots.txt or robots meta tags

pageoneresults - 2:59 pm on May 20, 2006 (gmt 0)


Instructions for utilizing the Robots META Tag

<meta name="robots" content="none">

Place the above directive (Robots META Tag) right after the <head> of your document.

<html>
<head>
<meta name="robots" content="none">
</head>

That should prevent those bots that obey the protocol from indexing and following links on the page.

Now, if you want to allow the links on that page to be followed, you might do this...

<html>
<head>
<meta name="robots" content="noindex, follow">
</head>

Both Google and MSN have developed specific Robots META Tags that you can use to keep just their bots away from those pages.

For Googlebot

<html>
<head>
<meta name="googlebot" content="none">
</head>

For MSNBot

<html>
<head>
<meta name="msnbot" content="none">
</head>

There are other robots terms that you can use to manipulate spider behavior.

Thread source:: http://www.webmasterworld.com/google/34402.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com