Page is a not externally linkable
- Yahoo
-- Deprecated - Altavista, Alltheweb.com
---- Fast not checking robots META tag


SeanL - 3:57 pm on Apr 15, 2003 (gmt 0)


Hello, all. This is my first time posting to this forum, though I have learned so much by reading it.

I have one page of links I do not want followed. I put in the following tag in the head:

<META name="robots" content="noindex,nofollow">

Unfortunately, Google and Fast seem not to obey it. I submited a bug report to Fast and the reply just came back that they would have their bot check my robots.txt again. They did not mention the issue I raised, which was the robots meta tag. I have altered my robots.txt, but the meta tag is supposed to work by itself, isn't it?

So, I figure this might be of interest to some here. I searched through the Fast forum to see if this issue had been mentioned, but I didn't see anything. Has anyone else noticed this behavior, or does their bot seem to behave appropriately on your site?

And also, does anyone see a flaw in the tag above?

-- SeanL


Thread source:: http://www.webmasterworld.com/altavista/861.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com