Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Fast not checking robots META tag



3:57 pm on Apr 15, 2003 (gmt 0)

10+ Year Member

Hello, all. This is my first time posting to this forum, though I have learned so much by reading it.

I have one page of links I do not want followed. I put in the following tag in the head:

<META name="robots" content="noindex,nofollow">

Unfortunately, Google and Fast seem not to obey it. I submited a bug report to Fast and the reply just came back that they would have their bot check my robots.txt again. They did not mention the issue I raised, which was the robots meta tag. I have altered my robots.txt, but the meta tag is supposed to work by itself, isn't it?

So, I figure this might be of interest to some here. I searched through the Fast forum to see if this issue had been mentioned, but I didn't see anything. Has anyone else noticed this behavior, or does their bot seem to behave appropriately on your site?

And also, does anyone see a flaw in the tag above?

-- SeanL


4:18 pm on Apr 15, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member

Hello Sean, welcome to the board.

I've not heard or seen that Fast should not honor the exclusion tags. Neither robots.txt nor meta name noindex.

At least that's what their FAQ says [fastsearch.com].

However they list the code for the meta tag with spaces between noindex and nofollow:


Could be the reason they still index your page?


Featured Threads

Hot Threads This Week

Hot Threads This Month