Is it important to put <meta NAME="robots" CONTENT="index,follow"> in all pages that you want to be indexed. I read somewhere that googlebot ignores this. Is it important to include this in all pages?
piatkow
6:55 pm on Jan 5, 2007 (gmt 0)
I understood that it was the default and could therefore be omitted but I don't think there is any disbenefit in using it.
I don't use it at all but with my business QA hat on I would always recommending future proofing things in case the rules change.
CainIV
7:23 pm on Jan 5, 2007 (gmt 0)
It isnt needed, and is simply another tag that one could 'mess up' if not done properly, so unless it's explicitly needed (which I cannot think of a reason why) it might be best to leave it out.
tedster
7:31 pm on Jan 5, 2007 (gmt 0)
Some really old SEO materials that you might still run into mention this. There was a theory years ago that "index,nofollow" was Inktomi's default behavior, so some webmasters started placing the invitation to "index,follow" on all their pages.
I agree with CainIV - I just leave it out and have for years with no problems that I can see.