Forum Moderators: mack
<meta name="robots" content="index,follow">
<meta name="GOOGLEBOT" content="INDEX,FOLLOW">
<meta NAME="GOOGLEBOT" CONTENT="NOARCHIVE">
The first two direct spiders to crawl the enire site, the second being the google spider specifically, the third to keep google from caching the page...correct?
Questions...
Is there any intrinsic advantage to any of these robot meta tags?
Can they in fact cause more harm than good?
Is this a valid tag... to keep other spiders from caching the page?
<meta name="robots" content="noarchive">
What other robot or similar tags might i consider?
Thanks
Here is a good guide to robots.txt:
[robotstxt.org...]
The first line of code is redundant too, though, because "index,follow" is the default behaviour if no robots tag is present.
The third one will prevent Google from showing a cached copy of the page, and may be useful for that and other reasons.
Jim