Robots META Tag for Googlebot

Another myth begins...

[pageoneresults]

<meta name="googlebot" content="index, follow">

I'm starting to see a proliferation of websites that are now utilizing the above Robots META Tag.

WHY?

Google's default indexing behavior is

index, follow

so why include this piece of metadata?

Nowhere in Google's guidelines does it suggest the use of this tag, nowhere!

SO WHY USE IT?

Googlebot obeys the

noindex, nofollow, and noarchive

Robots META Tag. If you place the tag in the head of your HTML/XHTML document, you can cause Google to not index, not follow, and/or not archive particular documents on your site.

Google does not recommend utilizing the Robots META Tag in the below manner. It is meant to specify indexing restrictions and that is all.

<meta name="googlebot" content="index, follow">

Can we stop this now before it becomes another piece of metadata junk that people start adding to their sites?

Additional information on specific Googlebot Robots META Tags can be found at Google's Webcrawler [google.com] page.

[rfgdxm1]

I guess they figure if they roll out the welcome mat for Googlebot she'll be inclined to spider more. Even though the idea is silly.

[pageoneresults]

lol, a Welcome Mat?

How many mats do you need? Server response 200 OK not enough? How about two Robots META Tags...

<meta name="robots" content="index, follow">
<meta name="googlebot" content="index, follow">

Or possibly even three, might as well not stop there, huh?

<meta name="robots" content="index, follow">
<meta name="googlebot" content="index, follow">
<meta name="msnbot" content="index, follow">

I'm seeing some really creative uses of the Robots META Tag. Many with incorrect syntax.

Here's a good one...

<meta name="googlebot" content="all, index, follow">

I guess the above person just wants to make doubly sure that Googlebot is going to index all of that page.

It amazes me what people will do to try and gain an edge. All the above stuff will do nothing other than shift the text to html ratio of the page.

Here, let's really confuse the issue...

<meta name="googlebot" content="all, index, follow, none, noindex, nofollow, noarchive">

[rfgdxm1]

<meta name="googlebot" content="I love you">

;)

[pageoneresults]

<meta name="googlebot" content="iloveyou">

Actually, Google clearly states that the robots-terms that they support are for preventing the indexing of content. So, the above should change to...

<meta name="googlebot" content="ihateyou">

[rfgdxm1]

I humbly stand corrected.

[Macguru]

>>all, index, follow, none, noindex, nofollow, noarchive

Sounds like a blind date, step by step. :)

[djgreg]

People using this are the ones calling themselves SEOs only because they managed to trade links with some sites and therefore achieved a good ranking in google. They don't know anything about Google nor about any algo not do they know how to read server logs. argh.

...need to calm down a bit... ;-)

greg

[mil2k]

I'm starting to see a proliferation of websites that are now utilizing the above Robots META Tag.

WHY?

Thank God I am not the only one.

In general, I have asked a few guys why they have this tag and it turns out that the SEO guys did this.

Which makes me think, is it a form of marketing tactic? We also optimize your Meta tags (which will enable better google indexing).

[GranPops]

<meta name="googlebot" content="have I seen you here before, do you come here often, would you like to come again in the future, is this the only bit you have found or have you explored a bit more, you have no idea how pleased I am to see you, it would break my heart if I thought this one only a one-night stand, are you just here for a quickie or do you think you might go the whole hog, I have spent many sleepless nights dreaming only of you">

and if that doesn't work - try threats

[mahlon]

<meta name="robots" content="index, follow">
<meta name="robots" content="noarchive">

<meta name="robots" content="index, follow">
<meta name="googlebot" content="noarchive">

[BigDave]

Ya know, after reading about the problems that people had with having a bad robots.txt file (if you have one, but google is not able to parse it correctly, they assume that you do not want to be spidered), I would be scared to put one of these in on the outside chance that google decides to do the same thing with the robots or googlebot meta tags.

Since the meta tags are used only to restrict activity, if google does not recognize the commands, they might just decide to play it safe and default to "noindex, nofollow".