Forum Moderators: goodroi

Message Too Old, No Replies

Robots meta tag *CORRECT* syntax

Space* or no space for "index,*follow", and case sensitive?

         

Clint

4:19 pm on Jun 22, 2005 (gmt 0)



I've wondered this for years. Can anyone please tell me which of these is the definitive CORRECT method?

<meta name="robots" content="noindex,nofollow">

<meta name="robots" content="noindex, nofollow">

Note the [space] between "noindex" and "nofollow" on the 2nd example. The robotstxt.org site itself has BOTH ways listed! So this is very confusing. It also lists case differences, should it be all lower case, all upper, or mixed case?
Thanks.

EBear

4:26 pm on Jun 22, 2005 (gmt 0)

10+ Year Member



I haven't an authoritive link for this, but this is what I remember seeing around:

  • Commas and spaces are both viewed as seperators, so you can use either. Using both is just done for readability, though it's wasteful in terms of file size (you may as well have two spaces)

  • The values are not case sensitive, but there's a spec somewhere that recommends that tags be in lowercase.
  • Clint

    4:34 pm on Jun 22, 2005 (gmt 0)



    Thanks. :)

    Clint

    4:45 pm on Jun 22, 2005 (gmt 0)



    Oh, regarding this:

    "Commas and spaces are both viewed as separators, so you can use either. Using both is just done for readability, though it's wasteful in terms of file size (you may as well have two spaces) "

    So using BOTH a comma and space you're saying is also OK? I ask because that sounds like it would be two "separators".

    EBear

    5:10 pm on Jun 22, 2005 (gmt 0)

    10+ Year Member



    Actually, it's what was recommended by the Spidering Birds Of A Feather Report [w3.org], the output of a Distributed Indexing/Searching Workshop held in 1996 and sponsored by the W3C, at which definitions for the meta robots and description tags were set with contribution from the major (where-are-they-now) engines of the time.

    sparky_132

    12:56 pm on Jun 28, 2005 (gmt 0)



    Good question Clint. I have also wondered the same.

    But I hope this will clear things up. The standard can be found at
    [robotstxt.org...]

    It states a couple of points which are of particular interest to us. I quote from that page:

    - "Note the "robots" name of the tag and the content are case insensitive."

    This tells me that the following two are equivalent:
    <meta name="robots" content="noindex,follow">
    <meta name="ROBOTS" content="NOINDEX,FOLLOW">
    as are any other possible combination.

    - "The content of the Robots META tag contains directives separated by commas"

    Also the formal syntax says:
    - "directives = directive ["," directives]"

    So I disagree with Ebear that "Commas and spaces are both viewed as seperators, so you can use either."

    The syntax states that a directive (the content part of the tag) is a comma separated list. So a space on its own is not a separator. E.g. content="noindex follow" is invalid syntax.

    The "," suggests to me that a space should NOT be placed after the comma separator.

    But I doubt whether any robot takes this literally and I feel that if you put a space after the comma the directive would still be observed.

    Anyone have any further comments please post!