Welcome to WebmasterWorld Guest from 34.201.121.213

Forum Moderators: goodroi

Message Too Old, No Replies

Robots meta tag *CORRECT* syntax

Space* or no space for "index,*follow", and case sensitive?

     
4:19 pm on Jun 22, 2005 (gmt 0)

Full Member

joined:Jan 12, 2004
posts:334
votes: 0


I've wondered this for years. Can anyone please tell me which of these is the definitive CORRECT method?

<meta name="robots" content="noindex,nofollow">

<meta name="robots" content="noindex, nofollow">

Note the [space] between "noindex" and "nofollow" on the 2nd example. The robotstxt.org site itself has BOTH ways listed! So this is very confusing. It also lists case differences, should it be all lower case, all upper, or mixed case?
Thanks.

4:26 pm on June 22, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Feb 12, 2002
posts:217
votes: 0


I haven't an authoritive link for this, but this is what I remember seeing around:

  • Commas and spaces are both viewed as seperators, so you can use either. Using both is just done for readability, though it's wasteful in terms of file size (you may as well have two spaces)

  • The values are not case sensitive, but there's a spec somewhere that recommends that tags be in lowercase.
  • 4:34 pm on June 22, 2005 (gmt 0)

    Full Member

    joined:Jan 12, 2004
    posts:334
    votes: 0


    Thanks. :)
    4:45 pm on June 22, 2005 (gmt 0)

    Full Member

    joined:Jan 12, 2004
    posts:334
    votes: 0


    Oh, regarding this:

    "Commas and spaces are both viewed as separators, so you can use either. Using both is just done for readability, though it's wasteful in terms of file size (you may as well have two spaces) "

    So using BOTH a comma and space you're saying is also OK? I ask because that sounds like it would be two "separators".

    5:10 pm on June 22, 2005 (gmt 0)

    Full Member

    10+ Year Member

    joined:Feb 12, 2002
    posts:217
    votes: 0


    Actually, it's what was recommended by the Spidering Birds Of A Feather Report [w3.org], the output of a Distributed Indexing/Searching Workshop held in 1996 and sponsored by the W3C, at which definitions for the meta robots and description tags were set with contribution from the major (where-are-they-now) engines of the time.

    sparky_132

    12:56 pm on June 28, 2005 (gmt 0)

    Inactive Member
    Account Expired

     
     


    Good question Clint. I have also wondered the same.

    But I hope this will clear things up. The standard can be found at
    [robotstxt.org...]

    It states a couple of points which are of particular interest to us. I quote from that page:

    - "Note the "robots" name of the tag and the content are case insensitive."

    This tells me that the following two are equivalent:
    <meta name="robots" content="noindex,follow">
    <meta name="ROBOTS" content="NOINDEX,FOLLOW">
    as are any other possible combination.

    - "The content of the Robots META tag contains directives separated by commas"

    Also the formal syntax says:
    - "directives = directive ["," directives]"

    So I disagree with Ebear that "Commas and spaces are both viewed as seperators, so you can use either."

    The syntax states that a directive (the content part of the tag) is a comma separated list. So a space on its own is not a separator. E.g. content="noindex follow" is invalid syntax.

    The "," suggests to me that a space should NOT be placed after the comma separator.

    But I doubt whether any robot takes this literally and I feel that if you put a space after the comma the directive would still be observed.

    Anyone have any further comments please post!