Welcome to WebmasterWorld Guest from 50.19.14.191

Forum Moderators: phranque

Message Too Old, No Replies

The <head> Cascade and Robots META Tags

     
3:50 pm on Sep 15, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12169
votes: 55


<head>
<meta content="noindex, nofollow" name="robots" />
<title></title>
<meta content="index, follow" name="robots" />
</head>

Would I be correct in taking an educated guess at the above implementation and stating that the bot will index and follow the page containing the above sequence of elements?

4:17 pm on Sept 15, 2008 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member encyclo is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 31, 2003
posts:9068
votes: 4


I don't have a definitive answer, but what happens if the bot is programmed to stop at a noindex rule and not parse the rest of the document? Technically-speaking would this be wrong? I don't think so. So with your example, only testing will tell. I don't believe that either behavior would be "wrong".
4:32 pm on Sept 15, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12169
votes: 55


I was wondering if one takes precedence over the other and if the cascade does not apply in this instance? Test? Arrrggghhh! Let me add to the other 100+ tests that may be going. Why can't this be in writing somewhere already? ;)
5:03 pm on Sept 15, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


*** what happens if the bot is programmed to stop at a noindex rule ***

In practice they don't. Google takes and holds a copy of the file, follows links out, assigns PR to the URL and so on.

They just don't show it in the public SERPs.

5:30 pm on Sept 15, 2008 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member encyclo is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 31, 2003
posts:9068
votes: 4


In practice they don't

Or rather, in practice Google don't. But what about the others? Live, Yahoo, etc.? As there is no real, formal "standard" either for robots.txt or for meta robots elements, the above example would be open to interpretation which could differ between bots.

7:10 pm on Sept 15, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12169
votes: 55


The above example would be open to interpretation which could differ between bots.

Therein lies my concern. If programming a bot I'd be following the rules of engagement in regards to the cascade. But, this is a Robots Directive and I'm 50/50.

Would it stop at the first directive and go no further?

Or, would it continue to index the entire <head></head> and honor the second directive which would override the first from a cascading perspective?

Bets? ;)

A little OT, I can confirm that G does not honor noindex, nofollow, noarchive. All three directives combined in that sequence. It will noarchive.

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members