homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Robot terms defined
The differences and similarities between nofollow, noindex and robots.txt
u to the sixth

10+ Year Member

Msg#: 558 posted 7:29 am on Feb 26, 2005 (gmt 0)

Greetings! I have just read several threads here relating to nofollow, noindex and robots.txt and was wondering if someone wouldn't mind checking my understanding for errors. These are certainly not exhaustive definitions but they articulate the aspects I was studying.

robots.txt - Blocks bots from indexing pages or folders. Pages linking to URLs in the robots file will still be placed in Google index, but as a bare link (no title, no description). Since the link is in the index, all necessary PR will applied to the pages involved.

meta noindex - The page will not be placed in the index. Should give the page PR-0 or null. Links on the page will still be followed, but will have no inbound PR added from the noindex page.

meta nofollow - The page is indexed as normal, but all links on the page are not followed. These links could very well be followed from another page in the site that does not have a nofollow tag. nofollow can be used to slightly improve PR for the page since it doesn't pass any outbound points.

Sound somewhat accurate? I appreciate any comments. Thanks!


Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved