homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

The effect of robots.txt exclusion on pagerank
pagerank robots.txt google newbie

10+ Year Member

Msg#: 17346 posted 2:28 pm on Oct 1, 2003 (gmt 0)

Could anyone help me out on a few questions I have?

Lets say I have a page with a pagerank of 6 and that page has links to 3 other pages.

If I understand things correctly. Each of the linked pages would be passed a pagerank of 2. Is this correct?

Now lets say that things change. One of the linked pages (Page 4) is removed from Google's index because it was added to the list of robots.txt excluded pages.

Does this affect the pagerank that is passed on? Are the other two pages now passed a pagerank of 3 or does it remain at 2?

Finally, if Page 4 had a link to another site, is the pagerank to that other site lost after it is excluded?



WebmasterWorld Senior Member 10+ Year Member

Msg#: 17346 posted 3:25 pm on Oct 1, 2003 (gmt 0)

Page Rank is logarithmic. The other pages will likely have a PR of 5.

Also, there is speculation that if you link TO the excluded page, that the PR is divided including that link, but they would still all be a about 5 (because it's logarithmic).

Can anyone confirm that the robots excluded page is counted in outgoing?


WebmasterWorld Senior Member ciml us a WebmasterWorld Top Contributor of All Time 10+ Year Member

Msg#: 17346 posted 4:47 pm on Oct 1, 2003 (gmt 0)

As jcoronella implies, /robots.txt excluded URLs count for PageRank purposes so they suck PR like 404s do (though 404s were ignored back in July).

If a page linking to you has three links, and one or two are /robots.txt excluded or return 404, then you get only 1/3 the PageRank (minus the 'rank source' factor). Google won't see any links on the /robots.txt excluded page, so it will pass no PageRank from it.

Due to the log scale these effects are very small in terms of Toolbar PR, so thry shouldn't cause too much alarm.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved