Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

The effect of robots.txt exclusion on pagerank

pagerank robots.txt google newbie

2:28 pm on Oct 1, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 28, 2003
votes: 0

Could anyone help me out on a few questions I have?

Lets say I have a page with a pagerank of 6 and that page has links to 3 other pages.

If I understand things correctly. Each of the linked pages would be passed a pagerank of 2. Is this correct?

Now lets say that things change. One of the linked pages (Page 4) is removed from Google's index because it was added to the list of robots.txt excluded pages.

Does this affect the pagerank that is passed on? Are the other two pages now passed a pagerank of 3 or does it remain at 2?

Finally, if Page 4 had a link to another site, is the pagerank to that other site lost after it is excluded?

3:25 pm on Oct 1, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 4, 2002
votes: 0

Page Rank is logarithmic. The other pages will likely have a PR of 5.

Also, there is speculation that if you link TO the excluded page, that the PR is divided including that link, but they would still all be a about 5 (because it's logarithmic).

Can anyone confirm that the robots excluded page is counted in outgoing?

4:47 pm on Oct 1, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member ciml is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 22, 2001
votes: 2

As jcoronella implies, /robots.txt excluded URLs count for PageRank purposes so they suck PR like 404s do (though 404s were ignored back in July).

If a page linking to you has three links, and one or two are /robots.txt excluded or return 404, then you get only 1/3 the PageRank (minus the 'rank source' factor). Google won't see any links on the /robots.txt excluded page, so it will pass no PageRank from it.

Due to the log scale these effects are very small in terms of Toolbar PR, so thry shouldn't cause too much alarm.