Forum Moderators: open

Message Too Old, No Replies

Losing PR to internal pages excluded in robots.txt

Will it happen and should I care?

         

amanda21

1:37 am on Nov 16, 2002 (gmt 0)

10+ Year Member



I have a site with about 40 internal links on the home page. About 30 of these go to pages which I plan to exclude in the robots.txt file.

Firstly, when passing out the pagerank will google include the excluded pages, reducing the amount allocated to the indexed pages?

Secondly, should I care? All the internal pages seem to wind up with a pr of one less than the home page anyway.

2_much

5:47 am on Nov 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



1. Yes. From what I understand, the calculation is based on the amount of PR the page has to give divided by the total # of outbound links. At that point Google doesn't care whether the pages you link to are indexable or not.

2. If you don't care about those pages having PR, then it shouldn't matter. However, you may consider not linking out to those pages that won't be indexed, or linking to them using javascript, so that the PR value may be increased for the pages that you do want indexed.

Yes, you are right that usually they are one less than the index, but this isn't always the case and ther are ways around that.

amanda21

7:13 am on Nov 16, 2002 (gmt 0)

10+ Year Member



Thanks 2_much

I've been thinking about using javascript but as many of the links in question have important keywords in them that brings up a whole new question:

Would the anchor text of a javascript link be as effective as that of a standard link at increasing ranking?

ie Is it the presence of the <a tag that makes it a link, or the ability of the spider to follow it?

I am guessing it is the former, but you can never tell with spiders.

conor

1:38 pm on Nov 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If the link is not followable by a spider, the the spider doesnt consider it to be a link, as a result the anchor text can be of no benefit.

ciml

6:03 pm on Nov 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The /robots.txt excluded URLs can be listed in the SERPs, and as 2_much points out they do suck PageRank (oddly, 404s do as well even though the URLs aren't listed). As far as I know, the anchor text should also boost the URL listing for those words.

Amanda, I'm not sure what you mean about Javascript links being as effective at increasing ranking as standard links. If the Javascript hides the link from Google then there is no anchor text.

amanda21

9:15 pm on Nov 16, 2002 (gmt 0)

10+ Year Member



ciml

Take the following link

<a href="index.htm" onClick="golink('parent','index.asp');return false">Link Text</a>

This is essentially a standard text link to index.htm, but actually is a link to index.asp (assuming the browser supports javascript).

I would assume that google will see it as text link and assign the appropriate weight to the link text, as well as following the link to index.htm.

Am I right here?

ciml

9:55 pm on Nov 16, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> Am I right here?

Good question. I don't use Javascript links so I've never run tests on them. I don't remember coming across spam using the method that you describe so I guess there's a good chance that Google don't filter out href links with an onClick attribute.

Does anyone here know of a page where they could test the idea? (No urls please, but it would be great if someone could check to see if it counts as a backlink)

Either way, I would be wary of that approach if it might look like spam to a human reviewer.

amanda21

10:50 pm on Nov 16, 2002 (gmt 0)

10+ Year Member



Use it to disguise backlinks - now that would be naughty ;)

There is a legitimate reason for including the href attribute, without it the cursor doesn't change to a hand when you move over the link and this will confuse many visitors.

Also, the page in the href attribute is one level above the actual page in the nav structure, hence it enables visitors with non js browsers to navigate the site.

I guess the problem would be that the js and href urls are different, but as this in itself wouldn't actually do anything to increase pr or rankings, what would they have to complain about?

Aside from preserving pr the only seo benefit to including the href attribute is to have the link text recognised, provided the text is genuine (and it is - we are talking product categories - the fact that the cat names are also keywords is just a happy coincidence) then surely this would help the robot to correctly assess the page and hence help the searcher - right?

Perhaps I am being a little too idealistic here?

ciml

12:11 am on Nov 17, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> too idealistic

I think not. Usability improvements ought not be penalised.

I would very much hope that the usage you describe wouldn't upset a human reviewer.

ikbenhet1

12:15 am on Nov 17, 2002 (gmt 0)

10+ Year Member




the hand can be set with style, style="cursor: hand;" that was the line i think, add it to the link, a hand will appear.

amanda21

1:04 am on Nov 17, 2002 (gmt 0)

10+ Year Member



...and as if by magic, a hand appears.

Cheers ikbenhet1