logic check on robots.txt

will robots.txt properly disallowing a links page prevent link pop penalty?

[wooden]

Please check my logic on this. I think I've got it, but need to make sure I've got my Monday head around this one.

I've got a page of links on my site to various folks for various reasons (friends, relatives, reciprocal linking, etc.). It's been said that who you link to can "harm you but never help you." (bad neighborhoods, PR0, etc.)

So, if I disallow this one page with robots.txt, I shouldn't have to worry about possible penalties in terms of PR/link pop, right? Spiders should never see the links page, and therefore, couldn't penalize me for any "bad" outbound links.

Am I thinking straight?

Thanks

[Terry_Plank]

Well, depends on your strategy?

If you are just wanting to provide links to your visitors for resources you value and don't care that the sites you link too benefit from increased popularity from your link, then you are fine with this.

If you are trying to get link partners (like you indicated), then you are creating a situation where you get popularity/PageRank value from their link to you, but you aren't offering the same popularity/PageRank value to them. If your page is not indexed, then it is not shown as a link to that partner. Hence you don't have a reciprocal link that is of value for popularity of PageRank.

The value, still remains for driving traffic to each other other, but you need to make clear to whom you are trading links to that you are not going to help their popularity/PageRank by your quality link to them.

Why don't you just put friends and relatives on a different page you keep robots from and then only associate with sites that you, your visitors, your link partners and the search engines would all benefit from knowing about?

[wooden]

Thanks, Terry. It's a tough one, for I want other sites to have the same link pop benefit if they are linking to me, but don't want to be hit with a penalty if their link pop/PR suddenly goes bad for whatever reason. You're right in saying that if I do disallow that page that I should be up front about it.

By the way, what is the consensus on whether or not reciprocal linking is a zero-sum game, meaning there is no gain or loss of link pop/PR?

[Terry_Plank]

wooden,

There is no question about the fact that links to you from quality sites in your theme area are helpful for PageRank and popularity. This is exactly why they started looking at links, to determine if anyone thinks you have value for a particular search term. If others link to you, then perhaps you are really a quality site for ranking higher for a particular search term query result.

Personally, I don't think that one or two links going south on you from a hundred you link to is going to significantly affect your ranking. Also, if someone is an eggregious offender in your field, you would proably know it and you can drop the link and ask them to stop linking to you.

And, be sure you look at their PageRank in the first place before you link to them. And, perhaps take a quick look at their source code and see if you see something questionable there. If you do, don't link to them.

By the way, part of the "sum gain" for our sites is that there are sites referring to us, regardless of the popularity issue. Search engine traffic is important, so is traffic from qualified referral sources. :-)

[wooden]

Thanks. What you say makes sense, but I am still wondering if reciprocal links just cancel each other out when an SE calculates a site's link pop, i.e., if an SE detects reciprocal links, does it just ignore those links in its link pop measurements?

[Terry_Plank]

I think you are mixing something up. By calling them "reciprocal" we mean that that I link to you and you link to me. The SE's are not making that distinction as far as I know. They are following my links out to other sites. They are following others sites to me.

There would not be any cancelling out set in the algorithm. If they started setting up that kind of thing it would destroy way more than it would help in trying to get people to stop improper reciprocal linking. But, there is nothing inherently wrong about the fact that we are willing to link to each other. After all, that's really what the Web is all about anyway, linking, linking, linking.

[wooden]

Thanks again, Terry. You and I are using the same definition of reciprocal linking.

I think that a few months back, I read a post somewhere that made me think that reciprocal links would (or could) cancel each other out in terms of link pop. Whether the poster meant that as a reality or just a possibility is the uncertain part for me.

Thanks for your help.

W

[Brett_Tabke]

When you block a page by robots.txt, you are telling the spider not to list the page in it's index. That doesn't mean it can not download the page and use the data for it's own purposes. Some se's have been found to do this. Thus, the links may end up known to an se anyway.

[Terry_Plank]

I certainly agree with Brett. There is definitely some strong indicatation se's may go ahead and follow the links, even though they will not index the page.

At the same time, I just wouldn't take a chance with something that I consider important for my site and would still look for alternatives to cover things if at all possible. ;)