DarkNova - 9:58 pm on Jul 12, 2007 (gmt 0)
I have thought about doing the robots.txt exclude, but here's what stopped me: if I link to copyright.html and have it excluded in the robots.txt page, then Googlebot won't index copyright.html, HOWEVER, won't the PageRank algorithm still pass on the same amount of PR to the copyright.html page as it would have anyway, but then that page acts as a dead end for the PR, not passing any on to other pages, making you worse off than before? I got this idea because I've read that some sites have a PageRank value even if the entire page is excluded in robots.txt...does this make sense or am I totally mistaken? I would love to do robots.txt if I could be sure it would work without passing on PR to the page. Thanks.