Hiding links from Google

[realmaverick]

I really need to hide links from Google.

I have over a million download pages and each page contains a download link. Despite there being no content at the end lf the link, google still tries to index it but then floods webmaster tools with errors such as "soft 404" and other crap.

I'd much prefer Google just couldn't see the links. Besides forbidding guests from downloading, I'm not sure what to do.

I've considered maybe an frame for the download button but I was concerned it may seem shady?

Any "clean" way to prevent google following these links?

[tedster]

Do the download links point to a script, then? If so you could disallow in robots.txt

[dstiles]

I wrap such links in a User-Agent test. If it's a real web browser (according to several tests in addition to the User-Agent) then the visitor gets the links. If not the links are removed.

I've been doing this for many years and so far there have been no problems.

Yes, I know, cloaking. But sometimes it's no business of google's or anyone apart from a real person. Typically I use the technique to hide forms (contact, search etc) or media links which can often be pirated by a) scrapers and b) google.

I would be wary of iframes and frames. Some (real) browser extensions turn them off, sometimes by default.

[realmaverick]

Dstiles I've considered that but just don't want to risk it.

Tedster, disallowing via robots.txt, would the link juice basically just evaporate in to the links? I guess either way, it's better than having them try index junk.

[tedster]

sure - but if they're already reporting a soft 404, the link equity is already evaporating, right?

[Robert Charlton]

The link juice from the download link is going to evaporate no matter what you do. You can probably minimize the loss through that link by including other useful nav links on the page to recirculate PR throughout your site.

In this situation, though, why not use rel="nofollow"? Google says they don't use it even for discovery, and I trust them on that, and I think it would accomplish what you want.

[deadsea]

One solution would be to assign each download an id. Then the link to download could be something like:

<a href="#" onclick="download(123456789);return false;">Download</a> where 123456789 is the id for the download. The javascript download function could be in a download.js file that is blocked by robots.txt. In my experimentation, it appears that Google does not drop pagerank for onclick events. The risk here is that Google indicated that having uncrawlable JS could be considered a sign of web spam.

Another way to do it is to send the download link through an internal redirector:
<a href="download_redirect.cgi?id=123456789">Download</a> where the download_redirect.cgi script is in robots.txt and issues a 302 redirect to the file on the other site. Linking to things on your site that are in robots.txt DOES drop PR on the floor just like nofollow, but it is internal pagerank. My testing seems to indicate that Google treats internal and external pagerank separately to some extent.