Forum Moderators: Robert Charlton & goodroi
As I continue to struggle to escape the dreaded -950, awhile ago I robot.txt'd my redirected links to affiliates.
This shows up in WMT as about 50,000 out-of-bounds pages, compared to (only coincidentally!) 50,000 pages indexed. Perhaps this ratio is to high? Perhaps G doesn't like so many out-of-bounds pages?
It's clear that G follows the links anyway (for awhile, I had the affiliate pages in the SERPs for several terms, which should never have happened) - how can I completely obfuscate those affiliate links? Or should I be more open about it instead?!?
AnonyMouse: how are your links formed?
I had the same technique once on a site I built for a client and we used something like /norobots/affil1.php?product=123456, google shows a few hundreds as disabled by robots.txt but not nearly as many as there are (with ~ 100.000 pages indexed, about 30.000 linking to those affiliate-redirects).
/norobots.php?productid=123456&othervars=etc
And in .htaccess:
RewriteRule (.*)$ /norobots.php?path=$1
And finally in robots.txt:
User-agent: *
Disallow: norobots.php
(Obviously the file is not called "norobots"!)
As I say, the strange thing is that for awhile I had the pages linked to actually appearing high up in the SERPS. G shd not have accessed the file, let alone followed the redirected links through to the final affiliate page.
Whereas beforehand I had thought that G wouldn't like to see loads of links out to affiliates, now I'm wondering if G also doesn't like it of you try to "hide" those links from them. Does that mean that it's impossible to rank an affiliate site? That once you include affiliate links in your site, it's game over? Oh, this -950 hell!
If that doesn't describe your site then dig down deep into it, tear it apart 'til you find what might be wrong. Read tedster's and other folks' posts on the -950 thingy and possible mega menu thingy. (I'm sure you probably did already, but ya' never know what you might find going back over them.)
I.e. Google is totally ignoring my disallow statement...