gmiller - 5:53 pm on Mar 8, 2005 (gmt 0)

This situation seems pretty simple when you get right down to it. SEs should never treat an URL that consists of a redirect as having any value. It needs to stay in the database for various reasons (passing PR, for example), but it shouldn't come up in search results or be treated as the original by a dupe filter. The fix is simply to implement this correctly. Complaining about Alexa legitimately using 302 redirects is just silly.

The really nasty thing here is that the scraper sites (please stop calling them "SKYSCRAPER" sites, OK?) aren't actually copying anything, allowing them to get the same results they would have if they'd actually scraped your page, but without having to violate copyright law in order to do it. Your best interim solution is probably to send the host a copyright complaint, as pure legal intimidation often works even when you have no real legal claim. If that doesn't work, you just have to wait for Google to finally fix the problem, and I wouldn't hold my breath given how long this has already been a problem.

As for sending referrers when spidering a page, that doesn't really make sense. If you have 5000 inbound links, do you really want Google to request the same page 5000 times with different referrers? Robots aren't browsers and they generally don't follow links. They just add the linked-to URL to their database (if it's not already there) and it'll get spidered along with all the rest at some future date. The problem is that Google is associating the content returned for the target URL with that URL and all URLs that link to it, rather than simply recording the fact that the redirect URLs are redirects and ignoring them when generating search results.