no-www to with-www redirect was a 302 - how to fix?

[1script]

Hello everyone!

I was doing an investigation on a large site that almost dropped out of Google (500,000+ pages down to 11,000+) index recently (late Jan 06) and found a strange result:

site:thesite.com brings thesite.com as the #1 result and is has very old cache dated Jan 2005. The rest of the results start with www.

site:www.thesite.com brings www.thesite.com as the #1 and the cache is very recent (two weeks old)

A look at thesite.com using HTTP header analyzers revealed that it was sending 302 redirects to www.thesite.com instead of 301.

Could this be the reason for such significant drop in number of indexed pages? It could be duplication penalty or the fact that in Jan 2005 a large portion of the site did not exist yet. Had anyone seen that sort of redirect mishap cause this much trouble?

Also, I have corrected the redirect and it now sends 301 status. How long do you think it takes them to realize the correction? The bot is at the site every day, although its activity significantly dropped after the site's indexed size went down.
Thanks!
D.

[tedster]

Yes, that certainly can create big troubles. Good for you to have found the issue.

I've seen the same thing many many times, and most often on IIS servers where sys admins don't easily notice the little "Permanent" checkbox. With Google especially, we cannot afford to be naive about 200/301/302/404 http headers.

The fix should be a 301, which you've already done, and then wait for a bit to let Google crawl and sort it out. Since Big Daddy has Google in a new moment, predictions of how long it might take are like rolling dice. And if things don't seem to be on the mend soon, then look for other possible issues, such as a "custom 404" page that actually returns a 200.

[1script]

Hi tedster, thank you for the quick reply.

interesting that you've mentioned 404 error pages. We do have error pages return 404 status yet there is a little bit of a text content showed on the page for troubleshooting purposes. Do you think we should return a simple no-content 404 error instead? Just out of curiosity: what is the adverse effect of a 200-status custom 404 error page?

Thanks!
D.

[Dayo_UK]

1script

Unfortunately sites have had this problem for a year or so now with the 301 in place and no fix in the Google rankings.

Google do keep talking up a fix for this issue and have been for a while now (talking about it) - but unfortunately as of today it seems to be all talk and no action.

Although if you look hard and squint your eyes you might be able to see some improvements in the way BLs and PR are being handled in these cases recently - but certianly no improvements in rankings and crawling.

Hope things work out OK.

[tedster]

You're right, Dayo, some sites are just stuck on Google. I didn't want to pass that dark cloud into the thread by mentioning the possibility, because I have also seen some sites get fixed by switching the 302 to a 301.

what is the adverse effect of a 200-status custom 404 error page?

It allows an infinite number of urls to resolve to the same content. Over time, this form of "duplicate content" really tangles up Google. As long as the 404 header is returned, a custom message is fine.

[1script]

I didn't want to pass that dark cloud into the thread by mentioning the possibility, because I have also seen some sites get fixed by switching the 302 to a 301.

Thanks for being so considerate, tedster ;-) If what I see here can be indicative of an improvement, exactly one week after changing the 302 to 301 I am seeing a slight increase in the number of pages indexed for the site for the first time since the crash occurred three months ago. New pages have been added this whole time, but only now some of them may have gotten picked up by Google. I keep my fingers crossed.