Redirect non-www to www - do we still need this?

[proboscis]

Is it still necessary to redirect non-www to www pages with .htaccess?

Especially since we can now set our "Preferred domain" in webmaster tools, isn't that enough?

[g1smd]

That only works for Google.

You still need this redirect for other searchengines.

You also need the redirect so that users 'see' the correct URL in their browser address bar - otherwise they will continue to cut and paste the wrong URLs to other sites, so linking back to you with your non-canonical form.

There are several other things you need to be aware of and fix. These are still all issues that you need to be aware of: [webmasterworld.com...]

[edited by: g1smd at 9:00 pm (utc) on Mar. 20, 2009]

[tedster]

I still use it - and Matt Cutts still recommends it, even if you are also using the new canonical tag.

It all depends on what you mean by "need". The 301 sort of forces Google (and other search engines - don't forget them!) to combine PR and link juice. Without the 301, you are 100% depending that Google will combine and calculate everything on the back end accurately for your domain.

With all the data that they move around, I have no such confidence. In Webmaster Tools they talk about "which version Google will display" and even then, there is some hedging language and no guarantees.

I will continue with the "sure thing."

[proboscis]

Thanks!

The reason I ask is because non-existent non-www pages that are within the cgi-bin return a 301 (moved permanantly to my 404 page)

Taking out the www redirect in htaccess makes all non-existent pages go 404 but I see that is not a good fix.

How else would I fix that problem or is it okay to have non-existent pages not go 404?

[g1smd]

What happens in Live HTTP Headers here?

Pages that do not exist should directly return a 404 status, or can return a 301 redirect then a 404 status.

There should not be a 200 OK response, or a 302, at any part of this.

[proboscis]

Live headers says 301 moved permanently, then it gives the location as my custom 404 page.

Then it shows a 304 for the 404 page itself.

[g1smd]

The 304 means the page is FOUND: 304 Not Modified.

So, you have an Infinite Duplicate Content scenario.

That can be a site killer. You need to alter the server configuration, and scripting so that a proper HTTP 404 Not Found response is returned in the HTTP header.

[proboscis]

Thanks g1smd!

My site is being killed, very slowly killed, maybe this is why.

I need non-www pages that are within the cgi-bin to return a 404 if they do not exist or I need my www redirect to not apply to pages within the cgi-bin.

Can I do that with htaccess?

Edit: Answering my own question. I will ask in the Apache section of the forums.

[edited by: proboscis at 12:39 am (utc) on Mar. 21, 2009]

[wheel]

Are there any sort of analysis tools that will review a site from this perspective?

[tedster]

analysis tools

You can use the Live HTTP Headers add-on with Firefox. Try a variety of "incorrect" urls in the browser, watching how your server responds.

I need non-www pages that are within the cgi-bin to return a 404 if they do not exist or I need my www redirect to not apply to pages within the cgi-bin.

You need any url that doesn't exist to generate a 404 status in the header.

[edited by: tedster at 12:44 am (utc) on Mar. 21, 2009]

[g1smd]

Use the Live HTTP Headers extension for Firefox on a selection of URLs and their variants: with and without www, with and without parameters, valid parameters but in a different order, with additional parameters, with and without port numbers, and all for a selection of valid and non-valid URLs.

You can also build a text file list of test URLs, and feed it to Xenu LinkSleuth.