Google displays mydomain.com without the www - despite a 301

[woop01]

Our site is listed in the SERPs as mydomain.com instead of www.mydomain.com. For keywords we dominate we’re actually listed as mydomain.com in the #1 spot and www.mydomain.com in the #2 spot.

I’m not sure this really matters but it’s one of those little things I’ve decided to obsess about in case it does.

We’ve got the canonical set on the home page… <link rel="canonical" href="http://www.mydomain.com" >

We’ve got mydomain.com 301’d to www.mydomain.com

Under ‘Google Webmaster Tools > Site Configuration > Settings’ we have the “Preferred domain” set to “Display URLs as www.mydomain.com” checked.

Why would Google still list us as domain.com in the SERPs?

[tedster]

Quick check - did you 301 all the internal pages too, or just the domain root? Did you change all internal links to include the "www"?

[woop01]

Thanks. We moved servers last October and it appears that while we remembered to 301 the home page, we hadn’t moved over the configuration for deeper URLs (domain.com/blahblah/).

Those URLS were actually just showing error pages so I’m not sure why it would have displayed the non www in the SERPs.

[g1smd]

We’ve got the canonical set on the home page… <link rel="canonical" href="http://www.mydomain.com" >

STOP!

The canonical URL for the site root is

http://www.example.com/

with a trailing slash.

That one extra character makes all the difference.

it appears that while we remembered to 301 the home page, we hadn’t moved over the configuration for deeper URLs (domain.com/blahblah/).

Huh? One rule can do the whole site:

RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

[woop01]

g1smd, thank you, I would have never thought of that.

Regarding the 301, it's IIS and you have to setup it a bit differently. When we moved over to the new server we setup a program on the default site of the server to 301 specific requests to the proper site. It was handling root domains properly (mydomain.com -> www.mydomain.com) but not interior pages (mydomain.com/blablahblah/ -> error page).

[g1smd]

it's IIS

::runs away -- very fast::

[woop01]

Hardie har har.

[tedster]

Permanent 301 canonical redirect for www in IIS

1. In Internet Services Manager, set up both www.example.com
(with-www) and example.com (no-www) as "websites".

2. Select the example.com website (no-www) in Internet Services
Manager and go into the properties.

3. In the Home Directory tab, change the option button
"When connecting to this resource the content should
come from" to be "A redirection to a URL".

4. In the "Redirect to" box, enter http://www.example.com$S$Q
(A note about the variables used here:
$S retains the requested URL's full filepath
$Q retains any query string present in the request.)

5. Check the checkbox that says "A permanent redirection
for this resource." This is a key step, or else you will
create a 302 redirect rather than a 301.

[phranque]

how do you handle the canonical redirect for IIS of a static default directory index document (example.com/index.html) without ISAPI_Rewrite?

[woop01]

tedster, I'm aware of that, it's how we setup the 301s on our old server (and now). When we switched to the new one we handled them programmatically. I'm not 100% positive this started at the same time but they were in the same ball park.

I'm still not sure why the fact that deeper URLs weren't 301'd would have caused the non www URL to show up in the SERPs. I assume that if that were the root of it, it would take some time to show up in the SERPs but as of now Google is still showing the mydomain.com instead of www.mydomain.com.

[woop01]

Digging deeper into it via Google Webmaster Tools…

If I setup a site for mydomain.com, it’s showing a lot of “Links to your site” under mydomain.com that are linking to www.mydomain.com. Those links are not showing up under the site profile for www.mydomain.com.

When looking at the details of those links it actually says it is linking to mydomain.com “Via this intermediate link: http://www.mydomain.com”.

[edited by: tedster at 4:24 am (utc) on Mar 3, 2011]
[edit reason] display hidden example url [/edit]

[tedster]

@phranque, if you have admin rights on an IIS 7 server, Microsoft offers a URL Rewrite extension that can be installed. Once you've done that, the syntax is pretty simple. Assuming the default index file for a directory is default.aspx --

<rule name="Redirect" patternSyntax="Wildcard" stopProcessing="true">
<match url="default.aspx" />
<action type="Redirect" url="/" redirectType="Permanent" />
</rule>

However, if you are on shared IIS hosting, or on IIS 6 even with Admin rights, then it's a mess. You end up creating an infinite loop. The desired redirect can still be achieved, I guess, because the 3rd party ISAPI Rewrite plug-in manages to handle IIS 6. But that wonderful workhorse plug-in works deep inside the server code to do its magic - not something mere mortals can do.

And if you're on shared IIS hosting, you're going to need the hosting company's help - it always takes Administrator rights.

The happy news today is that Google is well aware of these common IIS problems and seems to handle it on their back end. A canonical link tag is also a good practice for shared IIS hosting where you're just boxed into a corner. And one final precaution is never to create in internal link that points to default.aspx