www.example.com & example.com Conflicts

One site, 2 different indexing results,

[dibbern]

I know this has been asked before, but I can;t find the definitive thread on the topic. Please bear with me.

My e-com site fell from top10 to 'way down in 100's about a year ago. I've worked at most of the things suggested here to others like me. But just today I realized that Google returns only my index page for a search site:www.example.com, and ALL my page for site:example.com. Likewise, the few listings that rank OK in the SERPS point to the non-www version of my domain.

Has Google flagged me as running duplicate sites? Could there be a penalty attached to what they perceive as different (although they are not) domains running duplicate content?

Is this a case for a 301 redirect? My web host thinks I'm crazy. BTW, 5 other sites I run on the same host return identical results for the 'site:' searches using www. and non-www. No problem with their rankings, either.

Any help would be really appreciated. Thanks.

[edited by: ciml at 12:25 pm (utc) on Jan. 29, 2005]
[edit reason] Examplified [/edit]

[redleaf]

Hi there,
you might get some solutions here:
[webmasterworld.com...]

[rainborick]

I've became very familiar with this problem about 2 years ago when a client's site fell into this trap. The technical issue is that "www.somesite.com" and "somesite.com" are two distinct URLs and when Google initially encounters the two versions of the URL, it treats them as such - separate and distinct. All it takes is for a linking partner to decide to get creative with your link and you can fall into this pit. The problem is, of course, that Google's duplicate content sniffers quickly detect the situation and send one version or the other into "partially indexed" status - a state in which a page will appear in the SERPs as URL only and which rarely ranks for anything. It leads to all sorts of cascading problems that retard the rankings of the site for both URLs. It happens without warning, and you can go nuts trying to figure out why Google suddenly hates you as your rankings plummet without apparent cause.

GG addressed this issue briefly around that time and indicated that Google had an internal process ("canonicalization") that would reconcile these inadvertant duplicates into a single URL. As I recall, there was a hint that the process was run on an irregular basis, and not a part of their normal routine. The upshot is that, eventually Google should notice and self-correct the problem. Of course, "eventually" can sometimes mean several months.

The best solution is to set up code 301 redirects from the "bad" version of your URL to the "good" version. You can take your pick, but you need to choose. The key is to make sure your redirects work for all of the URLs for all of the pages on your site in order to make sure all pages' URLs get righted as soon as possible in addition to the root URL. The Apache .htaccess script in the thread that redleaf linked to in his message above will work fine. I don't know the fix on IIs servers. I'm sure there's references for them here in WebmasterWorld. But by setting up blanket 301's you will get your site back into Google's good graces within a much shorter time - often under a week for smallish sites.

I think everyone should set up these redirects as a preventative measure. I say this because I was recently embarassed to find one of my own sites fell into this trap. Once I installed the 301 fix, all was well in about 10 days.

[dibbern]

Thanks rainborick. That was a really valuable reply. I've traced the previous threads, including canonicalization, and found my worst worries seem true.

Oh well, its better than wondering. Thanks again.

[Ept103]

So lets say I have a site example.com which, according to the site:example.com has more indexed pages than www.example.com.

Since these numbers are not the same (the number of indexed pages that it returns from this command= site:example.com and site:www.example.com) should I be worried about setting up a 301 redirect on one of them?

Which one should I setup the 301 redirect for? Most of my links are to www.example.com

[Ept103]

Anyone?

[rainborick]

The method you describe is the acid test for determining if your site has fallen victim to what I call The Dreaded Missing WWWs Syndrome. If site:yoursite.com and site:www.yoursite.com produce different results, it means Google has not yet fully canonicalized the two URLs and you should immediately set up 301 redirects as mentioned above. Good luck!

[Ept103]

Ok. But how do I know which way to setup the 301 redirect?

www.example.com to example.com
or
example.com to www.example.com

the one with the most results on the site: test or do I just take my pick?

[rainborick]

Take your pick. On the whole, I tend to the more traditional www. version, but I suppose the best choice is the version used in the majority of the inbound links from other sites. Good luck!

[Frank_Rizzo]

On a similar theme what to do about indexed ip addresses?

I notice quite a few visitors landing on nnn.nnn.nnn.nnn (server's ip) rather than www.widgets.co.uk

Looking at site:nnn.nnn.nnn.nnn I see that 522 pages are indexed.

Should the ip address have a 301 redirect too?

-----

PS. That reminds me of another on a similar theme - same site for widgets.com and widgets.co.uk.

5 years ago I obtained widgets.co.uk and also widgets.com. Even though the site is based in the UK I thought it would be safer incase users just typed widgets in the url field and the browser automatically appends .com

Anyway, At that time I foolishly duplicated the .com and .co.uk by pointing the addresses to the same server. Therefore, if a user entered www.widgets.co.uk or www.widgets.com they would access the same duplicated pages on a virtual host of the same IP address.

After reading good posts at the time I realised it was a bad move and just changed the www.widgets.com to a holding page on the lines of "please visit our main site at www.widgets.co.uk"

That's fine over the years but I still get a lot of se's indexing the .com directories.

Recently I setup a redirect for all .com pages to redirect to the corresponding .co.uk. Was that the right way to go about it?

[Ept103]

figured it out

[dibbern]

I followed a recommended past thread about this, which led to another thread,etc, etc, and eventually found a post from GGuy inviting anyone who has this problem to write [e-mail]. It read as if Google can do some things on their end to correct this problem.

Not that anyone should NOT go ahead with the 301's, in any case. Just another possibilty to follow.

I'm trying that approach. Will post response, if any.

[pageoneresults]

Just for clarification...

example.com is referred to as the Root Domain. www.example.com is referred to as a Sub-Domain. Both are distinct entities. The search engines are pretty good about listing only one of the two. It is usually the one that has the most IBLs.

Yes, a 301 is a possible solution for this. But, if you ever plan on using the root domain, a 302 may be advisable.

[Ept103]

Speaking of the site command, I have one more concern:

site:www.example.com returns more results than I have pages on my web site

site:example.com also returns more results than I have pages on my web site

Is this possible? Does this mean my site is starting to get indexed twice as example.com and www.example.com?

[dibbern]

Google has responded.

The first part is a personnal message:
"We searched for your site and found that is is currently included in our results...Please visit the following link (link to my www. site)"

The rest is a canned message about not being able to make corrections to individual sites, recommendations to follow the listing guidelines, etc.

I really appreciate Google's effort to answer my dilemma, but I'm not sure it says anything.

My problem is vastly different indexing for www.domain.com VS domain.com, and what I believe (and has been pointed out by others) leads to a ranking nightmare.

I AM experiencing the nightmare, sitting in the 300s for an e-com site that used to be top 10. I guess I can only hope that my new 301's will force G to get this straight some day.