homepage Welcome to WebmasterWorld Guest from 54.205.7.136
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 42 message thread spans 2 pages: 42 ( [1] 2 > >     
Google Dumped Our www.domain Listings
Why would Google index a site with WWW?
NexDog




msg:133706
 3:25 pm on Jan 2, 2004 (gmt 0)

We compete in a fairly competitive market and 2003 was a real rollercoaster ride. High before March and then wiped away by the new algo in that update (Esmerelda I think) and then the slow crawl back up the rankings and getting back to #4 in September and hanging on to page one until December.

Since the beginning of December we have been slipping down almost constantly and are now on page 3. Unsure what's happening and many new players have enetered the niche but I always though our back linkage was strong enough. But now when I examine our listing, I see:

domain.com/ - 42k - Cached - Similar pages

Compared to 99% of:

www.otherdomain.com/ - 26k - Cached - Similar pages

All our backlinks point to WWW and now it looks like the WWW isn't even listed. Why would this happen and what can I do to get Google to index WWW as it does for everyone else? Why are we special? Is it worth asking Google and would they even reply or look at it?

 

dirkz




msg:133707
 4:36 pm on Jan 3, 2004 (gmt 0)

> All our backlinks point to WWW

This is something you really cannot know. Backlinks display is far from accurate.

The solution is to redirect from one version to the other via a 301 (in your case preferably from the WWW to the other one).

NexDog




msg:133708
 4:58 pm on Jan 3, 2004 (gmt 0)

There is no redirect or need of one. domain.com and www.domain are one and the same but why would google show just domain.com on a search for "Blue Widgets" and show www.domain.com on a search for "Cheap Blue Widgets"? I'm sure this has killed our ranking - there is absolutely no other reason for it.

Will Google fix itself in regards to this? I filled out their feedback form and informed them but I can't see why this would happen. It's like a fluke error that has killed us, lol.

dirkz




msg:133709
 5:04 pm on Jan 3, 2004 (gmt 0)

> There is no redirect or need of one.

I will install one instantly if I were you :-)
In order to get massive free traffic, you sometimes have to help Googlebot.

jdMorgan




msg:133710
 5:12 pm on Jan 3, 2004 (gmt 0)

NexDog,

www.example.com and example.com are not one and the same, even if they end up serving the same content. www.example.com is a subdomain of example.com, and is not necessarily the same site.

Therefore, GoogleBot has gotten confused and split your site across two domains.

As dirkz says, a 301 redirect to the domain with the majority of incoming links from the other domain will take care of this problem.

Topic search on WebmasterWorld [google.com]

Jim

[edited by: jdMorgan at 5:16 pm (utc) on Jan. 3, 2004]

Jabzebedwa




msg:133711
 5:15 pm on Jan 3, 2004 (gmt 0)

Until Florida I had thought that these, www.domain.com and domain.com were both one and the same. Then one of the earlier Florida theories congectured here was that this was the root issue for lost positions, and since then I remain confused. I wrote my ISP about it and they didn't seem to make any distinction between the two either. Should this redirect thing really play a part in a viable SEO strategy? There is quite a bit of conflicting info on this. If so, is it best to redirect the www.domiain.com to domain.com, or visa versa?

Thanks,

Jabzebedwa




msg:133712
 5:18 pm on Jan 3, 2004 (gmt 0)

While I was writing my post, jdMorgan posted his which fairly well answered my question. I'm off to redirect.

Thanks again,

jim_w




msg:133713
 7:43 pm on Jan 3, 2004 (gmt 0)

JD;

www is a sub-domain? I didnít realize that. I always thought that it more of a reference to the protocol type being World Wide WEB. Like ftp.mydomain.com or gopher.mydomain.com and what ever the news server is.

chinook




msg:133714
 7:51 pm on Jan 3, 2004 (gmt 0)

This topic actually raises a very good point. Most hosting companies automatically set up a site to respond with
example.com and www.example.com, I think this has been somewhat of a historical practice.

Consider the situation of:
www.example.com
www2.example.com
www3.example.com
usa.example.com
canada.example.com

Each of the above could point to a separate site, all it takes is a dns entry that then points where it needs to.

I suspect people think that www as the first part has some special significance when in fact it is not treated any differently than www2 or the other "host" names.

Now since googlebot could see www.example.com and example.com as being "potentially" two sites which have the same content, this could contribute towards a duplicate content count. Too much duplicate content and well you know what happens.

There seems to be two choices:
1. Set up your site with only 1 host (host header) ie www
2. redirect one of them (301 redirect)(your hosting company should be able to assist you with this)

hutcheson




msg:133715
 10:30 pm on Jan 3, 2004 (gmt 0)

>www is a sub-domain? I didnít realize that. I always thought that it more of a reference to the protocol type being World Wide WEB. Like ftp.mydomain.com or gopher.mydomain.com and what ever the news server is.

That is a convention, and (as you had noticed) in the old days, many larger sites set up a separate server (computer) with its own subdomain name running the server (program) for each protocol. Now, quite frequently one computer runs programs for multiple protocols or multiple websites -- or at least, one publicly visible router hides all the various servers.)

The significancs of omitting the "www" logically ought to be "hey, this server runs all the protocols we need." But it's only one convention; there are other conventions in use these days; and there's nobody to punish rogue webmasters whose subdomain names are chosen at random.

dasboot




msg:133716
 12:46 am on Jan 4, 2004 (gmt 0)

Therin lies the problem.

Technically www.site.com and site.com are different sites. Google, for some reason, respects this distinction. But in practice, the number of sites where not-www and www are distinct sites must be very small these days.

It would be better if, by default, Google regarded these as the same site - unless instructed otherwise.

What happens, in our experience, is that if the googlearsebot finds a not-www link; follows it, and does a deep crawl - those pages are now not-www as far as Google is concerned.

It doesn't seem to matter thereafter how often the Googlearsebot visits from a www link - these pages are a not-www in its opinion.

You might think that Google would spot the dupe content - but it doesn't. The reason is that between not-www and www crawls the pages have probably changed. And G is crap at spotting dupe content anyway - probably just goes by the 'last modified' info.

So, in short, if G lists your site incorrectly - there is the possibility you will be listed incorrectly almost indefinitely.

johannamck




msg:133717
 1:01 am on Jan 4, 2004 (gmt 0)

Sadly, DMOZ' example for submitting a site does not contain the "www". (Last time I looked.)

I wonder how many webmasters end up with the "non-www" link from DMOZ because they pattern it after the submission example, while all other links point to the "www." version of their site because that's what is commonly used.

dasboot




msg:133718
 1:09 am on Jan 4, 2004 (gmt 0)

Our slow death site is listed www in DMOZ . But G has it associated with not-www (removing the dupe content filter displays it with description and all)

Unfortunately G has consistently chosen to list the www version of our index. Hence no DMOZ description, or the perks that go with it due to it being a brilliant site ;)

www is actually our default - so that's ok, but G associates the DMOZ listing with not-www.

I hope that makes sense - because it increasingly makes no f*cking sense to us.

<edit> added a wink in case irony was misunderstood across the pond.

NexDog




msg:133719
 1:40 am on Jan 4, 2004 (gmt 0)

That still doesn't answer why would google shows just domain.com on a search for "Blue Widgets" and shows www.domain.com on a search for "Cheap Blue Widgets". Anyhow, I add a rewrite rule so domain.com redirects to www.domain.com as suggested. I think mod_rewrite deals with this best:

RewriteEngine On
RewriteCond %{HTTP_HOST} ^domain.com$
RewriteRule ^(.*)$ [domain.com...] [R]

Lets hope Googlebot comes back in a few days and we see a jump back to page one this month. Thanks for the help, fellaz. :)

dasboot




msg:133720
 1:51 am on Jan 4, 2004 (gmt 0)

It's showing different results because one site has displayed different content to the other in previous crawls. Get it fixed.

NexDog




msg:133721
 2:02 am on Jan 4, 2004 (gmt 0)

Hopefully that rewrite rule will fix it. Google visits every 2-3 days so do you think it's reasonable to thin that the bot crawls the site deeply within 2-3 days, our position will change within a week? I mean, we don't have to wait for the main update anymore?

dasboot




msg:133722
 2:10 am on Jan 4, 2004 (gmt 0)

Google's arsebot deep-crawls our 'slow death' site most days (usually between midnight and 3 AM UTC - I prefer GMT ;))

It's accepting the re-direct. But there's no sign of pages being returned to the index. Only the index page is holding out. Google is f*cked & Google knows it.

rfgdxm1




msg:133723
 3:34 am on Jan 4, 2004 (gmt 0)

>Technically www.site.com and site.com are different sites. Google, for some reason, respects this distinction. But in practice, the number of sites where not-www and www are distinct sites must be very small these days.

But sites like that exist, and Google has to accomodate that. Site owners can just redirect one to the other to eliminate possible problems. If they don't, technically they are running 2 mirror sites.

NexDog




msg:133724
 7:22 am on Jan 4, 2004 (gmt 0)

dasboot - are you sure all your pages redirect from domain.com to www.domain.com? JS, meta-refresh and .htaccess redirects will only do the index or the actual page with the code.

That's why the RewriteEngine is the best option as it filters down to every file on your site.

Chico_Loco




msg:133725
 7:34 am on Jan 4, 2004 (gmt 0)

NexDogs example of a 301 seems ok, but you need to add "!" before the "^" in the second line (that doesn't match), and you also need to set the R to 301, so make it [R=301,L] in the last line, L meaning last.

Without the R=301, you'd get a 302, which may screw things up more!

NexDog




msg:133726
 8:59 am on Jan 4, 2004 (gmt 0)

Nah, that's all superflous. The [R=301] provides the redirect code of 301 (which is does anyway)and the L tells it that's the last instruction for this condition (which it is anyway).

The code works without the! which I never really understood why was needed. The rule I wote is active and working perfectly on our site at the moment.

hutcheson




msg:133727
 9:16 am on Jan 4, 2004 (gmt 0)

>Sadly, DMOZ' example for submitting a site does not contain the "www". (Last time I looked.)

<trying hard to think of a non-sarcastic response, and apologizing in advance for what will probably fail>

It's hard to know where to begin.

In my experience, the vast majority of domain names work fine without the "www.", and there are good reasons to prefer the simplest form of the URL that will work.

Then, based on the evidence, only a small minority of submitters read the policy, so it's hard to see that this is causing a great deal of turmoil in the DNS world. People type e-mail addresses, bare domain names without any protocol, type the protocol twice, omit the TLD, put site titles in that field, add subcategory and file names [also omitted from the DMOZ example for numerous very good reasons!], deduce correctly, all on their own that ".co.uk" and other non-".com" TLDs can be used even though the example doesn't show it ...

In a word, I wish the submittal policies were half as influential as you seem to think they are.

And I'm further puzzled as to why you should care about this. Anyone with ... OK, I won't go there ... should be able to figure out that YOU type YOUR URL in. Whatever it is. If you can't figure that out, then you really should put your mouse away before you hurt yourself. And if someone else can't figure that out, neither you nor the directory is harmed, your competitor simply gets a lower page rank.

NexDog




msg:133728
 9:55 am on Jan 4, 2004 (gmt 0)

Not exactly on topic but, yea...hmmmm.....ok. ;)

Jabzebedwa




msg:133729
 12:01 pm on Jan 4, 2004 (gmt 0)

In my site's control panel, I have the option of doing a redirect. The first part of the redirect is hard-stated (I can't modify it) as [domain.biz...] file here]. Note the trailing slash. This is followed by "redirect to [here I add my destination URL]"
So I did the redirect within this limitation, and the site disappeared altogether with or without the w's. I deleted the redirect, got my site back, and have since been in touch with my ISP. They are generally good at tech support and I have now heard back from them that they are working on it.

<edit>Now I have gone through a number of my sites and have gotten a mixed response typing in the non-w domain. One site duplicates the www site, one doesn't exist, and one gives me an interesting file directory of a really old version of the site. Wild stuff. I am inclined to think that in developing these sites that I have done something, beyond not invoking the redirects, to cause this, since it is the same server, different effects.</edit>

I am looking forward to getting this straightened out to see what, if any, benefit comes from it. I am starting to think this may have had something to do with the whollop some of my sites took in the Florida update.

Saludos,

I just discovered that on one of the non-w domains, I was pulling up a directory on my local machine by that name. Sheesh! I now have shut off IIS and am traceroute pinging away. I have also noticed that when the two sites appear identical that they may have a different page rank.

dasboot




msg:133730
 3:41 pm on Jan 4, 2004 (gmt 0)

A really informative thread this one. I had never been clear which one (www or not-www) was the sub-domain of the other.

Bit off topic - but looking for help - can anyone sticky & recommend a reliable company to rent a dedicated server from in UK? Posted as such in comunity forum but hasn't appeared.

NexDog




msg:133731
 3:55 pm on Jan 4, 2004 (gmt 0)

WWW is usually just a CNAME in your dns zone file that points to domain.com. So it's not a subdomain in the physically hosted sense although it most certainly can be hence google's confusion.

Servers are cheaper in the USA and the speed difference is hardly noticeable if at all. I'd rather have a faster server in the US for the same price as slow server in the UK (bandwidth costs more in the UK).

chinook




msg:133732
 4:25 pm on Jan 4, 2004 (gmt 0)

WWW is usually just a CNAME in your dns zone file that points to domain.com. So it's not a subdomain in the physically hosted sense although it most certainly can be hence google's confusion.

Clarification:

1. It doesn't have to be a cname, www.site.com could be pointing to one ip address and site.com pointing to a completely different ip with different content, or they could be on the same ip with different content. (A records)

2. The www portion of the address is the "host" portion. It is very conceivable that there are many hosts within a "domain".

3. For example in the IIS world, sites typically get set up with host headers and the configuration would be for an individual site to respond to both www.site.com and site.com. ( by way of example a popular control panel for windows (en?m) sets up sites this way by default.

4. Google is not confused just getting picky.:)

dirkz




msg:133733
 6:08 pm on Jan 4, 2004 (gmt 0)

One 301 a day keeps Googlebot confusion away :-)

> 2. The www portion of the address is the "host" portion.

I thought so, by definition. Where is my "TCP/IP illustrated" :-)

NexDog




msg:133734
 12:25 am on Jan 5, 2004 (gmt 0)

We host over 10,000 websites where www is a cname.
It is the normal config on any linux control panel.

plumsauce




msg:133735
 8:49 am on Jan 5, 2004 (gmt 0)


4. Google is not confused just getting picky.:)

no, when it is a cname, google is confused
because the owner of the domain has clearly
defined one as an alias of the other. since
this is clearly specified in the dns response,
if they choose to ignore the information, then
it is they who are in the wrong.

this should not be rocket science for the
much vaunted scientists at the plex.

This 42 message thread spans 2 pages: 42 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved