homepage Welcome to WebmasterWorld Guest from 54.211.230.186
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 211 message thread spans 8 pages: < < 211 ( 1 2 3 4 [5] 6 7 8 > >     
"UK results only" - site: command shows index page missing
A problem in Google UK (UK results only filter) affects non .co.uk domains
fishfinger

5+ Year Member



 
Msg#: 3051237 posted 8:03 am on Aug 18, 2006 (gmt 0)

Except on these UK datacentres

64.233.183.107
64.233.187.99
66.249.85.107

If you do a site: search using Google UK's 'UK results only', the index pages of ten .com domains and three .net domains (out of 65 that I manage) are not listed.

1) All sites hosted in the UK across 4 different servers in 3 different DCs
2) No link exchanges, very little outward linking, no excessive inward linking. No inter-linking. These are not directories, MFA, affiliate. They are contact/services offered websites for UK companies and sole traders. Some are dynamic, some static. Size from under ten to under one hundred pages.
3) No canonical issues, no dupe content issues, no over-optimisation. I use the same techniques and links for all my sites. Number of affected sites has not grown since this problem was spotted. Unaffected sites re-cached this week.
4) Internal pages all listed AND RANKING for their terms.

Would other UK based webmasters like to share their experiences? Perhaps we can collectively contact Google and refer them to this thread.

 

hughie

10+ Year Member



 
Msg#: 3051237 posted 1:30 pm on Oct 30, 2006 (gmt 0)

Hi Wilburforce

i think a few people on this thread (it's been running since august!) had tried the same with decent results so i gave it a whirl.

Mine lists on a competitive "blue widgets" keyword no.2 in UK/web normally on google.co.uk and it was listing on "search the web" but not in UK.
(UK server, UK IP, .com Domain)

I didn't do too much to the content, mainly a basic re-jig, but i did add about 10 more location keywords into the body and title(i.e. UK, england, wales Britain etc.)

The site was crawled that day and the new version was in the index and stayed put within 36 hours.

It might have been a coincidence, but it's strange that this has worked a few times now.

Cheers,
hughie

fishfinger

5+ Year Member



 
Msg#: 3051237 posted 10:46 am on Oct 31, 2006 (gmt 0)

My view is that it is just coincidence.

One of my longest affected sites has "brighton" in the domain name and is optimised for "brighton", "west sussex", "uk". Others have always been optimised for "london", "kent" etc etc.

As far as changing content goes - two other affected sites dynamically update content every visit to the page.

Tweaking won't change anything. But it's always good to look at your site and ask yourself if it could be better though :)

Wibfision

5+ Year Member



 
Msg#: 3051237 posted 9:33 pm on Nov 4, 2006 (gmt 0)

Hi Guys, My homepage is now back within the UK results. I didn't change anything. :-)

M_Bison

5+ Year Member



 
Msg#: 3051237 posted 5:43 am on Nov 5, 2006 (gmt 0)

That's good to hear webfision.

This topic has received quite a bit of attention in the Google Webmaster Discussion Forum over the past week.

Funnily enough, if you browse through the topics form the past week, it seems that Google has removed the threads about the topic, but you can still access the disccusion directly if you have the URLs.

Needless to say, no one from Google has said a word about it.

Anyway, from the discussion, I have about 10 sites from both Australia and UK (including mine) that I shall be watching to see if they come back.

I am also now quite positive it is a canonical type issue as all the sites affected have links on their site to http://www.example.com/index.html or something like that.

M_Bison

5+ Year Member



 
Msg#: 3051237 posted 10:31 pm on Nov 5, 2006 (gmt 0)

OK, so I update https://www.example.com to 301 to http://www.example.com.

Google picks it up, but now they are indexing http://www.example.com/home.php (a PPC landing page only linked to in Google ads and formerly in my sitemap.xml file) in the AU index. Unfortunately home.php has the same <head> as www.example.com, thus Google treating them the same.

Anyway, I have significantly changed the h1 tag on www.example.com and added home.php to robots.txt.

This is becoming very frustrating.

Wilburforce

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3051237 posted 11:08 am on Nov 6, 2006 (gmt 0)

My homepage is also back this morning (#1 for primary search term), having been out since 3 October. Last crawled on 28 October. No changes at this end since previous crawl.

I don't think it is a canonical issue: my own site uses www.mysite.com throughout (no instances of mysite.com or www.mysite.com/index...), and has come and gone several times.

Let us hope Google now have it fixed (...he said, with no great conviction).

M_Bison

5+ Year Member



 
Msg#: 3051237 posted 11:15 am on Nov 6, 2006 (gmt 0)

I don't think it is a canonical issue: my own site uses www.mysite.com throughout (no instances of mysite.com or www.mysite.com/index...), and has come and gone several times.

yes, but have you linked to index.... within the past year? For my site, when "Pages from Australia" is selected, it returns a heap of previous supplemental results which haven't been redirected or 404'ed - eg http://www.example.com/blah/?id=xx. as opposed to http://www.example.com/blah/ which is the current link.

Without "Pages From Australia" the site command returns the proper results.

briggidere

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3051237 posted 11:33 am on Nov 6, 2006 (gmt 0)

our site has come back into the "pages from the UK" results now. it happend about 10 days ago.

We didn't do anything to make this happen, it just came back after 3-4 weeks of not appearing.

I think this is something buggy at googles end and just being patient will fix the problem for most people.

Wilburforce

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3051237 posted 12:14 pm on Nov 6, 2006 (gmt 0)

A couple of instances in the last year, yes. I cleaned it up in August, when the problem first affected me, although I don't think this has made any significant difference. Canonical issues wouldn't have a geo-specific effect.

I don't have (and haven't had) any problems with supplemental results. Generally, site: searches on the UK DC's have correctly returned all my current pages (other than the homepage) since August. Specifically the problem has been exclusion of the homepage (and only the homepage) from UK-only searches.

UK_Web_Guy

10+ Year Member



 
Msg#: 3051237 posted 8:56 am on Nov 7, 2006 (gmt 0)

Still seeing this issue coming and going for some sites - some have been affected by it for a long time now.

Another thing i've noticed this past few days, for certain keywords sites are not showing up in the SERPS, only for certain keywords, but when you add "&filter=0" to the end of the search string, they show up - i remember seeing this a lot a few months back.

Jez123

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3051237 posted 9:24 am on Nov 8, 2006 (gmt 0)

My site has been affected again! I will change the titles of the affected pages again as this caused (or seemed to) the index to come back last two times. I am noticing that it isn't just the main index page that can be affected by this - I have a few mini sites within the same site that have been hit as well. Not all are the classic index.html but can be descriptive-product-index.html as well.

My site is a .net and I am seeing .coms and .nets affected the same way in my SERP

I am getting a bit sick of this and am contemplating 301 redirect to a .co.uk domain but am really scared of risking it! Anyone taken drastic measures like this?

[edited spelling]

[edited by: Jez123 at 9:25 am (utc) on Nov. 8, 2006]

M_Bison

5+ Year Member



 
Msg#: 3051237 posted 10:05 am on Nov 8, 2006 (gmt 0)

I think a 301 to a new .co.uk domain would be the Google ranking equivilent of suicide.

My site is still affected, and I still believe this is a canonical/duplicate content issue.

My theory is that Google has introduced a new geotargeting filter which when duplicate content within the same site is discovered, it tries to decide which is more country specific (eg "More Australian" or "More British") based on inbound links, amongst other factors.

This duplicate content may come from the supplemental index, or may have been included in your Google sitemap, it doesn't matter - Google will still find it and index it in the country specific index, bumping your home page/other pages out of the index.

What this means is that if you have ever (past year maybe?) linked, or included in your Google sitemap a reference to www.example.com/index.htm, https://www.example.com, www.example.com/?blah=blah etc etc, it will still be in Google's index and will be ripe for indexing and inclusion in your country's specific index.

This means that you have to totally and utterly annihilate any canonical issues that you have, have had or will have in the future.

Things I have done in the past week/few days:

Redirecting non-www to www (or vice versa). If you have a DNS wildcard entry which goes to your site, you need to set up a redirect for that as well.
If a page has unneeded query parameters, do a redirect to the proper page.
Redirect https:// to http://
Disallow similar pages to my home page in robots.txt (eg PPC landing pages).

I base this theory on random searches, URLs given in Google's Webmaster Discussion Group and my own site. Hopefully I am right and my site will be back soon.

hughie

10+ Year Member



 
Msg#: 3051237 posted 11:38 am on Nov 8, 2006 (gmt 0)

i'm with you on the dupe content issue, whilst my site is still fine (fingers crossed) the effects were identical to a previous issue when some shark lifted the entire front page of my website.

Google totally ignored my home page on that occasion in the same way that it did with this "uk results" issue.

However that doesn't explain why it appears perfectly well in the "web" results.

fishfinger

5+ Year Member



 
Msg#: 3051237 posted 1:32 pm on Nov 8, 2006 (gmt 0)

I think a 301 to a new .co.uk domain would be the Google ranking equivilent of suicide.

I agree. Since BigDaddy cam along you will effectively sandbox your site if you move domains - doesn't matter whether you re-brand to a new domain or just switch extension. I've had two sites hit by this; nice of Google to tell us that 301s were no longer honoured wasn't it?

Re : Canonical Issues

I have had a few sites affected where on top of *never* linking internally to /index.htm or similar (I've not done this on a site for 3 years I always use the domain - in fact I always use absolute links, never relative for all pages on all sites) I use htaccess to avoid this ever happening.

It's a reasonable explanation but it just doesn't fit what I'm seeing on more than one site.

M_Bison

5+ Year Member



 
Msg#: 3051237 posted 9:52 pm on Nov 8, 2006 (gmt 0)

My site is back, as well as another site mentioned on the Google's discussion forum. Hopefully it stays that way.

M_Bison

5+ Year Member



 
Msg#: 3051237 posted 10:13 pm on Nov 8, 2006 (gmt 0)

I have had a few sites affected where on top of *never* linking internally to /index.htm or similar (I've not done this on a site for 3 years I always use the domain - in fact I always use absolute links, never relative for all pages on all sites) I use htaccess to avoid this ever happening.

It's a reasonable explanation but it just doesn't fit what I'm seeing on more than one site.

Remember it's not just a canonical issue, it is a duplicate content issue too. That means if, for example, you have had a page with the same <head> as your home page, then it could be bumping your home page out of your country's index.

I've seen this on my own site and when I do a search on google.com.au for the name of a large Australian company who runs many sites with TLDs - one site has duplicate titles and meta descriptions and is affected by this problem. All other sites have a unique <head> and aren't affected by this problem.

Jez123

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3051237 posted 12:52 pm on Nov 9, 2006 (gmt 0)

It's a reasonable explanation but it just doesn't fit what I'm seeing on more than one site.

Me too. I am not seeing this at all.

Wilburforce

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3051237 posted 8:52 am on Nov 10, 2006 (gmt 0)

UK & global #2 for key search term in my sector has gone missing in the UK again this morning. Info: finds the homepage but site: doesn't (i.e. the original problem conditions apply).

Clearly it isn't over yet.

fishfinger

5+ Year Member



 
Msg#: 3051237 posted 9:33 am on Nov 10, 2006 (gmt 0)

if, for example, you have had a page with the same <head> as your home page

M_Bison, again this just isn't what I'm seeing on the sites I have affected. Some only have 10 pages or less and the titles, descriptions and keywords are unique for all.

I have posted on the Google Webaster forums linking to this thread and one of W3bw0rks0p; I've also posted on Matt Cutt's blog (again, he did look at my last post but said there wasn't an issue) and I've even contacted our Google Adwords rep about this to see if anyone from Google will take an interest.

asher02

5+ Year Member



 
Msg#: 3051237 posted 2:08 pm on Nov 10, 2006 (gmt 0)

My site is is also effected as from today. A com site hosted in the US is no longer showing for pages from US. Only the home page is effected as other pages are in the index. Does anybody else with US hosting have the same problem when searching for US pages?

Jez123

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3051237 posted 9:46 am on Nov 14, 2006 (gmt 0)

Well, my usual trick of simply changing the index page title didn't work this time! 3rd time unlucky. Looks like I am going to suffer for a while.

I see another site in my SERP that has recovered after being affected for about a month or more - I don't think this is over for them though. This is another one of those tricky little google puzzles that we may never know the reason why.

lee_sufc

10+ Year Member



 
Msg#: 3051237 posted 2:48 pm on Nov 17, 2006 (gmt 0)

My site was affected by this about 1.5 months ago and came back after about 3 weeks.. Having hoping that was the end...the problem is back again!

I am a .net hosted in the UK aimed at UK clients but with inbound links from throughout the world.

mcdermc

10+ Year Member



 
Msg#: 3051237 posted 3:35 pm on Nov 17, 2006 (gmt 0)

One of my sites just returned yesterday to the uk results index (.com UK hosted) for the 2nd time since August. In both cases it was 'disappeared' for a whole month each time.

My view - don't worry: it's almost certainly nothing you've done. It's them! It's got to be.....hasn't it?

Now where was that darkened room?

kmick

10+ Year Member



 
Msg#: 3051237 posted 11:25 am on Nov 21, 2006 (gmt 0)

After 3 months missing from the UK search, our home page is back. Two of our main 2 words phrase search is now #1 and #2 in UK SERP.

Only two changes have been made since my last post on 28 Oct;

1. Update our new product data pages, which is a routine job for every 4 weeks, so I don't think this is the reason.

2 Change one internal link from mysite.com/index.html to mysite.com/

I don't know is this helped us back to UK search or not, but according to other topics in this forum about duplicate content and one URL per one page issues, it might make little sense. Or, simply it was Google bug?

Anyway, I just hope that our site not going to disappear again from UK SERP, and wish those who still out in the cool back to the UK SERP soon, good luck.

GingerTom1403

5+ Year Member



 
Msg#: 3051237 posted 1:47 pm on Nov 25, 2006 (gmt 0)

www homepage back in SERPS and site:www.domainname.com query results

Like others our main site has been greatly affected by this issue, but since yesterday evening things seem back to normal.

The actions we took, which may or may not have contributed to this, are as follows:-

1. Notified Google that we wanted their indexing to show www.example.com rather than example.com

2. Replaced all links to http://www.example.com/index.html on our website with http://www.example.com/

3. Replaced all links to http://www.example.com/index.html in Google Adwords with http://www.example.com/

The one thing we haven't done yet is a 301 redirect from http://example.com/ to http://www.example.com/

However, if we have similar probs in future it's top of my list of actions to try.

Hopefully others will find the steps we've taken work from them as well.

[edited by: tedster at 9:19 pm (utc) on Nov. 25, 2006]
[edit reason] use example.com [/edit]

UK_Web_Guy

10+ Year Member



 
Msg#: 3051237 posted 3:04 pm on Nov 25, 2006 (gmt 0)

loads of sites back showing index pages on .co.uk site: searches.

this was a google glitch which seems to have been fixed, until the next time.

lee_sufc

10+ Year Member



 
Msg#: 3051237 posted 3:12 pm on Nov 25, 2006 (gmt 0)

not fixed for us yet

mcdermc

10+ Year Member



 
Msg#: 3051237 posted 3:44 pm on Nov 25, 2006 (gmt 0)

My UK hosted dot com index page just disappeared again from the uk filtered index today for the 3rd time of asking...!

GingerTom1403 said - "The actions we took, which may or may not have contributed to this"
My input - I don't think that your actions had any effect, nor contributed to your return to the index!

UK_Web_Guy said, - "this was a google glitch which seems to have been fixed, until the next time."
My input - it was, is still, and will continue to be something wrong at the Google end!

swones

10+ Year Member



 
Msg#: 3051237 posted 10:06 am on Nov 27, 2006 (gmt 0)

Not back here yet either, the first time it happened I tried the suggestions that others made on this thread and it came back for about 2-3 weeks. Now it's been gone again for as long.

Has anyone actually made any progress in getting one of the Google reps around here look into this? It surely can't be something that they "want" to be happenning, it just wouldn't make any sense.

Simon.

UK_Web_Guy

10+ Year Member



 
Msg#: 3051237 posted 10:49 am on Nov 27, 2006 (gmt 0)

Sorry to hear not all have come back, some sites which have been missing for ages came back in this latest refresh.

I don't like it when it happens, but to be honest I don't know that it makes a huge difference to traffic, not many people click on pages from the UK, most just type in the query and hit enter.

Interested to hear if anybody thinks differently on this...

Wibfision

5+ Year Member



 
Msg#: 3051237 posted 12:33 pm on Nov 27, 2006 (gmt 0)

Hi Guys, Just checked, after being back for weeks my homepage has disappeared again from the within UK search.

This 211 message thread spans 8 pages: < < 211 ( 1 2 3 4 [5] 6 7 8 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved