Forum Moderators: Robert Charlton & goodroi
The rot started last year when traffic from the West Coast fell significantly, although the other US regions held up.
This year the site was hit twice by updates and lost all Google traffic, the last time on June 16. Traffic returned starting July 22 and for the UK, Europe, and Asia is back to normal. However the North American traffic has NOT returned. The East Coast and the Mid West have followed the West Coast into oblivion.
When I search for keywords on google.com or a specific US-based Google IP, the serps look the same as the UK results.
But I wonder if I am seeing the same thing as a user located in the US. Could there be a filter that is applied in the US depending on the IP address of the searcher? Perhaps implemented first in California last year?
The only reason I can think of for such a filter is to push down in serps those sites that are slow to load or difficult to access from the searchers region. And to get to a UK-hosted site there's that big, big hop across the Atlantic. I've noticed that even the best carriers can lose IP packets on these inter-continental links.
My own host's connectivity leaves a lot to be desired and I am thinking of moving to a better host with links to the premier carriers. This may help if Google is checking individual site's access times. But if the problem is simply that the site is hosted in the UK, that is not going to help.
Can anyone say if the above is true, or is there another explanation? More importantly what can I do about getting my US traffic back? I'm getting paranoid. :)
There are a couple of US sites from which I can run VisualRoute online. I compared my site with other UK hosting companies own sites. From the US my site doesn't seem any slower than the better hosted sites, although there are more hops. However the last leg to my site and another medium-cost hosting site blocks ICMP packets.
From the US occasionally the route to most hosting sites (even expensive ones) dropped IP packets. Although I never noticed this with one of the best which has Tier 1 connections.
I also tried running VisualRoute from within the UK (Surrey). The last leg to my site (in Manchester) sometimes lost IP packets. It was also a lot slower than sites hosted in London's Telehouse or those with Tier 1 ISPs. I am definitely going to move just because of that! :(
Doing the same thing from China all ISPs sometimes dropped packets, even the best. The big difference was that top-level hosted sites were routed via networks with fewer hops across the US. One of the medium-cost hosts I tried had 3 hops in San Jose, 2 in Dallas, and 2 in Washington. Whereas the best one I tried (and one of most expensive) went via France Telecom from China > Los Angeles > Ashburn > New York > London > Paris, where it picked up its own network straight to its UK servers.
I intend to move to a better hosting company with access to a Tier 1 network. But I doubt that my current situation is bad enough for Google to single it out. It's still a slow hop across the US and then the Atlantic to get to any UK hosting company.
I am still wondering if Google takes this cross-Atlantic transit time into account in its US serps. What would be good would be if someone in the US could compare their serps with my UK serps to see if all UK sites are being pushed out of sight - or if it's just me.
I think there may well be a latency/lossiness penalty, and with that in mind I now host mirrors of my site in the UK and the US. I certainly see some terrible link performance these days but it could be anything from my local WiFi router though contention on my SDSL link to poor routing from my ISP across the Atlantic, etc, etc.
By hosting in multiple locations I am hedging my bets.
(Note that my visitor numbers are roughly equal from North America and Europe, with a much smaller fraction from "other".)
And anyway, hosting in the US can be so cheap that you might as well have a mirror there, IMHO.
Rgds
Damon
How would I go about setting up a mirror site? I could for instance have a .com hosted in the US and a .co.uk hosted in the UK, but how would I avoid a duplicate page penalty?
All (3) of my mirrors are accessible by the same URL, so there is no "duplicate" content, just multiple servers for one site. The user's browser will pick one server essentially at random from the list offered it by DNS, also known as "round-robin". I have plans to create a geo-sensitive DNS to hand out the "closest" server to each caller, but for now I handle that sort of load balancing at a different level. For example I serve the bulky content explicitly from the "closest" site to each user for good performance even though they may be getting the HTML pages from any mirror they picked from DNS.
(The servers are accessible by individual mirror names too, but that does not seem to hurt in the SEs so far as I can tell.)
I've done all my mirroring code by hand in Java because that's my thang, but I'm sure there must be a dozen ways of doing it simply with free or commercial tools (eg rsync and its derivatives) depending on the structure and content of your site.
Rgds
Damon
I think your solution would be overkill for my site which is essentially an information site. It seems to me my options are:
a) move to a better host in the UK. But that won't help if Google is applying a filter in the US to all distant sites.
b) Move the site to the US. That way I would expect (hope!) to get my US traffic back. The downside is I would lose UK traffic from those users who search for "UK only".
c) Move the site to the US, but split of those pages that are specific to the UK into a .co.uk site. Downside is I would lose established links.
Before I do anything I need to be certain why the North American traffic has disappeared. Traffic everywhere else in the world is up, even allowing for the holiday season. Maybe the reason has nothing to do with its location, but it seems the most likely problem. It would be nice to know if anyone else with a .com site hosted in the UK has lost traffic from the US.
Anyone in the US care to do a test? A Google search on "latest news headlines" (without quotes) brings up the BBC as number 1 in the UK. What do you see in the US?
I am still wondering if Google takes this cross-Atlantic transit time into account in its US serps.
I am sure Google knows my site is hosted in the UK from "who is" info. It will also know it is a distant site by the response time when it crawls the site from California.
My suggestion is that Google may be using this information in order to provide what it considers as more relevant serps to its US users. It doesn't have this problem in other countries because there is a local search option. Local sites can be identified just by .uk, de, fr, etc., but a .com site can be anywhere.
It is could also in Google's interests to ensure that sites that are slow to load are kept way down in serps - so as not to give users a bad impression of the company.
From the fact that the BBC shows as #1 in the US and the UK suggests this might be done on an individual site basis.
I haven't emailed Google because I am sure they are not going to discuss their algorithm or filters. Although I might try it as a last resort. :)
Thanks for that. From what you say, it seems that if there is a filter it is being done on an individual basis. Can I sticky you with a couple of search terms relating to my own site to test?
You may well be right. But the only way I am going to know for certain is if someone can check the US serps for me.
My solution is probably an overkill for me too (!) but seriously, getting a decent US mirror host and replicating content to it periodically would probably be very cheap and easy, and would retain all your existing links, and overall improve performance for some or all of your visitors.
Rgds
Damob
I might try that route. But I would like to try and find out if hosting location is the problem first. I have lost all traffic twice this year - and I don't want to do anything Google might not like! I'm still not sure what caused the previous problems, although it affected many people.
If it's not a hosting or location problem, then there is the possibility that the filter that previously took away all my traffic, has been lifted everywhere but in the US. In which case moving hosting will not be the solution.
Well, I think that conclusive answers on this one will be just as hard to come by as when I tried to before going to the trouble of setting up my mirrors... Let us know if you do find out!
Rgds
Damon
Hello! Is there anyone out there in the US who could run a check? I have certain keywords in mind, but I don't think the mods would like me to post them here.
If your site has something to do with tourism your US traffic will be much lower since the events of the 7th and 21st.
In general the sites we power are down around 45% from the US.
TravelDog
Google serves up results based on where it perceives the web site is hosted. A .com hosted on a UK web server will come up in 'UK only' searches on google.co.uk. That's why I had to change web hosts to a UK one; got me more traffic.
If you want your .com to be recognised as American, host it there, and use American spellings [shudder] in your text.
Similarly, when I type [google.com...] in my browser location box, I'm redirected to [google.co.uk....] Google knows _my_ IP number is a UK one. If even 5% percent of people opt for ticking the 'UK only' button in their searches, that's 5% a US-hosted site will never see, unless ...
... your domain name is a .co.uk. That can be hosted anywhere, and come up in 'UK only' searches.
I'd bet that, likewise, Google favours US-hosted .com sites for US-specific searches. Why not? Easy to do.
So to answer your question (I think) Google does use geo-location.
Someone who lives in the US decides that he wants his cool US-related and US-only site to be hosted on a free host that does not insert any ads into his pages. He's also looking for unlimited bandwidth. He finds a free hosting company whose offer matches his criteria. He doesn't care where this company is located (it's in the UK).
Another example: Many international sites (including European) are hosted in the US.
If Google paid attention to geo location of the server on which your site is hosted, then their search results would often be pretty skewed and irrelevant. I find it hard to believe they would be so silly.
However do all users in the US see the same thing? Do people on the West Coast get the same results as on those on the East Coast?
Otherwise I am at a loss to explain my loss of US traffic. Currently the US is about 30%, the UK and Europe is about 40%, and Far East 15%. Going on earlier results, and the fact that most pages are of international interest rather than specific to the UK, I would expect the US traffic to be far greater. Currently the US traffic appears to be mainly from MSN, Yahoo, and Google images search.
Searching with keywords unrelated to travel my pages are often #1 and #2 in the "pages from UK" serps, and although they are lower in the "all the web" serps they are typically in the first 10, and therefore visible to the average user in the US.
However things are different when searching for terms which may be considered related to travel such as a well known foreign tourist attractions. For the top keywords I am nowhere in any serps, which is possibly to be expected. But for more niche keywords my pages are often within the first 10 in "pages from UK" serps, but are not in the first 100 for "all the web" serps.
It's not just my site. The #1 in "pages from UK" for certain keywords (think well-known marvel of the world supposedly visible from space) is #23 for "all the web", hence invisible to average users.
Some of this is clearly due to more competition within "all the web", but there are a LOT of very crappy pages ahead of mine. It's that *!%!ing travel algo again!
<added later> Thanks TravelDog for pointing me in the right direction </added>
Some of this is clearly due to more competition within "all the web"
