Can Google detect US VPNs? Mine are now redirected

[chrislloyd515]

I've been seeing some unsettling behaviour recently with accessing google.com through VPNs, presumed it was a bug at first, but it's been present for over 2 weeks now.

In order to get accurate USA positioning I have to view google.com though a USA VPN, I have access to 9 servers (a mix of a commercial VPN offerings and my own private servers... not a VPS, a private server!)

I have used this approach to get USA rankings for a few years now, but in the past few weeks I've noticed that google is redirecting google.com visits to google.com.hk or google.it or google.co.in. Essentially identifying my VPNs as Chinese, Italian or Indian.

If it is any help my servers are located in New York State, Texas and Los Angeles. All but one now being identified as a random non-USA visitor.

Thinking that google.com may have a way to identify that I'm visiting through a VPN I even used lynx on one of my Texas servers to access google directly, so no VPN involved, google.com still detected me as chinese.

The standard things I've checked:
- No cookies from Google at all, all browsing data removed when switching from direct connection to VPN, even used firefox's privacy mode
- Checked IPs of my servers against every geo-IP database and every one identifies me correctly as USA
- Tried OpenVPN and PPTP VPN protocols and a simple proxy, all showing same behaviour

Anyone else seen this, or have additional information?

[TheMadScientist]

Hmmm... You'd have to track down the thread (it's recent) but I remember reading about someone's iPhone being redirect to a different country the other day when they are and have been in the US. My guess is something's not working as intended...

Personally, I can't wait until they get out of their beta launch.

[chrislloyd515]

I thought it was a bug at first but I'm starting to think that it's a genuine attempt to target VPNs.

If it was a bug that was causing random people to be redirected the amount I'm seeing it (90% of the time) then I'd think it would have made much more news then it has as random people get caught up in the bug. (It's very noticeable when your results turn indian!)

It seems targeted to me! Just my two cents though!

[tedster]

Essentially identifying my VPNs as Chinese, Italian or Indian.

Are those countries related to where you are physically located?

[chrislloyd515]

No they are not, I am located in Singapore, so when no VPN is activated I default to .com.sg

[tedster]

I know that VPN technology is complex, but I'm not immediately aware of any way that such a connection is "leaked" to a web server.

Can you run a checker such as the Firefox LiveHTTPHeaders add-on and watch the chatter between your browser and the server? Then you could compare that with a regular connection's back and forth chatter to the server.

[chrislloyd515]

In order to remove all possible variables introduced by the VPN I installed a browser on 3 of my US based servers and used that, so absolutely no traffic was being sent to singapore by the browser, and I'm seeing the same behaviour.

So, the VPN isn't the issue, it's at the server level.

I see reports of this on other forums now aswell. (won't link as know that's against guidelines)

[indyank]

Are any of your servers located in those regions?

Could this be related to zombie traffic that many, starting with backdraft7, complain these days?

[chrislloyd515]

indyank: No, I have no connection to those regions at all, had never even been to google.in before january!

Ok, analysed the headers, when it works correctly firefox requests Google and gets a straight 200 response

When it goes wrong I'm getting the following 302 redirect

HTTP/1.1 302 Found
Location: [google.com.hk...]
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=bb26933625fd9b3a:FF=0:NW=1:TM=1298345721:LM=1298345721:S=0OnLBC3RTzUzOoSc; expires=Thu, 21-Feb-2013 03:35:21 GMT; path=/; domain=.google.com
Set-Cookie: NID=44=qBZQvbx-NVQQXPdqODFu0tltnhTnOulE6a40hXmk9gadSVBJkur9AXZ9bmDbqvpj4iPzLC-0kAKpEOx5wijZYTIE8VwhJYWvQdi6zoUE3nBBN1oDxBWv2ulAC-cG9ec9; expires=Wed, 24-Aug-2011 03:35:21 GMT; path=/; domain=.google.com; HttpOnly
Date: Tue, 22 Feb 2011 03:35:21 GMT
Server: gws
Content-Length: 376
X-XSS-Protection: 1; mode=block

When I look at the URL that I'm being sent to -

Location: [google.com.hk...]

Has anyone seen sa=p or pval=yes parameters before? p=proxy?


If you want to compare this to my normal direct internet connection... I'm based in singapore so google.com redirects to google.com.sg. The 302 I'm sent in this scenario is the following:

HTTP/1.1 302 Found
Location: [google.com.sg...]
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Date: Tue, 22 Feb 2011 03:44:16 GMT
Server: gws
Content-Length: 222
X-XSS-Protection: 1; mode=block

A very simple redirect, no extra parameters.

Seems deliberate to me?

[edited by: tedster at 4:43 am (utc) on Feb 22, 2011]
[edit reason] prevent side-scrolling [/edit]

[lammert]

Having Lynx from the command level at a Texas server being redirected to Google China is an indication that headers or other client side information isn't used in this particular redirection. It seems therefore to be an IP based redirection.

The IP you are using to access the Google servers is in a data center and not on a home/office location. Google probably has the knowledge to determine that. It could use recent traffic it sent to that server IP/IP block/data center to make a best guess which country is targeted by the servers in that data center and redirect visitor traffic from that data center to one of their local search engines based on that best guess.

Probably just one of the other Google attempts to serve the visitor what he needs, rather than what he wants (cough, cough)

[chrislloyd515]

the only pieces of data that I have to go against that lammert is that

1. It's not always the same google I get redirected to.
2. I have no visitors from those regions, to my server, as you say it may affect the data center as a whole, but that seems strange when it's a US hosting provider (linode.com for my small VPSs)

[indyank]

Even if google is trying to make the best guess, redirection to chinese (google.com.hk) and Indian (google.co.in) seem to be fairly close but not the Italian one! Are these the only three you are being redirected to everytime?

^{ps: I tried my best and one commonality I could see between these three nations is the popularity of their food recipes. :)}

Have you tried running this from a different computer? i.e. logging on to you server from a different computer and browsing google?

There are some rootkits and trojans known to cause redirections. So, you should ascertain that this is occuring even when you access your servers from other computers.

There is also one google help page on this.

[google.com...]

[chrislloyd515]

Accessed from multiple connections, from multiple macs (don't use windows) through multiple linux servers, I got results from Google Bahrain today, but only the once.

I'm running out of factors that aren't... Google just taking a dislike to me and many other servers, I see numerous complains on SEOBook about this issue.

[lammert]

Try to access Google with http://www.google.com/ncr This should stop the redirection to specific country versions of the search engine. (NCR = No Country Redirect)

[chrislloyd515]

Yes that does make it stay on Google.com, but it's not the USA google.com. It's the google.com as you see it from china/india/italy giving preference to chinese/indian/italian sites, and not giving preference to american sites, essentially very different results.

[tedster]

And that happens even when you access the NCR address through your VPN tunnel to a US server?

[engine]

I doubt it's connected, but i've seen some weird things happen when trying to log in. Suddenly, i'm redirected to a non-english language.
The only solution to stop that happening was to remove the cookies.

One other test to try: Look up your IPs to check they are showing correctly.

It does sound a little strange to be redirected to differing countries.

[deadsea]

My coworker was being redirected to Google Brazil from our US office this morning even though she has not used it before and does not have Brazil or Portuguese as any settings in her browser. I was not having a problem from the same IP address, still getting google.com.

Sounds like G introduced some bugs into their "should I redirect" algorithm recently.

[chrislloyd515]

And that happens even when you access the NCR address through your VPN tunnel to a US server?

Yes

One other test to try: Look up your IPs to check they are showing correctly.

They are, and they are being identified as USA IPs

[pr0s7]

So, it looks like it is a google issue and not a problem concerning any particular IP?