Forum Moderators: open

Message Too Old, No Replies

Google not indexing webpage - redirecting users based on browser language

         

yogapeace

11:32 pm on Apr 23, 2007 (gmt 0)

10+ Year Member



Hi,

My homepage redirects users with Chinese language set as the preferred language in their browser to the Chinese language website and takes all other users to a different webpage. Since I have made this change, google doesn't index the homepage. Baidu has been indexing it fine

I know the homepage hasn't been indexed because the "cache:" command shows data from months back before I made this change

Any ideas on how to fix this?

bill

12:28 am on Apr 24, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Personally I really don't like sites that redirect me based on my browser settings or IP address. Although a majority of your visitors may never say a thing these sorts of redirection are problematic in many ways. You're just seeing one of the main reasons now; Problems with search engine spiders.

How can you fix it? My suggestion would be to simply get rid of it and to think of an accessible site structure that will lend itself to browsing by humans and robots alike. However, you may want to keep it...and in that case you'd have to tell us how you're accomplishing the redirect.

yogapeace

3:24 am on Apr 24, 2007 (gmt 0)

10+ Year Member



Hi Bill,

I do need to keep the redirection. Basically I check the browser for "zh" language and if present I let it go to the homepage & if not present, I redirect it to another page.

The vbscript code is as follows (I took it off microsoft's website). I include this code on my .ASP homepage

<%
Dim strURL,strACCEPT
strACCEPT = GetAcceptLanguage()

' Change the language code below to customize the default.
If mid(strACCEPT,1,2) <> "zh" Then
strURL = "http://" & Request.ServerVariables("HTTP_HOST")
strURL = strURL & "/<english.htm>"
Response.Redirect strURL
End If

' This function returns the first accept-language.
Function GetAcceptLanguage()
Dim strHTTP, strLANG, strTEMP, intTEMP
strHTTP = LCase(Request.ServerVariables("ALL_RAW"))
If InStr(strHTTP,"accept-language:") Then
strHTTP = Trim(Mid(strHTTP,16+InStr(strHTTP,"accept-language:")))
For intTEMP = 1 to Len(strHTTP)
strTEMP = Mid(strHTTP,intTEMP,1)
If IsAlpha(strTEMP) Or strTEMP = "-" Then
strLANG = strLANG & strTEMP
Else
GetAcceptLanguage = strLANG
Exit Function
End If
Next
End If
End Function

' Determine if the character passed is a letter.
Function IsAlpha(strCHAR)
If (Asc(strCHAR)>=65 And Asc(strCHAR)<=90) Or _
(Asc(strCHAR)>=97 And Asc(strCHAR)<=122) Then
IsAlpha = -1
Else
IsAlpha = 0
End If
End Function
%>

DamonHD

7:50 am on Apr 24, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The solution I use is to localise the page content to Chinese when the browser settings indicate Chinese, to English for English browser settings, etc, ie the PAGE and URL remains the same, but the content adapts to the correct language for the user (from the limited range that I can support).

Furthermore, when the SE's bot gets to my Beijing site I have the site default to zh_CN (simplified), but when the same bot visits my London servers they see British English. There's even American English on the US server, and slightly 'Indianised' Indian English (en_IN) on the Bombay server.

No redirects at all.

As you'll have noticed, SE bots/spiders often/usually do not set a language at all.

Rgds

Damon

yogapeace

11:27 pm on Apr 26, 2007 (gmt 0)

10+ Year Member



Hi Damon,

If you don't mind, can you please share the code for how you do it?

DamonHD

7:16 am on Apr 27, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi,

The entire code for my site is published, but I'd need the mods' permission to post the URL here. You can use Google's 'codesearch' to dig out some of the code around my simple example below anyway.

(Woz/bill: just say...)

But there's nothing especially clever about it. I use the standard i18n (internationalisation) mechanisms built into Java (and similar ones exist in other languages).

When generating a page, for example, instead of the first link being hard-wired to say "Home", instead it has code to look up the message called "org.hd.pg2k.Home", and the message catalogue I look that up in depends on the user's 'Accept-Language' header sent by their browser. The Accept-Language browser says which language(s) they'd like to see the Web page in.

In this case, for my home link:

For English the result is: Home

For Chinese (zh) of any flavour the result is: &#x4e3b;&#x9875;

(You can decode those UNICODE characters yourself!)

My system uses JSP (Java Servlet Pages), but even plain Apache has a mechanism to support i18n of individual static pages, so that they appear at the same URL, but selected/customised for the user's language. No redirection needed.

The whole process is called 'Content Negotiation'.

Rgds

Damon

DamonHD

7:20 am on Apr 27, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi,

Sorry: just re-read your post, so I realised that you are aware of 'Accept-Language' (good!).

The thing to do in your ASP code is look up messages (parts of your page text) in a different catalogue depending on the language, rather than redirecting to a different page.

Rgds

Damon

[edited by: DamonHD at 7:21 am (utc) on April 27, 2007]

yogapeace

10:28 pm on Apr 28, 2007 (gmt 0)

10+ Year Member



Thanks Damon,

I will give it a try ...

GrendelKhan TSU

7:02 am on May 1, 2007 (gmt 0)

10+ Year Member



Personally I really don't like sites that redirect me based on my browser settings or IP address. Although a majority of your visitors may never say a thing these sorts of redirection are problematic in many ways. You're just seeing one of the main reasons now; Problems with search engine spiders.
How can you fix it? My suggestion would be to simply get rid of it and to think of an accessible site structure that will lend itself to browsing by humans and robots alike. However, you may want to keep it...and in that case you'd have to tell us how you're accomplishing the redirect.

totally agree. I can't stand sites that do that. pet peeve. MS does it a lot, even for updates. and its impossible (or annoyingly difficult) to get back to the english versions.

with all the international travellers around and ex-pats all around, I would think it would actually more beneficial to give users the choice.

again, that could be just venting again though. ^^

59box

7:24 am on May 3, 2007 (gmt 0)

10+ Year Member



Hi Damon,

I had been thinking to use content-negotiation.

Since the same page appears as different languages, how does the search engine sees it? There is probably a default language, so does the other languages page gets index?

Not everyone has a server in each country so what you mentioned (different default in each country) may not work.

DamonHD

11:12 pm on May 3, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Well, I also have a link on each page to switch to a different language for that page, which is just a query tacked onto the page's URL, eg?lang=zh_CN or?lang=en

This isn't needed by a user with their browser language settings correctly enabled, but makes it possible for badly-set-up browsers and SEs to follow the link to see the alternate language(s).

Rgds

Damon

yogapeace

7:09 am on May 7, 2007 (gmt 0)

10+ Year Member



Based on Damon pointing out that Google doesn't pass a default language, I added a default language in my ASP code, when none is passed by the browser & lo & behod, Google is indexing the page again.

I waited till Google came back to report this. Thanks much Damon!

DamonHD

9:48 am on May 7, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Kewl! I'm gald it works for you!

Of course, that was my secret sauce! B^>

Rgds

Damon