Correct Way to Create Mobile Pages and Avoid Duplicate Content?

[jpservicez1]

I am trying to setup mobile pages for my main site and am running into google's duplicate content penalty filter it seems.

Initially when I tried to incoporate my mobile pages into my main site. i didn't follow the industry standard approach which after research some believe it to be a sub-folder on your main site like www.example.com/mobile/index.html or believe you just create a sub-domain either mobile.example.com or wap.example.com.

I incurred a duplicate content penalty for doing it wrong and when i switch to using sub-domain, it appears google don't like it my traffic nosedived. I blocked both googlebot and google-mobile bot to the sub-domain and traffic went up again.

So my first question what's the correct way from google's prespective of creating mobile pages.

and do you allow or block googlebot from your mobilepages.

Also maybe because of my initial mistake. My main site pages [non-mobile pages] are now showing in google mobile search engine.

Need help

[edited by: Robert_Charlton at 6:32 am (utc) on June 19, 2008]
[edit reason] changed to example.com - it can never be owned [/edit]

[tedster]

Google recommends that you submit a Mobile Sitemap in your Webmaster Tools account.

Request headers from Google's mobile crawl will always use the HTTP "Accept" header [w3.org] to explicitly tell your site that it should return documents with mobile content types, if available, rather than standard HTML. If your site respects this standard, your site will return mobile content correctly to our mobile crawl.

In some cases, Accept headers are ambiguous. For example, text/html is the content type for both cHTML, which is appropriate for certain types of mobile devices, and HTML, which is generally intended for desktop computers. Google's mobile crawl does its best to appear to be a mobile device, so if your site tries to detect mobile devices in other ways in a case like this, it will probably work for Google's mobile crawl as well.

Google's mobile crawl will send a User-agent header that contains the string "Googlebot-Mobile".

[google.com...]

You do need to allow "Google-Mobile" user agent to spider your mobile pages. In fact, from the information I read here: [google.com...] - you should not block any Google spidering.

It looks like returning mobile content types, plus submitting a specific mobile sitemap, should give Google all the data they need to sort things out. I recommend pouring over the technical information on these Google Mobile pages and lining up with everything that you can.

[eltercerhombre]

After doing my own tests I would like to advise that if you block normal Googlebot from crawling your "pda" subdomain, you won't be able to submit your sitemap Webmasterstools.

To be true I hope it really handles Content-types as it should (I'm being really specific and sending application/vnd.wap.xhtml+xml; charset: UTF-8), because if not I'll end up with a lot of duplicate content.

[jdMorgan]

I've had some problems too, though not this severe.

Do be sure you are returning the correct mobile Content-type header, and that you have a DocType which specifies a valid mobile doctype at the top of your mobile pages.

Google provides quite a few useful hints, spread out across several (perhaps too many) "My mobile site in Google" webmaster help pages.

One thing that is often helpful is to check the HTTP headers and view the HTML/XML/WML source of Google's (and Yahoo's, and others') mobile pages. This can clarify things quite a bit if you're not sure what the help pages are telling you.

Jim