mack - 3:59 pm on Oct 15, 2009 (gmt 0)

I think the duplicate content issue here is very real. The safest option would to build each site from a clean slate so that each site is very different, although serves the same purpose.

This approach is the only real option when you develop country level sites that are based on a different language.

Another potential issue with multiple sites is ensuring the relevant site ranks in front of its intended users. For example you build the .com intended for mainly US users, and the .co.uk for UK users. What if the .co.uk outranks the .com on Google?

One way around this is to base all regional sites within sub domains of folders of the main domain. Lets say you have the main .com site. When a user arrives it gets details of the user based on their IP address and forwards them to their related country page. For example if I am from the UK and see your site in the serps. When I click through I am forwarded to example.com/uk or uk.example.com

This is quite a common approach, but does have its downsides, It could be seen as cloaking, but this is a perfect example of cloaking being used for the right reasons. I would tend not to use xy.example.com because this may be seen as a different site and be crawled in its own right, this again would cause a duplicate content issue.

As for your existing domains set them up as 301 redirects to their locations, example.co.uk redirected to example.com/uk and so on.

The question now is what do we allow the bots to crawl. My suggestion would be to allow everything on the main .com site to be spidered but block the country level folders in robots.txt except the non English folders. If you have good anchor text inks pointing to the regional folders the index page will still show up in the serps even although the page wont be crawled. It will simply be shown as a title with no text snippet. “View site name version for the UK”

This is just my suggestion, I'm sure others will have very different ideas. There is no simple approach to this issue.

Mack.