Forum Moderators: open
Basically I have 3 sites. One in the US and 2 in the UK, all hosted on different servers. The main 2 sites in the US and UK have the exact same design, but with about a 50% difference in content. I am mainly concerned about the US site b/c it has been established longer so on pages on the UK site where the content is duplicated, I block those pages with robots.txt. These pages are directly linked from the index page, but I am not sure I did the robots.txt correctly so I am pasting a portion below. FYI I did run it through the webmasterworld validator, and it was correct.
Ex.
User-agent: *
Disallow: /convertMainuk.htm
Now in regards to the second UK site, it has been around the longest, and it has the best links in the search engines. Actually it has been around so long that we recently updated the design to make it look like our current designs. It is not a duplicate design however. I did not touch any of the existing content, but I did add some duplicate content from both the US and UK sites. I once again blocked these pages in the robots.txt. I would say the content on this site is 75% different than the the other 2 sites.
All 3 sites are linked together from the Contact Us page on each site where the physical address appears. This is simply to direct someone to our UK site that lives in the UK and vice versa. I really have no other external links on the sites other than some links to a company who we do some reselling for.
As I stated above, I have no desire of trying to use duplicate content to gain extra positioning in the SEs. It would do us no good anyway. I bascially want someone to find us in the US if they are searching for "US widgets" and vice versa. The same way IBM or any other international company addresses people in another country/region with a site specific to that country/region.
Sorry for all the words, but I needed to give some background before I got to my dilemma.
My dilemma in relation to Google: With our US site, I have been struggling to crack the top 20 on one of our main key phrases for many many months. The second UK site, the one that has been around the longest, has been pretty consistent in remaining in the top 20 for this particular key phrase. That is until this last update. The second UK site started out August in the top 20 until about a week ago when it was suddenly knocked back to the 80s. It appears this way in all 3 Google sites so it is not moving (at least until the next update) My US site now has the 31 and 32 position (home page and the key phrase page)
Coincidentally I changed the second UK site a month ago to its current design that has some duplicate content, but as I said I blocked those pages in the robots.txt.
None of the Google PRs have changed. I have a PR 6 for the US and the second UK site (the one that has been around the longest). And I have a PR 5 for the first UK site.
So my basic question; am I doing something wrong in addressing the duplicate content on the 3 sites? I am at a loss to find out what could be the reason behind the drop of my second UK site, and my inability to move up the US site. In relation to the top 30 sites, I generally have more external links than all but 2 of the sites, and I have a greater amount of content than all but one of the sites. And I always make sure to get external links using the key phrase in the link.
I basically think I am doing everything right, but there has to be some small thing that I am missing so any help would be greatly appreciated.
And I took down the new site and put back up the original version of the oldest site. It does not look pretty, but it did rank very well.
I understand how some people might try and take advantage of duplicate pages to get more than one ranking, but instead of penalizing people why doesn't Google just take the page from the site with the highest PR and add that one and drop the other page.
Take IBM or any larege coproration for example. They have many different country sites with some duplicate content. I doubt very seriously that Google penalizes them.
