Forum Moderators: phranque

Message Too Old, No Replies

Content Duplication

         

shadowen

11:30 am on Sep 8, 2004 (gmt 0)

10+ Year Member



Hi everyone,

I was hoping someone might be able to clarify / help me with this. Here are some details for the site:

The server being used is iPlanet Web Server Enterprise Edition 6.0
The site is running on Unix Sun box
Programming Language = Java (JSP)

The current site generates URLs like:
[domain.com...]

The site is being setup to offer regional versions of itself to better target specific areas in the country.

URL: [domain.com...]

Now this is all well and good and the regional pages have different top images etc. although use the same template. What the problem is however is that the regional sites link to information that already appears on the national original version, and this appends an additional variable to the URL:

[domain.com...]

Therefore the content at:

[domain.com...]

would be identical just with a different top image.

Could someone please tell me the best way (quick fixes and long term solution) to making sure this doesn't harm the sites rankings in the search engines, and if it is a problem, preventing the search engine spiders from seeing the duplicate content.

Thought about using the robots.txt to exclude the pages, but there would be 1000's of them. Is there a more effective way?

Many thanks in advance.