Msg#: 3551062 posted 8:25 pm on Jan 17, 2008 (gmt 0)
I'm worried that my recordset navigation may be causing duplicate content. My "Widget" site displays widgets 20 at a time so at the bottom of the page I have the 1 etc. page navigation. Once you navigate to another page the '1' link becomes offset=0 resulting in duplicate content.
http://www.example.com/widget.asp?color=green&offset=0 AND http://www.example.com/widget.asp?color=green have the same content
Msg#: 3551062 posted 9:33 pm on Jan 18, 2008 (gmt 0)
You can put that disallow statement in your robots.txt file, but the links would still exist on your site.
There are two negative effects from that: 1. They waste PageRank flow within your site; PageRank that could have been channeled elewhere. 2. They URLs are still visible to visitors who may well cut and paste those URLs in to the page content on other sites, thereby creating links to your site that are useless to you for PageRank.