Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Duplicate Content


joyjesters - 9:34 pm on Oct 29, 2011 (gmt 0)


Shoot. I did a search on Google like this for my site...

site:widgets.com/page

And over 1,000 pages came up indexed on Google! Thats so much duplicate content. How do i get rid of it? What's the best way to configure my robots.txt file to block that content?

Alex.


Thread source:: http://www.webmasterworld.com/robots_txt/4381219.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com