homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Pages with Partially Syndicated Content
No indexing of page elements

10+ Year Member

Msg#: 4584970 posted 3:51 pm on Jun 17, 2013 (gmt 0)

Looking at incorporating some syndicated User Generated Content (UCG) through some third-party feeds together with unique and well-written content. Obviously want the unique content spidered and indexed, but also don't want to run the risk of Dupe Content penalties for the syndicated content.

The intent here is to provide a better experience for the user with additional content, such as user reviews, Twitter posts, etc., along with the unique product descriptions.

I know several years ago Yahoo proposed a norobots-content, but Google never implemented it.

What are people doing in this situation? I don't want to use plain JS or image content for this as the reasons are legitimate and don't want to be seen as cloaking.

The obivous option would be to just load everything third-party with AJAX, but given that Googlebot does read (execute?) JS is that sufficient to ensure that it isn't dupe content or appears illegitimate?



WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

Msg#: 4584970 posted 4:59 pm on Jun 22, 2013 (gmt 0)

Perhaps you can load the UGC content into a separate page and frame that page within the page you want to show it on. Then you can either noindex framed URL or block it via robots.txt

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved