homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Robots.txt vs Nofollow.

Msg#: 4197813 posted 4:43 pm on Sep 7, 2010 (gmt 0)

Which is the best to use for large sections of site that have low quality pages?

Will eventually build these pages out but in the meantime want to block them from consuming valuable pagerank and being indexed. Some of these sections of the site have close to 100k pages. Don't want to trip any alarms while doing this either.



WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 4197813 posted 12:57 am on Sep 8, 2010 (gmt 0)

welcome to WebmasterWorld [webmasterworld.com], slasher20!

in your situation i would generally suggest meta robots noindex.
otherwise the urls may be indexed even though the pages are excluded by robots.txt, in which case the SE may choose something outside of your control to show for a snippet.


Msg#: 4197813 posted 12:53 am on Sep 10, 2010 (gmt 0)

noindex, nofollow or noindex,follow? keeping in mind within the next 6-12 months we'll be building these pages out and will then want them indexed.

is it helpful to also add these to robots.txt?

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved