homepage Welcome to WebmasterWorld Guest from 54.81.170.186
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robots.txt vs Nofollow.
slasher20




msg:4197815
 4:43 pm on Sep 7, 2010 (gmt 0)

Which is the best to use for large sections of site that have low quality pages?

Will eventually build these pages out but in the meantime want to block them from consuming valuable pagerank and being indexed. Some of these sections of the site have close to 100k pages. Don't want to trip any alarms while doing this either.

 

phranque




msg:4198106
 12:57 am on Sep 8, 2010 (gmt 0)

welcome to WebmasterWorld [webmasterworld.com], slasher20!

in your situation i would generally suggest meta robots noindex.
otherwise the urls may be indexed even though the pages are excluded by robots.txt, in which case the SE may choose something outside of your control to show for a snippet.

slasher20




msg:4199773
 12:53 am on Sep 10, 2010 (gmt 0)

noindex, nofollow or noindex,follow? keeping in mind within the next 6-12 months we'll be building these pages out and will then want them indexed.

is it helpful to also add these to robots.txt?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved