Welcome to WebmasterWorld Guest from 50.16.24.12

Forum Moderators: goodroi

how to let robot crawl the links at a page but not index this page?

index, robot

   
12:13 am on Jan 26, 2004 (gmt 0)

10+ Year Member



But I want google to crawl the links at this page. Only don't want google to index or archive this page.

For exapmle, this page is siteA.com/pageA.html and it include the links pageA1.html and pageA2.html, will these two pages pageA1.html and pageA2.html be crawled and archive, indexed by google?

(there will not put any <meta name=robots value=noindex> in these two pages pageA1.html and pageA2.html)

Thanks,

11:39 pm on Jan 26, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



"noindex,follow" should be what you're looking for
[robotstxt.org...]
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved