| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|How would I disallow Google...|
...and allow all others?
I have some duplicate content that I don't want Google (to keep) penalizing me for, and I was wondering how to accomplish the following with a robots.txt:
1) Disallow Google totally.
2) Allow Googlebot into index page (and links pages), but disallow to other sub-pages.
for the second u may use meta tags
Is there a way to disallow subpages with a robots.txt? Changing individual pagess can be a bit tedious.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved