Welcome to WebmasterWorld Guest from 54.166.102.61

Forum Moderators: goodroi

How would I disallow Google...

...and allow all others?

   
11:44 pm on Nov 27, 2004 (gmt 0)

10+ Year Member



I have some duplicate content that I don't want Google (to keep) penalizing me for, and I was wondering how to accomplish the following with a robots.txt:

1) Disallow Google totally.

2) Allow Googlebot into index page (and links pages), but disallow to other sub-pages.

Thanks!

11:53 pm on Nov 27, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



totally

User-agent: googlebot
Disallow: /

for the second u may use meta tags

5:35 pm on Nov 28, 2004 (gmt 0)

10+ Year Member



Is there a way to disallow subpages with a robots.txt? Changing individual pagess can be a bit tedious.

Thanks!

 

Featured Threads

Hot Threads This Week

Hot Threads This Month