| How would I disallow Google...
...and allow all others? |
Rollo
msg:1525242 | 11:44 pm on Nov 27, 2004 (gmt 0) |
I have some duplicate content that I don't want Google (to keep) penalizing me for, and I was wondering how to accomplish the following with a robots.txt: 1) Disallow Google totally. 2) Allow Googlebot into index page (and links pages), but disallow to other sub-pages. Thanks!
|
benevolent001
msg:1525243 | 11:53 pm on Nov 27, 2004 (gmt 0) |
totally User-agent: googlebot
Disallow: / for the second u may use meta tags
|
Rollo
msg:1525244 | 5:35 pm on Nov 28, 2004 (gmt 0) |
Is there a way to disallow subpages with a robots.txt? Changing individual pagess can be a bit tedious. Thanks!
|
|
|
