| How to exclude from crawling subdomain
exclude the crawling subdomain |
fendy
msg:3248506 | 3:02 pm on Feb 10, 2007 (gmt 0) |
I have a problem. I want to prevent search bots to index all the pages from subdomain.example.com because that is not a unique content and I am afraid that it will harm our rankings. The text of those pages is something many sites have it on their sites so it is definately a dupliacte content. Can I used the robots.txt file and how to do it please thank you [edited by: tedster at 3:21 pm (utc) on Feb. 10, 2007]
[edit reason] use example.com, remove specifics [/edit]
|
|
