Forum Moderators: phranque

Message Too Old, No Replies

Exlcuding googlebot, allowing AdSense

How is it done?

         

hdpt00

6:31 pm on May 28, 2005 (gmt 0)



For one of my large content sites I am using a ew public domain articles (about 5% of total site content) and want to allow AdSense on the site, but not get punished for having duplicate content. What is the easiest way to do this? Should I use meta tags or put it in robots.txt or what?

Lets say the page name is www.example.com/cat/subcat/article.html how would I do it for that page?

Thanks for the help!

Span

10:09 pm on May 28, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member




User-agent: Googlebot
Disallow: /cat/subcat/article.html

..in your robots.txt will tell only Googlebot not to fetch the article.html page.
The Mediapartners-Google adsense bot is not affected.

hdpt00

10:29 pm on May 28, 2005 (gmt 0)



Thanks!

hdpt00

10:48 pm on May 28, 2005 (gmt 0)



How come on all the robots.txt generator pages they don't say you can just stop certain pages, but only directories?