Forum Moderators: goodroi
I am trying to allow google's adsense bot to spider my site so it can target ads to my content. However, I also have a couple pages I do not want indexed. Here is my robots.txt. Could someone please check to make sure I have it right and the adsense bot will spider my site properly and I have blocked the normal googlebot from the page? Thanks!
User-agent: Mediapartners-Google*
Disallow:
User-agent: Googlebot*
Disallow: /resources.html
You can check if your robots.txt file is valid here [searchengineworld.com]