Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: goodroi
I am trying to allow google's adsense bot to spider my site so it can target ads to my content. However, I also have a couple pages I do not want indexed. Here is my robots.txt. Could someone please check to make sure I have it right and the adsense bot will spider my site properly and I have blocked the normal googlebot from the page? Thanks!
You can check if your robots.txt file is valid here [searchengineworld.com]