Welcome to WebmasterWorld Guest from 18.104.22.168 , register , free tools , login , search , subscribe , help , library , announcements , recent posts , open posts Subscribe and Support WebmasterWorld
Possible to have a page disalowed by robots.txt but indexed by adsense Possible to do this? brakkar msg:1528916 11:12 am on Nov 5, 2004 (gmt 0) Hi, I have some pages that I don't want indexed by google search (disalowed in robots.txt) yet, I want adsense on them.
But it is not possible right? As robots txt will prevent adsense? Any workaround?
dmorison msg:1528917 9:16 pm on Nov 5, 2004 (gmt 0)
No problem; just add... User-agent: Mediapartners-Google* Disallow:
...to your robots.txt.
On a side issue, I have absolutely no idea why Mediapartners-Google bothers with robots.txt at all. By using the AdSense code on a page you are by definition giving the bot permission to retrieve that page in order to do the job that YOU want it do. robots.txt shouldn't come into it!