Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Possible to have a page disalowed by robots.txt but indexed by adsense

Possible to do this?

11:12 am on Nov 5, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 17, 2003
votes: 0

I have some pages that I don't want indexed by google search (disalowed in robots.txt) yet, I want adsense on them.

But it is not possible right? As robots txt will prevent adsense? Any workaround?


9:16 pm on Nov 5, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 3, 2003
votes: 0

No problem; just add...

User-agent: Mediapartners-Google* 

...to your robots.txt.

On a side issue, I have absolutely no idea why Mediapartners-Google bothers with robots.txt at all. By using the AdSense code on a page you are by definition giving the bot permission to retrieve that page in order to do the job that YOU want it do. robots.txt shouldn't come into it!