homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Possible to have a page disalowed by robots.txt but indexed by adsense
Possible to do this?

 11:12 am on Nov 5, 2004 (gmt 0)

I have some pages that I don't want indexed by google search (disalowed in robots.txt) yet, I want adsense on them.

But it is not possible right? As robots txt will prevent adsense? Any workaround?




 9:16 pm on Nov 5, 2004 (gmt 0)

No problem; just add...

User-agent: Mediapartners-Google* 

...to your robots.txt.

On a side issue, I have absolutely no idea why Mediapartners-Google bothers with robots.txt at all. By using the AdSense code on a page you are by definition giving the bot permission to retrieve that page in order to do the job that YOU want it do. robots.txt shouldn't come into it!

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved