Welcome to WebmasterWorld Guest from

Forum Moderators: incrediBILL & martinibuster

Message Too Old, No Replies

Targeted or charity for non-indexed pages?



10:36 pm on Aug 21, 2003 (gmt 0)

10+ Year Member

I would like to place AdSense on all the webpages on my site, but I don't want a section of it appearing at all in the SERPS. Will the ads on pages I don't want indexed show targeted or charity ads after Mediabot visits? Once he visits, will the pages in this section automatically be considered for crawling by Googlebot?

I know the two bots are completely different in their roles but would like to know if they share info on the existence of pages. Also, I'd like to know whether the section I don't allow Googlebot in will be penalized by AdSense into showing only charity ads.


10:56 pm on Aug 21, 2003 (gmt 0)

WebmasterWorld Senior Member jenstar is a WebmasterWorld Top Contributor of All Time 10+ Year Member

You can use robots.txt to disallow the regular googlebot and to allow the adsense mediabot.

User-agent: Mediapartners-Google*

would allow the AdSense bot to visit, and begin provided targeted ads.

If you currently exclude all bots from those pages, the AdSense bot WILL obey your robots.txt and not index those pages, resulting in those pages only showing PSA ads.

Word from Google is that using AdSense doesn't affect results in the Google index.


3:46 am on Aug 22, 2003 (gmt 0)

10+ Year Member

Okay, thanks.

Featured Threads

Hot Threads This Week

Hot Threads This Month