Mediapartners-Google vs. Googlebot

[akogo]

If the User-agent Mediapartners-Google crawls the AdSense Publisher's site, does that mean the crawled pages will also be included in Google's search engine index if it wasn't already in? Or is this only the function of Googlebot?

[Jenstar]

No, the only function of the Mediapartners-Google bot is to provide targeted ads for AdSense. To be included in the regular Google search engine index, you will need to be indexed by Googlebot.

[4eyes]

the only function of the Mediapartners-Google bot is to provide targeted ads for AdSense.

... or at least thats what Google have lead us to believe.

Its not what I see in practice.

[Jenstar]

I remembered some initial posts about the possibility of a crossover when Google initially launched, but I haven't seen any posts on it since.

None of the sites/pages I am running AdSense on seem to be affected in the regular index. But then, maybe I am not checking or noticing what 4eyes is seeing.

[akogo]

Its not what I see in practice

What do you see?

[GrantNZ]

I changed a domain name to completely content the other day. Google only had the front page in its index and it was showing contextual based on its *previous* content, however, the site's sub-pages - which google did not have indexed - were correctly contextual, so I'm thinking that Google adsense must be using something of Google's index when it needs to, until it's index/adsense is updated?

[4eyes]

What do you see?

I see many pages from my sites appearing in Google after only being visited by Mediapartners-Googlebot

[justageek]

4eyes - what you see is common. Seems that Google does decide whether to show a site or not but I'm not sure why. I've seen client logs that show the googlebot being there almost daily but even trying to find the url in the index doesn't work. Do you run a site that Google could consider a threat, no matter how small, to their core business? That might explain it. If they can make money from your content then I don't see why they wouldn't index you for adsense. So maybe that's it.

[4eyes]

Yes its common enough. I have more than enough evidence to support this - multiple sites and thousands of pages. I have no query in my mind as to what is happening - nor am I complaining about it;)

[shrirch]

Common sense would say that if you've been accepted as an Adsense publisher, you have some quality content.

Then, why would google want to waste bandwidth and spider your sites twice? And by wasting bandwidth, we're talking about the millions of pages which show adsense ads. Just does not make sense to spider twice.

[justageek]

Actually, Google *must* spider the adsense sites often or the advertisements will no longer be relevant. Even if you have a static site Google does not know that so they still *must* spider much more often than ever before.