Welcome to WebmasterWorld Guest from

Forum Moderators: incrediBILL & martinibuster

Message Too Old, No Replies

Mediapartners-Google Bot and Duplicate Pages

Does Mediapartners-Google/2.1 bot cause duplicate page in Google?

1:59 pm on Aug 30, 2006 (gmt 0)

New User

10+ Year Member

joined:May 27, 2006
votes: 0

My site is mostly dynamic pages. I've tried very hard to ensure that Google bot can only access the most critical dynamic pages via one URL. However, because I run Adsense on my site, I allow Mediapartners-Google/2.1 full access to all pages to make sure targeted ads show on all pages when visitors access ones that are denied to Google bot.

The reason for this separation is because the basic data available can be viewed in a number of helpful reports and views that users find valuable. But these multiple views cause duplicate problems for my site with Google. I do not want to only show ads on the pages accesses by Google bot as my analysis of visitor activity shows that most of my ad clicks come from these various views.

My logs show that the bots are accessing the pages as I have indicated (Google bot only accesses limited pages via a single URL and Mediapartners-Google/2.1 accesses all pages on my site). However, I have massive supplemental pages because Google bot has only been restricted for a couple of months, since I've learned of the duplicate page impact on my site.

My question is,
1. Is allowing Mediapartners-Google/2.1 full access to my site exacerbating the duplicate page issue on my site? And

2. If it is does anyone have a suggestion on how to control the duplicate page issue while allowing Mediapartners-Google/2.1 access to provide targeted ads?

Thanks in advance for you help.

2:38 pm on Aug 30, 2006 (gmt 0)

Senior Member

joined:Nov 29, 2003
votes: 0

The two bots share cache, so I suppose it's possible. Interesting question though.
3:33 pm on Aug 30, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 21, 2005
votes: 0

Yes, scince few months googlebot and mediapartners rely on each other for spidering...

Anyway, read here:


for a robots.txt based solution

6:13 pm on Aug 30, 2006 (gmt 0)

New User

10+ Year Member

joined:May 27, 2006
votes: 0

Thanks Frox,

I had read that before posting but wasn't sure if they were including the cross-over information between the Metapartners-Google Bot and the normal Google Bot. But reading it again I believe that is what they are trying to convey.


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members