homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google AdSense
Forum Library, Charter, Moderators: incrediBILL & martinibuster

Google AdSense Forum

Poll: Should Google add urls to its index from the MediaBot crawler?
At first I thought it did, now I think it doesn't

WebmasterWorld Senior Member 10+ Year Member

Msg#: 495 posted 3:42 pm on Aug 28, 2003 (gmt 0)

Any time someone visits a page where you have adsense code on it Google sends a MediaBot crawler to the page to determine relevant ads for that page. I think the bot only gets sent if that page is not already in the cache from a previous MediaBot crawl OR from the normal crawl.

The Poll question is: Do you think Google should use the Mediabot crawls in order to add urls to its index?

The pros:
1. They have the data already.
2. It gives a good idea of pages that are actually viewed.
3. They can selectively choose to add only pages viewed many times by many ip addresses.
4. More pages would make it into the index. They could zero out the relevancy of Mediabot-only pages so results would go to the back of the SERPS and only be useful when no other page is relevant.

The cons:
1. Lots more duplicate content might result.
2. Unfair to those who don't use adsense (who cares? heh heh).
3. Opens Google up to abuse (however, see Item 3 in "Pros" for a possible solution)

My vote:
Add 'em.

Your vote?



WebmasterWorld Senior Member 10+ Year Member

Msg#: 495 posted 4:12 pm on Aug 28, 2003 (gmt 0)

It already does.

I have few thousand pages in the index that still haven't seen the proper Googlebot yet.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google AdSense
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved