homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Google / Google AdSense
Forum Library, Charter, Moderators: incrediBILL & jatar k & martinibuster

Google AdSense Forum

How often does MediaBot crawl?
New pages have ads targeted for he home page

 12:50 pm on Aug 3, 2003 (gmt 0)

Hello all,

I have added several new pages to my site, each with unique content (plenty enough text and keywords to entice proper targeting by Adsense). The adwords being diaplayed are targeted for the home page. When should I expect MediaBot to come along and spider the new pages and start serving relevant ads? Some of these new pages have been up for about a week already, still without targeting. No PSAs though, just the wrong targeting...



 12:56 am on Aug 4, 2003 (gmt 0)

OK, I'll take a stab at this one. The AdSense FAQs say that mediabot will come within "a few hours or more" after you put the AdSense ads on the page - ie, for pages that google doesn't have in its index. I think some folks have been saying that it's been taking longer lately.

If google already has the page in its index and you make changes to the page to try to improve the targeting, I've heard that it may take 2 or three weeks before the mediabot comes back to check the content. I think this last bit came from someone in this forum who emailed AdSense with the question.

Could someone confirm this last paragraph if it's true - or let us know if I've got it wrong.



 1:03 am on Aug 4, 2003 (gmt 0)

Thanks for the info Beth, but it appears that the mediabot hasn't crawled my new pages at all. Google just put ads on the new pages that are optimized for the home page.


 2:05 am on Aug 4, 2003 (gmt 0)

I've been having similar problems, in that AdSense has been placing ads on six new pages that pertain to that section of the site, but have nothing at all to do with those pages.

I wrote them and got back, "Thank you for bringing this issue to our attention. We are currently working as quickly as we can to address this problem. As
soon as we have more information for you, we will email you again."

I'm not sure if it's an email written to me... seems kind of form-letter like, so maybe it's just a standard reply. Who knows?!

In my message to them, I did ask if there was a way to force the page to be re-spidered, but that didn't get answered.

I suppose one could just put the page up again at a diff URL and that would make the bot come round again, but it seems that it's off on all of the new pages, so I'm not sure if that would work right now (it has in the past).


 2:09 am on Aug 4, 2003 (gmt 0)

I have AdSense on my message boards, and I can watch the AdSense bot as it travels and gets specific threads. It is usually there within a minute of a new thread, and showing ads (non-PSA ads) a few minutes after that.


 4:39 am on Aug 4, 2003 (gmt 0)

i have adsense on my forums and it used to be quick as hell...

however, it has now not been updated for like.. 4-5 days.. my CTR is down to ridiculously low levels... to the point of almost zero... i'm mildly annoyed since I've got literally thousands of PSA impressions and almost no real ad impressions lately... but I am trying to be patient, heh...

I think the last time my site got new targeted ads was something like July 30th or so... maybe even the 29th


 4:54 am on Aug 4, 2003 (gmt 0)

Hmmm... this is interesting. I just checked - I've had the AdSense bot all over my site today (four different ones right as I type this), but none of the new threads from today are showing targeted ads, just the PSA ones. I was consistent with checking for this up until a couple of days ago, as I was making some changes to my message boards and wanted to make sure it wasn't affecting AdSense.

Others have been reporting much lower CTRs since the 1st of the month.

Visit Thailand

 5:02 am on Aug 4, 2003 (gmt 0)

I emailed Google over the weekend about this as all our new pages since around midday 30 July (Thailand time) have not had targetted ads where they should.

Still awaiting a reply but by the looks of it it is a problem overall.


 9:04 am on Aug 4, 2003 (gmt 0)

I emailed Google over the weekend about this as all our new pages since around midday 30 July (Thailand time) have not had targetted ads where they should.

please let us know if you will get something from Google about this very nasty problem!

Visit Thailand

 9:07 am on Aug 4, 2003 (gmt 0)

Yes it is a nasty problem as I feel like I am advertising PSA's! Have not heard yet from Google but hope to do so by tomorrow Thailand time.

Would love the ability to choose which PSA's we can run, that way we could have at least PSA's we support or which are more localised.


 9:35 am on Aug 4, 2003 (gmt 0)

I am advertising PSA's! Have not heard yet from Google but hope to do so by tomorrow Thailand time.

it is really getting annoying - after so many hours (close to 12) still I have no targeted AdSense ads on my totaly new web site with new domain!

GOOGLE, please do something! I am a very sad Polish guy...

Visit Thailand

 9:40 am on Aug 4, 2003 (gmt 0)

Polish Guy you need to be patient as I said i can confirm that for our site it has been since July 30 around midday or so. I am sure Google are working on the problem if there (which I think there is) is one.

Be patient and if you cannot take it remove the code until it is fixed.

Visit Thailand

 1:38 am on Aug 5, 2003 (gmt 0)

Received the reply as I expected I would this morning. Basically Google is aware of the problem and is working as fast as they can to resolve it.


 3:15 am on Aug 5, 2003 (gmt 0)

I got a less encouraging reply myself...

It said basically that they're unable to control how often the crawlers come by, but it is usually done within 30 minutes...

problem is, my pages haven't gotten targeted content in 5 days now...

Visit Thailand

 3:34 am on Aug 5, 2003 (gmt 0)

twiggy so when did targetting stop for you? Is this on new pages or exisitng pages?


 3:57 am on Aug 5, 2003 (gmt 0)

well, as I said I have it in my forums.. so it's new pages.. they stopped on July 30th I believe...


 7:23 am on Aug 5, 2003 (gmt 0)

Try the tip on this page...
[webmasterworld.com ]

I was having the same problem stated on this thread -- new pages showing ads not specifically related to the page, but with the addition of "?a=1" to the end of the links leading to the page, I've now got targeted ads on these pages.

Visit Thailand

 7:36 am on Aug 5, 2003 (gmt 0)

Nice idea ahsanshami but I think this is a different problem.

For example you were not having a problem with PSA's showing but just that the ads were not as targetted as you wanted.

This problem for me at least only affects pages that were added after July 30 around midday Thailand time. These new pages are only showing PSA's whereas the hundreds or thousands of other 'AdSense' pages which had the code added prior to midday 30 July are all showing targetted ads.

As someone else in this thread confirmed as well that s/he saw the same problem as this also on the same day would indicate a larger problem.


 8:44 am on Aug 5, 2003 (gmt 0)

Well, what was happening was this... the first 10 odd articles in the section related to two unrelated subjects and when Mediabot first came around (end june), it must have decided that all of the pages in that section (directory) related to just these two topics.

As the section's subject matter expanded to cover six topics, AdSense continued offering ads targeted to the individual pages.

However, in the last 5-7 days, all newly created pages started to carry ads relating to just the initial two topics, often having nothing to do with the topic on the page.

I think the difference (as opposed to Polish Guy's new site) is that our site had been spidered initially, so when new pages aren't looked at by mediabot, the ads just default to what AdSense believes the section (or site) to be about. Therefore, we don't get PSA ads, but completely mistargeted ads.

The problem, I believe, is still the same -- mediabot isn't looking at (some) pages with the same speed as we all had seen initially.

As to Visit Thailand's issue on getting PSAs on new pages at a site that's been serving paid ads prior to Jun 30, I'm not really sure I have an idea about what's going on!

I did write to Google a few days back (pg1 post of this thread) and got a standard, "we're aware and are trying to fix" response.


 12:01 pm on Aug 5, 2003 (gmt 0)

This is indeed very annoying! I run a news site, where freshnes is the key. New pages get the most traffic, and since August 2 no new pages has been displaying anything but charity ads.

I usually add between 3 and 5 new articles to my news site per day. But I was away between July 27 and August 2 and since I got back not a single new article has been targeted by Google. However, I have the feeling that older pages gets new, targeted ads. Also, index pages where new pages are listed gets ads targeted for the new articles headlines and caption text.

I have also seen a dip in the Mediabot visits. Especially around July 31/August 1, but it still comes round.


 1:21 pm on Aug 5, 2003 (gmt 0)

It seems to be fixed!

I just published a new article on my site, and it only took a couple of minutes before the charity ads where gone. Older articles (published after August 1) also shows something that seems more targeted than charity (if not perfect matches).

Now, all I hope is that this will start showing in the CTR rates and click results as well. They have been miserable for the last week.

Visit Thailand

 2:15 pm on Aug 5, 2003 (gmt 0)

That is great!

I also can confirm that all our newly added pages that were uploaded after around July 30 midday now all have targetted ads.

The pages uploaded prior to then were showing targetted ads anyway.

I hope it stays this way! I will sleep well tonight.


 2:30 pm on Aug 5, 2003 (gmt 0)

I also confirm :) All my new pages from the past few days are now targeted ads.


 2:33 pm on Aug 5, 2003 (gmt 0)

me too.. looks like they finally fixed it.. yay!

too bad about all the lost revenue, but.. oh well... for me it's not a lot of money each day anyhow...

Visit Thailand

 7:14 am on Aug 13, 2003 (gmt 0)

Is anyone seeing this problem again.

New pages added in the last four hours only show PSA's even though in past days they have been targetted within minutes.


 7:34 am on Aug 13, 2003 (gmt 0)

Seeing this here, too :( It seems to be back for the last 12 or so hours. I just saw the AdSense MediaBot on my site though, but only one. Usually when I am running around checking to see if ads are showing targeted or PSA ads, at least two or three will immediately show up. And this bot appeared to only grab one URL (perhaps two) out of all the new ones showing PSAs.

Visit Thailand

 7:37 am on Aug 13, 2003 (gmt 0)

Thanks Jenstar - Well if I am not the only one it may be the same problem as last time, let's hope it does not take as long for them to fix it this time.

Visit Thailand

 9:10 am on Aug 13, 2003 (gmt 0)

Jenstar - to let you know targetted ads have now just appeared for the new pages we added today. You seeing the same?


 9:19 am on Aug 13, 2003 (gmt 0)

Yes, they are now targeted on my pages as well.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google AdSense
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved