January 2018 AdSense Earnings & Observations

[ivok]

Happy New Year !
I wish you have high CTR, CPC and RPM throughout the year!

Let's start the discussion about the January earnings :)

[Grapetimes]

How much of a drop in coverage are people seeing that have been affected by the new updates from the brand policy?

[sdksjdksjd]

This policy change doesn't explain why on the page some of the ads spots are filled, but some are not. Obviously the page is crawled, if some ads are on.

Also in the second part of December I repeatedly saw huge amount of errors in the browser console, related to the 3rd party networks. Ads were blank and console was stuffed with Javascript errors. Errors are gone, but still less ads.
It looks to me, that we have two situations here - policy related and the other ...

[Grapetimes]

What percentages in coverage drop have you seen because of this update?

[sdksjdksjd]

My coverage almost did not drop. Was 100%, now 92-93%.
But I see blank ads and huge drop in CPC/CPM.

[Cralamarre]

My one page from yesterday that was suddenly showing no ads, including the Matched Content section at the bottom, is back to showing ads again today, with coverage at 100%. Just as suddenly as the ads disappeared, they've returned. I don't know what's going on.

[azlinda]

My CPC is 0.00 this morning.

[MisterTux]

People... it comes more and more curious...
Since Monday I'm running a simple split test: "300x250 Rectangle vs. Responsive Linkblock"
I use a very simple script:

$random = rand(1,2);
if($random == 1) {
 // 300x250 Rectangle
} else {
 // Responsive Linkblock
}

I use Piwik to track my page. Piwik says that the subpage where the split test takes place has 900 views for today so far.
Adsense-Report says that 300x250 ad has ~400 impressions, the linkblock only 20(!)

So my first idea was that the linkblock has a terrible coverage.... but it's marked with 91%

[NickMNS]

@sdksjdksjd welcome to Webmaster World.

This policy change doesn't explain why on the page some of the ads spots are filled,

Your are correct, the reason is likely to do with 3rd party ads. My understanding, and I am not certain about this but is based on the wording of the infamous email sent to me and many others by AdSense support, the policy is an AdWords not AdSense policy and what is stated is that AdWords will not bid on impressions that have not been previously crawled by the AdSense bot. What this means is that uncrawled ad slots may still be filled by 3rd party ad networks if the demand exists, sometime it does other times not so that ad-slot will go un-filled.

The other impact this has is that the 3rd party ads are winning auctions in which there is very little demand and tremendous supply. The winning bid prices are very low, so as advertisers the even though those ads are filled we will be getting pennies on the dollar for those impressions.

Publishers are being thrown under the bus by AdSense. Now if you are an advertiser, I would recommend going to Criteo or AppNexus or one of the many other 3rd party networks because you will be able to get impressions and clicks almost for free on many high quality websites.

What a mess....


@grapetimes

What percentages in coverage drop have you seen because of this update?

This will depend on what percentage of your impressions are from uncrawled pages, it will vary from site to site depending on your level of exposure. As explained above coverage isn't everything, if you are running 3rd party ads you may be seeing more coverage than expected but earnings will be impacted by low prices from those ads.


Note: Over time as advertisers realize that there deals to be had using the 3rd parties they will move away from AdWords a go to the 3rd parties, so this mess has potential to infect the entire ad market place. The faster that the exodus occurs the sooner that Google will fix the mess. #ShortGOOG

[sdksjdksjd]

@NickMNS
Is this AdWords ad format or 3rd party : [prnt.sc...]

[ember]

But once a page is crawled, which seems to be triggered by a visit, ads will show. So visiting all the pages in your site - I know this might be impossible for very large sites - should trigger ads for subsequent visitors. Right now many sites still have uncrawled pages because this new policy only started 3 weeks ago, but once they are crawled, the problem should be over. At least I would think so.

[Cralamarre]

I know I'm going to feel really dumb for asking this, but could someone explain how a site would have many pages that are only viewed by a small number of people? Are the pages generated dynamically for each visitor? My site is just a small static site so I'm not understanding how a site with many pages that almost no one visits would work.

[NickMNS]

@sdksjdksjd, next time can you please use a better name, than some random key strokes.

There is no way to tell from a screen cap you need to check your stats in the "Ad Networks" report.

@ember
Yes in principal that is correct. I tested it and it works see my posts about that here: [webmasterworld.com...]
The only issue is how long is a page kept in the index?

[sdksjdksjd]

Popularity of the topic became less = less visitors. For example everything related to mobile phones. Same about books and movies. Who would be interested today in content on "Avatar" movie? Almost nobody. Should it be dumped then? No.

[sdksjdksjd]

@NickMNS, excuse me? Take care of your name. It's not somehow better.

You said "uncrawled ad slots". There is no such thing. Bot crawls page for the on page content, not slot. If content is sensitive, then ads will be not shown according to new policy. But ALL ads, including 3rd party. They are shown under the Google brand, so there is no chance they will be treated differently.

If page is half filled with ads, then new policy explains nothing.

[kelsheikh]

but once they are crawled, the problem should be over. At least I would think so.

@ember that is definitely the big question

@NickMNS if our uncrawled urls are not kept in the index for at least a month, then we are all doomed. I would hope that just one visit should request the crawler to come index. For every visitor afterwards, they should be served ads going forward. At least this is how I would think it would work.

Im just constantly wondering if this new crawler is now for the first time indexing on a url by url basis, that it must be extremely backed up and so much slower than I think Google thought it would be.

Lets say you have a site that has 10k in dynamically generated urls. If you get 1k in visits a day to different urls. Shouldn't you start seeing an increase in coverage percentage?

[UpOnOne]

What worries me is how long the Adsense crawler caches a page. If I make a change to a page and the last modified date changes, does Adsense still consider this page crawled? Also, how long does a new page stay in the index? I have a fair number of pages making me money that are visited infrequently. If no one visits them for a month let's say, does that mean ads may never show?

On another note, my coverage is still way down and earnings have now gone below 50%. This is definitely not my usual January slow down. I have a script in my fall back ads that count when Adsense ads do not display. On a normal day, this counter might get to 10 or 20. Since Dec 18th, I am seeing numbers around 10,000.

[sdksjdksjd]

@MisterTux
Interesting experiment. Keep us updated please.

[riccarbi]

My current though is this is something which went out of Google's control, somewhat.
If AdSense needs to always crawl a page and its content in order to provide "compliancy" for brand protection purposes some serious problems arise:

1) How the crawler can determine such compliancy? If i write on a page "this page doesn't contain #*$!ography" does the crawler read the world "#*$!ography" and marks the page unsuitable for AdSense and/or AdWords? (please note that even this forum's automatic control has censored "that word" without understanding the way I was using it...)

2) If I update some info in a page (i.e. a change of date of an event) does AdSense needs to re-crawl that page? And how long and how much computational power are required for it to recrawl millions of such pages a day?

3) What happens with news sites? No Ads on their "breaking news" pages until Adsense has crawled them?

4) Do Google search engine crawler and AdSense crawler coincide? Are they different things? Are they are different things but share information somehow?

[edited by: riccarbi at 4:14 pm (utc) on Jan 10, 2018]

[NickMNS]

@sdksjdksjd
The page is crawled yes, when I said crawled ad slot, I meant ad-slots on the crawled pages.

If page is half filled with ads, then new policy explains nothing.

I explained why you are seeing some blank ad slots and some filled ad slots. I tested it. Simply block third party ad networks you will see all those ads disappear, and your coverage will plummet further. Also if you are using ad-balancer that may cause some AdWords ad slots to remain un-filled.

if our uncrawled urls are not kept in the index for at least a month, then we are all doomed.

I agree, I estimate my exposure to be 25% of all impression being impacted at one month, that falls to 16% at three month. My big question is, are we seeing 0 month now? Is AdSense starting with an empty index and populating on an ongoing basis or was the index populated and then switched on the 15th?

it must be extremely backed up and so much slower than I think Google thought it would be.

The crawl occurs only a few minutes at most after visiting the page. Last night when I tested it, it took about an hour for the index to be updated and ads to show after the crawl. Is this slower than usual? Who knows! One thing is sure, simply refreshing the page will not be sufficient to start showing AdWords ads. So if user lands on a page with no ads then navigates around and returns to the page, there will still be no ads when the user returns. (I probably have move exposure than first thought)

[kelsheikh]

@riccarbi
Not sure about #1.
2) They say reindexing takes 1-2 weeks on their doc at the bottom. Which to me is so slow.
[support.google.com ]
3) Your absolutely right about this. If I was a news site or blogger, I would be so mad that my breaking news articles or posts aren't serving ads immediately. Whats the point anymore!
4) Googlebot and adsense crawler dont coincide so they dont hammer us on bandwidth. In their doc it says:

The two crawlers are separate, but they do share a cache. We do this to avoid both crawlers requesting the same pages, thereby helping publishers conserve their bandwidth.

[support.google.com ]

[MisterTux]

How the crawler can determine such compliancy? If i write on a page "this page doesn't contain #*$!ography" does the crawler read the world "#*$!ography" and marks the page as unsuitable for AdSense and/or AdWords?

That's also my fear. Yesterday I've read a post in a German forum. There's a guy with a medical infopage which also includes sexual diseases. There are no ads shown on these pages since ~3 weeks. In other words: A Pulmonary is allowed to become monetized, an erectile dysfunction isn't.

If I update some info in a page (i.e. a change of date of an event) does AdSense needs to re-crawl the page? And how long and how much computational power are required for it to recrawl millions of such pages a day?

Also a good question. I've the feeling that every little single change on a page needs a "re-crawl". It's crazy!

Let's get a little cynical: The new slogan of Adsense/Google should be: Let's caress the advertisers and kick publishers straight in the face.

[ember]

There is no way to tell from a screen cap you need to check your stats in the "Ad Networks" report.

You can tell by using the Adsense toolbar. If you hover over an ad, it shows the advertiser and network. I've only ever seen Adwords as the network, but I'm assuming pages with low coverage will show other networks now.

[NickMNS]

@kelshiekh and @riccarbi

2) They say reindexing takes 1-2 weeks on their doc at the bottom. Which to me is so slow.

I would not put too much credence on that support post. Pages are crawled each time a user visits a page. You can check you logs to see this occurring.

3) Your absolutely right about this. If I was a news site or blogger, I would be so mad that my breaking news articles or posts aren't serving ads immediately. Whats the point anymore!

Not really, as soon as the page is viewed it is crawled. My test showed that it took about an hour or less to get ads to start appearing but that was for a site with little to no traffic. I assume that if there are many crawl requests in a short time span that the process will be accelerated. So the impact should be be minimal to negligible.

) Googlebot and adsense crawler dont coincide

Googlebot is Google for search-indexing and ranking, and MediaPartners-Google is the bot for Adsense. They are completely unrelated. Having your site index in search is in no way related to AdSense. You can have millions of indexed pages and not a single one will show ads if the AdSense bot hasn't crawled and indexed them as well.

I re-checked my exposure based on unique-page-views I am looking at 20% at 3mo. ttl index or 32% at 1mo. ttl index.

[sdksjdksjd]

@NickMNS
Sure coverage will plummet, when you disable ads ) What does this explain?
According to your explanation, Google doesn't want to fill ad space on some pages (whatever reason) and forces 3rd networks to fill them (this or the other way). How Google does benefit from this?
And again, 3rd networks act under the Google brand. If Google, supposedly, didn't crawl page yet and didn't want to show his native ads, then why would it show 3rd party networks ads, risking to be accused absolutely the same way, since ads brand is the same?

Your explanation makes no sense at all.

[sdksjdksjd]

@ember
I'm asking about the exactly this ad format. I believe this is AdWords only. Can somebody confirm this?

[Cralamarre]

I just updated one of my older articles and changed the Published Date to today, and the ads are still appearing as normal. So it seems that updating content has no effect on ads.

[kelsheikh]

@Cralamarre thanks for the test. Thats at least good to know for urls that have already been crawled. It would be terrible if making a small change resulted in blank ads. So its really the uncrawled urls that we are all waiting to have indexed for that first time.

[NickMNS]

Look it is simple, this has to do with AdWords not AdSense. If the impression comes up for auction and the page where the impression is to appear hasn't been crawled AdWords will not bid on the impression. Any other party involved in the auction can still bid on the impression. Since AdWords accounts for much of the demand the resulting bid price (supply & demand) for that impression is very low. In fact in many situation, as we see, demand is so low without AdWords it ends up being zero and no ad is displayed.

Here is the quote from the AdSense support email:

AdWords and DoubleClick Bid Manager have adopted more restrictive bidding on ad requests coming from URLs that are uncrawled.

"more restrictive bidding" does mean blocking ads, it means not participating in the auction and AdWords is not AdSense. AdSense provides the framework through which AdWords ads are served on your pages but the two are different entities. AdSense also allows "3rd Party" networks to be served and depending on your site and account you could be serving ads from up to 3000 to 6000 different networks. In general demand from these networks is very low so you rarely see any ads. But with the current situation this has changed, now if you go to "Ad Review" and check the various ads by clicking on the preview it will show which network the ad is coming from. The most common network I see after AdWords is Criteo. It is impossible to tell from a screen cap what network served the ad, it is only possible by looking at the ad-code or through your account.

[ember]

And once crawled I think pages will stay in the index for some time. Otherwise, Google is going to lose advertisers to other display networks where their ads will actually show. I think this will all calm down once more pages are crawled.

We got no notice about this because it is an Adwords policy, not an Adsense policy.

[Maximum44]

Today is looking pretty decent for us (for the month of January).

We've had no coverage issues this whole time. I have yet to come across a blank ad (even on new pages). We have auto ads enabled and slider at 90%.