Adsense and WordPress

[browndog]

I switched my site from Joomla (for desktop) and Duda (mobile) to WordPress with the Enfold template. The very day of the switch, my Adsense pageviews and revenue dropped by 2/3rds. I thought it was just a glitch due to the move, but these numbers have stayed that way. Analytics shows a huge jump in traffic the day my site changed, which accurately reflects my desktop and mobile numbers.

I contacted Google to see if they might have a clue, she said mobile traffic is down by 2/3rds (can see this in the report). I said that my Analytics showed traffic was up, and she said that shows all traffic, Adsense only shows valid traffic. Her suggestion, post on social media, add more content and invest in some marketing, which was so unbelievably patronising.

I'm clueless as to why Analytics shows good numbers but Adsense doesn't, and how switching to WordPress can cause my Adsense pageviews and Adsense to drop by 2/3rds.

I've looked at Webmaster tools, fixed a few errors, but nothing which would affect numbers so badly. This is literally going to cost us our house :(

[TorontoBoy]

What "Analytics" are you talking about? Google Analytics, AWStats?

Are you monitoring your raw access log? Any WP site is like a magnet for bots, both bad and good. This is a major issue with Wordpress. But it takes time for bad bots to find you, not the day after you move platforms.

Is it possible that your Joomla site was attracting bots and increasing your page views to the Joomla pages but now they are receiving 404s (File not found)? I may be way off base...

[browndog]

I'm looking at Google Analytics and Awstats. I haven't looked at the raw access log, I don't know what that is...

The thing is, the traffic in Analytics and Awstats is normal, it reflects what my traffic numbers usually are. But Google are telling me my sudden drop in pageviews in Adsense is down to less people visiting the site. We're talking down from 10,000 a day to around 3,000. Exactly the same day the site moved to Wordpress. Even when I was hit with Panda years ago I didn't get a drop like that. I get that traffic changes, but that is a huge drop overnight. And I'm quite shocked she said Analytics traffic isn't actually accurate. So why do they push us to use it then?

[not2easy]

Assuming your old pages were 301 redirected to the new URLs, there should be no sudden drop.

Your raw access logs are the records of everyone (and everything) that accesses your files. AWStats uses your logs to show you their analysis of those logs. But for details when it matters, you can download your logs via ftp if you like and view them line by line, as a text file or as an EXcel sheet. Some of us only use access logs to keep track of actual usage.

[TorontoBoy]

Did you do 301 redirects? For a large site that would be quite time consuming. Check to ensure your ads are rendering and are in the same place as on Joomla. While your aggregate traffic volume could look normal, if a bot is not finding your Joomla page and gets a 404 page not found, that will still count in your traffic as a valid request. Google would see a 404 differently. Something is amiss.

A compare of your access logs before and after your move might give you an indicator of a change in traffic, or might confirm there has been no traffic changes. You have a monthly log for June. Check the day before against the day after the move.

AWStat will not tell you about anonymous bots, only the large search engine bots. It is not grainy enough. Google Analytics will tell you a bit about referrer spam, but also not about anonymous bots.

[browndog]

I set up redirects. We switched from m.dot to responsive and there were issues. But I have spent the past week getting Google to re-crawl, and pretty much all the url's in Google are now correct. However there has been absolutely no increase in Adsense pageviews. It's like they are not counting the mobile traffic at all.

I have now checked every single article on the site, forced a recrawl of those still showing m.dot, re-checked and 99% are fine.

I can understand the drop when switching from m.dot to responsive, but would have expected revenue to go back up once the urls were re-indexed. But nope, it's still the same as the day we switched.

[browndog]

I switched from Joomla to Wordpress and my Adsense dropped to 1/3rd of what it was, which has financially devastated us. I spoke to Google and they said my mobile traffic dropped significantly which would be the result of switching from m.dot to responsive. I waited for Google to re-index pages, after 3 weeks I forced them to re-index and it would seem the bulk have been now.

Despite this, my Adsense still hasn't recovered. It isn't a matter of if, we are going to lose our house over this fiasco, which is absolutely gut wrenching (we have kids and pets). I could go into the emotional toll of that, but I won't.

I keep looking at my webmaster tools for crawl errors. When I search for topics, it all looks fine on both mobile and desktop, but I am seeing lots of this...

mysite.com/?s&no_redirect=true
mysite.com/html?s=

Does anybody know what that means?

[edited by: not2easy at 4:10 am (utc) on Jul 23, 2017]
[edit reason] cleanup [/edit]

[keyplyr]

Are you positive the m.dot traffic is getting redirected to the responsive version? I ask because "no_redirect=true" would suggest otherwise.

[browndog]

I don't know? When I search on my mobile it all looks fine. Looking at my stats I see the following...

Duda (m.dot site) from May 2017: 10,000 visitors a day
Responsive from the past 30 days, since the change: 6,300 a day

But even with that, it doesn't explain a drop by 2/3rds

Obviously I'm not allowed to divulge Adsense stats, but will say this...

I went from 4 to 1 in regards to pageviews. Google (and I verified it by looking at it myself) show it's all from a drop in mobile traffic.

[engine]

Sorry to hear that browndog.

Just a couple of things spring to mind...
How many pages are we talking about: Sometimes it can take months for the change to be reflected.
Yes, check that redirect works correctly - you may be seeing a cached version. Oh, wait, you said it was weeks ago - cancel that.
Do you have GSC or G analytics to view? What does that tell you?

[not2easy]

It would be normal if you no longer have two versions of a site to see lower numbers because visitors are not being sorted. That is, they no longer arrive at pageA and get sent to pageA.m so rather than counting 2 page views for their initial arrival, they are only seeing 1 page.

mysite.com/?s&no_redirect=true
mysite.com/html?s=

Does anybody know what that means?

The /?s is WP search syntax, you should have a line in robots.txt to disallow any such URLs.

Disallow: /?s=

example.com/?s&no_redirect=true on the other hand looks like it is due to some plugin or setting you are using. It is not seen due to WP itself.

To see exactly what a visitor or robot sees, the best way to document the server responses and verify that no (default) 302s are introduced is to check the live headers. Using an old sitemap from your Drupal version, visit those old URLs and see the server responses as you are redirected.

Compare the old sitemaps to the newer WP sitemaps. This ensures you are not trying to index all the WP taxonomy formats which would be duplicates. Personally, I index only pages and posts, without any variations. This avoids submitting duplicate versions such as WP produces by default.

[browndog]

Thank you both for your responses.

This week I pulled up my old site sitemap (which I can still see in a different folder), and with my mobile phone I went through every single url on the site to see if Google showed me the old m.dot site or my new site. I then asked Google via WMT to re-crawl any m.dot sites, which would have been in the range of 300-400 (my site totals around 800 articles). To their credit, they are pretty fast at re-crawling, and while I haven't checked every url again, the ones I have checked (probably around 100) are now pointing to my new/responsive site. But still my Adsense is down by 2/3rds. I would have expected it to slowly increase day by day as more pages are indexed.

I find the Wordpress sitemap really confusing. It's an XML one, I wanted a regular one for my visitors too because I know when I visit a site I like, I will often go to their sitemap to see what else they have. With the XML, the articles look like this.

http://www.example.com.au/sitemap-pt-post-2017-07.html

But the visitor would actually see this url...

http://www.example.com.au/blue-widgets.html

I have noticed with the no_redirect=true, the page bounces.
So I have a genuine article http://www.example.com.au/blue-widgets.html, which works fine, but if the no_redirect=true is on the end it bounces.

I am wondering if I should set up a redirect on them all. See if that sorts it out.

[edited by: not2easy at 9:13 pm (utc) on Jul 13, 2017]
[edit reason] exemplified/readability [/edit]

[not2easy]

I changed the examples to example.com because using example.com prevents linking so we can all read what was posted. (mysite-com might belong to someone)

In WP the sitemap options are open to you depending on what you use to generate them. The URL format is also open to you in your Settings file (see permalinks). It is best to make any needed adjustments early in the life of the site and not try to alter URL syntax for established URLs. I build a sitemap page for WP visitors which you can do from the .xml sitemap, but not in the format yours is in. WP is easy to use, but with the nearly unlimited choices of tools (plugins) it can be hard to find the right tool for your purposes. One idea would be to search through the discussions here, or just visit the Recommended Plugins Thread [webmasterworld.com]

[browndog]

Thanks not2easy.

The url's shown in Google look like this...

www.example.com/blue-widgets.html

It's just when you look at the site map that it looks like this...

www.example.com/sitemap-pt-post-2017-07.html

I'll do some digging, see what I can fix.

I would like to have a site map for my visitors, as I said, I find them useful.

Another thing I've noticed is when I go to categories, it has a summary of each article, which makes the page look a bit cluttered and the list of articles spread over many pages. I would prefer the following...

Category Page (say, Widget Colours)

www.example.com/blue-widgets.html
www.example.com/green-widgets.html
www.example.com/white-widgets.html

Not the first paragraph of each page. Is it possible to do that?

[not2easy]

Your theme probably comes with defaults, but you can change just about any aspect of WP to display as you prefer. Have you looked into sitemap generation that shows the URL structure you have set up? If the sitemaps are not showing your actual URL structure according to your permalink settings, it is time to look into a more competent sitemap generator.

[TorontoBoy]

Another thing I've noticed is when I go to categories, it has a summary of each article, which makes the page look a bit cluttered and the list of articles spread over many pages. I would prefer the following...

Category Page (say, Widget Colours)

www.example.com/blue-widgets.html
www.example.com/green-widgets.html
www.example.com/white-widgets.html

This would be a customization of your WP theme that can be written to your liking, in addition to what comes standard with your theme. Make a child theme of whatever theme you are using and then customize the child theme. Then when your main theme gets updated, which could be often, your child theme will inherit the main theme's changes.

[browndog]

I switched from Joomla/m.dot to a responsive Wordpress site in June. My Adsense plunged, as I semi-expected due to the change from m.dot to responsive. Initially all the mobile traffic was shunted to the homepage, but eventually the m urls changed to the current format.

My traffic is still down, but not by a huge amount, however my Adsense is still rock bottom even though all the mobile urls are working.

This is an example of the stats (obviously not giving proper figures as I don't want to upset Google)

Last 30 days
Pageviews 2 Impressions 9

30 days before that (May/June)
Pageviews 10 Impressions 10

So as you can see, the impressions remain similar, but the pageviews have dropped from 10 to 2. When I questioned that with Adsense saying my traffic remained similar (in Analytics), they said the only TRUE stats are from Adsense and reporting relates to VALID traffic. So all of a sudden, the traffic I am getting is invalid, when it wasn't before? It just doesn't make sense. Awstats and Google Analytics show a slight dip, but nothing drastic. Why would I suddenly receive so much invalid traffic when I didn't in the past? It's like they're not counting the mobile as valid anymore.

[edited by: not2easy at 4:09 am (utc) on Jul 23, 2017]
[edit reason] cleanup [/edit]

[NickMNS]

Pageviews by itself doesn't really count for much either way. You are only being paid for impressions, and given that impression are 90% of what they were it seems like your not too far off. Now the question remains why the change in PVs? My guess would be some technical glitch in your previous implementation, where pages may have reloaded before the ads had a chance to be fully loaded. Thus counting each actual PV for two or three.

How many ad-units do you have on each page? Did this change from one version to the next?

[browndog]

I have three ads per page, that hasn't changed.

Adsense and Analytics have been linked for a few months now, but when I was using Duda, I never got around to adding Analytics tracking code. So my Adsense data in Analytics only related to my desktop site.

I looked at the figures yesterday, and my July income is exactly the same as January to May. It's like I'm only being paid for my desktop traffic. When I spoke to Google about it a few weeks ago, she said they only count valid traffic, so it's as if they are suddenly invalidating the mobile. I'm going to try contact them again, only problem is they keep rejecting my publisher id on their contact form despite me directly copying and pasting from my Adsense page.

[not2easy]

Others have been complaining about communication problems this week with regards to AdSense: [webmasterworld.com...]

If your old "m." URLs are being sent to the same responsive pages as other traffic that would account for having the same earnings with fewer page views. You have eliminated their initial duplicate page views (from desktop to "/m." pages).

[browndog]

I managed to contact them by manually typing in the pub-id instead of copying and pasting.

When I had the m.dot site, if you were on a mobile, they'd see and go to m.example.com/widgets, or desktop would show example.com/widgets. So two sites, two urls. Google have now flushed out the m urls, so they all go to example.com/widgets.

What initially happened was the guy who designed the new site just sent all mobile traffic to the homepage, so Google may not have considered that to be valid, as they weren't actually reaching the page they searched for. I waited for Google to clear the mobile pages, but after 3 weeks many were still showing, so I spent several days on my mobile and laptop looking at all pages, and forcing Google to re-index the ones still showing m.dot. That was 2 weeks ago and within days, the m.dot urls vanished and the correct url showed for mobile, BUT my Adsense still hasn't increased. Analytics shows a huge increase in traffic now the mobile is counted.

When I looked at Adsense via analytics, which until June didn't include my mobile data/income the figures didn't change. This is desktop income only (I made more than those figures from Jan-May as mobile was included with Adsense, just not show in Analytics) until June, when I switched to responsive. Again, not giving actual figures, but the same pattern.

Jan $15 desktop only
Feb $13 desktop only
Mar $13 desktop only
Apr $15 desktop only
May $13 desktop only
June $12 desktop and mobile from 20th
July $12 (projected) desktop and mobile

So since I swapped to responsive, nothing changed?