Anatomy of a HTML to CMS Upgrade; How Google is responding to the new site...

[samwest]

Original site started in 2000, did awesome for 10 years until May Day 2010, since then it's been a slow, bumpy ride down hill. I understand many of the reasons, mobile devices, the proliferation of freebie versions of my material on Ads sites, the "evil" G wanting everyone's traffic, the economy etc, etc.

I've been working on a blog for a few years. put it in a /m directory.
Google hardly noticed it there. Always wanted to upgrade the site to CMS. The html with the blog never indexed. They just didn't like to see the two side by side.

About three weeks ago, I scrapped the 70 pages of old html and moved all those pages to Wordpress. Moved everything to the root and have a permalink program catching all possible 301s. Cleaned up all broken links. Three weeks later and G has indexed about 350 of 3037 pages and posts and 468 of 675 images. Each day they index a few more.
Now it's all under one roof.

Site:mydomain.com returns 2940 results...all appear to be valid URL's

I also ran a rank check on about 100 of my top key phrases. 95% rank on page 1, most in the 1, 2 or 3 position.

So, here I am, looking a Google Anlytics Real Time Live display and there's a big goose egg on the display. ZERO!

Every once in a while I get a visit...maybe two, but never more. I'm on a dedicated server with plenty of horsepower. Site loads quickly and gets a A/B rating on GTmetrics.

If I type my top key phrases into G, I get my site, so where are all the visitors? Am I in some kind of strange sand box?

[samwest]

Seems like a paradox to be indexed, appearing on page 1 and running an adwords campaign yet getting such a trickle of traffic...I welcome any observations. TIA!

BTW - title for this topic should read "Autopsy" instead of anatomy.

[not2easy]

Have you tried to visit some of your old URLs and checked headers for the redirects?

Is your AW QS in line with the rankings report?

_{<stupid, basic question>}Have you double checked the GA changes to new CMS? Is GA tracking known visits?_</sbq>

[samwest]

Google has picked up the changes, even my site links are correct. All other stragglers are being handled by an auto 301 utility. Tracking has been verified and I'm running additional monitoring to track page traffic. Seeing minor bumps but mostly 1 or 2 visitors at a time is it.

[JD_Toims]

Interesting -- I've seen something similar recently myself and don't have any answers yet, but it's good to know there are others seeing the same type of thing, so thanks for sharing.

[aakk9999]

A few ideas to throw in:

- Have you changed your titles and meta descriptions?

- Is Google rewriting page titles and meta descriptions (and previously might have not done so)? Basically, see if your CTR is suffering since the changes

- Have you verified your rankings on the page 1 in WMT? What you are seeing may not be what others are seeing. What do your impressions/clicks say there?

- When did you do your site changes? If less than 3 months ago, then you could compare before/after impression and clicks data in WMT and maybe you learn something there.

[rish3]

Is Google rewriting page titles and meta descriptions (and previously might have not done so)? Basically, see if your CTR is suffering since the changes

That's interesting. Is there consensus that Google only rewrites page titles and meta descriptions when they view the original as needing improvement (from a CTR standpoint) ?

[trabis]

I've noticed on another post that you had another website forwarding requests to your site/server. What is the current situation on that? Perhaps you have been hacked? Perhaps your server/.htaccess is now redirecting to that other site under certain circumstances?

[Shepherd]

Is this an ecom site?

here I am, looking a Google Anlytics Real Time Live display and there's a big goose egg on the display. ZERO!

When you migrated the site did the analytics code break?

[JD_Toims]

That's interesting. Is there consensus that Google only rewrites page titles and meta descriptions when they view the original as needing improvement (from a CTR standpoint) ?

Good question -- I'm not sure there's a consensus, but I can tell you I've personally designed titles to be what I *thought* Google would change, but the titles stood out in the SERPs as they were and, interestingly to me, did *not* get changed algorithmically except in a few instances even though I purposely designed them to "likely be edited" by the algo, so my guess is Google's reasonably/fairly good at only changing titles when it helps the CTR.

[samwest]

GA is installed properly, site switch was about 2 weeks ago. New site converts well when it gets traffic, but G's not passing much.It's now 9pm and GA's been at 2 or 3 all day, no more, it's as if they are walking visitors in single file.

Here's a possible big problem. WP was set up with authorship on. It quickly accepted my authorship, but since I use a pseudonym in my writing projects and don't necessarily want my real name on every public listing for all the hackers to attack, I changed the last name on my G+ account to my pseudonym. I'm wondering if they are now considering the previously indexed content with the other name a duplicate, even though that name no longer exists on their system.

[JD_Toims]

GA is installed properly, site switch was about 2 weeks ago.

Personally, I'd wait a bit before I changed/reverted anything -- Google has decided to "take some time" to fully trust/apply algorithmic rankings through redirects, including 301's. Last I heard/remember, it can take up to 3 weeks for Google to fully transfer all "ranking signals" through a redirected URL, so I'd wait another week or two and see what happens.

[aakk9999]

I agree with JD.

Two weeks is far too little time to make any conclusions when you have this kind of site re-launch. You have changed URLs. This means that Google has crawled new URLs, but probably still has old ones in index too. Until it recrawls the old URL, it will not see the redirect. I presume there has been some change in site structure too.

The site trust will temporary suffer with the influx of many "new" URLs with what is seen as a duplicate content (until Google processes 301).

You haven't said what is happening with your impressions and CTR for the last 2 weeks comparing with the same length of period before the re-launch. I would also compare the number of queries before/after, to see if you have perhaps lost some long tail.

I would say that what you are experiencing is expected as I would expect ranking to suffer temporarily with such a move. But what confuses me is that you said you are ranking well, but no clicks. This is why I would focus on WMT Queries section to see if this can tell me anything.

[samwest]

@JD & aakk - thanks for the heads up on that! I would agree and it looks like the "sand box" is indeed where I am right now. Listings are all there, but traffic is nowhere. Strange. re-index continues, 351 of 3,345 indexed so far.

@aakk - I did take a big hit on queries. It seems to have found the bottom and popped back up a little, but now it's just going sideways.

There's no going back. I plan to just sit tight and watch what happens. I'll keep you posted as to my progress. Cheers!

[samwest]

Really strange day today.
I used Chrome incognito mode to manually verify dozens of key phrases that are still showing on page 1. Cleared local cache and cookies, flushed DNS and still my listings show prominently across my niche as if nothing had ever changed (although their URL's are now pointing to my new site). As I mentioned before, I'm also running a $20/day adwords campaign.

So...what's the result? I'm sitting on GA Live right now and there are absolutely ZERO visitors. (and before you ask, yes, GA code is properly installed and stats are backup up by PIWIK and server logs)

Anyone who can answer this with a solid explanation get's a cookie out of petty cash.

[netmeg]

Run your phrases through a tool like SEMRush (paid but you can do a small number of searches for free) and see if they show the same placements you see (even in incognito mode)

[samwest]

@netmeg - I did a quick check on SEMrush and find 20 of my top keywords in the number 1 position. It shows 108 key phrases for page 1. I'm really "nichey" so those 100 covers a lot of ground.

Thanks for the tip, I never knew about that site. Still doesn't explain why the extremely choked off traffic. I've been at this for 14 years, so I know my traffic pretty well. This just feels like a sandbox, which is odd, I thought domain age and competitive keywords forced to sandbox state. I'm old and my niche is not that competitive....except for one of G's ad network pal sites.


BTW - SEMrush also shows 484 organic listings. All time high was back in 2010 when it was 2,159. This year started at 839 and has dropped to half that.

[samwest]

also for the past few weeks I'm seeing the same window open up during the day. 11pm to 6am seems to produce a few conversions. Daytime conversion is zero, simply because there is no traffic to convert. It's as if I'm totally shut down during daytime hours. An entire dedicated server with nothing to do anymore.

[netmeg]

Well if you're still ranking for normal keywords, it's hard to pin lack of traffic on Google. What about Bing and Yahoo, or any other channels? Is traffic normal there? Are there any external factors (seasonality, something in the news) that could be affecting your traffic? When you were on SEMRush, did you happen to notice the timeline for the expected searches on your main keywords (to see if the trend has fallen) ?

[samwest]

Thanks Netmeg!

Not pinning it on G or B or Y....but the occams razor thing always kinda points in those directions. Traffic should be on the upswing right now. It was, until I did the migration to a CMS. I have 10 years of data, so I pretty much know what to expect in coming months. If I'm way off target and there is no correction, I'll know something is very wrong.

As additional data to chew on, I've also run WP through all the speed tests. I have all the common, recommended plugins doing their thing. Even tweaking javascript to make G's pagespeed happy. Broken links eliminated, 404s redirected. It all looks good, but damn if I'm not missing something.

Can I believe what GA reports? It's nearly identical to what PIWIK is reporting. Could caching somehow be throwing my data off? That's a possibility.

Either way, I'm still making enough on adsense to buy a Subway footlong every day (I got that goin' for me, which is nice) but I'm not quite sure how that's happening with this little trickle of traffic. PIWIK shows a lot of "one and done" hits. bounce ranges from low 40's to a high of 55.

Luckily I'm still getting a conversion every 6 to 12 hours on this light volume, so the new responsive site is def converting well.

Last Sunday the lights came on and traffic flowed like normal. One visit a minute is all it takes. Conversions on the hour or better. at 6pm Sunday it shut right down. Sunday nights are typically the busiest of the week.

Oh, one more thing that I prob mentioned... The former site was 70 pages. That's all G ever had indexed. The new site is 3350 pages of which 368 have been indexed so far (about 2 weeks). Should be more traffic, but there's less.

Patience...

Keep the great tips coming and thanks again!

[samwest]

MASSIVE re-crawl last night. 8,600 pages and 111Mb for the day. Biggest spike I've ever seen. Also seeing some very nice positive moves for key terms. One moved from the bottom of page 1 to the #3 position over nite. Things are looking up!

[edited by: samwest at 1:12 pm (utc) on Aug 22, 2014]

[simonlondon]

From what I read here, the key thing is the fact that you have rankings, but not traffic according to Google. You have provided sufficient evidence to prove that 1, the search volume can't have just vanished; 2, your rankings are real and 3, your traffic monitoring tools are working fine.

Sorry I have nothing valuable to add here, but the discussions seems to have all focused on the wrong side of things.

I suppose if you don't mind and if aakk9999 allows, maybe you can publish your site URL here and we check it for you.

[netmeg]

Site reviews are actually in the contributors section.

[simonlondon]

Apologies. I didn't mean to break the rules.

[Planet13]

I'm a little confused...

"Three weeks later and G has indexed about 350 of 3037 pages and posts and 468 of 675 images. Each day they index a few more. "

As of today, is google showing the NEW URLS for all the pages that moved? Or is it still showing the old URLs + 301 redirect in the index?

If it is still indexing some OLD URLs, then use an old browser like IE8, go to google, do a search that would bring up an old URL (that has been 301 redirected), then click that result and see what happens.

My first-hand experience has been that even if every http headers tool - including fetch as googlebot - tells you that your 301 redirect is correct, it could still be malformed.

And believe me; I am an EXPERT in malformed redirects...

[samwest]

@Planet13 - OMG - the biggest problem I am having is redirects.
I feel like the dutch boy with all his fingers (and toes) in the dike.

If I enter mydomain.com in search, it pulls up a nice listing with site links. All the sitelinks (except one) contain the new URLs...

I am using plugins to enter manual redirects, but I think I'm going to use the cPanel redirect tool instead. I just don't trust these plugins.

I was using a "permalink finder" plugin, but after a few days it starts making a mess of things. It has even messed up admin links.

Looking for best approach to doing redirects. TIA!

[samwest]

OK, question for any WP expert - why don't server based redirects work (those entered in cpanel and written to .htaccess) when they are clearly there? I believe 99% of my current traffic problems are related to redirection.

[netmeg]

I have had a couple traumatic experiences with redirect plugins - dunno if it was the plugin or me, but as a result I have completely stopped using them, and won't do it again.

Me, I just add them directly to the .htaccess file, either via the edit files function in WordPress SEO, or just ftp'd to the server. I don't know how the CPanel stuff works, but it should be pretty straightforward.

Make sure you are testing your redirects - get a browser extension like HTTP Live Headers which will tell you whether or not your 301s are working properly.

[samwest]

once again, thanks for a great tip netmeg! I'm always wary of those plugins.

[aakk9999]

Have you tried browsing your site using proxy and watching it on Google Analytics Real Time? Seeing whether you see your own hits? Visiting various pages and see if this reflects in Real Time?

As netmeg pointed out, you should check your redirects are technically correct. There are numerous tools you can use for this. This site has some in Free tools [freetools.webmasterworld.com] section or you can use site crawlers, some of which are listed in our 2013 Favourite SEO Tools [webmasterworld.com] and Favourite SEO Tools [webmasterworld.com] threads.