Google Webmaster Tools vs Google Analytics - urgent help

[shaunm]

Hi All,

I have been through some weird instances and I need your help very badly.

Google webmaster tools: (Last 3 months data)
Reports the URL: example.com/products/ having regular visits until a couple of weeks ago and fall drastically (ex 1000 to 10) for the last two weeks.
Reports no traces of visits for example.com/products/overview.aspx

Google Analytics: (Last 3 months data)
Reports the URL: example.com/products/overview.aspx having regular visits until this moment.
Reports no traces of visits for example.com/products/

My problem is:

The page at: example.com/products/ has long been redirected to example.com/products/overview.aspx

1. If so, why Google webmaster tools continue to report for example.com/products/?
2. Why Google wemaster tools doesn't have ANY data for example.com/products/overview.aspx even though GA has the data for this?
3. Last and main thing is, do we actually have any traffic loss or not?


Thanks a lot for all your help!

[aakk9999]

First a question:

When you do search that should bring your page up, which URL shows in SERPs? Is it example.com/products/ or example.com/products/overview.aspx ? Can you verify (by using Live HTTP Headers FF extension or something like this) that the URL to which Google initially sends visitors is the target URL and not the URL that redirects to target URL? And if it is the overview.aspx URL, are you sure this was the URL shown in SERPs before the "drastic fall"?

This is my speculation:

I am asking the above because the one thing that crosses my mind is that the redirect is perhaps not a permanent redirect (i.e. that it is 302 and not 301). Or alternatively, even if the redirect is 301, Google has not yet digested this redirect and still sends visitors from SERPs to the old URL.

This would explain why WMT reports impressions/visits for example.com/products/ (if this was URL in SERPs) but Google Analytics shows the URL of the target page example.com/products/overview.aspx as the second one is the page visitors see and the page on which the tracking code is placed. The first page (that redirects) would of course not show in GA as it was not visited (it served redirect instead).

With regards to drop of impressions/visits in WMT, if Google started "changeover" because of redirect, the example.com/products/ may have been removed from SERPs, (whilst perhaps still showing in some data centres) hence the huge drop in WMT, but with still some visits showing.

From GA it would appear you have not lost Google traffic (assuming the "regular visits until this moment" you are refering to, are visits reported to be from Google and not inclusive of directs/referrals which may skew the figures).

It is worth remembering that WMT Search Query report is based on URLs shown on SERPs that are shown/clicked. GA report shows the pages on which visitors landed. These may not be always the same and they will certainly not be the same if a redirect is implemented, but not yet processed in Google SERPs OR if page that redirects is blocked by robots, but has enough links pointing to it for Google to rank it in SERPs

[lucy24]

overlapping aakk, so I hope we didn't wildly contradict each other

3. Last and main thing is, do we actually have any traffic loss or not?

You're quoting information from two different third-party sources. What do the site logs say?

webmaster tools doesn't actually log visits to a page. It only reports clicks on search results. So it's not always an accurate measure of total visits.

GA on the other hand is based on code located on physical pages. Presumably you have a page called /products/overview.aspx. But do you-- or did you-- have a page called /products/index.xtn? Or is overview.aspx the index page of the /products/ directory?

Tangential but necessary question: Why on earth are you redirecting from a shorter prettier URL to a longer uglier one? An index page doesn't have to be called index.html.

[shaunm]

Thank you so much @aakk9999 & lucy24 :-)

When you do search that should bring your page up, which URL shows in SERPs? Is it example.com/products/ or example.com/products/overview.aspx ?

I'm so much confused.

When I do the search for the keyword, the top 1 result is mine. So, when I click on the'cached' it shows absolutely different page, the first characters are capitalized. Like example.com/Products/Overview.aspx and the last crawled date was 10 days before.

So, here's what the direct search for the URLs brings up in SERPs

Search / Cached (Date are same)
example.com/products/ - example.com/Products/Overview.aspx
example.com/products/overview.aspx - example.com/Products/Overview.aspx
example.com/Products/Overview.aspx - example.com/Products/Overview.aspx

The search for the main URL example.com/products/overview.aspx brings 8 sitelinks underneath it in SERPs.

Can you verify (by using Live HTTP Headers FF extension or something like this) that the URL to which Google initially sends visitors is the target URL and not the URL that redirects to target URL?

Both,
example.com/products/
example.com/Products/Overview.aspx are getting redirected to example.com/products/overview.aspx and the redirect is 301.

Btw, how can you see what URL that Google sends visitors initially? Google isn't showing the URL in SERPs but more of like a breadcrumb type of links?

And if it is the overview.aspx URL, are you sure this was the URL shown in SERPs before the "drastic fall"?

I am sure it's overview.aspx because as I said, the redirect from /products/ to /products/overview.aspx performed almost 6 months back. My confusion started as Google webmaster tool reported as example.com/products/ is having regular traffic until the fall whereas adobe sitecatayst reported example.com/products/overview.aspx.

In either case the fall started in the SAME WEEK
In either case they have no data for the other URL

This would explain why WMT reports impressions/visits for example.com/products/ (if this was URL in SERPs) but Google Analytics shows the URL of the target page example.com/products/overview.aspx as the second one is the page visitors see and the page on which the tracking code is placed. The first page (that redirects) would of course not show in GA as it was not visited (it served redirect instead).

That's interesting, indeed! So why does GA says in the tip box right above the URLs columns 'The pages through which visitors entered your site'? If it's the old URL through which the visitors entered the site, it should also display that right? Can you please correct me?

But again the redirect was performed almost 6 months back :(

GA on the other hand is based on code located on physical pages. Presumably you have a page called /products/overview.aspx. But do you-- or did you-- have a page called /products/index.xtn? Or is overview.aspx the index page of the /products/ directory?

We never had a page like /products/index.xtn. Yes overview.aspx is the index page.

Tangential but necessary question: Why on earth are you redirecting from a shorter prettier URL to a longer uglier one? An index page doesn't have to be called index.html.

I cannot make any changes to the process :-)


Guys, I know I might make it bit of messy to understand the situation. To summarize it,

Adobe Sitecatalyst:
Traffic down: example.com/products/overview.aspx
No data at all for: example.com/products/

Google webmaster tools:
Traffic down: example.com/products/ (Same week as adobe even the drop ratio is same)
No data at all for: example.com/products/overview.aspx

Google Analytics:
Regular Traffic: example.com/products/overview.aspx
No data at all for: example.com/products/

The URL at example.com/products/ redirects to example.com/products/overview.aspx and the redirect was performed 6 moths back.

Once again, thank you so much for all your kind help :-)

[aakk9999]

So, when I click on the'cached' it shows absolutely different page, the first characters are capitalized. Like example.com/Products/Overview.aspx

You are on IIS and using .NET and you have canonicalisation problem - such common problem for IIS. All of incorrectly capitalised URLs need to redirect to the your main URL version that you need indexed.

Btw, how can you see what URL that Google sends visitors initially?

As I said, use Firefox Live HTTP Headers extension or Firefox HttpFox. Activate either of these, then do a search on Google and click on your URL from SERPs. You will see the request/response exchange so if there was a redirect, you will see it. Alternatively, you may check your logs after clicking on result in SERPs.

So why does GA says in the tip box right above the URLs columns 'The pages through which visitors entered your site'? If it's the old URL through which the visitors entered the site, it should also display that right? Can you please correct me?

It cannot show the old URL if old URL is redirecting. For a page to show in GA, there must be a tracking code on the page that gets executed when the page is loaded in browser. But the page that redirects is never loaded in browser and the tracking code never executed. So for this reason it cannot show in GA.

The URL at example.com/products/ redirects to example.com/products/overview.aspx and the redirect was performed 6 moths back.

For WMT figures, it is not how long ago you implemented redirect, it matters when Google discovered redirect and replaced your URL in SERPs with the target URL.

[lucy24]

Edit: That's twice!

I cannot make any changes to the process

If you can code a redirect-- assuming htaccess, not in-page-- you can make another htaccess for the /products/ directory and let it say

DirectoryIndex overview.aspx

Or, I guess, IIS equivalent. (Don't look at me.)

So, when I click on the'cached' it shows absolutely different page, the first characters are capitalized. Like example.com/Products/Overview.aspx and the last crawled date was 10 days before.

That's normal. If your name is WebmasterWorld, the cached page may be from the crawl done half an hour ago, as opposed to the more recent crawl from five minutes ago. Ordinary mortals take longer.

'The pages through which visitors entered your site'? If it's the old URL through which the visitors entered the site, it should also display that right? Can you please correct me?

I think you may not understand what happens physically.

#1 user comes to your site, requesting /oldname.php
#2 user never sets eyes on this page, because site redirects them to (= "tells the browser to ask for") /newname.php
#3 user makes a fresh request, this time for /newname.php, and is allowed to reach the page
#4 /newname.php leads to various supporting files, including the google analytics code --which might be physically in the page's html, or maybe in a nearby file linked via a <script src> tag
#5 this in turn sends information to GA's database somewhere in Cupertino or Redmond or, uh, wherever they keep it

GA has no way of knowing that the user originally requested /oldname.php. Each request is an island. Information about redirects isn't normally included in a request header. ("Hi! Remember me? I was here two nanoseconds ago asking for /oldname.php and you told me to come over here and ask for /newname.php instead." This does not happen.) The new request comes with the same referer as the old request, even if the referer (google dot com, for example) actually told you to go to /oldname.php.

Can you verify (by using Live HTTP Headers FF extension or something like this) that the URL to which Google initially sends visitors is the target URL and not the URL that redirects to target URL?

Both,
example.com/products/
example.com/Products/Overview.aspx are getting redirected to example.com/products/overview.aspx and the redirect is 301.

No, that wasn't the question. There's no doubt the redirect is taking place. The question is what URL the google SERP sends you to. In fact you should be able to check without using any kind of Live Headers; just hover over the link and your browser will say what it is.

[shaunm]

@aakk9999 Thanks :-)

You are on IIS and using .NET and you have canonicalisation problem - such common problem for IIS. All of incorrectly capitalised URLs need to redirect to the your main URL version that you need indexed.

That incorrectly capitalized URL is now redirecting to the target URL. But how come that capitalized URL stayed in Google's cache? Does that mean that capitalized URL were there live for some days before it were actually redirected? And Google still has the capitalized URL but not the target URL because the redirect recently happened?

For WMT figures, it is not how long ago you implemented redirect, it matters when Google discovered redirect and replaced your URL in SERPs with the target URL.

But how come WMT reports a redirected URL until couple of weeks ago even though the redirect was done 6 months back? Have you gone through such a case before where Google is not capturing the new destination and sticking with the old URL?

@lucy24 Thanks :-)

That's normal. If your name is WebmasterWorld, the cached page may be from the crawl done half an hour ago, as opposed to the more recent crawl from five minutes ago. Ordinary mortals take longer.

&

No, that wasn't the question. There's no doubt the redirect is taking place. The question is what URL the google SERP sends you to. In fact you should be able to check without using any kind of Live Headers; just hover over the link and your browser will say what it is.

So, when I hover over the link it says 'example.com/Products/Overview.axpx a incorrectly capitalized version of the URL as aak said.

To both,

Thank you for your kind help in this regard. Btw, I have no clue why Sitecatalyst report for a longer version while WMT reports for the shorter version. The drop happens exactly in the same week. In reality there isn't a shorter version which was only extinct 6 months ago. So, if there isn't a shorter version, how come the drop aligns with the data of the longer version as reported by Sitecatalyst?


Thanks a TON!

[aakk9999]

But how come WMT reports a redirected URL until couple of weeks ago even though the redirect was done 6 months back? Have you gone through such a case before where Google is not capturing the new destination and sticking with the old URL?

Let me try to explain this differently :)

Lets say you lived on "FirstStreet 10" and then you decide to move a house. So, on the door of your old house (which is in "FirstStreet10") you stick a note with a message "I am now living in SecondStreet 20".

I know your old address, but I haven't visited you for a while. Whoever asks me where do you live, I reply "FirstStreet 10" because this is what I know - as I haven't visited, I have not seen your note that you have moved.

A year after your move, I decide to visit. I come to "FirstStreet 10" and I see the note that you moved to "SecondStreet 20". I go and visit you at "SecondStreet 20" and seeing you are there, I can update my address book. And only AFTER this, when someone asks for your address, I can reply that it is "SecondStreet 20".

So in this case for the whole year I was sending a visitors to your old address. But they could not enter, they saw the note and they then went to your new address. But I did not know that. So if I was counting how many visitors I sent you, I would report to all these visitors sent to your OLD address - because this is where I thought you live - even though you have moved ages ago.

[shaunm]

@aakk9999

Thank you so much for such a wonderful explanation of how WMT differs from GA. And kudos for the write up :-)

Best,