Page Speed Discrepancy - Alexa vs. Google Webmaster Tools

[Sally Stitts]

On one of my sites (others are very similar) -

Alexa always reports -
0.78 seconds - 90% of sites are slower

GWT always reports -
2.5 seconds - faster than 57% of sites

This is a 3X difference. Why such a large difference?
Who is more accurate? Alexa or GWT?
How can Alexa get the page in 1/3 the time it takes Google?

[tedster]

The reported numbers are not about tests from any single location or set of servers. Google's numbers are primarily based on Google Toolbar users, and I assume Alexa is based on their installed toolbar base (I couldn't locate specific information about that, but it's a reasonable guess.)

There is probably more detailed methodology behind each set of reported numbers as well, but the difference in the installed user base demographics probably accounts for a big piece of the difference.

[aristotle]

In my opinion Google's method of performing the measurement is seriously flawed, and the results do not represent what the average user experiences.

[Sally Stitts]

" . . the difference in the installed user base demographics probably accounts for a big piece of the difference."

Are you saying that those using the Alexa toolbar, in general, have much faster access than those using the Google toolbar? I can see a small difference, but 3 TIMES?

Your suggestion about different methodologies being employed, begs the question of just how different they might be, as in 3 TIMES? Hard for me to imagine. I think that Aristotle might have something there.

[levo]

Not the connection speed, geographic disparity. Also I think Google reports visits with empty cache.

[icedowl]

Take this for what it's worth. I spent last week on vacation where I could only get online with a dial-up connection. During that week I visited my own sites using ie8 with the google toolbar. The speed reported in WMT slowed down immensely, from ~1.4 seconds up to near the 5.0 second mark. Since I've been back at home and connecting with DSL the speed has returned nearly to where it had been.

[Sally Stitts]

levo -
So, the Google toolbar users are, in general, in the boondocks?
And the Alexa toolbar users are concentrated in urban areas?

I can't really go for this explanation, either.

[KenB]

The answer may lay with sample size. If the sample sizes for the two toolbars are too small then they are going to have a huge margin of error. As icedowl pointed out from his personal experience, it might only be a few users skewing the averages.

I really don't think these page speed stats ban be very accurate unless the sample size is at least 1,000 data points.

[helpnow]

one of my sites is a local real estate site - it is listed at less than a second per page. Most users are physically close to me, and being based in Toronto, Canada, most here have decent high-speed access.

Another of my sites is an international site - it is listed at 4.4 seconds per page. Vistors to this site are from everywhere, including the boondocks. ; )

So, there may be a coorelation vis-a-vis google vs. alexa toolbar, not sure on their geographics, but there may be something to the rural vs urban, local vs intl concept in understanding reported site speed.

Just my 1 cent worth... I don`t think it makes sense to worry about this metric. Implement google`s suggestions on how to speed up your site, for the benefit of your users, and leave it at that.

[KenB]

I have a very significant amount of international traffic on one of my sites and I'm now using geo location to turn off some of my ad slots that wouldn't be serving ads in a given country anyways. This seems to help counter the slowness introduced because of distance.

[tedster]

Here's one difference between the two toolbars - Alexa only released a Firefox toolbar in November 2009.

Whatever the differences in the installed user base are, for several dozen sites that I checked in WMT, it does seem that Google's page speed metrics are consistently on the high side.

[Sally Stitts]

I wonder if Alexa just measures the loading of the regular body text on the page, but Google measures the loading of the page, PLUS the loading of the ads?

I suggest this because of the number of times that I look down at my status bar and see "waiting for ads.doubleclick . ." This might be a partial explanation.

Also, I thought I saw a thread somewhere that Google suggested that removing the ads will speed up load time (or maybe I just dreamed it).

[rainborick]

I don't know that there's much to worry about in terms of the differences in the two services' accuracy. Even if we knew exactly how they both operated and could discuss the merits of one approach over the other, the debate would still rage on. Like most such things, both of these reports seem to be best considered as metrics to use to monitor changes over time, rather than a source of accurate information.

[levo]

Sally Stitts, my point is the latency difference. With a 150-250ms latency the connection speed doesn't help much if you're not using keep-alive, or have many small images etc.

[aristotle]

Quote:
Alexa always reports -
0.78 seconds - 90% of sites are slower

GWT always reports -
2.5 seconds - faster than 57% of sites

Look at the difference in % of slower sites 90% (Alexa) vs 57% (Google). How do you explain that?

I still say that Google's method, whatever it is, is seriously flawed.

[londrum]

google's version must actually wait for the javascript on your page to run, because it takes some of its data from analytics. and given that analytics is always placed at the very end of the page, it would have to wait for all the html and other javascripts to download first.

i'm not sure how alexa's toolbar works, but if it just downloads the javascript without running it (is that possible?) then that will probably take a whole chunk of time off.

[levo]

AFAIK it doesn't use Analytics data. They use Google Toolbar for performance data and "Page Speed" plugin for "Page Speed Suggestions" data.

[icedowl]

google's version must actually wait for the javascript on your page to run, because it takes some of its data from analytics. and given that analytics is always placed at the very end of the page, it would have to wait for all the html and other javascripts to download first.

I'm not running analytics on my site that I visited while on dial-up. Google only had the toolbar to work with.

[anallawalla]

In my opinion Google's method of performing the measurement is seriously flawed, and the results do not represent what the average user experiences.

I tend to agree. WMT tells us that our average page load time is around 6 seconds, which is slower than 80% of the world's sites. When I check our pages from home or elsewhere, my rough timing is closer to 2-3 seconds tops with a full reload to avoid caching.

[tedster]

Some of the really skewed numbers I've been seeing are for sites on a [major] Content Distribution Network - and one of the suggestions I see is the site should use a CDN! That certianly gives me something to chew on.