Lapizuli - 11:07 pm on Apr 9, 2010 (gmt 0)
Great post, Lapizuli.
Just because something is bigger doesn't mean that it's more accurate.
Yes, I agree. And sorry, yes, statistics are a complex science and my example is woefully simplified.
But I still maintain that with greater sample quantity comes greater statistical usefulness. Note that I say usefulness, because in my book it's well nigh meaningless to say "accurate," since it's far trickier to measure accuracy than usefulness. If we could easily test accuracy, we'd just go ahead and be accurate, instead of take statistics. I mean, if we could go to Google and say, "Did you change the earnings algorithm?" and ask our users "Why did you bounce on the page?" and trust everyone's answers to be factual and such we'd do that and not bother with a lot of data collecting fuss. I think the real benefit of stats analysis is for decision making, and what matters is the outcome. If we make decisions based on less useful data, chances are the results will be more variable and less predictable than if we base it on more useful data. All relative.
Anyway, the "greater quantity" I'm talking about may come from a combination of number of pages, longevity of content, number of visitors, range of content types, range of content quality, how much content can dance the rhumba, or any number of other weird and unexpected dimensions. A good analysis will take as many measurable dimensions as possible into account. It's all a guessing game and the questions asked can shape the answers, and yes, statistical data is not remotely perfect.
But the issue here is comparing statistical relevance for two websites that might differ in ten dimensions (and so be apples and oranges, as fearlessrick points out), or in just one. If all else is equal, I'd base my future actions on stats coming from a website that has more numbers. And since online statistical data is possibly one of the most messed-up, organic, and complex set of data there is, it's rare for all else to be equal. But one hopes some is more equal than others.
I often wonder if people would follow prescribed medical recommendations based on the latest health research study if they realized the study was done on 38 female undergraduate students in a private midwestern college or whatever. The media often doesn't differentiate between that kind of study and one that lasted decades and tracked half of Europe. Both studies have something to teach, but both do not have the exact same predictive value.
He makes assumptions that aren't always accurate.
Heh-heh. I'm not a he. But anyway, that's the thing about assumptions. They're never accurate. They're not meant to be. They're for establishing a common starting point in order to make resultant arguments clear.
And to revert to topic...my stats two days ago sucked for half the day, then caught up to average. My stats yesterday were lovely. Today is awful.