Forum Moderators: Robert Charlton & goodroi
Let's say you test something on a minimum 30 or so sites and it works on all 30 sites.
Is that how such tests can be - and have been - conducted using 30 test sites? Everything is completely identical other than the one factor being tested, right?
[edited by: tedster at 7:14 pm (utc) on Dec. 18, 2008]
Ah but... then you are falling towards the 'test it and see [TIAS]' camp WRT advising others. Effectively you are saying "this works withing my quantum field, thus it must work within yours". They whole point of Methodology Testing is that it produces 'stateless' results.
hmmm think we are talking 2 different "octaves" here.
There is no such thing as "stateless" results.
Theres no COMPLETELY OBJECTIVE universe that exists OUTSIDE the observer.
(lol, ponder this one. It's paradoxical but ultimately necessary for deconstructing the algo intuitively.)
The "unconscious bias" is EASILY reduced when creating test sites if one knows about it to begin with. Even if it's something as simple as using different CMSs or templates or coding.
Certainly nothing to OVER-estimate when testing a PROBABILITY.
It's similar to when they ask polling questions by not rotating questions.
Do you like Candidate X,Y,Z? Z,Y,X? Y,Z,X? etc.
Yes, there is a SMALL Bias found within the order the questions or options are given, but it rarely skews results considerably.
You don't get 60% "favorability" rating when asked one way and then a 40% the other way.
Perhaps a 55% v 60% is likely however.
[edited by: whitenight at 8:56 pm (utc) on Dec. 22, 2008]
One of the ways in which the try-it-on-one-site approach fails is that people try all kinds of things at basically the same time....So I recommend keeping a site changelog for key changes - especially sitewide changes, server configurations and that kind of thing. It can be a lifeline
excellent advice!
Our statistician chose 180 domain names at random and grouped them into three groups of 60:
The last two groups were linked from about 10 well-established sites - some were verticals and the rest were not topic-specific. Each site had content matching the domain name, but being randomly chosen, there were some domain names with no search volume to speak of, e.g. [grammatical term].com and some with high search volumes, e.g. [finance term].com.
As the sites had no content history other than the parking history, they were fairly new domains and were spidered almost at the same time, being linked from a handful of sites.
Google indexed them the fastest, Yahoo next and Live not worth mentioning. Traffic was directly in proportion to a site's theme and number of pages, e.g. a sports theme site with a .com domain got a lot of visits from people presumably looking for the .net official site. Similarly, an airline named widget-airlines was launched around then and a domain named after the widget got a lot of visits.
As a testing methodology, this had statistical rigor, but the SEO in me said that we would not prove anything by selecting domain names for exotic topics that were rarely searched. I'd have picked 60 random domain names from highly competitive niches and 60 from moderately competitive niches.
I accept that Google's algos change all the time, as does the economy, and many other factors, so each domain name will show different results at different times. At any given time you have to pick a methodology and give it your best shot.
Crush, wasn't it you in one of the recent threads asking for assistance with one of your websites that either went -950, or was substantially hit?
When people have developed websites for years and written thousands of unique pieces of content, simply writing off issues as 'unexplained' doesn't cut it.
Then there are others who sincerely love the subject of SEO, statistics and linguistics. It is those people on this forum that fuel much of the active theories and advancements in SEO.
Also the site in question came back when I did nothing to it.
Here's an area where my I feel my interest in Google's patents (and information retrieval altogether) helps me understand. In recent years, Google patents described penalties that are based on going over a threshold. That threshold can be set automatically baased on their own statistical testing of factors, and then recalculated every so often.
So it makes sense that a site's rankings could bounce back, even if you changed nothing. Google also releases filters that have been imposed on sets of sites at various times - such an action was just mentioned a few weeks ago. So there's another reason why sites can bounce back without any changes. This is not an area where we could easily test without outrageous resources, however.
--------
A more valuable way that I feel patents can help is suggesting WHICH factors to test. Sometimes you only get mixed results from your tests, and nothing is really conclusive. The patents can give some hints about other factors that could be affecting the SERPs that you may not have considered. These factors can be worth testing to see if the patented approach could actually be in use.
The Historical Data [webmasterworld.com] patent was that kind of an eye opener for many people. In the past, some SEOs were used to tweaking a page every day until they got a better ranking for it. As some recent threads here show, today you can overdo the frequency of on-page changes in sensitive areas. The laundry list of historical factors in that patent really freaked some people out when it was first published in 2005 [webmasterworld.com]
As some recent threads here show, today you can overdo the frequency of on-page changes in sensitive areas. The laundry list of historical factors in that patent really freaked some people out when it was first published in 2005
<mumbles about patents> lol
You kinda illustrate 2 main points I have about "patents".
A. "freaked out people in 2005".
Really?
People freaking out 2 full years before I even got a whiff of the historical data having a REAL impact on rankings?
2-3 years is a long time to be looking for the "end of the world" when one could have been STILL taking advantage of an "non-yet-patent-implemented" algo.
B. People are noticing this fully NOW, but it started in earnest about 6-9 months ago.
How do you solve BOTH of this issues to be up-to-date IMMEDIATELY on what the ALGO is doing?
TESTING. :)
In recent years, Google patents described penalties that are based on going over a threshold. That threshold can be set automatically based on their own statistical testing of factors, and then recalculated every so often. So it makes sense that a site's rankings could bounce back, even if you changed nothing.
Imagine sites are set to be judged like students. The class has a bell curve for grading. Relative performance becomes the issue. If your site stays the same but others change, you move on the bell curve.
Somebody speculated a while back (don't recall if it was based on a patent) that Google looks at averages or benchmarks for industries and from the base they develop standards to judge other sites within that industry.
Some webmasters may have got carried away, however, guessing the amount of detail to which the historical algo would be set. We also don't know how often it is adjusted, i.e., how frequently the base is determined and sites rechecked against the new base. I suspect, nevertheless, the frequency is probably greater now than in the past.
So much of Google ranking is a flux now it's very difficult to make sense of what is going on. By the time you finish running a test the data could be obsolete. I figured out how to solve this problem. Buy Google and write the algo! :/
Speaking of patents, I really need to start reading them more, but having said that, I don't know why Google keeps applying for them. The Google algo is not like other IP technology where you can open it up and see how it works then copy it. And how would Google even be able to tell if another SE copied its ideas and incorporated them into their own algo?
p/g
And how would Google even be able to tell if another SE copied its ideas and incorporated them into their own algo?
I would imagine they can reverse engineer their own algo, and I suspect they know the boundries and conditions where the normal rules break down. If I were them, I would have 'honeypot' sites exploiting the boundries, and then be prepared to sue on suspicion and confirm in Discovery.
@whitenight
hmmm think we are talking 2 different "octaves" here.There is no such thing as "stateless" results.
Obviously in a ranking situation (where by definition several things are being compared) you cannot escape dependencies- no recourse to ideas borrowed from quantum (or indeed meta~) physics required. That said, knowledge that is universally applicable without reference to the context is the (unacheivable) ideal. However, to couch every statement in qualifiers makes it too opaque to follow on boards where rapid digestion is the key to dialog- otherwise your post gets skipped by a good proportion of readers. To recap the conversation (paraphrased):
Ted: Testing only your sites mean you include personal bias
WN: Personal bias is irrelevant 98% of the time, as you are you are applying the results on personal sites
Shad: Yes, but then you lose the right to say your results are unqualifiably tranferable to others
WN: <replies with critique of statelessness and the bidirectional nature of testing>
edit: spelling and punctuation
[edited by: Shaddows at 9:51 am (utc) on Dec. 24, 2008]
Somebody speculated a while back (don't recall if it was based on a patent) that Google looks at averages or benchmarks for industries and from the base they develop standards to judge other sites within that industry.
It's in the patents. There's one "main" historical data patent that's been granted, and several related patents that are published, but still in the application stage.
Some webmasters may have got carried away, however, guessing the amount of detail to which the historical algo would be set.
We also don't know how often it is adjusted, i.e., how frequently the base is determined and sites rechecked against the new base. I suspect, nevertheless, the frequency is probably greater now than in the past.So much of Google ranking is a flux now it's very difficult to make sense of what is going on. By the time you finish running a test the data could be obsolete.
I figured out how to solve this problem. Buy Google and write the algo! :/
Nope, definately same octave, but you are misunderstanding me. I was trying to advance the point about advising others and used oversimpification as rhetorical device, while carelessly allowing the post to be interpretted as a statement of intent about testing methodology
lol sorry shaddows. Thought you realized you, tedster, and I were ALL talking on 2 different octaves.
(The theory itself and the theorizing of theorizing - from before)
I'll try to curb my "unspoken conversations" during this thread =P
(But it's sooo fun!)
----------------
only one change be clearly seen after only 3-4 days. More than once.
M: only one change be clearly seen after only 3-4 days. More than once.
WN: This sounds about right for on-site changes.
Re Historical Data:
"freaked out people in 2005".
Really?
People freaking out 2 full years before I even got a whiff of the historical data having a REAL impact on rankings?
2-3 years is a long time to be looking for the "end of the world" when one could have been STILL taking advantage of an "non-yet-patent-implemented" algo.
B. People are noticing this fully NOW, but it started in earnest about 6-9 months ago.
A more valuable way that I feel patents can help is suggesting WHICH factors to test. Sometimes you only get mixed results from your tests, and nothing is really conclusive. The patents can give some hints about other factors that could be affecting the SERPs that you may not have considered. These factors can be worth testing to see if the patented approach could actually be in use.
That's fertile ground for factors to test. And testing doesn't always need to be as complicated and arcane (and difficult) as some people, for whatever their reasons are, try to make it seem; sometimes (often) it can be very simple - and quick.
[edited by: Marcia at 6:11 pm (utc) on Dec. 24, 2008]
only one change be clearly seen after only 3-4 days
What kind of change was it if you could possibly mention? An addition of a link, more body text.
Trying to see what kind of changes' affects will show in several days.
When people have developed websites for years and written thousands of unique pieces of content, simply writing off issues as 'unexplained' doesn't cut it.
and i see plenty of good SEO's on these forums recommending the line of "cut the old site and set up new sites" .
For many established sites supporting a brand based business with a "cut and run" approach just isn't possible.
Persistance and networking with experts on forums like this over a multitude of experiences is the only way to gather sufficient knowledge to pin point issues, and even then it doesn't come easily and has a lot of risk associated with it. Disclosures rarely come in one piece however or come in "one size fits all".
On the other hand, Google likely observes these discussions to counteract any moving consensus that would allow a significant proportion of site owners to master their own results to any great scale and consistency.
Google's commercially too smart for that.
